Home Sign Up Sign In
Auston
V2EX  ›  问与答

有了解 fast tokenizer 的老哥吗？怎么将 baichuan2-13b 的 tokenizer 转成 fast tokenizer?

  •   
  •   Auston · Apr 20, 2024 · 1313 views
    This topic created in 817 days ago, the information mentioned may be changed or developed.
    目前在用 vllm 做 baichuan2-13b-chat 模型的推理框架，想进一步加速推理速度，怎么将 slow tokenizer 转成 fast tokenizer 呢？
  • Fast
  • tokenizer
  • 转换
    1 replies
    Auston
        1
    Auston  
    OP
       Apr 23, 2024
    大佬们，能给个思路吗
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Solana   ·   1052 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 40ms · UTC 18:54 · PVG 02:54 · LAX 11:54 · JFK 14:54
    ♥ Do have faith in what you're doing.