verl icon indicating copy to clipboard operation
verl copied to clipboard

Qwen2Tokenizer has no attribute all_special_tokens_extended

Open iqiancheng opened this issue 2 months ago • 0 comments

Qwen3--vl-30b-a3b verl:vllm011:latest

verl/workers/rollout/vllm_rollout/vllm_async_server.py", line 347, in run_server
    engine_client = AsyncLLM.from_vllm_config(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/utils/__init__.py", line 1572, in inner
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 207, in from_vllm_config
    return cls(
           ^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 114, in __init__
    self.tokenizer = init_tokenizer_from_configs(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/tokenizer.py", line 286, in init_tokenizer_from_configs
    return get_tokenizer(
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/tokenizer.py", line 256, in get_tokenizer
    tokenizer = get_cached_tokenizer(tokenizer)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/tokenizer.py", line 99, in get_cached_tokenizer
    tokenizer.all_special_tokens_extended)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/transformers/tokenization_utils_base.py", line 1329, in __getattr__
    raise AttributeError(f"{self.__class__.__name__} has no attribute {key}")
AttributeError: Qwen2Tokenizer has no attribute all_special_tokens_extended. Did you mean: 'num_special_tokens_to_add'?

iqiancheng avatar Nov 28 '25 08:11 iqiancheng