-
Notifications
You must be signed in to change notification settings - Fork 565
Description
我使用这个命令启动ERNIE-4.5-VL-28B-A3B-Paddle模型,CUDA_VISIBLE_DEVICES=3,4,5,6 FASTDEPLOY_MOE_BACKEND=paddle python -m fastdeploy.entrypoints.openai.api_server --model /workspace/LlmFile/ERNIE-4.5-VL-28B-A3B-Paddle --port 8180 --max-model-len 20000 --tensor-parallel-size 4 --gpu-memory-utilization 0.75
遇到这报错:/usr/local/lib/python3.10/dist-packages/_distutils_hack/init.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
[2025-07-15 05:57:54,291] [ INFO] - Loading configuration file /workspace/LlmFile/ERNIE-4.5-VL-28B-A3B-Paddle/generation_config.json
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/entrypoints/openai/api_server.py", line 385, in
main()
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/entrypoints/openai/api_server.py", line 376, in main
if load_engine() is None:
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/entrypoints/openai/api_server.py", line 85, in load_engine
if not engine.start(api_server_pid=os.getpid()):
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/engine/engine.py", line 188, in start
self.data_processor = self.input_processor.create_processor()
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/input/preprocess.py", line 77, in create_processor
self.processor = DataProcessor(
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/input/text_processor.py", line 174, in init
self.tokenizer = self._load_tokenizer()
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/input/text_processor.py", line 548, in _load_tokenizer
return AutoTokenizer.from_pretrained(self.model_name_or_path,
File "/usr/local/lib/python3.10/dist-packages/paddleformers/transformers/auto/tokenizer.py", line 389, in from_pretrained
raise ValueError(
ValueError: Tokenizer class Ernie4_5_VLTokenizer does not exist or is not currently imported.
请问怎么解决,谢谢!