FastDeploy icon indicating copy to clipboard operation
FastDeploy copied to clipboard
verified publisher icon PaddlePaddle

自编译的 fastdeploy_python-1.0.7-py3.10-linux-x86_64 使用 openvino 作为backend 报错, paddle, ort 工作正常

Open jianweimama opened this issue 10 months ago • 2 comments

温馨提示：根据社区不完全统计，按照模板提问，可以加快回复和解决问题的速度

环境

  • 【FastDeploy版本】： 说明具体的版本， fastdeploy_python-1.0.7-py3.10-linux-x86_64

  • 【编译命令】如果您是自行编译的FastDeploy，请说明您的编译方式（参数命令） 参考: https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/cpu.md git clone https://github.com/PaddlePaddle/FastDeploy.git git checkout origin/release/1.0.7

cd FastDeploy/python export ENABLE_ORT_BACKEND=ON export ENABLE_PADDLE_BACKEND=ON export ENABLE_OPENVINO_BACKEND=ON export ENABLE_VISION=ON export ENABLE_TEXT=ON export OPENCV_DIRECTORY=/usr/lib/x86_64-linux-gnu/cmake/opencv4

python setup.py build python setup.py bdist_wheel

-- ***FastDeploy Building Summary

-- CMake version : 3.27.2 -- CMake command : /usr/local/lib/python3.10/dist-packages/cmake/data/bin/cmake -- System : Linux -- C++ compiler : /usr/bin/c++ -- C++ compiler version : 12.3.0 -- CXX flags : -Wno-format -g0 -O3 -- EXE linker flags : -- Shared linker flags : -- Build type : Release -- Compile definitions : _GLIBCXX_USE_CXX11_ABI=1;FASTDEPLOY_LIB;CMAKE_BUILD_TYPE=Release;ENABLE_ORT_BACKEND;ENABLE_PADDLE_BACKEND;ENABLE_OPENVINO_BACKEND;ENABLE_VISION;ENABLE_TEXT;ENABLE_PADDLE2ONNX -- CMAKE_PREFIX_PATH : -- CMAKE_INSTALL_PREFIX : /usr/local -- CMAKE_MODULE_PATH :

-- FastDeploy version : 1.0.7 -- ENABLE_ORT_BACKEND : ON -- ENABLE_RKNPU2_BACKEND : OFF -- ENABLE_HORIZON_BACKEND : OFF -- ENABLE_SOPHGO_BACKEND : OFF -- ENABLE_PADDLE_BACKEND : ON -- ENABLE_LITE_BACKEND : OFF -- ENABLE_POROS_BACKEND : OFF -- ENABLE_TRT_BACKEND : OFF -- ENABLE_OPENVINO_BACKEND : ON -- ENABLE_BENCHMARK : OFF -- ENABLE_VISION : ON -- ENABLE_TEXT : ON -- ENABLE_ENCRYPTION : OFF -- ENABLE_FLYCV : OFF -- ENABLE_CVCUDA : OFF -- WITH_GPU : OFF -- WITH_IPU : OFF -- WITH_OPENCL : OFF -- WITH_TESTING : OFF -- WITH_ASCEND : OFF -- WITH_DIRECTML : OFF -- WITH_TIMVX : OFF -- WITH_KUNLUNXIN : OFF -- WITH_CAPI : OFF -- WITH_CSHARPAPI : OFF -- ONNXRuntime version : 1.12.0 -- Paddle Inference version : 0.0.0.660f781b77 -- OpenVINO version : dev.2023.03.2 -- Python executable : /home/test/py310_paddle_cpu/bin/python -- Python includes : /usr/include/python3.10

  • 【系统平台】: Linux x64(Ubuntu 22.04)
  • 【硬件】： 说明具体硬件型号，Xeon-SP CPU
  • 【编译语言】： Python(3.10）

    python list

Python 3.10.12 openvino 2024.6.0 openvino-dev 2024.6.0 openvino-telemetry 2025.1.0

问题日志及出现问题的操作流程

测试, 参考: https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/vision/ocr/PP-OCR/cpu-gpu/python 看log, openvino 作为backend 已经初始化完成.

FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python# python infer.py --det_model ch_PP-OCRv3_det_infer --cls_model ch_ppocr_mobile_v2.0_cls_infer --rec_model ch_PP-OCRv3_rec_infer --rec_label_file ppocr_keys_v1.txt --image kt.png --device cpu --backend paddle WARNING: Logging before InitGoogleLogging() is written to STDERR W0419 17:08:07.493630 923328 analysis_config.cc:971] It is detected that mkldnn and memory_optimize_pass are enabled at the same time, but they are not supported yet. Currently, memory_optimize_pass is explicitly disabled [INFO] fastdeploy/runtime/runtime.cc(273)::CreatePaddleBackend Runtime initialized with Backend::PDINFER in Device::CPU. [INFO] fastdeploy/runtime/runtime.cc(273)::CreatePaddleBackend Runtime initialized with Backend::PDINFER in Device::CPU. [INFO] fastdeploy/runtime/runtime.cc(273)::CreatePaddleBackend Runtime initialized with Backend::PDINFER in Device::CPU. det boxes: [[6,4],[664,4],[664,18],[6,18]]rec text: KTransformers（KVCache,Al出品）是一个灵活的本地 LLM 推理优化框架，旨在通过高级内 rec score:0.914088 cls label: 0 cls score: 0.940954 det boxes: [[6,29],[664,29],[664,44],[6,44]]rec text: 核优化和模型部署/井行策略提升Transformer模型推理性能(GitHub -kvcache- rec score:0.931263 cls label: 0 cls score: 0.967136 det boxes: [[6,53],[665,53],[665,68],[6,68]]rec text: ai/ktransformers: A Flexible Framework for Experiencing Cutting-edge LLM rec score:0.914593 cls label: 0 cls score: 0.760472 det boxes: [[6,77],[664,78],[664,93],[6,92]]rec text: InferenceOptimizations)。它以注入式设计嵌入在HuggingFaceTransformers接口之 rec score:0.926265 cls label: 0 cls score: 0.960029 det boxes: [[7,102],[665,102],[665,117],[7,117]]rec text: 上，用一行代码即可将模型替换为优化版本，提供与HuggingFace接口兼容的使用方式，以及 rec score:0.941941 cls label: 0 cls score: 0.995538 det boxes: [[7,127],[665,127],[665,142],[7,142]]rec text: OpenAl APl兼容的REsT接和简易 Web Ul (GitHub - kvcache-ai/ktransformers: A rec score:0.904875 cls label: 0 cls score: 0.932776 det boxes: [[6,151],[653,151],[653,166],[6,166]]rec text: Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations). rec score:0.955298 cls label: 0 cls score: 0.802574

FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python# python infer.py --det_model ch_PP-OCRv3_det_infer --cls_model ch_ppocr_mobile_v2.0_cls_infer --rec_model ch_PP-OCRv3_rec_infer --rec_label_file ppocr_keys_v1.txt --image kt.png --device cpu --backend ort [INFO] fastdeploy/runtime/runtime.cc(300)::CreateOrtBackend Runtime initialized with Backend::ORT in Device::CPU. [INFO] fastdeploy/runtime/runtime.cc(300)::CreateOrtBackend Runtime initialized with Backend::ORT in Device::CPU. [INFO] fastdeploy/runtime/runtime.cc(300)::CreateOrtBackend Runtime initialized with Backend::ORT in Device::CPU. det boxes: [[6,4],[664,4],[664,18],[6,18]]rec text: KTransformers（KVCache,Al出品）是一个灵活的本地 LLM 推理优化框架，旨在通过高级内 rec score:0.914088 cls label: 0 cls score: 0.940954 det boxes: [[6,29],[664,29],[664,44],[6,44]]rec text: 核优化和模型部署/井行策略提升Transformer模型推理性能(GitHub -kvcache- rec score:0.931263 cls label: 0 cls score: 0.967136 det boxes: [[6,53],[665,53],[665,68],[6,68]]rec text: ai/ktransformers: A Flexible Framework for Experiencing Cutting-edge LLM rec score:0.914593 cls label: 0 cls score: 0.760473 det boxes: [[6,77],[664,78],[664,93],[6,92]]rec text: InferenceOptimizations)。它以注入式设计嵌入在HuggingFaceTransformers接口之 rec score:0.926265 cls label: 0 cls score: 0.960029 det boxes: [[7,102],[665,102],[665,117],[7,117]]rec text: 上，用一行代码即可将模型替换为优化版本，提供与HuggingFace接口兼容的使用方式，以及 rec score:0.941942 cls label: 0 cls score: 0.995538 det boxes: [[7,127],[665,127],[665,142],[7,142]]rec text: OpenAl APl兼容的REsT接和简易 Web Ul (GitHub - kvcache-ai/ktransformers: A rec score:0.904874 cls label: 0 cls score: 0.932776 det boxes: [[6,151],[653,151],[653,166],[6,166]]rec text: Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations). rec score:0.955297 cls label: 0 cls score: 0.802574

FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python# python infer.py --det_model ch_PP-OCRv3_det_infer --cls_model ch_ppocr_mobile_v2.0_cls_infer --rec_model ch_PP-OCRv3_rec_infer --rec_label_file ppocr_keys_v1.txt --image kt.png --device cpu --backend openvino [INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(218)::InitFromPaddle number of streams:1. [INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(228)::InitFromPaddle affinity:YES. [INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(240)::InitFromPaddle Compile OpenVINO model on device_name:CPU. [INFO] fastdeploy/runtime/runtime.cc(286)::CreateOpenVINOBackend Runtime initialized with Backend::OPENVINO in Device::CPU. [INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(218)::InitFromPaddle number of streams:1. [INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(228)::InitFromPaddle affinity:YES. [INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(240)::InitFromPaddle Compile OpenVINO model on device_name:CPU. [INFO] fastdeploy/runtime/runtime.cc(286)::CreateOpenVINOBackend Runtime initialized with Backend::OPENVINO in Device::CPU. [INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(218)::InitFromPaddle number of streams:1. [INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(228)::InitFromPaddle affinity:YES. [INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(240)::InitFromPaddle Compile OpenVINO model on device_name:CPU. [INFO] fastdeploy/runtime/runtime.cc(286)::CreateOpenVINOBackend Runtime initialized with Backend::OPENVINO in Device::CPU. Traceback (most recent call last): File "/home/test/FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python/infer.py", line 211, in result = ppocr_v3.predict(im) File "/home/test/py310_paddle_cpu/lib/python3.10/site-packages/fastdeploy_python-1.0.7-py3.10-linux-x86_64.egg/fastdeploy/vision/ocr/ppocr/init.py", line 958, in predict return self.system_.predict(input_image) RuntimeError: Primitive descriptor was not found for node linear_1.tmp_1.

jianweimama avatar Apr 19 '25 09:04 jianweimama

补充一下: 对于 AI model 我使用的还是 paddle 格式, 如果使用转成 OV 的IR 格式, 直接报错: FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python# python infer.py --det_model ch_PP-OCRv3_det_infer_ov --cls_model ch_ppocr_mobile_v2.0_cls_infer_ov --rec_model ch_PP-OCRv3_rec_infer_ov --rec_label_file ppocr_keys_v1.txt --image kt.png --device cpu --backend openvino Traceback (most recent call last): File "/home/test/FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python/infer.py", line 178, in det_model = fd.vision.ocr.DBDetector( File "/home/test/py310_paddle_cpu/lib/python3.10/site-packages/fastdeploy_python-1.0.7-py3.10-linux-x86_64.egg/fastdeploy/vision/ocr/ppocr/init.py", line 209, in init self._model = C.vision.ocr.DBDetector( RuntimeError: [ NETWORK_NOT_READ ] Unable to read the model: ch_PP-OCRv3_det_infer_ov/inference.pdmodel Please check that model format: pdmodel is supported and the model is correct. Available frontends: ir tflite onnx paddle tf pytorch

jianweimama avatar Apr 19 '25 09:04 jianweimama

使用官方编译的镜像fastdeploy:1.0.7-cpu-only-21.10,也出现同样的问题，CPU型号也是Xeon

pioneer12345 avatar May 06 '25 08:05 pioneer12345