caofx0418

高手，在高通8gen3上，llama2-7B推理性能如何？ 多少token/s ?

Llama 7b 有20token 太棒了！ 使用llama.cpp在8gen3的cpu上4线程推理llama2-7B最多只有5token

I also have the same issue. when i use gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu to compile opencv-4.10, CMakeFiles/opencv_dnn.dir/src/layers/cpu_kernels/conv_winograd_f63.cpp.o: In function `std::_Function_handler::_M_invoke(std::_Any_data const&, cv::Range const&)': conv_winograd_f63.cpp:(.text._ZNSt17_Function_handlerIFvRKN2cv5RangeEEZNS0_3dnn13runWinograd63ERKNS0_11_InputArrayES8_RKNS0_12_OutputArrayERK NS0_3PtrINS5_8FastConvEEEiffPNS5_14dnn4_v2024052115ActivationLayerEbEUlS3_E_E9_M_invokeERKSt9_Any_dataS3_+0x304): undefined reference to `cv: :dnn::opt_NEON::winofunc_BtXB_8x8_F32(float const*, int, float*,...

> I also have the same issue. when i use gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu to compile opencv-4.10, > > CMakeFiles/opencv_dnn.dir/src/layers/cpu_kernels/conv_winograd_f63.cpp.o: In function `std::_Function_handler::_M_invoke(std::_Any_data const&, cv::Range const&)': conv_winograd_f63.cpp:(.text._ZNSt17_Function_handlerIFvRKN2cv5RangeEEZNS0_3dnn13runWinograd63ERKNS0_11_InputArrayES8_RKNS0_12_OutputArrayERK NS0_3PtrINS5_8FastConvEEEiffPNS5_14dnn4_v2024052115ActivationLayerEbEUlS3_E_E9_M_invokeERKSt9_Any_dataS3_+0x304): undefined reference to `cv: :dnn::opt_NEON::winofunc_BtXB_8x8_F32(float...