ncnn icon indicating copy to clipboard operation
ncnn copied to clipboard

[feat](benchmark): replace load_param with load_param_mem

Open 0130w opened this issue 5 months ago • 4 comments

Previous implementation required benchncnn to be run from the directory containing the model files. This often caused failures and inconvenience when executed from an arbitrary working directory.​​

​This change embeds the model files directly into the executable by compiling them as C arrays. Consequently, benchncnn now becomes fully self-contained and can be run from any location without dependency on external model files.​

0130w avatar Aug 26 '25 19:08 0130w

The binary size change of libncnn.so (bytes)

architecture base size pr size difference
x86_64 15328640 15212648 -115992 :kissing_heart:
armhf 6229904 6206656 -23248 :kissing_heart:
aarch64 9527568 9524560 -3008 :kissing_heart:

github-actions[bot] avatar Aug 26 '25 19:08 github-actions[bot]

CLA assistant check
All committers have signed the CLA.

tencent-adm avatar Aug 31 '25 18:08 tencent-adm

run

cd build
cmake -DCMAKE_BUILD_TYPE=Debug -DNCNN_VULKAN=ON -DNCNN_BUILD_EXAMPLES=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ..
make -j$(nproc)
cd benchmark
./benchncnn 4 8 0 -1 1

result before

./benchncnn 4 8 0 -1 1
loop_count = 4
num_threads = 8
powersave = 0
gpu_device = -1
cooling_down = 1
fopen squeezenet.param failed
network graph not ready
          squeezenet  min =    0.00  max =    0.00  avg =    0.00
fopen squeezenet_int8.param failed
...

after

loop_count = 4
num_threads = 8
powersave = 0
gpu_device = -1
cooling_down = 1
          squeezenet  min =   12.68  max =   13.12  avg =   12.94
     squeezenet_int8  min =    8.39  max =    9.11  avg =    8.88
           mobilenet  min =   16.69  max =   17.01  avg =   16.83
      mobilenet_int8  min =   15.41  max =   16.59  avg =   15.86
        mobilenet_v2  min =   16.33  max =   17.03  avg =   16.67
        mobilenet_v3  min =   13.08  max =   15.00  avg =   14.34
          shufflenet  min =   10.04  max =   12.69  avg =   10.88
       shufflenet_v2  min =   10.14  max =   13.40  avg =   11.20
             mnasnet  min =   12.67  max =   16.30  avg =   14.48
     proxylessnasnet  min =   14.56  max =   16.55  avg =   15.68
     efficientnet_b0  min =   23.54  max =   25.99  avg =   24.56
   efficientnetv2_b0  min =   25.69  max =   28.93  avg =   27.90
...

0130w avatar Aug 31 '25 19:08 0130w

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 95.91%. Comparing base (5154f22) to head (41f463d). :warning: Report is 22 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6286      +/-   ##
==========================================
+ Coverage   95.89%   95.91%   +0.02%     
==========================================
  Files         841      844       +3     
  Lines      266337   267021     +684     
==========================================
+ Hits       255409   256122     +713     
+ Misses      10928    10899      -29     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov-commenter avatar Sep 01 '25 00:09 codecov-commenter