flow-go icon indicating copy to clipboard operation
flow-go copied to clipboard
verified publisher icon onflow

Add bitswap metrics

Open synzhu opened this issue 3 years ago • 8 comments

closes https://github.com/onflow/flow-go/issues/2679

synzhu avatar Aug 04 '22 20:08 synzhu

bors merge

synzhu avatar Aug 16 '22 19:08 synzhu

Build failed (retrying...):

bors[bot] avatar Aug 16 '22 20:08 bors[bot]

Canceled.

bors[bot] avatar Aug 16 '22 20:08 bors[bot]

bors merge

synzhu avatar Aug 16 '22 21:08 synzhu

FVM Benchstat comparison

This branch with compared with the base branch onflow:master commit f14494afc08463017025cf1de300c3acd12e378d

The command (for i in {1..10}; do go test ./fvm ./engine/execution/computation --bench . --tags relic -shuffle=on --benchmem --run ^$; done) was used.

Collapsed results for better readability

old.txtnew.txt
time/opdelta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/get_signer_vault-253.4ms ± 3%55.2ms ± 2%+3.40%(p=0.001 n=9+10)
RuntimeTransaction/reference_tx-242.6ms ± 5%42.8ms ± 6%~(p=0.968 n=10+9)
RuntimeTransaction/convert_int_to_string-246.8ms ±13%46.4ms ± 5%~(p=0.853 n=10+10)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-248.2ms ± 4%48.4ms ± 4%~(p=0.360 n=8+10)
RuntimeTransaction/get_signer_address-244.3ms ± 5%46.0ms ±15%~(p=0.549 n=9+10)
RuntimeTransaction/get_public_account-249.2ms ± 7%49.2ms ±11%~(p=0.971 n=10+10)
RuntimeTransaction/get_account_and_get_balance-2686ms ± 2%683ms ± 3%~(p=0.684 n=10+10)
RuntimeTransaction/get_account_and_get_available_balance-2557ms ± 2%555ms ± 2%~(p=0.853 n=10+10)
RuntimeTransaction/get_account_and_get_storage_used-254.8ms ± 2%53.8ms ± 8%~(p=0.280 n=10+10)
RuntimeTransaction/get_account_and_get_storage_capacity-2520ms ± 3%522ms ± 3%~(p=0.579 n=10+10)
RuntimeTransaction/get_signer_receiver-267.7ms ± 3%67.6ms ± 2%~(p=0.971 n=10+10)
RuntimeTransaction/transfer_tokens-2249ms ± 4%249ms ± 5%~(p=1.000 n=10+10)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-254.4ms ± 5%53.8ms ± 5%~(p=0.529 n=10+10)
RuntimeTransaction/load_and_save_long_string_on_signers_address-296.8ms ± 7%96.1ms ± 3%~(p=0.905 n=10+9)
RuntimeTransaction/create_new_account-21.10s ± 2%1.09s ± 1%~(p=0.573 n=10+8)
RuntimeTransaction/call_empty_contract_function-248.1ms ± 6%47.6ms ± 4%~(p=0.436 n=10+10)
RuntimeTransaction/emit_event-264.6ms ± 3%64.3ms ± 4%~(p=0.529 n=10+10)
RuntimeTransaction/borrow_array_from_storage-2153ms ± 4%151ms ± 5%~(p=0.353 n=10+10)
RuntimeTransaction/copy_array_from_storage-2157ms ± 6%158ms ± 6%~(p=0.739 n=10+10)
RuntimeNFTBatchTransfer-2139ms ±15%143ms ±21%~(p=0.780 n=9+10)
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/1/cols/16/txes-273.5ms ± 3%73.0ms ± 4%~(p=1.000 n=9+9)
ComputeBlock/1/cols/32/txes-2127ms ± 4%128ms ± 4%~(p=0.436 n=10+10)
ComputeBlock/1/cols/64/txes-2247ms ± 4%251ms ± 3%~(p=0.105 n=10+10)
ComputeBlock/1/cols/128/txes-2494ms ± 4%488ms ± 1%~(p=0.460 n=10+8)
ComputeBlock/4/cols/16/txes-2257ms ± 3%257ms ± 3%~(p=0.971 n=10+10)
ComputeBlock/4/cols/32/txes-2496ms ± 3%494ms ± 4%~(p=0.684 n=10+10)
ComputeBlock/4/cols/64/txes-2948ms ± 1%946ms ± 1%~(p=0.579 n=10+10)
ComputeBlock/4/cols/128/txes-21.85s ± 2%1.84s ± 2%~(p=0.684 n=10+10)
ComputeBlock/16/cols/16/txes-2988ms ± 3%993ms ± 4%~(p=0.912 n=10+10)
ComputeBlock/16/cols/32/txes-21.88s ± 1%1.88s ± 2%~(p=1.000 n=10+10)
ComputeBlock/16/cols/64/txes-23.67s ± 1%3.66s ± 1%~(p=0.842 n=9+10)
ComputeBlock/16/cols/128/txes-27.21s ± 1%7.18s ± 1%−0.49%(p=0.016 n=8+10)
 
computationdelta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/reference_tx-2202 ± 0%202 ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-2502 ± 0%502 ± 0%~(all equal)
RuntimeTransaction/get_signer_address-2302 ± 0%302 ± 0%~(all equal)
RuntimeTransaction/get_public_account-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_balance-21.00k ± 0%1.00k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_available_balance-22.60k ± 0%2.60k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_used-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_capacity-21.30k ± 0%1.30k ± 0%~(all equal)
RuntimeTransaction/get_signer_vault-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/get_signer_receiver-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/transfer_tokens-23.50k ± 0%3.50k ± 0%~(all equal)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/load_and_save_long_string_on_signers_address-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/create_new_account-2202 ± 0%202 ± 0%~(all equal)
RuntimeTransaction/call_empty_contract_function-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/emit_event-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/borrow_array_from_storage-22.60k ± 0%2.60k ± 0%~(all equal)
RuntimeTransaction/copy_array_from_storage-22.60k ± 0%2.60k ± 0%~(all equal)
 
interactionsdelta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/reference_tx-244.4k ± 0%44.4k ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string-244.4k ± 0%44.4k ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-244.4k ± 0%44.4k ± 0%~(all equal)
RuntimeTransaction/get_signer_address-244.4k ± 0%44.4k ± 0%~(all equal)
RuntimeTransaction/get_public_account-244.4k ± 0%44.4k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_balance-216.8M ± 0%16.8M ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_available_balance-25.28M ± 0%5.28M ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_used-248.0k ± 0%48.0k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_capacity-25.27M ± 0%5.27M ± 0%~(all equal)
RuntimeTransaction/get_signer_vault-244.7k ± 0%44.7k ± 0%~(all equal)
RuntimeTransaction/get_signer_receiver-245.0k ± 0%45.0k ± 0%~(all equal)
RuntimeTransaction/transfer_tokens-245.8k ± 0%45.8k ± 0%~(all equal)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-244.8k ± 0%44.8k ± 0%~(all equal)
RuntimeTransaction/load_and_save_long_string_on_signers_address-249.7k ± 0%49.7k ± 0%~(all equal)
RuntimeTransaction/create_new_account-28.53M ± 0%8.53M ± 0%~(all equal)
RuntimeTransaction/call_empty_contract_function-244.6k ± 0%44.6k ± 0%~(all equal)
RuntimeTransaction/emit_event-244.6k ± 0%44.6k ± 0%~(all equal)
RuntimeTransaction/borrow_array_from_storage-249.8k ± 0%49.8k ± 0%~(all equal)
RuntimeTransaction/copy_array_from_storage-249.8k ± 0%49.8k ± 0%~(all equal)
 
alloc/opdelta
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/1/cols/64/txes-287.3MB ± 5%89.8MB ± 0%+2.84%(p=0.002 n=10+7)
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/load_and_save_long_string_on_signers_address-255.2MB ± 1%55.7MB ± 2%+1.02%(p=0.008 n=9+10)
RuntimeTransaction/get_account_and_get_storage_capacity-2181MB ± 4%182MB ± 0%+0.96%(p=0.028 n=10+9)
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/4/cols/128/txes-2484MB ± 0%489MB ± 1%+0.89%(p=0.027 n=8+10)
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/reference_tx-236.5MB ± 4%36.6MB ± 3%~(p=1.000 n=10+10)
RuntimeTransaction/convert_int_to_string-238.0MB ± 6%38.5MB ± 6%~(p=0.436 n=10+10)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-239.0MB ± 2%38.7MB ± 4%~(p=0.436 n=10+10)
RuntimeTransaction/get_signer_address-237.5MB ± 4%37.5MB ± 4%~(p=0.968 n=10+9)
RuntimeTransaction/get_public_account-239.7MB ± 8%39.9MB ± 6%~(p=0.905 n=10+9)
RuntimeTransaction/get_account_and_get_balance-2237MB ± 0%237MB ± 0%~(p=0.436 n=10+10)
RuntimeTransaction/get_account_and_get_available_balance-2185MB ± 3%185MB ± 1%~(p=0.912 n=10+10)
RuntimeTransaction/get_account_and_get_storage_used-241.6MB ± 6%40.4MB ± 7%~(p=0.165 n=10+10)
RuntimeTransaction/get_signer_vault-240.6MB ± 3%40.9MB ± 3%~(p=0.393 n=10+10)
RuntimeTransaction/get_signer_receiver-245.4MB ± 5%45.8MB ± 4%~(p=0.684 n=10+10)
RuntimeTransaction/transfer_tokens-286.7MB ± 6%85.8MB ± 6%~(p=0.460 n=8+10)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-241.1MB ± 6%40.2MB ± 6%~(p=0.190 n=10+10)
RuntimeTransaction/create_new_account-2258MB ± 2%257MB ± 2%~(p=0.481 n=10+10)
RuntimeTransaction/call_empty_contract_function-239.0MB ± 6%38.4MB ± 6%~(p=0.315 n=10+10)
RuntimeTransaction/emit_event-245.4MB ± 3%44.7MB ± 5%~(p=0.190 n=10+10)
RuntimeTransaction/borrow_array_from_storage-263.1MB ± 5%62.6MB ± 8%~(p=0.353 n=10+10)
RuntimeTransaction/copy_array_from_storage-276.2MB ± 5%76.3MB ± 4%~(p=0.549 n=9+10)
RuntimeNFTBatchTransfer-258.4MB ± 4%57.9MB ± 4%~(p=0.481 n=10+10)
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/1/cols/16/txes-246.2MB ± 6%46.4MB ± 5%~(p=0.853 n=10+10)
ComputeBlock/1/cols/32/txes-257.8MB ± 4%58.3MB ± 6%~(p=0.481 n=10+10)
ComputeBlock/1/cols/128/txes-2145MB ± 2%146MB ± 3%~(p=0.280 n=10+10)
ComputeBlock/4/cols/16/txes-2117MB ± 4%114MB ± 5%~(p=0.053 n=9+10)
ComputeBlock/4/cols/32/txes-2174MB ± 1%174MB ± 3%~(p=0.211 n=9+10)
ComputeBlock/4/cols/64/txes-2279MB ± 3%281MB ± 3%~(p=0.315 n=10+10)
ComputeBlock/16/cols/16/txes-2392MB ± 2%396MB ± 2%~(p=0.123 n=10+10)
ComputeBlock/16/cols/32/txes-2600MB ± 3%599MB ± 2%~(p=0.912 n=10+10)
ComputeBlock/16/cols/64/txes-21.01GB ± 1%1.02GB ± 1%~(p=0.481 n=10+10)
ComputeBlock/16/cols/128/txes-21.84GB ± 1%1.84GB ± 1%~(p=0.218 n=10+10)
 
allocs/opdelta
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/1/cols/64/txes-2921k ± 0%921k ± 0%+0.03%(p=0.043 n=10+10)
ComputeBlock/16/cols/16/txes-23.67M ± 0%3.67M ± 0%+0.02%(p=0.034 n=8+10)
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/get_signer_address-2128k ± 0%128k ± 0%+0.00%(p=0.003 n=9+8)
RuntimeTransaction/reference_tx-2123k ± 0%123k ± 0%~(p=0.195 n=10+10)
RuntimeTransaction/convert_int_to_string-2137k ± 0%137k ± 0%~(p=0.745 n=9+9)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-2151k ± 0%151k ± 0%~(p=0.768 n=9+10)
RuntimeTransaction/get_public_account-2159k ± 0%159k ± 0%~(p=0.827 n=10+9)
RuntimeTransaction/get_account_and_get_balance-23.17M ± 0%3.17M ± 0%~(p=0.268 n=9+10)
RuntimeTransaction/get_account_and_get_available_balance-22.55M ± 0%2.55M ± 0%~(p=0.424 n=10+10)
RuntimeTransaction/get_account_and_get_storage_used-2181k ± 0%181k ± 0%~(p=0.342 n=10+10)
RuntimeTransaction/get_account_and_get_storage_capacity-22.39M ± 0%2.39M ± 0%~(p=0.744 n=8+10)
RuntimeTransaction/get_signer_vault-2175k ± 0%175k ± 0%~(p=0.926 n=10+10)
RuntimeTransaction/get_signer_receiver-2264k ± 0%264k ± 0%~(p=0.868 n=10+10)
RuntimeTransaction/transfer_tokens-21.03M ± 0%1.03M ± 0%~(p=0.736 n=10+9)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-2173k ± 0%173k ± 0%~(p=0.955 n=10+10)
RuntimeTransaction/load_and_save_long_string_on_signers_address-2276k ± 0%276k ± 0%~(p=0.223 n=10+10)
RuntimeTransaction/create_new_account-23.59M ± 0%3.59M ± 0%~(p=1.000 n=10+10)
RuntimeTransaction/call_empty_contract_function-2140k ± 0%140k ± 0%~(p=0.615 n=10+10)
RuntimeTransaction/emit_event-2185k ± 0%185k ± 0%~(p=0.897 n=10+10)
RuntimeTransaction/borrow_array_from_storage-2412k ± 0%412k ± 0%~(p=0.643 n=10+10)
RuntimeTransaction/copy_array_from_storage-2368k ± 0%368k ± 0%~(p=0.986 n=10+10)
RuntimeNFTBatchTransfer-2333k ± 1%333k ± 1%~(p=0.796 n=10+10)
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/1/cols/16/txes-2235k ± 0%235k ± 0%~(p=0.203 n=10+9)
ComputeBlock/1/cols/32/txes-2464k ± 0%464k ± 0%~(p=0.579 n=10+10)
ComputeBlock/1/cols/128/txes-21.84M ± 0%1.84M ± 0%~(p=1.000 n=10+9)
ComputeBlock/4/cols/16/txes-2922k ± 0%922k ± 0%~(p=0.247 n=10+10)
ComputeBlock/4/cols/32/txes-21.84M ± 0%1.84M ± 0%~(p=0.393 n=10+10)
ComputeBlock/4/cols/64/txes-23.67M ± 0%3.67M ± 0%~(p=1.000 n=10+10)
ComputeBlock/4/cols/128/txes-27.32M ± 0%7.32M ± 0%~(p=1.000 n=9+9)
ComputeBlock/16/cols/32/txes-27.33M ± 0%7.33M ± 0%~(p=0.447 n=10+9)
ComputeBlock/16/cols/64/txes-214.6M ± 0%14.6M ± 0%~(p=0.739 n=10+10)
ComputeBlock/16/cols/128/txes-229.3M ± 0%29.3M ± 0%~(p=0.661 n=10+9)
 
us/txdelta
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/1/cols/16/txes-24.59k ± 3%4.57k ± 4%~(p=1.000 n=9+9)
ComputeBlock/1/cols/32/txes-23.98k ± 4%4.00k ± 4%~(p=0.436 n=10+10)
ComputeBlock/1/cols/64/txes-23.86k ± 4%3.92k ± 3%~(p=0.105 n=10+10)
ComputeBlock/1/cols/128/txes-23.86k ± 4%3.81k ± 1%~(p=0.420 n=10+8)
ComputeBlock/4/cols/16/txes-24.02k ± 3%4.02k ± 3%~(p=0.954 n=10+10)
ComputeBlock/4/cols/32/txes-23.87k ± 3%3.86k ± 4%~(p=0.699 n=10+10)
ComputeBlock/4/cols/64/txes-23.70k ± 1%3.70k ± 1%~(p=0.591 n=10+10)
ComputeBlock/4/cols/128/txes-23.61k ± 2%3.59k ± 2%~(p=0.699 n=10+10)
ComputeBlock/16/cols/16/txes-23.86k ± 3%3.88k ± 4%~(p=0.912 n=10+10)
ComputeBlock/16/cols/32/txes-23.68k ± 1%3.68k ± 2%~(p=1.000 n=10+10)
ComputeBlock/16/cols/64/txes-23.58k ± 1%3.58k ± 1%~(p=0.889 n=9+10)
ComputeBlock/16/cols/128/txes-23.52k ± 1%3.50k ± 1%−0.50%(p=0.012 n=8+10)
 

github-actions[bot] avatar Aug 16 '22 21:08 github-actions[bot]

Build failed (retrying...):

bors[bot] avatar Aug 16 '22 22:08 bors[bot]

Build failed (retrying...):

bors[bot] avatar Aug 16 '22 22:08 bors[bot]

Canceled.

bors[bot] avatar Aug 16 '22 22:08 bors[bot]

bors merge

synzhu avatar Aug 17 '22 19:08 synzhu

Build failed (retrying...):

bors[bot] avatar Aug 17 '22 22:08 bors[bot]

Build failed:

bors[bot] avatar Aug 18 '22 00:08 bors[bot]

bors merge

synzhu avatar Aug 19 '22 19:08 synzhu

Codecov Report

Merging #2932 (f8bf2a7) into master (f14494a) will decrease coverage by 0.00%. The diff coverage is 3.44%.

@@            Coverage Diff             @@
##           master    #2932      +/-   ##
==========================================
- Coverage   54.66%   54.65%   -0.01%     
==========================================
  Files         717      717              
  Lines       66405    66430      +25     
==========================================
+ Hits        36298    36307       +9     
- Misses      27063    27081      +18     
+ Partials     3044     3042       -2
Flag Coverage Δ
unittests 54.65% <3.44%> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
cmd/scaffold.go 17.34% <0.00%> (-0.05%) :arrow_down:
network/p2p/blob_service.go 0.00% <0.00%> (ø)
network/p2p/middleware.go 0.00% <0.00%> (ø)
network/test/testUtil.go 94.57% <100.00%> (+0.02%) :arrow_up:
consensus/hotstuff/eventloop/event_loop.go 73.46% <0.00%> (-1.37%) :arrow_down:
engine/consensus/compliance/core.go 75.10% <0.00%> (ø)
insecure/corruptible/network.go 67.48% <0.00%> (+0.82%) :arrow_up:
network/p2p/dns/resolver.go 96.36% <0.00%> (+4.84%) :arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov-commenter avatar Aug 19 '22 19:08 codecov-commenter

Build succeeded:

bors[bot] avatar Aug 19 '22 20:08 bors[bot]