ityfuzz icon indicating copy to clipboard operation
ityfuzz copied to clipboard

feat: add to corpus when any comparisons are closer

Open plotchy opened this issue 1 year ago • 3 comments

This duplicates the CmpFeedback effort on the corpus.

The idea being that adding values to the corpus when the input reverts can still be useful to overcome comparisons.

ie: in this toy example, the magic value of 8650 is difficult to guess (without concolic).

pragma solidity ^0.8.13;
contract debug {
    function foo(uint x) public {
        if (x == uint(200000 / uint(23))) {
            assert(false);
        }
    }
}

However, if the input gets closer, the corpus will keep that input and mutate off that.

Concerns:

  • The corpus will probably hugely over-inflate. I only tested this on the toy example.
  • The map comparison takes a significant time. I went from ~62k exec/sec to ~50k exec/sec on this pr. This pr is dumb in the sense that it does all the comparisons once in Infant feedback and then again in Corpus feedback.

Pros:

  • It works really well alongside the gaussian mutator. In debug target I went from finding the magic number in 6m22s to 29s with both

plotchy avatar May 14 '24 06:05 plotchy

Found: 19

Project Name Vulnerability Found Time Taken Log
BIGFI_exp.txt ✅ Price Manipulation 0h-0m-51s Log File
Shadowfi_exp.txt -1 Log File
SEAMAN_exp.txt ✅ Fund Loss 0h-1m-54s Log File
BEGO_exp.txt ✅ Fund Loss 0h-0m-9s Log File
cftoken_exp.txt ✅ Price Manipulation 0h-0m-21s Log File
Carrot_exp.txt -1 Log File
MBC_ZZSH_exp.txt -1 Log File
AUR_exp.txt -1 Log File
SellToken_exp.txt ✅ Fund Loss 0h-0m-52s Log File
ROI_exp.txt ✅ Fund Loss 0h-0m-22s Log File
GPT_exp.txt -1 Log File
OLIFE_exp.txt -1 Log File
THB_exp.txt ✅ Fund Loss 0h-4m-11s Log File
VerilogCTF.txt ❌‼️ Crashed -1 Log File
CS_exp.txt ✅ Price Manipulation 0h-0m-16s Log File
MintoFinance_exp.txt ✅ Fund Loss 0h-0m-11s Log File
Novo_exp.txt ✅ Price Manipulation 0h-4m-17s Log File
SELLC03_exp.txt ✅ Fund Loss 0h-2m-0s Log File
Yyds_exp.txt ✅ Fund Loss 0h-0m-54s Log File
DYNA_exp.txt -1 Log File
EAC_exp.txt ✅ Fund Loss 0h-0m-22s Log File
Annex_exp.txt -1 Log File
PLTD_exp.txt ✅ Price Manipulation 0h-0m-45s Log File
ApeDAO_exp.txt ✅ Price Manipulation 0h-0m-22s Log File
GSS_exp.txt ✅ Fund Loss 0h-1m-10s Log File
Axioma_exp.txt ✅ Fund Loss 0h-0m-35s Log File
RFB_exp.txt ✅ Fund Loss 0h-0m-33s Log File
HEALTH_exp.txt ✅ Price Manipulation 0h-0m-9s Log File

fuzzland-bot avatar May 16 '24 17:05 fuzzland-bot

drafting this bc this isn't a good solution. it inflates the corpus too much and should have logic for corpus replacement on closer comparisons.

plotchy avatar May 21 '24 21:05 plotchy

After experimenting more I think doing this in should_replace() is probably better. No need for a Feedback

https://github.com/fuzzland/ityfuzz/blob/2ffa214e8eb67490366ac8f27ffe81ab8196bee8/src/fuzzer.rs#L169-L188

plotchy avatar May 23 '24 12:05 plotchy

I force pushed a new implementation. This adds a new metadata that is also checked in the should_replace function. It no longer inflates corpus. Now it replaces items with the same coverage map that have a better comparison map.

Here is a run with an extra trace that showed when a corpus was replaced INFO Replacing testcase 1 ....

It solves the original example immediately whereas without this it takes some minutes.

================ EVM Fuzzer Start ===================


 INFO Deploying contract: test/debug.sol:debug
 INFO Contract test/debug.sol:debug deployed to: 0x4d4f5b9982db86a2fabcdc1ce0c4dbb734e578bb
 INFO Deployed all contracts

 INFO [Stats #0] run time: 0h-0m-0s, clients: 1, corpus: 0, objectives: 0, executions: 0, exec/sec: 0.000
 INFO ============= New Corpus Item =============
 INFO Reverted? false 
 Txn:
[Sender] 0x35c9dfd76bf02107ff4f7128Bd69716612d31dDb
   └─[1] 0x4D4f5b9982DB86A2fabcDc1cE0c4dbB734e578bB.foo(98112)

 INFO ==========================================
 INFO [Testcase #0] run time: 0h-0m-0s, clients: 1, corpus: 2, objectives: 0, executions: 1, exec/sec: 0.000
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO ============= Coverage Summary =============
 INFO test/debug.sol:debug(0x4d4f5b9982db86a2fabcdc1ce0c4dbb734e578bb): 60.24% Instruction Covered, 50.00% Branch Covered
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO [Stats #0] run time: 0h-0m-0s, clients: 1, corpus: 2, objectives: 0, executions: 1, exec/sec: 0.000
 INFO ============= New Corpus Item =============
 INFO Reverted? true 
 Txn:
[Sender] 0x35c9dfd76bf02107ff4f7128Bd69716612d31dDb
   ├─[1] 0x4D4f5b9982DB86A2fabcDc1cE0c4dbB734e578bB.foo(8695)
   │  └─ ← 0x4e487b710000000000000000000000000000000000000000000000000000000000000001

plotchy avatar May 28 '24 15:05 plotchy

Found: 18

Project Name Vulnerability Found Time Taken Log
BIGFI_exp.txt ✅ Price Manipulation 0h-0m-40s Log File
Shadowfi_exp.txt -1 Log File
SEAMAN_exp.txt ✅ Fund Loss 0h-1m-1s Log File
BEGO_exp.txt ✅ Fund Loss 0h-2m-4s Log File
cftoken_exp.txt ✅ Price Manipulation 0h-0m-32s Log File
Carrot_exp.txt ✅ Arbitrary Call 0h-0m-17s Log File
MBC_ZZSH_exp.txt ✅ Fund Loss 0h-3m-48s Log File
AUR_exp.txt -1 Log File
SellToken_exp.txt ✅ Fund Loss 0h-0m-32s Log File
ROI_exp.txt ✅ Fund Loss 0h-0m-11s Log File
GPT_exp.txt -1 Log File
OLIFE_exp.txt -1 Log File
THB_exp.txt -1 Log File
VerilogCTF.txt ❌‼️ Crashed -1 Log File
CS_exp.txt ✅ Price Manipulation 0h-0m-12s Log File
MintoFinance_exp.txt ✅ Fund Loss 0h-0m-9s Log File
Novo_exp.txt -1 Log File
SELLC03_exp.txt ✅ Fund Loss 0h-1m-30s Log File
Yyds_exp.txt ✅ Fund Loss 0h-0m-45s Log File
DYNA_exp.txt -1 Log File
EAC_exp.txt -1 Log File
Annex_exp.txt -1 Log File
PLTD_exp.txt ✅ Price Manipulation 0h-1m-3s Log File
ApeDAO_exp.txt ✅ Price Manipulation 0h-0m-15s Log File
GSS_exp.txt ✅ Fund Loss 0h-0m-46s Log File
Axioma_exp.txt ✅ Fund Loss 0h-0m-17s Log File
RFB_exp.txt ✅ Fund Loss 0h-0m-19s Log File
HEALTH_exp.txt ✅ Price Manipulation 0h-0m-6s Log File

fuzzland-bot avatar May 28 '24 15:05 fuzzland-bot

Found: 20

Project Name Vulnerability Found Time Taken Log
BIGFI_exp.txt ✅ Price Manipulation 0h-1m-31s Log File
Shadowfi_exp.txt -1 Log File
SEAMAN_exp.txt ✅ Fund Loss 0h-2m-1s Log File
BEGO_exp.txt ✅ Fund Loss 0h-0m-17s Log File
cftoken_exp.txt ✅ Price Manipulation 0h-0m-10s Log File
Carrot_exp.txt ✅ Arbitrary Call 0h-0m-17s Log File
MBC_ZZSH_exp.txt ✅ Fund Loss 0h-3m-24s Log File
AUR_exp.txt -1 Log File
SellToken_exp.txt ✅ Fund Loss 0h-0m-20s Log File
ROI_exp.txt ✅ Fund Loss 0h-0m-19s Log File
GPT_exp.txt -1 Log File
OLIFE_exp.txt -1 Log File
THB_exp.txt ✅ Fund Loss 0h-3m-14s Log File
VerilogCTF.txt ❌‼️ Crashed -1 Log File
CS_exp.txt ✅ Price Manipulation 0h-0m-10s Log File
MintoFinance_exp.txt ✅ Fund Loss 0h-0m-19s Log File
Novo_exp.txt ✅ Price Manipulation 0h-1m-49s Log File
SELLC03_exp.txt ✅ Fund Loss 0h-1m-15s Log File
Yyds_exp.txt ✅ Fund Loss 0h-0m-42s Log File
DYNA_exp.txt -1 Log File
EAC_exp.txt -1 Log File
Annex_exp.txt -1 Log File
PLTD_exp.txt ✅ Price Manipulation 0h-0m-41s Log File
ApeDAO_exp.txt ✅ Price Manipulation 0h-0m-17s Log File
GSS_exp.txt ✅ Fund Loss 0h-0m-32s Log File
Axioma_exp.txt ✅ Fund Loss 0h-0m-37s Log File
RFB_exp.txt ✅ Fund Loss 0h-2m-16s Log File
HEALTH_exp.txt ✅ Price Manipulation 0h-0m-12s Log File

fuzzland-bot avatar May 28 '24 16:05 fuzzland-bot