📎 Fuzz Testing for Parser

[Boshen]

Description

After #1909 is done, we should probably look into Fuzz Testing to make the Parser panic resilient.

Setup

https://github.com/Boshen/tools/commit/c0f33d66b8e2b2f114310e945b64400a061d4526

cargo install cargo-fuzz
cd crates/rome_js_parser
cargo fuzz init && cd fuzz
rm fuzz_targets/fuzz_target_1.rs
cargo fuzz add parser

Add to fuzz_targets/parser.rs:

#![no_main]
use libfuzzer_sys::fuzz_target;
fuzz_target!(|data: &[u8]| {
    if let Ok(s) = std::str::from_utf8(data) {
        if s.chars().all(|s| !s.is_control()) {
            rome_js_parser::parse_module(s, 0);
        }
    }
});

Run cargo +nightly fuzz run parser -- -only_ascii=1

Panics Found

• 2022.01.14: (:
• 2022.01.24: 0E+
• 2022.04.09:
  • rome_js_parser/src/parser.rs:213
    • /=0*_:m/=/*_:|
    • ([=[(p[=[(p%]>([=[(p[=[(
  • rome_js_parser/src/parser/rewrite_parser.rs:70
    • (--(((--$?:<(
    • (-- @ (--(-((QQQQQ $
    • = 8?0 @ (--(-( A S=
• 2022.04.11:
  • rome_js_parser/src/lexer/mod.rs:512:9
    • +\\u{A +\\u{C
• 2022.04.14 b0b0b36
  • rome_js_parser/src/syntax/assignment.rs:71:61
    • (([zAgRvz=[=(e{V{
    • ({[>[[ =\\c{

[Boshen]

@MichaReiser found 2 crashing points for you to play with 😁

• 2022.04.09:
  • /=0*_:m/=/*_:| crashing point rome_js_parser/src/parser.rs:213
  • ([=[(p[=[(p%]>([=[(p[=[( crashing point rome_js_parser/src/parser.rs:213
  • (--(((--$?:<( crashing point rome_js_parser/src/parser/rewrite_parser.rs:70
  • (-- @ (--(-((QQQQQ $ crashing point rome_js_parser/src/parser/rewrite_parser.rs:70
  • = 8?0 @ (--(-( A S= crashing point rome_js_parser/src/parser/rewrite_parser.rs:70

[Boshen]

@MichaReiser

• 2022.04.11:
  • rome_js_parser/src/parser/rewrite_parser.rs:87:9
    • ++0 [{+
    • ++ \\ [--()()(-<(
    • --(-is[--(!is[{)
    • --({ *[ )
  • rome_js_parser/src/syntax/assignment.rs:75:59
    • (H =0 0(=H({ ={ =#000[

[Boshen]

@xunilrj since you are working on the lexer

• 2022.04.11:
  • rome_js_parser/src/lexer/mod.rs:512:9
    • +\\u{A +\\u{C

[MichaReiser]

@MichaReiser

* 2022.04.11:
  
  * `rome_js_parser/src/parser/rewrite_parser.rs:87:9`
    
    * ` ++0 [{+`
    * `++ \\   [--()()(-<(`
    * `--(-is[--(!is[{)`
    * `--({ *[ )`
  * `rome_js_parser/src/syntax/assignment.rs:75:59`
    
    * `(H =0       0(=H({ ={ =#000[`

These should all be fixed in the latest version on main.

These have been super useful. It would have taken me hours to find these otherwise (with larger code samples).

[Boshen]

@MichaReiser Assignment is hard and no unwrap is safe 😢

• 2022.04.14 b0b0b36
  • rome_js_parser/src/syntax/assignment.rs:71:61
    • (([zAgRvz=[=(e{V{
    • ({[>[[ =\\c{

[Boshen]

The parser seems pretty solid now, the fuzzer can still detect some panics but the input is super long: panicked at 'The parser is no longer progressing. Stuck at '?' QUESTION:8..9

• ({[({= ?:::({[({=
• ({[({= :?:::({((-$$$= ::\"

I can move on to fuzzing the formatter, but I need an easy entry point such as format(source_code, options) because I can't find all the pieces to glue everything together 😢 .

[leops]

The parser and formatter crates are left purposefully unaware of each other (so they can be compiled in parallel), so such an entry point would be rome_js_formatter::format(options, &rome_js_parser::parse(source_code, 0, source_type).syntax())