Parsers are not 100% compliant - 9 YAML Test Suite failures

[aSemy]

The README says "100% compliant YAML 1.2 parsers" and "The parsers pass 100% of the YAML Test Suite!"

However, when I run make test I observe 9 test failures.

not ok 4 - (yaml-test-suite) 2AUY - Tags in Block Sequence -- '   - !!str a␤   - b␤   - !!int 42␤   - d␤tree: |␤  +STR␤   +DOC␤    +SEQ␤     =VAL <tag:yaml.org,2002:str> :a␤     =VAL :b␤     =VAL <tag:yaml.org,2002:int> :42␤     =VAL :d␤    -SEQ␤   -DOC␤  -STR␤json: |␤  [␤    "a",␤    "b",␤    42,␤    "d"␤  ]␤dump: |␤  - !!str a␤  - b␤  - !!int 42␤  - d␤'
not ok 35 - (yaml-test-suite) 4RWC - Trailing spaces after flow collection -- '    [1, 2, 3]␣␣␤  ␣␣∎␤tree: |␤  +STR␤   +DOC␤    +SEQ []␤     =VAL :1␤     =VAL :2␤     =VAL :3␤    -SEQ␤   -DOC␤  -STR␤json: |␤  [␤    1,␤    2,␤    3␤  ]␤dump: |␤  - 1␤  - 2␤  - 3␤'
not ok 62 - (yaml-test-suite) 6HB6 - Spec Example 6.1. Indentation Spaces -- '    # Leading comment line spaces are␤     # neither content nor indentation.␤  ␣␣␣␣␤  Not indented:␤   By one space: |␤      By four␤        spaces␤   Flow style: [    # Leading spaces␤     By two,        # in flow style␤    Also by two,    # are neither␤    —»Still by two   # content nor␤      ]             # indentation.␤tree: |␤  +STR␤   +DOC␤    +MAP␤     =VAL :Not indented␤     +MAP␤      =VAL :By one space␤      =VAL |By four\n  spaces\n␤      =VAL :Flow style␤      +SEQ []␤       =VAL :By two␤       =VAL :Also by two␤       =VAL :Still by two␤      -SEQ␤     -MAP␤    -MAP␤   -DOC␤  -STR␤json: |␤  {␤    "Not indented": {␤      "By one space": "By four\n  spaces\n",␤      "Flow style": [␤        "By two",␤        "Also by two",␤        "Still by two"␤      ]␤    }␤  }␤dump: |␤  Not indented:␤    By one space: |␤      By four␤        spaces␤    Flow style:␤    - By two␤    - Also by two␤    - Still by two␤'
not ok 90 - (yaml-test-suite) 8G76 - Spec Example 6.10. Comment Lines -- '    # Comment␤  ␣␣␣␤  ↵␤  ↵␤tree: |␤  +STR␤  -STR␤json: ''␤dump: ''␤'
not ok 96 - (yaml-test-suite) 93JH - Block Mappings in Block Sequence -- '   - key: value␤     key2: value2␤   -␤     key3: value3␤tree: |␤  +STR␤   +DOC␤    +SEQ␤     +MAP␤      =VAL :key␤      =VAL :value␤      =VAL :key2␤      =VAL :value2␤     -MAP␤     +MAP␤      =VAL :key3␤      =VAL :value3␤     -MAP␤    -SEQ␤   -DOC␤  -STR␤json: |␤  [␤    {␤      "key": "value",␤      "key2": "value2"␤    },␤    {␤      "key3": "value3"␤    }␤  ]␤dump: |␤  - key: value␤    key2: value2␤  - key3: value3␤'
not ok 149 - (yaml-test-suite) DK95-03 - Tabs that look like indentation -- '   ———»␤  foo: 1␤tree: |␤  +STR␤   +DOC␤    +MAP␤     =VAL :foo␤     =VAL :1␤    -MAP␤   -DOC␤  -STR␤json: |␤  {␤    "foo" : 1␤  }␤emit: |␤  ---␤  foo: 1␤'
not ok 150 - (yaml-test-suite) DK95-03 - Tabs that look like indentation -- '   ———»␤  foo: 1␤tree: |␤  +STR␤   +DOC␤    +MAP␤     =VAL :foo␤     =VAL :1␤    -MAP␤   -DOC␤  -STR␤json: |␤  {␤    "foo" : 1␤  }␤emit: |␤  ---␤  foo: 1␤'
not ok 160 - (yaml-test-suite) F2C7 - Anchors and Tags -- '   - &a !!str a␤   - !!int 2␤   - !!int &c 4␤   - &d d␤tree: |␤  +STR␤   +DOC␤    +SEQ␤     =VAL &a <tag:yaml.org,2002:str> :a␤     =VAL <tag:yaml.org,2002:int> :2␤     =VAL &c <tag:yaml.org,2002:int> :4␤     =VAL &d :d␤    -SEQ␤   -DOC␤  -STR␤json: |␤  [␤    "a",␤    2,␤    4,␤    "d"␤  ]␤dump: |␤  - &a !!str a␤  - !!int 2␤  - &c !!int 4␤  - &d d␤'
not ok 163 - (yaml-test-suite) F8F9 - Spec Example 8.5. Chomping Trailing Lines -- '   # Strip␤    # Comments:␤  strip: |-␤    # text␤  ␣␣␤   # Clip␤    # comments:␤␤  clip: |␤    # text␤  ␣␤   # Keep␤    # comments:␤␤  keep: |+␤    # text␤␤   # Trail␤    # comments.␤tree: |␤  +STR␤   +DOC␤    +MAP␤     =VAL :strip␤     =VAL |# text␤     =VAL :clip␤     =VAL |# text\n␤     =VAL :keep␤     =VAL |# text\n\n␤    -MAP␤   -DOC␤  -STR␤json: |␤  {␤    "strip": "# text",␤    "clip": "# text\n",␤    "keep": "# text\n\n"␤  }␤dump: |␤  strip: |-␤    # text␤  clip: |␤    # text␤  keep: |+␤    # text␤␤  ...␤'

Failed 9/309 subtests
 
Test Summary Report
-------------------
test/parse-suite.tml (Wstat: 0 Tests: 309 Failed: 9)
  Failed tests:  4, 35, 62, 90, 96, 149-150, 160, 163
Files=8, Tests=471,  5 wallclock secs ( 0.03 usr  0.01 sys +  4.46 cusr  0.33 csys =  4.83 CPU)
Result: FAIL

It would be appreciated if the README was up to date.

Suggestions

• add a date to the claims in the README, e.g. "As of 2022/01/31 the parsers are 100% complaint".
• The tests should be reproducible and pinned to a specific version of the YAML Test Suites, rather than using the latest versions https://github.com/yaml/yaml-reference-parser/blob/f00319457419feacddf743d75eb317dacf5caf4b/.common.mk#L31 This should also be specified in the README, e.g. "The parsers pass 100% of the YAML Test Suite v2022-01-17"