feat(developer): next generation KMN compiler - parser 🤔

[markcsinclair]

A recursive-descent parser with AST generation for the next generation KMN compiler, based on a modified port of Fowler's parser (see Domain Specific Languages by Martin Fowler and Rebecca Parsons, Ch.21, 24)

see Next Generation KMN Compiler #13349

The parser will initially be developed for positive path only i.e. no error messages, and with no backward compatability

Test-bot: skip Build-bot: skip

To do list (following first full parse of khmer_angkor.kmn):

• [ ] Use correct nodes in AST for layer(), platform() and baselayout()`[handle in semantic layer]
• [x] Refactore system and variable store rules into separate file
• [ ] Reduce rule object construction/destruction overhead using builder pattern [do later]
• [x] Include full range of (current) language elements
  • [x] CALL
  • [x] CAPSALWAYSOFF
  • [x] CAPSONONLY
  • [x] DEADKEY
  • [x] NOTANY
  • [x] NUL (on LHS)
  • [x] RESET
  • [x] RETURN
  • [x] SAVE
  • [x] SET
  • [x] SHIFTFREESCAPS
  • [ ] CLEARCONTEXT
  • [ ] FIX
• [x] Include named constants and Hangeul syllables (needs lexer support)
• [x] Check whitespace/continuation in rules for consistency and coverage
• [x] Check LINE nodes moved to correct places in AST
• [ ] Ensure virtual keys are included only in keystroke part of rules [Leavbe to semantic layer]
• [x] Include virtual character keys
• [ ] Check keywords are only recognised where they are allowed
• [x] Include permitted keywords in variable store and group names
• [x] Tests for tree-construction.ts
• [ ] Check test coverage

[keymanapp-test-bot[bot]]

User Test Results

Test specification and instructions

User tests are not required

Test Artifacts

• Developer
  • Keyman Developer - build : all tests passed (no artifacts on BuildLevel "build")
  • Compiler Regression Tests - build : all tests passed (no artifacts on BuildLevel "build")
  • Keyman Developer (old PRs) - build : all tests passed (no artifacts on BuildLevel "build")
  • kmcomp.zip - build : all tests passed (no artifacts on BuildLevel "build")
  • kmcomp.zip (old PRs) - build : all tests passed (no artifacts on BuildLevel "build")
• Keyboards
  • Test Keyboards - build : all tests passed (no artifacts on BuildLevel "build")

[mcdurdin]

@srl295 process note: please don't approve PRs while still in draft.

[srl295]

@srl295 process note: please don't approve PRs while still in draft.

accidental- thought i dismissed it.

[srl295]

@markcsinclair looking good. I think the Rule family tree is a lot easier to follow now as far as what's expected from each rule.

[markcsinclair]

Hmm ... merging master seems to have been a mistake sigh