probabilistic-earley-parser-javascript icon indicating copy to clipboard operation
probabilistic-earley-parser-javascript copied to clipboard

Left-recursive grammar is not accepted?

Open gimite opened this issue 7 years ago • 2 comments

Looks like this package doesn't accept left-recursive grammar. Is it expected? I was wondering because I heard that Earley parser accepts arbitrary context-free language. It looks it accepts right-recursive grammar, though.

parse_test.js:

const {Grammar} = require('probabilistic-earley-parser');

const NP = 'NP';
const N = 'N';
const foo = (token) => !!token.match(/foo/i);
const and = (token) => !!token.match(/and/i);
const grammar = Grammar.builder('test')
  .addNewRule(1.0, NP, [NP, and, N])
  .addNewRule(1.0, NP, [N])
  .addNewRule(1.0, N, [foo])
  .build();
$ node parse_test.js
./node_modules/probabilistic-earley-parser/grammar/left-corner.js:42
                throw new Error("Matrix was not invertable");
                ^

Error: Matrix was not invertable
    at invert (./node_modules/probabilistic-earley-parser/grammar/left-corner.js:42:23)
    at Object.getReflexiveTransitiveClosure (./node_modules/probabilistic-earley-parser/grammar/left-corner.js:113:17)
    at new Grammar (./node_modules/probabilistic-earley-parser/grammar/grammar.js:45:46)
    at GrammarBuilder.build (./node_modules/probabilistic-earley-parser/grammar/grammar.js:111:16)
    at Object.<anonymous> (./parse_test.js:11:4)
    at Module._compile (module.js:649:30)
    at Object.Module._extensions..js (module.js:660:10)
    at Module.load (module.js:561:32)
    at tryModuleLoad (module.js:501:12)
    at Function.Module._load (module.js:493:3)

gimite avatar Mar 31 '18 09:03 gimite

Known issue, see URL below. Sloppiness on my part: I meant to normalize probabilities so that they sum to 1, but forgot to implement. Again, will fix when I get behind a computer (on holiday now).

Work-around until I've implemented this is to ensure the rules in your grammar sum to 1! Should not change semantics as long as probabilities are the same relative to each other.

https://github.com/digitalheir/java-probabilistic-earley-parser/issues/12

cacfd3a avatar Mar 31 '18 11:03 cacfd3a

Thanks! The workaround worked.

gimite avatar Mar 31 '18 14:03 gimite