lexer freezes on this Python code

[joshi1983]

Steps to reproduce:

1. Try to run lexer("print('\'m')")
2. Notice that it freezes.

I traced the freeze to an infinite do-while loop in the lexer.
In the following code, input is set to the code "print('\'m')" but current variable reaches into the billions, currentChar === undefined, and validator.test(currentChar) is returning false when currentChar is undefined.

  /**
     * 过滤（提取） 引号中的内容
     */
    // eslint-disable-next-line
    const matchQuotation = (currentChar, validator, TokenType) => {
        do {
            if (currentChar === '\n') {
                line++;
            }
            currentChar = input[++current];
        } while (!validator.test(currentChar));
        ++current;
    };

I wonder if this has anything to do with the backslashed quotation mark. Maybe the grammar doesn't define STRING properly for escaped quotes.

I edited this issue because I first thought the bug was reproducible by parsing but it actually requires running the lexer function on the code.

[joshi1983]

HaydenOrz do you want to try this? I ask because you did a great job with another lexer bug a couple weeks ago.