Improving expression handling

[ArashPartow]

Hi @texodus , @sc1f

Was wanting to let you know the latest version of ExprTk adds new features which add protections when compiling and evaluating expressions from unverified or otherwise malicious end-points, which might be a use-case for perspective.

Issues that can arise are:

1. Stack overflows either during the parsing or evaluation phase
2. Loops that never complete either entirely or within a reasonable amount of time.
3. Invalid memory accessing either in string or vector contexts

---

For first point: stack protection the following section of the readme has the necassary details:

https://github.com/ArashPartow/exprtk/blob/master/readme.txt#L1283

Simply put after having instantiated the parser but before compiling all that is required is to call the following:

parser_t parser;

parser.set_max_stack_depth(100);
parser.set_max_node_depth (200);

The values passed in are the limits in terms of stack-frame level size (not byte size) that will be checked for during parsing and as the AST is being constructed (aka node depth).

---

For the second point: loop run-time protections, after having instantiated the parser all that is required is to call the following:

 #define exprtk_enable_runtimechecks
 #include "exprtk.hpp"

 using loop_runtime_check_t = exprtk::loop_runtime_check;

 parser_t parser;

 loop_runtime_check_t loop_runtime_check;

 loop_runtime_check_t loop_runtime_check;
 loop_runtime_check.loop_set = loop_runtime_check_t::e_all_loops;
 loop_runtime_check.max_loop_iterations = 100000;

 parser_t parser;

 parser.register_loop_runtime_check(loop_runtime_check);

The above, will inject checks within the loop iteration code to ensure any single loop does not exceed the max_loop_iterations, which in this case will be 100000 iterations.

An example of the loop RTC being extended to enforce a maximum expression evaluation time can be found here:

https://github.com/google/oss-fuzz/blob/master/projects/exprtk/exprtk_fuzzer.cpp#L22

---

For the third point, protecting against invalid memory accesses, via run-time checks simply requires having the following define exprtk_enable_runtimechecks enabled before the first include of the ExprTk header

The first protection is all done at compile time and has no effect on expression evaluation performance. However the other two protections both have, as expected, a run-time overhead.

My recommendation would be to have two instances of the parser be used, one for expressions from trusted sources and another instance, with these protections enabled, for expressions from untrusted/unverified sources.

I'd be happy to provide assistance in integrating these changes in perspective.

[hegdeadithyak]

Hey @timkpaine , just wanted to clarify: do we compile a parser with verified sources without any extra runtime enabled, and a parser with unverified sources with extra runtime enabled, to avoid issues like stack overflow and infinite loops? Am I on the right?