[python] a simple rule crashes if large dict fed in

[GergelyMolnar]

Hi, I'm very new to this promising tool. I've been running some test-snippet to see if durable_rules can help to make a distinction on some JSON object retrieved from an API.

It seems promising (I'm trying to avoid writing several hundreds of IF clause) a simple rule worked fine on a sparse handmade object. But when I fed in an actual object from the API containing over 70 fields (not too large actually) it crashed.

I've investigated the data format and that isn't the issue. The JSON objects in python stored as dict() so that's OK. The only extra is the objects have a nested object. But that's been simulated in my handmade dict and it works like a charm.

What I found the library crashes if it gets a dictionary more than 28 keys, considering the nested dict for a single value.

I've tried to decrease an actual dict read from the API by deleting one key:value pair at a time. Then I used this crude mockup with arbitrary fields keeping the data short.

from durable.lang import *

with ruleset('test'):
    @when_all((+m.Id) & (m.attributes.type == 'type_1'))
    def is_a_and_ea(c):
        print(f"Yupee! <Id> is {c.m.Id} AND <attributes.type> is {c.m.attributes.type}")

def dr_test(r):
    jj = {'attributes': {'type': 'type_1', 'url': '/partial/url'}, 'Id': 'id-10001'}
    for i in range(3, r):
        jj.update({f'field{i}': i})
    post('test', jj)

looping the dr_test() function

>>> for i in range(24, 30):
...     print(i)
...     dr_test(i)
... 
24
Yupee! <Id> is id-10001 AND <attributes.type> is type_1
25
Yupee! <Id> is id-10001 AND <attributes.type> is type_1
26
Yupee! <Id> is id-10001 AND <attributes.type> is type_1
27
Yupee! <Id> is id-10001 AND <attributes.type> is type_1
28
Yupee! <Id> is id-10001 AND <attributes.type> is type_1
29
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "<stdin>", line 5, in dr_test
  File "/usr/local/lib64/python3.6/site-packages/durable/lang.py", line 666, in post
    return get_host().post(ruleset_name, message, complete)
  File "/usr/local/lib64/python3.6/site-packages/durable/engine.py", line 786, in post
    return self._handle_function(rules, rules.assert_event, message, complete)
  File "/usr/local/lib64/python3.6/site-packages/durable/engine.py", line 770, in _handle_function
    rules.do_actions(func(args), callback)
  File "/usr/local/lib64/python3.6/site-packages/durable/engine.py", line 329, in assert_event
    return self._handle_result(durable_rules_engine.assert_event(self._handle, json.dumps(message, ensure_ascii=False)), message)
durable_rules_engine.error: Could not assert event, error code: 302
>>> 

The same error thrown with the actual data, too.

I'm not sure if this is a bug or just a ~~pointless?~~ limitation but it takes away a good measure of flexibility of this awesome tool.

I hope it's just an easy fix and not a planned limitation. Thanks for your effort guys!

Gergely

[jruizgit]

Hi, thanks for reporting this issue. The number of object properties is limited to 32 (including nested object properties). To achieve higher performance in the rules engine, objects are allocated in the stack. I have increased the number of properties to 255 (the manners benchmark for node.js did regress slightly, from 420ms to 450ms. The python benchmark stayed at 630ms). Please use version 2.0.18.

[GergelyMolnar]

I've installed version 2.10.19 and it works perfectly. Thanks a lot 255 is plenty in my case, though I can imagine a situation when that might not be enough.

Just an idea. Would it make sense to allow an arbitrary size JSON/dict input for any rule. Instead of passing down the original dictionary assign only the relevant key/value pairs to a temporary dict and work on that? And keeping the original in case some may want to reference other key/values pairs in the action-function.

[ruslanolkhovsky]

Thanks! Helped me too... updated to 2.10.19, solved the same issue.