checklist icon indicating copy to clipboard operation
checklist copied to clipboard

Reproducibility Issues When Templating

Open ChatBotMatt opened this issue 2 years ago • 0 comments

Because template values are in part based on the order of the keys (e.g. {city1}, {city2} vs {city2}, {city1} will output two different texts even given the same sets of options in the same order), and the order of keys is both undefined and non-seedable (since keys is a set generated by find_all_keys, it's impossible to get proper reproducibility when templating texts. This is quite inconvenient for proper unit testing, reproducibility of experiments, etc.

Something simple like calling sorted on the return values of find_all_keys in Editor.template can fix this, and it'd be nice if it was at least a parameter that could be set.

ChatBotMatt avatar Oct 31 '23 12:10 ChatBotMatt