BrendanChambersBourgeois

👍

Updated it to just do defang. ``` └─$ python evals/cli/oaieval.py gpt-3.5-turbo defang_domain_check.dev.v0 ... [2023-05-08 09:36:30,998] [oaieval.py:147] Final report: [2023-05-08 09:36:30,998] [oaieval.py:149] accuracy: 0.703125 ```

Added checks for is defang or fanged. View evals in JSON ### Eval ```jsonl {"input": [{"role": "system", "content": "Respond with only a 1 or 0 to signify if the user's...

More information on defanging: (ChatGPT-4 output ^_^) The main issue with GPT-3 and GPT-4 is that it often does not fully defang things. e.g. subdomain.example[.]com is not defanged as subdomain.example...

Also please let me know if being more generic is better for the model... I'm not sure if this would be better for defang_domain_check.dev.v0. View evals in JSON **more generic**...