ODM icon indicating copy to clipboard operation
ODM copied to clipboard

Implement testing infrastructure

Open dakotabenjamin opened this issue 10 years ago • 2 comments

Currently none exists for ODM. We need to be able to test every aspect of the code, from the python front-end to the C++ modules running the toolchain. Debugging manually is time consuming and leads to missed errors, and right now my "testing" is simply running the code against test data and looking for relevant changes.

With an object oriented code rewrite coming, I think we should first focus our efforts on the toolchain.

dakotabenjamin avatar Nov 16 '15 16:11 dakotabenjamin

The way this project is put together, this seems like an impossible task. Maybe someone could get a local LLM up and running, write a test spec for it and leave it running for a year and check back.

Otherwise, maybe e2e tests could take a base docker image and try running each stage. Then you could just mount the files & stage you're working on and fix that. You could start by modifying the benchmark function to take a snapshot of current project files. This would allow you to work on any stage that's being benchmarks.

Really, the biggest issue is simply how much test data is required per stage.

disarticulate avatar Aug 25 '25 20:08 disarticulate

We need the tests to actually work and test properly, not approximate what a proper test might look most like, so I don't see LLMs helping here.

Saijin-Naib avatar Aug 26 '25 16:08 Saijin-Naib