CarperAI
Standardize Testing, Datatests, Unittests, Integration-Tests
Discuss, standardize and track how we want to test the submodules of the Code-Pile here.
Atm. we tend towards using pytest through github actions. We want some sample input data and the target output data, for each Processor. One current way, would be to use parquet files with some "dummy data" to test against our intermediate data.
Having both real testdata and some edge case synthetic testdata would be best. For example, real testdata to ensure question and answer pairs are still matched. For example, use synthetic testdata to check if weird unicode characters are properly kept.
I don't think we need to integration test everything, but if a dependency isn't straightforward, we might want to add some sanity integration-tests against it too.