TopoStats icon indicating copy to clipboard operation
TopoStats copied to clipboard

Separated Disordered Tracing into own module

Open Max-Gamill opened this issue 1 year ago • 4 comments

This is the first part of the mega PR to refactor the 800-better-tracing branch.

This PR in particular does:

  • Creates a smoke test for me to check the refactored workflow works (once it's finished)
  • Adds to one of the tests to allow the comparison between np.nan and np.nan which previously failed
  • Moves smoothing, skeletonisation and pruning workflow into it's own module. This works on the image but processes each grain separately, producing a dictionary of the grain's crop data (skeletonisation stages and bounding boxes). The individual crops are then compiled into a "full_image" to plot as the processing / diagnostic images. This differs from previously where the trace coordinates were stored / passed into dnatracing (this step will move into dnatracing as nodestats prefers skeletons)
  • Moves the smoothing, skeletonisation and pruning parameters into it's own section in the config (and accommodates this in validation.py
  • Adds the "disordered_trace" step in process_scan.
  • Works along-side the current dnatracing pipeline and is NOT connected to anything on the output.

Some docstrings may not be right but I've ran (some) pre-commit checks so they at-least should be present (but I was going to check these together after the pipeline works as i/o's etc may change)

*when making changes to this, and other stacked branches, will need to rebase <branch_tip> --update-refs and that should update 800-better-traces-refactor too

Max-Gamill avatar Jul 18 '24 14:07 Max-Gamill

nodestats tracing .topostats test fails in the comparison because previously the keys in the .topostats file for the new changes aren't saved out yet as the pipeline is not complete.

Max-Gamill avatar Jul 19 '24 15:07 Max-Gamill

@llwiggins has moved out all the tests associated code from this PR into it's own branch (llwiggins/800-better-tracing-876_tests) and PR (#880) so this one can solely focus on the disordered traces step and I have reverted the commits associated with that.

Max-Gamill avatar Jul 24 '24 15:07 Max-Gamill

@ns-rse Thank you for amending the pre-commit complaints, some of these I don't think can be helped and when I tried to add the ignoring statements it didn't like them but that's probably a user error :)

The general exception was from older code and might be useful to look into when writing the tests?

Max-Gamill avatar Aug 07 '24 20:08 Max-Gamill

@ns-rse regarding the git-blame fixes, I've read your blog and it seems my commits were not atomic enough - the commit Moved disordered traces out from dnatracing includes: • The disordered tracing functions moved out from dnatracing into disordered_tracing.py • The updates to the config + validation • My own functions to run the disordered tracing pipeline With my reason being to ensure the pipeline worked so I wasn't making a load of tiny fix commits

So my question is, if I do put this commit in the .git-blame-ignore-revs file, how will it handle the new lines of code in the disordered_tracing.py file that wasn't there before? As it's a new filename, git wont associate those moved functions with their counterparts in the dnatracing file so there will be no blame references right, does this still make it worth doing?

Max-Gamill avatar Aug 14 '24 10:08 Max-Gamill

This has been added in #889 (sorry)

Max-Gamill avatar Aug 30 '24 10:08 Max-Gamill