Add a way to persist dataset even if exception is thrown
Refactoring upon the changes at #494 from the comments, this introduces the way to allow a way to persist dataset even if exception is thrown. With this change, the cleanup is now done at the active context. So, we can use the session context as checkpoints now. Example implementation is as:
https://github.com/iterative/datachain/blob/029e677e596c84bb8c2ecb56603d164adde03fe8/tests/scripts/feature_class_exception.py#L1-L34
In this file, local_test_datachain should only persist since it is the only dataset where the exception doesn't occur in current context.
Closes #500
Deploying datachain-documentation with
Cloudflare Pages
| Latest commit: |
501ad01
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://87fed5ff.datachain-documentation.pages.dev |
| Branch Preview URL: | https://amrit-persist-error.datachain-documentation.pages.dev |
Codecov Report
Attention: Patch coverage is 95.83333% with 1 line in your changes missing coverage. Please review.
Project coverage is 87.11%. Comparing base (
e0c654e) to head (501ad01). Report is 3 commits behind head on main.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| src/datachain/query/session.py | 95.45% | 0 Missing and 1 partial :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #504 +/- ##
=======================================
Coverage 87.10% 87.11%
=======================================
Files 92 92
Lines 9852 9832 -20
Branches 1350 1349 -1
=======================================
- Hits 8582 8565 -17
Misses 917 917
+ Partials 353 350 -3
| Flag | Coverage Δ | |
|---|---|---|
| datachain | 87.08% <95.83%> (+<0.01%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Approving to move forward, but this design Session -> Catalog -> Sessions looks a bit weird to me.
Thanks. Modified the chain logic.