fides icon indicating copy to clipboard operation
fides copied to clipboard

[ENG-2185]

Open JadeCara opened this issue 1 month ago • 2 comments

Ticket ENG-2185

Description Of Changes

This ticket is asking for consent manual tasks. We are going forward with a work around #7102 because consent task graphs and access/erasure task graphs are different and have different capabilities - especially around conditional dependencies.

In order to make conditional dependencies work for consent tasks and access/erasure tasks we need to be able to assign conditions to specific manual task instances/submissions rather than to the whole task. (There was another request to be able to assign users the same way).

In addition the original update model where conditions would be updated individually was replaced by a full delete replace on conditions making individual condition search/retrieval unnecessary. The logging being used by the front end is almost entirely the general audit logs so the manual task log table and utilities are now redundant.

This PR is the first step to getting manual tasks and conditions into shape for the above requirements. It is primarily clean up based on the first few rounds of customer use and learning actual usage patterns.

  1. single row for conditional dependencies:
  • Migration to move all rows for conditional dependencies under into a JSONB in a single row.
    • This will save time when reading, writing, deleting etc.
    • This also sets us up nicely for future work on verifying that conditions do not negate each other (for policies coming up)
    • Right now the structure is limited to a single leaf or AND -> [leaves] The migration can handle this very easily.
    • Right now there are a couple customers using this but they don't have a LOT of conditions created so it should be easily run as a migration.
  • Update to the ORM removing the old parent/child column and adding the condition tree column
  • Updates to all tests

Code Changes

  • .fides/db_dataset.yml - update new column structure
  • src/fides/api/alembic/migrations/versions/xx_2025_12_12_1313_c37e8fde5a71_migrate_to_one_row_conditional_.py New migration, does a data migration from multi row to single row structure
  • src/fides/api/models/conditional_dependency/conditional_dependency_base.py updated to use new JsonB column removing old parent col etc
  • src/fides/api/models/digest/conditional_dependencies.py updated to use new inherited base
  • src/fides/api/models/manual_task/conditional_dependency.py updated to use new inherited base
  • src/fides/api/task/manual/manual_task_conditional_evaluation.py updated to pull field addresses from tree rather than rows
  • src/fides/api/task/manual/manual_task_utils.py removed the enum for the old row based conditional dependencies and added field address extraction function.
  • Updated all tests using the new model

Steps to Confirm

  1. There should be NO change in functionality.
  2. Manual tasks, conditional dependencies etc should continue to function as before.
  3. Please test with FidesPlus 2918
  4. Run FidesPlus branch above pointed at this branch.
  5. Create a ManualTask with several submissions required and several conditions, both dataset and privacy request based.
  6. Create privacy requests that meet and do not meet the requirements. Verify that they create/do not create as expected.
  7. Verify that you can complete the manual tasks with no errors.
  8. Verify that you can see the skipped logs and the manual task and manual task actions appear on the DSR activity log.
  9. Verify that you receive the access package with all expected manual task inclusions (attachments and text)
  10. Create a digest - The conditions are currently hardcoded for this so it should send as normal/expected. (Tip: you can adjust the rate for sending using the API so you dont have to wait until Monday at 9am to receive the digest)

Pre-Merge Checklist

  • [ ] Issue requirements met
  • [ ] All CI pipelines succeeded
  • [ ] CHANGELOG.md updated
    • [ ] Add a https://github.com/ethyca/fides/labels/db-migration label to the entry if your change includes a DB migration
    • [ ] Add a https://github.com/ethyca/fides/labels/high-risk label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
    • [ ] Updates unreleased work already in Changelog, no new entry necessary
  • UX feedback:
    • [ ] All UX related changes have been reviewed by a designer
    • [ ] No UX review needed
  • Followup issues:
    • [ ] Followup issues created
    • [ ] No followup issues
  • Database migrations:
    • [ ] Ensure that your downrev is up to date with the latest revision on main
    • [ ] Ensure that your downgrade() migration is correct and works
      • [ ] If a downgrade migration is not possible for this change, please call this out in the PR description!
    • [ ] No migrations
  • Documentation:
    • [ ] Documentation complete, PR opened in fidesdocs
    • [ ] Documentation issue created in fidesdocs
    • [ ] If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • [ ] No documentation updates required

JadeCara avatar Dec 12 '25 13:12 JadeCara