nifi icon indicating copy to clipboard operation
nifi copied to clipboard

NIFI-13929: Removed Provenance Repository from the stateless Reposito…

Open markap14 opened this issue 1 year ago • 0 comments

…ryContextFactory and added it to the DataflowTriggerContext. This was necessary because the previous design overlooked the possibility of many threads concurrently running the same dataflow. They all shared the same StatelessProvenanceRepository, but the code was designed as if only a single thread would be using the repository. As a result, the events that were registered with the stateless prov repo were being copied many times into NiFi's underlying provenance repository. This refactoring also led to the discovery of some old Java 8 syntax that could be cleaned up, and it led to the discovery of some methods that were no longer being used and could be cleaned up. Finally, in testing, I found that when a Stateless Group was scheduled, it scheduled the triggering of the stateless group before marking the state as RUNNING; as a result, the second thread could run, determine that the state is STARTING instead of RUNNING, and return without triggering the stateless group. This was addressed by ensuring that we set the state to RUNNING before triggering the stateless group to be triggered.

Summary

NIFI-00000

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • [ ] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • [ ] Pull Request based on current revision of the main branch
  • [ ] Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • [ ] Build completed using mvn clean install -P contrib-check
    • [ ] JDK 21

Licensing

  • [ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • [ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • [ ] Documentation formatting appears as expected in rendered files

markap14 avatar Oct 24 '24 21:10 markap14