nifi icon indicating copy to clipboard operation
nifi copied to clipboard

NIFI-10073 Adding fluid load balancing strategy

Open simonbence opened this issue 3 years ago • 1 comments

NIFI-10073

This is a possible approach to add "fluid" load balancing strategy which among others, tries to split the load between nodes based on the actual processing capacities. Please keep that in mind that this is currently in a DRAFT phase! Thanks!

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • [x] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • [ ] Pull Request based on current revision of the main branch
  • [ ] Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • [ ] Build completed using mvn clean install -P contrib-check
    • [x] JDK 8
    • [x] JDK 11
    • [ ] JDK 17

Licensing

  • [ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • [ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • [ ] Documentation formatting appears as expected in rendered files

simonbence avatar Jul 19 '22 20:07 simonbence

In principle, this looks like it should solve a problem that's vexed my team for a while. We have nodes frequently not getting new data because the current approaches seem to decide to stop sending data to "slower nodes." Am I reading this Jira/PR right?

MikeThomsen avatar Aug 14 '22 16:08 MikeThomsen