epirust icon indicating copy to clipboard operation
epirust copied to clipboard

Engines pick up stale simulation request

Open akshaydewan opened this issue 5 years ago • 1 comments

Steps to reproduce:

  1. Run a simulation in multi-engine mode with, say, 2 engines.
  2. Let the simulation finish. Orchestrator and engines will terminate.
  3. Start another multi-engine simulation with 3 or 4 engines (> 2)
  4. The new engines will pick up the old simulation request and complain that no work was found for engines3 and 4.

Cause: Since we're adding new consumers (engines), Kafka delivers the older messages to the engines.

Possible fix: Let the orchestrator delete the simulation requests after the simulation finishes.

akshaydewan avatar Apr 07 '20 10:04 akshaydewan

Small-term fix: add fromOffset: latest in the consumer config.

Calling this small term-fix as it is now creating multiple records of consumer vs offset on the kafka-server for each run, which is not the desired behaviour.

swakhandekar avatar Apr 13 '20 05:04 swakhandekar