shield icon indicating copy to clipboard operation
shield copied to clipboard
verified publisher icon shieldproject

[BUG] SHIELD has not marked a task as failed, even though the agent restarted

Open thomasmitchell opened this issue 6 years ago • 4 comments

Describe the bug The agent restarted, and its not doing that task anymore. SHIELD has not marked the task as failed in the database (it is still running). This may cause future attempts to run this task to stay pending.

To Reproduce Steps to reproduce the behavior:

  1. Schedule a long-running backup
  2. Restart the agent running it
  3. Check the task list

Expected behavior The backup should have been marked as failed.

Screenshots If applicable, add screenshots to help explain your problem.

SHIELD versions (please complete the following information):

  • SHIELD Core: 8.5.0
  • SHIELD CLI: 8.5.0

Browser version(s) (please complete if reporting a web UI bug): n/a

Additional context Add any other context about the problem here.

thomasmitchell avatar Nov 13 '19 03:11 thomasmitchell

I believe there is a (default, but configurable) 12h timeout after which time the task will be marked as cancelled.

jhunt avatar Nov 14 '19 00:11 jhunt

This appears to be what eventually occurred.

thomasmitchell avatar Nov 14 '19 01:11 thomasmitchell

Is there any way that a task can become successful if the SSH session is terminated before task completion?

thomasmitchell avatar Nov 14 '19 01:11 thomasmitchell

Hey, I never said it was a great solution 😁

If the SSH connection fails, we should fail the task immediately. I can't wait to get this into a lab and repro it to see how you pulled this one off, @thomasmitchell ...

jhunt avatar Nov 14 '19 03:11 jhunt