Progress report for pulling images and session creation

[achimnol]

Let's add explicit progress reports when creating a new session.

For cluster sessions, multiple agents may try to pull the image at the same time, so the pull progress should include both per-agent progress and total progress.

• Manager
  • [ ] Spawn a background task when creating a new session, returning the task ID to the client after max_wait seconds.
  • [ ] Let the background task relay the kernel_pull_progress events.
• Agent
  • [ ] Add a progress reporter argument to AbstractAgent.pull_image() and let the concrete implementations report the progress from the underlying backend (e.g., the docker API).
  • [ ] Implement the progress report to produce kernel_pull_progress events.
• Client (Python & Javascript)
  • [ ] Check the existence of the task ID in the create-kernel API response.
  • [ ] If it exists, show a progress bar reflecting its progress stream.

FYI: Using the following code snippet, we can keep track of the overall progress in percent: https://gist.github.com/achimnol/11fd2b379ad186cf624307b2c7e0953f

[youngjun0627]

@achimnol I implement kernel-pull-progress. please review my code.

result images:

[fregataa]

Suggenstion. How about creating association_agents_images table and let it determine Image.is_installed, which has been saved in Redis? Plus, saving bgtask_id in the association_agents_images would be good to track the long-term pulling for each agent.