dockercloud-agent icon indicating copy to clipboard operation
dockercloud-agent copied to clipboard
verified publisher icon docker-archive

My nodes Digital Ocean are UNREACHABLE from docker cloud

Open jjmonsalveg opened this issue 9 years ago • 1 comments

Hello, I have worked with docker cloud for a month. My nodes are created as described at https://docs.docker.com/docker-cloud/infrastructure/link-do/ for a Digital Ocean account. But I've decided to update the operating system of the node (I access through ssh https://docs.docker.com/docker-cloud/infrastructure/ssh-into-a-node/) $ sudo apt update && sudo apt upgrade

Then increase the swap memory as described here: https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-14-04

I finally reboot the system: $ sudo reboot

But after this my system is already inaccessible from Docker cloud as shown in the image.

When running $ docker version I get:

Docker version 
Client: 
 Version: 1.11.2-cs5 
 API version: 1.23 
 Go version: go1.5.4 
 Git commit: d364ea1 
 Built: Tue Sep 13 15:26:43 2016 
 OS / Arch: linux / amd64 
Can not connect to the Docker daemon. Is the docker daemon running on this host?

log /var/log/dockercloud/agent.log

2016/12/21 15:58:56 Got signal: terminated 2016/12/21 15:58:56 Docker daemon is running 2016/12/21 15:58:56 Starting to shut down docker daemon gracefully 2016/12/21 15:59:26 Running dockercloud-agent: version 1.1.0 2016/12/21 15:59:26 Create pid file(/var/run/dockercloud-agent.pid): 1166 2016/12/21 15:59:26 Checking if config file exists 2016/12/21 15:59:26 Loading Configuration file 2016/12/21 15:59:26 Registering in Docker Cloud via PATCH: https://cloud.docker.com/api/agent/v1/node/7b74789b-ba06-427b-b3ed-b548466d949d 2016/12/21 15:59:27 PATCH error 401 :either UUID (7b74789b-ba06-427b-b3ed-b548466d949d) or Token is invalid 2016/12/21 15:59:27 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 15:59:27 Cannot register node in Docker Cloud: unauthorized. Please try again with a new token. 2016/12/21 15:59:27 Removing the invalid token from config file 2016/12/21 15:59:37 Docker Cloud agent is terminated 2016/12/21 15:59:37 Running dockercloud-agent: version 1.1.0 2016/12/21 15:59:37 Create pid file(/var/run/dockercloud-agent.pid): 1820 2016/12/21 15:59:37 Checking if config file exists 2016/12/21 15:59:37 Loading Configuration file 2016/12/21 15:59:37 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 15:59:37 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 15:59:37 Running dockercloud-agent: version 1.1.0 2016/12/21 15:59:37 Create pid file(/var/run/dockercloud-agent.pid): 1860 2016/12/21 15:59:37 Checking if config file exists 2016/12/21 15:59:37 Loading Configuration file 2016/12/21 15:59:37 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 15:59:37 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 15:59:37 Running dockercloud-agent: version 1.1.0 2016/12/21 15:59:37 Create pid file(/var/run/dockercloud-agent.pid): 1902 2016/12/21 15:59:37 Checking if config file exists 2016/12/21 15:59:37 Loading Configuration file 2016/12/21 15:59:37 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 15:59:37 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:24:01 Running dockercloud-agent: version 1.1.0 2016/12/21 16:24:01 Create pid file(/var/run/dockercloud-agent.pid): 2259 2016/12/21 16:24:01 Checking if config file exists 2016/12/21 16:24:01 Loading Configuration file 2016/12/21 16:24:01 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:24:01 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:31:49 Running dockercloud-agent: version 1.1.0 2016/12/21 16:31:49 Create pid file(/var/run/dockercloud-agent.pid): 2319 2016/12/21 16:31:49 Checking if config file exists 2016/12/21 16:31:49 Loading Configuration file 2016/12/21 16:31:49 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:31:49 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:31:50 Running dockercloud-agent: version 1.1.0 2016/12/21 16:31:50 Create pid file(/var/run/dockercloud-agent.pid): 2359 2016/12/21 16:31:50 Checking if config file exists 2016/12/21 16:31:50 Loading Configuration file 2016/12/21 16:31:50 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:31:50 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:31:50 Running dockercloud-agent: version 1.1.0 2016/12/21 16:31:50 Create pid file(/var/run/dockercloud-agent.pid): 2399 2016/12/21 16:31:50 Checking if config file exists 2016/12/21 16:31:50 Loading Configuration file 2016/12/21 16:31:50 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:31:50 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:31:50 Running dockercloud-agent: version 1.1.0 2016/12/21 16:31:50 Create pid file(/var/run/dockercloud-agent.pid): 2441 2016/12/21 16:31:50 Checking if config file exists 2016/12/21 16:31:50 Loading Configuration file 2016/12/21 16:31:50 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:31:50 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:32:03 Running dockercloud-agent: version 1.1.0 2016/12/21 16:32:03 Create pid file(/var/run/dockercloud-agent.pid): 2494 2016/12/21 16:32:03 Checking if config file exists 2016/12/21 16:32:03 Loading Configuration file 2016/12/21 16:32:03 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:32:03 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:32:03 Running dockercloud-agent: version 1.1.0 2016/12/21 16:32:03 Create pid file(/var/run/dockercloud-agent.pid): 2534 2016/12/21 16:32:03 Checking if config file exists 2016/12/21 16:32:03 Loading Configuration file 2016/12/21 16:32:03 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:32:03 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:32:03 Running dockercloud-agent: version 1.1.0 2016/12/21 16:32:03 Create pid file(/var/run/dockercloud-agent.pid): 2574 2016/12/21 16:32:03 Checking if config file exists 2016/12/21 16:32:03 Loading Configuration file 2016/12/21 16:32:03 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:32:03 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:32:03 Running dockercloud-agent: version 1.1.0 2016/12/21 16:32:03 Create pid file(/var/run/dockercloud-agent.pid): 2614 2016/12/21 16:32:03 Checking if config file exists 2016/12/21 16:32:03 Loading Configuration file 2016/12/21 16:32:03 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:32:03 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:32:59 Running dockercloud-agent: version 1.1.0 2016/12/21 16:32:59 Create pid file(/var/run/dockercloud-agent.pid): 2631 2016/12/21 16:32:59 Checking if config file exists 2016/12/21 16:32:59 Loading Configuration file 2016/12/21 16:32:59 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:32:59 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:34:36 Running dockercloud-agent: version 1.1.0 2016/12/21 16:34:36 Create pid file(/var/run/dockercloud-agent.pid): 2676 2016/12/21 16:34:36 Checking if config file exists 2016/12/21 16:34:36 Loading Configuration file 2016/12/21 16:34:36 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:34:36 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:34:36 Running dockercloud-agent: version 1.1.0 2016/12/21 16:34:36 Create pid file(/var/run/dockercloud-agent.pid): 2717 2016/12/21 16:34:36 Checking if config file exists 2016/12/21 16:34:36 Loading Configuration file 2016/12/21 16:34:36 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:34:36 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:34:36 Running dockercloud-agent: version 1.1.0 2016/12/21 16:34:36 Create pid file(/var/run/dockercloud-agent.pid): 2758 2016/12/21 16:34:36 Checking if config file exists 2016/12/21 16:34:36 Loading Configuration file 2016/12/21 16:34:36 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:34:36 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:34:36 Running dockercloud-agent: version 1.1.0 2016/12/21 16:34:36 Create pid file(/var/run/dockercloud-agent.pid): 2799 2016/12/21 16:34:36 Checking if config file exists 2016/12/21 16:34:36 Loading Configuration file 2016/12/21 16:34:36 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:34:36 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:54:29 Running dockercloud-agent: version 1.1.0 2016/12/21 16:54:29 Create pid file(/var/run/dockercloud-agent.pid): 1164 2016/12/21 16:54:29 Checking if config file exists 2016/12/21 16:54:29 Loading Configuration file 2016/12/21 16:54:29 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:54:29 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:54:30 Running dockercloud-agent: version 1.1.0 2016/12/21 16:54:30 Create pid file(/var/run/dockercloud-agent.pid): 1300 2016/12/21 16:54:30 Checking if config file exists 2016/12/21 16:54:30 Loading Configuration file 2016/12/21 16:54:30 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:54:30 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:54:30 Running dockercloud-agent: version 1.1.0 2016/12/21 16:54:30 Create pid file(/var/run/dockercloud-agent.pid): 1395 2016/12/21 16:54:30 Checking if config file exists 2016/12/21 16:54:30 Loading Configuration file 2016/12/21 16:54:30 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:54:30 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:54:31 Running dockercloud-agent: version 1.1.0 2016/12/21 16:54:31 Create pid file(/var/run/dockercloud-agent.pid): 1560 2016/12/21 16:54:31 Checking if config file exists 2016/12/21 16:54:31 Loading Configuration file 2016/12/21 16:54:31 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:54:31 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:54:31 Running dockercloud-agent: version 1.1.0 2016/12/21 16:54:31 Create pid file(/var/run/dockercloud-agent.pid): 1652 2016/12/21 16:54:31 Checking if config file exists 2016/12/21 16:54:31 Loading Configuration file 2016/12/21 16:54:31 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:54:31 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:54:31 Running dockercloud-agent: version 1.1.0 2016/12/21 16:54:31 Create pid file(/var/run/dockercloud-agent.pid): 1689 2016/12/21 16:54:31 Checking if config file exists 2016/12/21 16:54:31 Loading Configuration file 2016/12/21 16:54:31 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:54:31 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/21 16:54:32 Running dockercloud-agent: version 1.1.0 2016/12/21 16:54:32 Create pid file(/var/run/dockercloud-agent.pid): 1731 2016/12/21 16:54:32 Checking if config file exists 2016/12/21 16:54:32 Loading Configuration file 2016/12/21 16:54:32 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/21 16:54:32 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 04:01:35 Running dockercloud-agent: version 1.1.0 2016/12/22 04:01:35 Create pid file(/var/run/dockercloud-agent.pid): 1163 2016/12/22 04:01:35 Checking if config file exists 2016/12/22 04:01:35 Loading Configuration file 2016/12/22 04:01:35 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 04:01:35 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 04:01:36 Running dockercloud-agent: version 1.1.0 2016/12/22 04:01:36 Create pid file(/var/run/dockercloud-agent.pid): 1304 2016/12/22 04:01:36 Checking if config file exists 2016/12/22 04:01:36 Loading Configuration file 2016/12/22 04:01:36 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 04:01:36 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 04:01:36 Running dockercloud-agent: version 1.1.0 2016/12/22 04:01:36 Create pid file(/var/run/dockercloud-agent.pid): 1398 2016/12/22 04:01:36 Checking if config file exists 2016/12/22 04:01:36 Loading Configuration file 2016/12/22 04:01:36 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 04:01:36 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 04:01:37 Running dockercloud-agent: version 1.1.0 2016/12/22 04:01:37 Create pid file(/var/run/dockercloud-agent.pid): 1567 2016/12/22 04:01:37 Checking if config file exists 2016/12/22 04:01:37 Loading Configuration file 2016/12/22 04:01:37 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 04:01:37 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 04:01:38 Running dockercloud-agent: version 1.1.0 2016/12/22 04:01:38 Create pid file(/var/run/dockercloud-agent.pid): 1651 2016/12/22 04:01:38 Checking if config file exists 2016/12/22 04:01:38 Loading Configuration file 2016/12/22 04:01:38 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 04:01:38 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 04:01:38 Running dockercloud-agent: version 1.1.0 2016/12/22 04:01:38 Create pid file(/var/run/dockercloud-agent.pid): 1692 2016/12/22 04:01:38 Checking if config file exists 2016/12/22 04:01:38 Loading Configuration file 2016/12/22 04:01:38 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 04:01:38 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 04:01:38 Running dockercloud-agent: version 1.1.0 2016/12/22 04:01:38 Create pid file(/var/run/dockercloud-agent.pid): 1735 2016/12/22 04:01:38 Checking if config file exists 2016/12/22 04:01:38 Loading Configuration file 2016/12/22 04:01:38 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 04:01:38 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 12:30:58 Running dockercloud-agent: version 1.1.0 2016/12/22 12:30:58 Create pid file(/var/run/dockercloud-agent.pid): 3913 2016/12/22 12:30:58 Checking if config file exists 2016/12/22 12:30:58 Loading Configuration file 2016/12/22 12:30:58 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 12:30:58 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 12:30:58 Running dockercloud-agent: version 1.1.0 2016/12/22 12:30:58 Create pid file(/var/run/dockercloud-agent.pid): 3953 2016/12/22 12:30:58 Checking if config file exists 2016/12/22 12:30:58 Loading Configuration file 2016/12/22 12:30:58 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 12:30:58 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 12:30:58 Running dockercloud-agent: version 1.1.0 2016/12/22 12:30:58 Create pid file(/var/run/dockercloud-agent.pid): 3995 2016/12/22 12:30:58 Checking if config file exists 2016/12/22 12:30:58 Loading Configuration file 2016/12/22 12:30:58 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 12:30:58 The token is empty. Please run 'dockercloud-agent set Token=xxx' first! 2016/12/22 12:30:58 Running dockercloud-agent: version 1.1.0 2016/12/22 12:30:58 Create pid file(/var/run/dockercloud-agent.pid): 4036 2016/12/22 12:30:58 Checking if config file exists 2016/12/22 12:30:58 Loading Configuration file 2016/12/22 12:30:58 Registering in Docker Cloud via POST: https://cloud.docker.com/api/agent/v1/node/ 2016/12/22 12:30:58 The token is empty. Please run 'dockercloud-agent set Token=xxx' first!

How can I recover these nodes? I appreciate your help.

seleccion_006

jjmonsalveg avatar Dec 22 '16 12:12 jjmonsalveg

@jjmonsalveg

There was an issue on our side that marked your node token invalid. We have found the issue and fixed it, but unfortunately, your node was affected before the fix.

If there is no important data/volume on the node, you can simply remove the node and create a new one. Our scheduler will automatically spawn related containers to the new nodes.

If you DO need recover the node, please tell me the best way to reach you, and we can help you recover it step by step. Alternatively, you can open a support ticket at support.docker.com, tell me the ticket number, and I can follow you there.

Thank you

tifayuki avatar Dec 22 '16 17:12 tifayuki