Can't view interaction with private registry to troubleshoot

[dwatrous]

I would like to be able to see the detailed interactions with a docker v2 registry.

I'm using Artifactory to provide a docker v2 registry. It is configured to allow anonymous push and works on most of the computers in our office. About 1 in 3 computers (usually Macs) get an error

$ docker push stash.company.com:8443/pmd
The push refers to a repository [stash.company.com:8443/pmd]
24af4657da8d: Preparing
...
d17d48b2382a: Waiting
unauthorized: BAD_CREDENTIAL

I have been unable to figure out what is different between these computers that causes some computers to get this error and other to work as expected. I have compared ~/.docker/config.json to ensure there isn't a credential being passed.

What I have tried:

I tried adding --debug, -D to the docker push command, but it doesn't show any details about the interaction with the registry, so I can't see if it's trying to send a credential when it shouldn't be.

I also tried following the Docker daemon logs to see if they provided some details about the interaction with the registry, but it only shows a DNS lookup.

I have asked three or four times on the slack channel and I posted this question on SO: https://stackoverflow.com/questions/45172693/docker-push-how-to-see-detailed-interaction-with-registry

It would seem that this isn't possible, so this is probably a feature request. I would like to be able to view detailed logs showing how my local docker is interacting with a registry.

Output of docker version:

$ docker version
Client:
 Version:      17.06.0-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:31:53 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.06.0-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:51:55 2017
 OS/Arch:      linux/amd64
 Experimental: true

Output of docker info:

$ docker info
Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 2
Server Version: 17.06.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a
runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.36-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.952GiB
Name: moby
ID: RYYU:QDZ6:YNDV:7GSO:VEEV:YLXD:KPJW:RB4T:CIB6:AQKI:V6ET:ZF32
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 18
 Goroutines: 30
 System Time: 2017-07-24T14:46:57.11149011Z
 EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.): MacOS Sierra (10.12.5) with native install of Docker CE Stable

[thaJeztah]

/cc @dmcgowan @n4ss @simonferquel PTAL

[n4ss]

@dwatrous can you please:

• show the output ofdocker-credential-osxkeychain version
• check that your server is present in docker-credential-osxkeychain list output with "https" protocol?

[thaJeztah]

@n4ss I think the question here is also to have a way to monitor / track what's happening. For credentials, a log-entry that shows if credentials were found (and sent) to the registry (obviously, not without printing the content of the credentials)

[dwatrous]

@n4ss , @thaJeztah is right. I have looked at my local credentials, including in the keychain, and everything looks fine. I'm interested in seeing the details of the interaction with the registry. Since the docker v2 registry spec is REST based, I would expect to see the HTTP requests and responses. This would tell me what was being sent.

As @thaJeztah mentioned, any sensitive data would need to be masked before being logged.

I actually don't care if it's logged, but there should be some way to view those details, and right now there isn't (at least not that I can find).