receiving http error: 400

[dawaj71]

Hello,

As you can see with the title i got this error "receiving http error: 400" with some images that i want to scan, but with others images it scans well and without errors.

Both kind of images are local that i pull from my registry before i do the scan, the command that i use for the scan is : clairctl analyze -l animage

I used this command to see what are the logs : clairctl analyze -l --log-level debug animage

2018-03-09 15:01:32.913560 D | config: No config file used 2018-03-09 15:01:32.915134 D | dockercli: docker image to save: animage 2018-03-09 15:01:32.915202 D | dockercli: saving in: /tmp/clairctl/animage 2018-03-09 15:01:47.371101 I | config: retrieving interface for local IP 2018-03-09 15:01:47.371213 D | config: no interface provided, looking for docker0 2018-03-09 15:01:47.372097 D | server: Update local server port from "0" to "53700" 2018-03-09 15:01:47.372147 I | server: Starting Server on X.X.X.X 2018-03-09 15:01:47.377015 I | config: retrieving interface for local IP 2018-03-09 15:01:47.377065 D | config: no interface provided, looking for docker0 2018-03-09 15:01:47.377479 I | clair: using http://X.X.X.X/local as local url 2018-03-09 15:01:47.377524 I | clair: Pushing Layer 1/11 [17efd5fdbef8] 2018-03-09 15:01:47.377741 D | clair: Saving 17efd5fdbef8f0b0dcedd31beb409982edfcae8a6a9b9dbc45f7517e129e6497[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.423263 I | clair: Pushing Layer 2/11 [002fafa83f3b] 2018-03-09 15:01:47.423502 D | clair: Saving 002fafa83f3b6f94b3f44b3df6b1828bb8bd8fb8ded6b68fc2565dbaaef28fa6[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.431985 I | clair: Pushing Layer 3/11 [a43065cd7262] 2018-03-09 15:01:47.432204 D | clair: Saving a43065cd72626b5a3495cb4c53c391a9beb5add727b1ce824eae4394e3b28776[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.439857 I | clair: Pushing Layer 4/11 [35e854e2bf81] 2018-03-09 15:01:47.440034 D | clair: Saving 35e854e2bf8171c89ec6e99d849846137651e28c6ec0f5f39a5de1cfec724232[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.444071 I | clair: Pushing Layer 5/11 [723b01ba3a25] 2018-03-09 15:01:47.444224 D | clair: Saving 723b01ba3a25ca4df42da84356ec6c776e64e0c5967acc08aadae84ae3a05e1b[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.448404 I | clair: Pushing Layer 6/11 [ad2ae30a3f16] 2018-03-09 15:01:47.448534 D | clair: Saving ad2ae30a3f1625a5f5688d0e6ccdaff56abbba45c296c96741ab6379a4b032c7[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.457143 I | clair: Pushing Layer 7/11 [169b8319d348] 2018-03-09 15:01:47.457295 D | clair: Saving 169b8319d3481f95e77e13436276b0e27d0cc2a4c5b8ae8171f5ff3efb75aadc[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.461696 I | clair: Pushing Layer 8/11 [702a418e95b6] 2018-03-09 15:01:47.461837 D | clair: Saving 702a418e95b6793f8e37ba6ac09ed110df4cd83fd2b61d6ae5ff18ed72ba7990[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.466871 I | clair: Pushing Layer 9/11 [6d5e09df55e7] 2018-03-09 15:01:47.467038 D | clair: Saving 6d5e09df55e7ce71e3811be0a43cc8a8b4269caf01802dfc0404f6bc6fcb53b0[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.476673 I | clair: Pushing Layer 10/11 [66d2113695b9] 2018-03-09 15:01:47.476837 D | clair: Saving 66d2113695b97bc90b91abd4645d81cbf25258b3c7391727366d0aa1986a9b88[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.481223 I | clair: Pushing Layer 11/11 [bb4732f083a2] 2018-03-09 15:01:47.481363 D | clair: Saving bb4732f083a2aa5cd1ad07f3427fbdf201ff0f496ed0f04a5f1914a49f675f4c[https://registry-1.docker.io/v2] 2018-03-09 15:01:47.488110 I | config: retrieving interface for local IP 2018-03-09 15:01:47.488146 D | config: no interface provided, looking for docker0 2018-03-09 15:01:47.488585 I | clair: using http://X.X.X.X/local as local url 2018-03-09 15:01:47.652981 I | clair: analysing layer [bb4732f083a2] 1/11 2018-03-09 15:01:47.680441 I | clair: analysing layer [66d2113695b9] 2/11 2018-03-09 15:01:47.701403 I | clair: analysing layer [6d5e09df55e7] 3/11 2018-03-09 15:01:47.720298 I | clair: analysing layer [702a418e95b6] 4/11 2018-03-09 15:01:47.737839 I | clair: analysing layer [169b8319d348] 5/11 2018-03-09 15:01:47.756311 I | clair: analysing layer [ad2ae30a3f16] 6/11 2018-03-09 15:01:47.773774 I | clair: analysing layer [723b01ba3a25] 7/11 2018-03-09 15:01:47.792441 I | clair: analysing layer [35e854e2bf81] 8/11 2018-03-09 15:01:47.809847 I | clair: analysing layer [a43065cd7262] 9/11 2018-03-09 15:01:47.834309 I | clair: analysing layer [002fafa83f3b] 10/11 2018-03-09 15:01:47.851754 I | clair: analysing layer [17efd5fdbef8] 11/11 2018-03-09 15:01:47.852069 D | cmd: Using priority filters:

and here the logs from a scan without errors

2018-03-09 14:41:39.643721 D | config: No config file used 2018-03-09 14:41:39.647539 D | dockercli: docker image to save: animage 2018-03-09 14:41:39.647612 D | dockercli: saving in: /tmp/clairctl/animage 2018-03-09 14:42:19.298488 I | config: retrieving interface for local IP 2018-03-09 14:42:19.301301 D | config: no interface provided, looking for docker0 2018-03-09 14:42:19.312669 I | config: retrieving interface for local IP 2018-03-09 14:42:19.312767 D | config: no interface provided, looking for docker0 2018-03-09 14:42:19.316913 D | server: Update local server port from "0" to "53364" 2018-03-09 14:42:19.316970 I | server: Starting Server on X.X.X.X 2018-03-09 14:42:19.318944 I | clair: using http://X.X.X.X/local as local url 2018-03-09 14:42:19.319343 I | clair: Pushing Layer 1/11 [894c1784ca81] 2018-03-09 14:42:19.321202 D | clair: Saving 894c1784ca81caa3fd379b360c3194cef4043aeb8122d1cd2fc4d3736d43f825[https://registry-1.docker.io/v2] 2018-03-09 14:42:19.419638 I | clair: adding layer 1/11 [894c1784ca81]: receiving http error: 400 client quit unexpectedly 2018-03-09 14:42:19.419861 C | cmd: pushing image "canimage": receiving http error: 400

here my docker compose file :

version: '2.1' services: postgres: image: postgres:9.6 restart: unless-stopped volumes: - ./docker-compose-data/postgres-data/:/var/lib/postgresql/data:rw environment: - POSTGRES_PASSWORD=ChangeMe - POSTGRES_USER=clair - POSTGRES_DB=clair - http_proxy=myproxy - https_proxy=myproxy - no_proxy=/var/run/docker.sock,localhost,127.0.0.1 clair: group_add: - 244 image: quay.io/coreos/clair:latest restart: unless-stopped environment: - http_proxy=myproxy - https_proxy=myproxy - no_proxy=/var/run/docker.sock,localhost,127.0.0.1 ports: - "6060:6060" - "6061:6061" privileged: true volumes: - ./docker-compose-data/clair-config/:/config/:ro - ./docker-compose-data/clair-tmp/:/tmp/:rw depends_on: postgres: condition: service_started command: [--log-level=debug, --config, /config/config.yml] clairctl: group_add: - 244 image: jgsqware/clairctl:latest restart: unless-stopped environment: - http_proxy=myproxy - https_proxy=myproxy - no_proxy=/var/run/docker.sock,localhost,127.0.0.1 environment: - DOCKER_API_VERSION=1.3 volumes: - ./docker-compose-data/clairctl-reports/:/reports/:rw - /var/run/docker.sock:/var/run/docker.sock:ro depends_on: clair: condition: service_started

I have no errors from the logs of the three containers (postgres clair and clairctl)

[lukeb2e]

This pull request fixes the issue you are encountering: https://github.com/jgsqware/clairctl/pull/112

It is basically the same issue already documented in https://github.com/jgsqware/clairctl/issues/110

[dawaj71]

Hello, Thanks for your answer, i pulled with this command

git clone -b fix-temp-path https://github.com/glookie1/clairctl.git

and i have the same error :

2018-03-20 09:46:52.472923 D | config: No config file used 2018-03-20 09:46:52.473545 D | dockercli: docker image to save: serverweb:latest 2018-03-20 09:46:52.473584 D | dockercli: saving in: /tmp/clairctl/serverweb/latest/blobs 2018-03-20 09:47:02.196738 I | config: retrieving interface for local IP 2018-03-20 09:47:02.196848 D | config: no interface provided, looking for docker0 2018-03-20 09:47:02.197882 D | server: Update local server port from "0" to "50153" 2018-03-20 09:47:02.197936 I | server: Starting Server on X.X.X.X 2018-03-20 09:47:02.202660 I | config: retrieving interface for local IP 2018-03-20 09:47:02.202709 D | config: no interface provided, looking for docker0 2018-03-20 09:47:02.203060 I | clair: using http://X.X.X.X/local as local url 2018-03-20 09:47:02.203123 I | clair: Pushing Layer 1/2 [313a85ae8f22] 2018-03-20 09:47:02.203412 D | clair: Saving 313a85ae8f22ccda740b507b12894c4695ae3f75d819ad8cc31e33f413c47809[https:///v2] 2018-03-20 09:47:02.212524 I | clair: adding layer 1/2 [313a85ae8f22]: receiving http error: 400 client quit unexpectedly 2018-03-20 09:47:02.213402 C | cmd: pushing image "serverweb:latest": receiving http error: 400

here my dockercli.go files

import ( "compress/bzip2" "compress/gzip" "context" "encoding/json" "fmt" "io" "os" "strings" "syscall" "github.com/artyom/untar" "github.com/coreos/pkg/capnslog" "github.com/docker/distribution" "github.com/docker/distribution/manifest/schema1" "github.com/docker/docker/client" "github.com/docker/docker/image" "github.com/docker/docker/layer" "github.com/docker/docker/reference" "github.com/jgsqware/clairctl/config" "github.com/opencontainers/go-digest" ) var log = capnslog.NewPackageLogger("github.com/jgsqware/clairctl", "dockercli") func parseImage(imageName string) (reference.NamedTagged, error) { n, err := reference.ParseNamed(imageName) if err != nil { return nil, err } var image reference.NamedTagged if reference.IsNameOnly(n) { image = reference.WithDefaultTag(n).(reference.NamedTagged) } else { image = n.(reference.NamedTagged) } return image, nil } func tempImagePath(image reference.NamedTagged) string { return fmt.Sprintf("%s", image.FullName()) } //GetLocalManifest retrieve manifest for local image func GetLocalManifest(imageName string, withExport bool) (reference.NamedTagged, distribution.Manifest, error) { image, err := parseImage(imageName) if err != nil { return nil, nil, err } var manifest distribution.Manifest if withExport { manifest, err = save(image) } else { manifest, err = historyFromCommand(image) } if err != nil { return nil, schema1.SignedManifest{}, err } +more lignes

and dockercli_test.go

package dockercli import ( "testing" "github.com/docker/docker/reference" ) func TestImageParsing(t *testing.T) { images := map[string]string{ "ubuntu:14.04": "docker_io/library/ubuntu/14_04", "ubuntu/ubuntu:14.04": "docker_io/ubuntu/ubuntu/14_04", "registry.com/ubuntu:14.04": "registry_com/ubuntu/14_04", "registry.com/ubuntu/ubuntu:14.04": "registry_com/ubuntu/ubuntu/14_04", "registry.com:5000/ubuntu:14.04": "registry_com_5000/ubuntu/14_04", "registry.com:5000/ubuntu/ubuntu:14.04": "registry_com_5000/ubuntu/ubuntu/14_04", } for value, expected := range images { n, err := reference.ParseNamed(value) if err != nil { t.Error("Error:", err, expected) } var image reference.NamedTagged if reference.IsNameOnly(n) { image = reference.WithDefaultTag(n).(reference.NamedTagged) } else { image = n.(reference.NamedTagged) } result := tempImagePath(image) if result != expected { t.Errorf("Expecting %s, got %s", expected, result) } } }

[lukeb2e]

Hey,

For some weird reason you have to copy my source in the original directory because some paths are hardcoded somewhere.

Currently I only have access to my phone else I could provide you with my binary.

Am 20.03.2018 10:49 schrieb dawaj71 [email protected]:

Hello, Thanks for your answer, i pulled with this command

git clone -b fix-temp-path https://github.com/glookie1/clairctl.git

and i have the same error :

2018-03-20 09:46:52.472923 D | config: No config file used 2018-03-20 09:46:52.473545 D | dockercli: docker image to save: serverweb:latest 2018-03-20 09:46:52.473584 D | dockercli: saving in: /tmp/clairctl/serverweb/latest/blobs 2018-03-20 09:47:02.196738 I | config: retrieving interface for local IP 2018-03-20 09:47:02.196848 D | config: no interface provided, looking for docker0 2018-03-20 09:47:02.197882 D | server: Update local server port from "0" to "50153" 2018-03-20 09:47:02.197936 I | server: Starting Server on X.X.X.X 2018-03-20 09:47:02.202660 I | config: retrieving interface for local IP 2018-03-20 09:47:02.202709 D | config: no interface provided, looking for docker0 2018-03-20 09:47:02.203060 I | clair: using http://X.X.X.X/local as local url 2018-03-20 09:47:02.203123 I | clair: Pushing Layer 1/2 [313a85ae8f22] 2018-03-20 09:47:02.203412 D | clair: Saving 313a85ae8f22ccda740b507b12894c4695ae3f75d819ad8cc31e33f413c47809[https:///v2] 2018-03-20 09:47:02.212524 I | clair: adding layer 1/2 [313a85ae8f22]: receiving http error: 400 client quit unexpectedly 2018-03-20 09:47:02.213402 C | cmd: pushing image "serverweb:latest": receiving http error: 400

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/jgsqware/clairctl/issues/113#issuecomment-374518987, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AMVr_p69xfaC-PxymBMaxY2mfGaaMohxks5tgMKKgaJpZM4SkZis.

[dawaj71]

Thanks for your time.

I don't understand the

you have to copy my source in the original directory because some paths are hardcoded somewhere

You mean that I have to download it and paste it in my directory instead of using git clone ?

PS : I'm not in this directory btw '$GOPATH/src/github.com/jgsqware/clairctl"

PS1; I tried to gitclone the master branch from jsqawe and just replace the 2 files changed in /docker/dockercli and build with dockerfile and it does not work

[lukeb2e]

Yes

I didn't have time to debug the issue so it currently is as is.

Am 21.03.2018 09:05 schrieb dawaj71 [email protected]:

Thanks for your time.

I don't understand the

you have to copy my source in the original directory because some paths are hardcoded somewhere

You mean that I have to download it and paste it in my directory instead of using git clone ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/jgsqware/clairctl/issues/113#issuecomment-374848803, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AMVr_hKQFgsk4m3WNWrsS-in6s8D-YCRks5tgfukgaJpZM4SkZis.

[dawaj71]

Ok so what I did is :

copy the master branch from the jqsware projet and change only the 2 files that are affected by your pull request. Same error ^^

I have time no worries if you don't it is ok :-)

PS: in the docker file i changed ARG CLAIRCTL_VERSION=${CLAIRCTL_VERSION:-master} by ARG CLAIRCTL_VERSION=${CLAIRCTL_VERSION:-fix-temp-master}

PS2: clair logs :

{"Event":"could not download layer: expected 2XX","Level":"warning","Location":"driver.go:135","Time":"2018-03-22 07:28:02.991983","status code":404} {"Event":"failed to extract data from path","Level":"error","Location":"worker.go:122","Time":"2018-03-22 07:28:02.992164","error":"could not find layer","layer":"17efd5fdbef8f0b0dcedd31beb409982edfcae8a6a9b9dbc45f7517e129e6497","path":"http://X.X.X.X/local//planningdb/blobs/17efd5fdbef8f0b0dcedd31beb409982edfcae8a6a9b9dbc45f7517e129e6497/layer.tar"} {"Event":"Handled HTTP request","Level":"info","Location":"router.go:57","Time":"2018-03-22 07:28:02.993259","elapsed time":17228471,"method":"POST","remote addr":"172.19.0.1:54468","request uri":"/v1/layers","status":"400"}

PS3: now it doesnt work for every images not just for some images like before

[NigelWan]

@dawaj7 my issue is the same as you. let me know if you had found out the way to fix. thanks

Update: use v1.2.6 in https://github.com/jgsqware/clairctl/releases temporary fix my problem

[lukeb2e]

I always have my images tagged with the full name (registry/namespace/name:tag) - it might not work if you only have set a name for your image...

I also always have to push the image first before I can generate a report.

clairctl push -l ${container}
clairctl report --format html -l ${container}

Can you try to tag your image with a domain & namespace as well?

[pengyaoxu]

I also meet this problem.My config file below:

version: '2.1'
services:
  postgres:
    container_name: clair_postgres
    image: postgres:9.6
    restart: unless-stopped
    environment:
      POSTGRES_PASSWORD: password

  clair:
    container_name: clair_clair
    image: quay.io/coreos/clair:v2.0.0
    restart: unless-stopped
    depends_on:
      - postgres
    ports:
      - "6060-6061:6060-6061"
    links:
      - postgres
    volumes:
      - /tmp:/tmp
      - ./clair_config:/config
    command: [-config, /config/config.yaml]

When i run clairctl analyze -l --log-level debug centos:

2018-04-17 02:53:43.098375 D | config: No config file used
2018-04-17 02:53:43.098702 D | dockercli: docker image to save: centos:latest
2018-04-17 02:53:43.098721 D | dockercli: saving in: /tmp/clairctl/centos/latest/blobs
2018-04-17 02:53:46.727978 I | config: retrieving interface for local IP
2018-04-17 02:53:46.728022 D | config: no interface provided, looking for docker0
2018-04-17 02:53:46.728987 D | server: Update local server port from "0" to "40347"
2018-04-17 02:53:46.729003 I | server: Starting Server on 172.17.0.1:40347
2018-04-17 02:53:46.734048 I | config: retrieving interface for local IP
2018-04-17 02:53:46.734090 D | config: no interface provided, looking for docker0
2018-04-17 02:53:46.734512 I | clair: using http://172.17.0.1:40347/local as local url
2018-04-17 02:53:46.734552 I | clair: Pushing Layer 1/1 [6ce355201742]
2018-04-17 02:53:46.734756 D | clair: Saving 6ce35520174293337cf4fa0039f04e5cecfcaceb5b968d89fcc9c6fd7b952d97[https:///v2]
2018-04-17 02:53:46.742209 I | clair: adding layer 1/1 [6ce355201742]: receiving http error: 400
client quit unexpectedly
2018-04-17 02:53:46.742254 C | cmd: pushing image "centos:latest": receiving http error: 400

Is that anything wrong with my config file?

[lukeb2e]

Did you push your image before you triggered the analyze?

clairctl push -l ${container}

[djl197]

I too am having similar issues.
I am analysing Centos based containers and pushing 'local' images into clair works for some layers but for others I get the same messages as above. I have tried 1.2.8 and the 'fix-temp-path' branch but with no difference in behaviour

[pengyaoxu]

Now is ok with my clairctl.I pull the latest clairctl resource code and compile.It works.

[animale66]

I am also getting HTTP 400s while doing clairctl push or clairctl analyze with images that are publically available on dockerhub. The issue appears to me that clairctl is not honoring http_proxy or https_proxy variables (and likely, no_proxy too).

Clairctl version 1.2.8

None of the machines (either the clair server, which I run as a containerized instance), nor the clairctl client have direct non-proxy access to the real-world. I am able to run clairctl pull without any problems, just push and analyze don't work.

My ~/clairctl.yaml contains the following:

clair: port: 6060 healthPort: 6061 uri: http://127.0.0.1 report: path: /opt/data/httpd/htdocs/clair_reports/ format: html

luser@bigbox ~ $ http_proxy=http://myproxyserver.domain.net:8080 https_proxy=http://myproxyserver.domain.net:8080 clairctl push rancher/dns:v0.15.3

2018-06-14 18:38:20.612975 E | clair: response error: Head https://registry-1.docker.io/v2/rancher/dns/blobs/sha256:b3e1c725a85f0953e81815b7c7aabfad9ebfd90af53f99248981282b8045d787: dial tcp 34.200.28.105:443: getsockopt: connection refused

client quit unexpectedly

2018-06-14 18:38:20.614321 C | cmd: pushing image "rancher/dns:v0.15.3": receiving http error: 400

Clair has it both as environment variables passed via launch, as well as in it's config.yaml, so if I'm missing it somewhere I have no idea where I'm supposed to set it.

/usr/bin/docker run --net bridge -m 0b --net=host --userns=host -e http_proxy=http://myproxyserver.domain.net:8080 -e https_proxy=http://myproxyserver.domain.net:8080 -e no_proxy=localhost,*.domain.net -p 6060-6061:6060-6061 -v /opt/data/clair_config:/config \ --name clair \ clair

And the tail part of my clair-config lists the proxy server:

{stock stuff at the top}

notifier: # Number of attempts before the notification is marked as failed to be sent attempts: 3

# Duration before a failed notification is retried
renotifyinterval: 2h

http:
  # Optional endpoint that will receive notifications via POST requests
  endpoint:

  # Optional PKI configuration
  # If you want to easily generate client certificates and CAs, try the following projects:
  # https://github.com/cloudflare/cfssl
  # https://github.com/coreos/etcd-ca
  servername:
  cafile:
  keyfile:
  certfile:

  # Optional HTTP Proxy: must be a valid URL (including the scheme).
  proxy: http://myproxyserver.domain.net:8080

I feel this error is with the clairctl client, since I'm able to pull vulnerability data from the Internet via the proxy server.

[chirangaalwis]

I am also facing a similar issue. Also explained in https://github.com/jgsqware/clairctl/issues/74.

Is this fixed yet? @jgsqware is Clairctl deprecated as I find no development or support for existing issues in recent times?