GVM-Docker icon indicating copy to clipboard operation
GVM-Docker copied to clipboard
verified publisher icon Secure-Compliance-Solutions-LLC

Upgrade Greenbone to debian-master (Report outdated / end-of-life Scan Engine)

Open devMarc5598 opened this issue 4 years ago • 42 comments

Hello, I want to update from my current alpine version (21.4.2) to the newest debian version. Is there a guide with some simple steps to follow? A detailed guidance would be very good!?

I would be happy about any help.

GVM versions gsad: Greenbone Security Assistant 21.4.2 gvmd: Greenbone Vulnerability Manager 21.4.3 openvas-scanner: OpenVAS 21.4.2 gvm-libs: gvm-libs 21.4.2

devMarc5598 avatar Mar 11 '22 09:03 devMarc5598

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

It use POSTGRESQL 13.

What are meaning the tags:

docker pull deineagenturug/gvm:latest               # is an not initialized basic image, that first start its initialization on the first start (short: no db & no PDF report support)
docker pull deineagenturug/gvm:latest-full          # is an not initialized basic image, that first start its initialization on the first start (short: no db, PDF report is support)
docker pull deineagenturug/gvm:latest-data          # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db, but no PDF report support)
docker pull deineagenturug/gvm:latest-data-full     # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db & PDF report is support)

It can be used as direct replacement for the here provided image, with the same arguments on docker run.

⚠️ I have not tested the migration from the alpine image, but this should work out of the box now. Maybe you can give a notice it it works as I expected.

Dexus avatar Mar 14 '22 11:03 Dexus

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

It use POSTGRESQL 13.

What are meaning the tags:

docker pull deineagenturug/gvm:latest               # is an not initialized basic image, that first start its initialization on the first start (short: no db & no PDF report support)
docker pull deineagenturug/gvm:latest-full          # is an not initialized basic image, that first start its initialization on the first start (short: no db, PDF report is support)
docker pull deineagenturug/gvm:latest-data          # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db, but no PDF report support)
docker pull deineagenturug/gvm:latest-data-full     # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db & PDF report is support)

It can be used as direct replacement for the here provided image, with the same arguments on docker run.

warning I have not tested the migration from the alpine image, but this should work out of the box now. Maybe you can give a notice it it works as I expected.

Why do you switch from https://hub.docker.com/r/securecompliance/gvm to https://hub.docker.com/r/deineagenturug/gvm?

devMarc5598 avatar Mar 16 '22 09:03 devMarc5598

Why do you switch from https://hub.docker.com/r/securecompliance/gvm to https://hub.docker.com/r/deineagenturug/gvm?

Because I feel like I'm the only one working on it. So I created my own fork so that I am able to push new versions and build all Docker images directly and correctly without having to wait for a merge. Unfortunately, that's what we've seen here in the past and again currently, no one cares. So someone has to do it who also has all the rights. And since I don't work for SCS LLC, I have no choice but to use my own fork and hardware so I can do my work and support best possible.

Dexus avatar Mar 16 '22 10:03 Dexus

I'm using the "docker pull deineagenturug/gvm:latest-data-full" but when I go to check the openvas scan added, the service appears unavailable, does anyone have a link to openvas correct or have you faced this problem?

Kelvinloucosta avatar Mar 17 '22 12:03 Kelvinloucosta

I think I have had the same or a similar issue. The gvmd logs shows "osp_scanner_feed_version: failed to get scanner_feed_version. OSPd OpenVAS is still starting"

@Dexus - Any thoughts? Thanks for your efforts keeping this project going!

FiveOFive avatar Mar 19 '22 01:03 FiveOFive

I think I have had the same or a similar issue. The gvmd logs shows "osp_scanner_feed_version: failed to get scanner_feed_version. OSPd OpenVAS is still starting"

I will investigate, since It build the latest-data-full without problems I think not there is a problem.

But will check this.

I'm using the "docker pull deineagenturug/gvm:latest-data-full" but when I go to check the openvas scan added, the service appears unavailable, does anyone have a link to openvas correct or have you faced this problem?

How did you run the Image? What is your run command? While it is building the image successful I think there is a unknown problem which I can't currently reproduce.

Dexus avatar Mar 20 '22 14:03 Dexus

Thanks for your efforts, @Dexus! Unfortunately I'm unable to start up deineagenturug/gvm:latest-full and deineagenturug/gvm:latest-data-full (versions published today) on top of my existing install. I will continue to use securecompliance/gvm:debian-master-data-full in the meantime.

Hopefully this info helps...

With latest-data-full, it throws repo signature errors: E: The repository 'http://deb.debian.org/debian bullseye-backports InRelease' is not signed. W: GPG error: http://apt.postgresql.org/pub/repos/apt bullseye-pgdg InRelease: At least one invalid signature was encountered.

With latest-full, it throws postgres startup errors: 2022-03-21 15:08:32,173 INFO spawned: 'postgresql' with pid 653 2022-03-21 15:08:32,188 INFO exited: postgresql (exit status 1; not expected) .... 2022-03-21 15:08:52,251 INFO gave up: postgresql entered FATAL state, too many start retries too quickly

Docker-compose:

services:
    gvm:
#        image: securecompliance/gvm:debian-master-data-full
        image: deineagenturug/gvm:latest-data-full
    restart: unless-stopped
    volumes:
      - /gvm_data/database:/opt/database
      - /gvm_data/plugins:/var/lib/openvas/plugins
      - /gvm_data/gvm:/var/lib/gvm
      - /gvm_data/ssh:/etc/ssh
    ports:
      - "443:9392"
      - "5432:5432"
      - "2222:22"
    environment:
      - DB_PASSWORD="passwordhere"
      - USERNAME="admin"
      - PASSWORD="passwordhere"
      - TZ="America/Toronto"
      - SSHD=true
      - HTTPS=false
      - AUTO_SYNC=true
      - RELAYHOST=172.17.0.1
      - SMTPPORT=25

xenago avatar Mar 21 '22 15:03 xenago

Thanks for your efforts, @Dexus! Unfortunately I'm unable to start up deineagenturug/gvm:latest-full and deineagenturug/gvm:latest-data-full (versions published today) on top of my existing install. I will continue to use securecompliance/gvm:debian-master-data-full in the meantime.

Hopefully this info helps...

With latest-full, it throws postgres startup errors: 2022-03-21 15:08:32,173 INFO spawned: 'postgresql' with pid 653 2022-03-21 15:08:32,188 INFO exited: postgresql (exit status 1; not expected) .... 2022-03-21 15:08:52,251 INFO gave up: postgresql entered FATAL state, too many start retries too quickly

hi @xenago i think you have to migrate your DB something like: https://www.kostolansky.sk/posts/upgrading-to-postgresql-12/

currently we have no automatic ways for this - what makes me a bit unhappy but without full time work on it, it will take a while :(

If you use latest-data-full or latest-data together with volumes, you can simply run latest-full or latest as image tag. Because you don't will benefit from the pre-initalized images.

With latest-data-full, it throws repo signature errors: E: The repository 'http://deb.debian.org/debian bullseye-backports InRelease' is not signed. W: GPG error: http://apt.postgresql.org/pub/repos/apt bullseye-pgdg InRelease: At least one invalid signature was encountered.

When you get this message? On my local system I never get this message.

Dexus avatar Mar 22 '22 10:03 Dexus

Ok thanks for the info regarding the DB upgrade, I appreciate it!

When you get this message?

I see the repo errors in the docker logs when running the docker-compose as-included in my previous comment. I'm not sure if it has anything to do with the environment variables or volumes I used or if it has something to do with the image itself.

xenago avatar Mar 22 '22 14:03 xenago

Ok thanks for the info regarding the DB upgrade, I appreciate it!

When you get this message?

I see the repo errors in the docker logs when running the docker-compose as-included in my previous comment. I'm not sure if it has anything to do with the environment variables or volumes I used or if it has something to do with the image itself.

are you behind an proxy? I'm not able here to reproduce this, maybe a newer image will solve this issue.

Dexus avatar Mar 24 '22 07:03 Dexus

For further support, use https://github.com/DeineAgenturUG/greenbone-gvm-openvas-for-docker

Dexus avatar Mar 24 '22 12:03 Dexus

@Dexus Using the image deineagenturug/gvm:latest-full worked for me. Now I have the version 21.4.3. 1) But I cant do Scans without an error... It shows "Task interrupted unexpectedly "and "Scan process Failure" after I run a task. 2) I cant start the container with a preinitialized volume... So I don't have some of the old data in the current version.

devMarc5598 avatar Mar 24 '22 13:03 devMarc5598

@devMarc5598 to point 1: your feed sync is done before you start the scan? to point 2: do you have a DB problem? did you use before the debian image with pg11/12 or the alpine image? if the debian with pg11/12 then you need to upgrade the DB. But I'm no DB Admin so I can only ref to https://www.kostolansky.sk/posts/upgrading-to-postgresql-12/

Edit:

I will try, to provide a Docker Image to Upgrade the DB. But this may take some days.

Dexus avatar Mar 24 '22 13:03 Dexus

@Dexus no, I created a simple task and started it. -> it stoped after a view seconds and shows this errors in the report: image

Before: deineagenturug/gvm:latest-data I was using the Alpine Version securecompliance/gvm:21.4.3-v1 from 7 months ago

devMarc5598 avatar Mar 24 '22 14:03 devMarc5598

@Dexus no, I created a simple task and started it. -> it stoped after a view seconds and shows this errors in the report: image

Before: deineagenturug/gvm:latest-data I was using the Alpine Version securecompliance/gvm:21.4.3-v1 from 7 months ago

what is the status of <url>/feedstatus are there any sync in progress?

Dexus avatar Mar 24 '22 14:03 Dexus

plase exec docker exec -ti <container_id/name> cat /var/log/gvm/* and docker exec -ti <container_id/name> cat /var/log/supoervisor/* this will showing some problems maybe.

Dexus avatar Mar 24 '22 14:03 Dexus

My Main-Problem is: I want to update from my current greenbone alpine version to the newest version because I get this issue -> Report outdated / end-of-life Scan Engine / Environment (local)

But I dont want to use the debian version because of the db downgrade and some other reasons.

So the first small part of the solution is: Using a Image of https://github.com/DeineAgenturUG/ But now I need to get this work with my old alpine data.

devMarc5598 avatar Mar 24 '22 14:03 devMarc5598

My Main-Problem is: I want to update from my current greenbone alpine version to the newest version because I get this issue -> Report outdated / end-of-life Scan Engine / Environment (local)

But I dont want to use the debian version because of the db downgrade and some other reasons.

So the first small part of the solution is: Using a Image of https://github.com/DeineAgenturUG/ But now I need to get this work with my old alpine data.

This should not be a problem since the alpine version was with PG 13.3 so it should work out of the box with the current debian based image with PG 13.6.

Dexus avatar Mar 24 '22 14:03 Dexus

My Main-Problem is: I want to update from my current greenbone alpine version to the newest version because I get this issue -> Report outdated / end-of-life Scan Engine / Environment (local)

But I dont want to use the debian version because of the db downgrade and some other reasons.

So the first small part of the solution is: Using a Image of https://github.com/DeineAgenturUG/ But now I need to get this work with my old alpine data.

@devMarc5598 I have found 2 bugs, that may have impact of the things you mentioned. I have have a run open to publish a fix for it - but it only releases an amd64 image for now.

Dexus avatar Mar 24 '22 15:03 Dexus

But I cant do Scans without an error... It shows "Task interrupted unexpectedly "and "Scan process Failure" after I run a task.

@devMarc5598 i mean this point of failure

Dexus avatar Mar 24 '22 15:03 Dexus

I have a doubt, I just uploaded two dockers, one with openvas and the other with gvm, I now want to install an openvas on the clients so that I can manage everything in a single panel that would be GSA, is this possible? Point to the public address of the GVM?

Kelvinloucosta avatar Mar 24 '22 17:03 Kelvinloucosta

@Kelvinloucosta

I have a doubt, I just uploaded two dockers, one with openvas and the other with gvm, I now want to install an openvas on the clients so that I can manage everything in a single panel that would be GSA, is this possible? Point to the public address of the GVM?

you can use the gvm image as "MASTER" and the openvas-scanner image as "SENSOR" in your DMZ. Use the Images from https://hub.docker.com/u/deineagenturug

Once you registered the "SENSOR" in "MASTER" you are able to select the Scanner for your new Scans.

Dexus avatar Mar 25 '22 09:03 Dexus

thanks @Dexus , but I would like to install openvas-scanner on another network by public ip. it's possible?

Can anyone help me explain this design to me, sorry about that but I would like to know, why use redis? what is the function of postgresql? what does ospd do?

Kelvinloucosta avatar Mar 25 '22 12:03 Kelvinloucosta

thanks @Dexus , but I would like to install openvas-scanner on another network by public ip. it's possible?

Yes this would work, but I would recommend a connection via VPN. if you make the SSH port from MASTER listen public, you can connect the openvas-scanner to it.

Can anyone help me explain this design to me, sorry about that but I would like to know, why use redis? what is the function of postgresql? what does ospd do?

If you need more details for this, please look around at Greenbone

Dexus avatar Mar 25 '22 12:03 Dexus

thanks again @Dexus

Kelvinloucosta avatar Mar 25 '22 12:03 Kelvinloucosta

@Kelvinloucosta you're welcome. But you should know, that the performance via WAN is not the same as via LAN. So you should be know that this may have a quit big different.

Dexus avatar Mar 25 '22 12:03 Dexus

yeah you right

Kelvinloucosta avatar Mar 25 '22 12:03 Kelvinloucosta

I have the same problem. I have found this link https://community.greenbone.net/t/report-outdated-end-of-life-scan-engine-environment-local/5430/2

"You can edit the following script to turn off the message:

/var/lib/openvas/plugins/gcf/gb_outdated_scan_engine.nasl

Insert the versions for your scanner and libs, mine is 5.1.3 and 9.0.3:

#expected_gce_ver = “6.0.7”;
expected_gce_ver = “5.1.3”;
#expected_libs_ver1 = “10.0.2”;
expected_libs_ver1 = “9.0.3”;
#expected_libs_ver2 = “11.0.1”;
expected_libs_ver2 = “9.0.3”;

Now it won’t report the scanner.

An alternative is to set an override for the script."

This workarround work but the file change after stop and restart container. Override doesn't work for me. I don't have find how to update scan engine :(

gdanldo avatar Mar 31 '22 16:03 gdanldo

@gdanldo do read the full thread https://community.greenbone.net/t/report-outdated-end-of-life-scan-engine-environment-local/5430/21

Dexus avatar Mar 31 '22 17:03 Dexus

Hi,

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

Sorry, nice try, but there are too many problems with this container - the most recognizeable one is, that all reports are empty - only logging information is shown in the reports. There are many small problems with their container additionaly. For me, it is not useable as a drop in replacement at all :-(

4920441 avatar Apr 01 '22 06:04 4920441