Migrating from sameersbn/gitlab to gitlab/gitlab-ce

[almereyda]

In course of #2103 it also appears good to consider migrations to the official container image at https://hub.docker.com/r/gitlab/gitlab-ce.

This concerns migrating from this repository's gitlab.yml template to the GITLAB_OMNIBUS_KEY containing gitlab.rb configuration.

A guide to translating the settings can be found in doc/settings/gitlab.yml.md · master · GitLab.org / omnibus-gitlab · GitLab.

[almereyda]

My observations include that one will want to

• map the host_* files from git/ssh into /etc/gitlab/
• copy the secrets from the environment to /etc/gitlab/gitlab-secrets.json - which you could copy out of a local omnibus container and fill with your variables
• setting nginx not to listen for https
• adding a trusted reverse proxy
• disabling most of the included services in the OMNIBUS_CONFIG

registry['enable'] = false
grafana['enable'] = false
alertmanager['enable'] = false
prometheus['enable'] = false
node_exporter['enable'] = false
redis_exporter['enable'] = false
postgres_exporter['enable'] = false
gitlab_exporter['enable'] = false
prometheus_monitoring['enable'] = false
postgresql['enable'] = false
redis['enable'] = false

Then we need some directories, and have them the right permissions.

mkdir config
mkdir data
mkdir data/gitlab-ci
mkdir data/git-data
mkdir data/gitlab-rails

and certain permissions

chown -R 998:0 git
chown -R 998:0 data
chown -R 0:0 git/ssh

with these new host volume mappings

      - ./config:/etc/gitlab
      - ./git/ssh/ssh_host_ecdsa_key:/etc/gitlab/ssh_host_ecdsa_key
      - ./git/ssh/ssh_host_ecdsa_key.pub:/etc/gitlab/ssh_host_ecdsa
_key.pub
      - ./git/ssh/ssh_host_ed25519_key:/etc/gitlab/ssh_host_ed25519
_key
      - ./git/ssh/ssh_host_ed25519_key.pub:/etc/gitlab/ssh_host_ed2
5519_key.pub
      - ./git/ssh/ssh_host_rsa_key:/etc/gitlab/ssh_host_rsa_key
      - ./git/ssh/ssh_host_rsa_key.pub:/etc/gitlab/ssh_host_rsa_key.pub
      - ./logs:/var/log/gitlab
      - ./data:/var/opt/gitlab
      - ./data/gitlab-ci:/var/opt/gitlab/gitlab-ci
      - ./data/git-data:/var/opt/gitlab/git-data
      - ./data/gitlab-rails:/var/opt/gitlab/gitlab-rails
      - ./git/backups:/var/opt/gitlab/backups
      - ./git/builds:/var/opt/gitlab/gitlab-ci/builds
      - ./git/repositories:/var/opt/gitlab/git-data/repositories
      - ./git/shared:/var/opt/gitlab/gitlab-rails/shared
      - ./git/uploads:/var/opt/gitlab/gitlab-rails/uploads

I'm not sure if I forgot something, but these hints might help some people already.

[justacec]

Is this what I should be doing. I just tried to update my gitlab and have now been getting the CRIT errors on the supervisor. Will there be a full guide with step-by-step instructions on how to do this? I really do not know docker very well. (or at all)

[justacec]

Ok, I have, I believe, successfully transitioned to the official GitLab-CE images. These are the steps that I executed to make things work. Some of them are detailed, some are more general steps. I did not attempt to make parity with the sameersbn directory mappings. My server is for my personal use and therefore is not in a corporate/enterprise environment.

General idea of the transition was to generate a backup of the existing install, install the same version as the existing from the official packages, restore from the backup, and then upgrade that to the most current.

Ok, with all of that said, here is what I did:

Backup existing install

1: Log into the running docker container

docker exec -i -t <CONTAINER ID> bash

2: Execute the backup

bin/rake gitlab:backup:create

3: "exit" from the shell

4: Stop the currently running container

docker container stop <CONTAINER ID>

5: Make a backup of container settings

docker container inspect <CONTAINER ID> > ~/old_gitlab_container_settings.txt

Install old version of Official Image

1: Download the official GitLab-CE version for your currently running gitlab from Sammerson (in my case it was 12.4.2)

docker pull gitlab/gitlab-ce:12.4.2-ce.0

2: Rename the old external gitlab directory to gitlab-old

3: Create a new empty gitlab directory in its place

4: Create subdirectories for "config", "logs", "data" under the new empty gitlab directory

5: Start the docker container pointing to this new directory

docker run --detach \
  --hostname XXX.XXX.XXX.XXX \
  --publish 4043:443 --publish 800:80 --publish 202:22 \
  --name gitlab \
  --restart always \
  --volume /volume1/docker/gitlab/config:/etc/gitlab \
  --volume /volume1/docker/gitlab/logs:/var/log/gitlab \
  --volume /volume1/docker/gitlab/data:/var/opt/gitlab \
  gitlab/gitlab-ce:12.4.2-ce.0

6: Log into the running container

docker exec -i -t <CONTAINER ID> bash

7: Update the secret key base and the db key base from the origninal environment to the /etc/gitlab/gitlab-secrets.json file (Not sure if this is necessary?)

8: Follow the instructions on page: (https://docs.gitlab.com/ee/raketasks/backup_restore.html)[https://docs.gitlab.com/ee/raketasks/backup_restore.html]

Upgrade to the most current Gitlab-ce

1: Stop the existing docker container

docker container stop <CONTAINER ID>

2: Grab the lastest GitLab-CE

docker pull gitlab/gitlab-ce:latest

3: Rename the old temporary container to avoid a name collision

docker container rename <CONTAINER ID> gitlab-old

4: Start a new container with the new image using the same parameters as the temporary

docker run --detach \
  --hostname XXX.XXX.XXX.XXX \
  --publish 4043:443 --publish 800:80 --publish 202:22 \
  --name gitlab \
  --restart always \
  --volume /volume1/docker/gitlab/config:/etc/gitlab \
  --volume /volume1/docker/gitlab/logs:/var/log/gitlab \
  --volume /volume1/docker/gitlab/data:/var/opt/gitlab \
  gitlab/gitlab-ce:latest

5: Configure the appropriate options in the gitlab.rb file (Do this while logged into the running container)

6: Update all of the subordinate configurations though the reconfigure command

gitlab-ctl reconfigure

7: Create new backup (Might have to do this as root through an su)

docker exec -t <CONTAINER ID> gitlab-backup create

8: Make any changes to the container setting, such as exposed or mapped ports based on your changes in the gitlab.rb file

9: Start and then restart the container

docker container stop <CONTAINER ID>
docker container start <CONTAINER ID>

[cwildfoerster]

@justacec Thank you, will give it a try in the next days and post my results.

[jvanlunenburg]

@justacec How is the performance? I understand the official gitlab-ce image is a monolithic container. Does it support the same set of environment variables? In particular I care about the relative root path.

[sameersbn]

I hope you we're able to successfully migrate to gitlab official (no hard feelings). We're working on ramping up out efforts on maintaining the image and would really appreciate any help we could get. Please ping me on kubernetes slack @sameersbn if you are interested.

[mpbzh]

I'm having issues migrating from sameersbn/gitlab to gitlab/gitlab-ce.

Here's what I've done:

• Created a backup by following the documentation in this repository.
  • I used docker-compose run --rm gitlab app:rake gitlab:backup:create SKIP=registry due to a Gitlab issue with restoring.
• I followed the official guide to migrate to omnibus
  • Create a backup, create new gitlab instance, restore backup
  • I copied the secrets from /home/git/gitlab/config/secrets.yml and /home/git/gitlab-shell/.gitlab_shell_secret to /etc/gitlab/gitlab-secrets.json. I've also tried including workhorse and pages secrets.
  • I ran gitlab-ctl reconfigure and restarted the container
• I'm getting the new installation up and running, but every time a access Admin > Shared Runners or any repository's Settings > CI/CD i get a 500 error. According to the stack trace it appears that there is an issue decrypting the variables.
  • The steps described here help with getting to the shared runners, but I still get the same error on repository CI settings.

Even though I already spent far too many hours researching this I cannot find out which secret is used to decrypt CI variables. I still have the old (sameersbn) instance running.

Does anyone have an idea what I'm missing? What additional information can I copy to get this working?