Confused by "source" and "destination" folder and mappings

[henkisdabro]

Don't know how best to describe this, but I'm confused with the namings/mapping of folders.

• why is there a /source mentioned in every docker compose example - is that meant to map somewhere on the machine, or is it a template example folder?
• what do I need to add to the config to get my docker volumes backed up, not enough documentation to explain the various scnearios
• what if I want to back up the parent folder under which the nautical folder+config is also stored, as I need to map that parent folder to a folder in the /app/source folder?

Overall, I wish the documentation was more (crystal) clear?

[mrjk]

Hey,

Q1: The general idea is nautical-backup will scan the /source/ dir, and if a folder match the name of an existing container, then it start to backup the directory. To make it work, you have to mount the volume(s) of the container you want to backup inside the nautical-backup container (otherwise nautical can't access files).

Q2: Basic howto: Create /source/CONTAINER_NAME to enable the backup of the container called CONTAINER_NAME (output of docker ps). Everything in this folder will be backuped. If a directory does not match to a container, then nautilcal-backup ignore the folder. Obviously, everything is overridable by config, and source means 'what to backup' and destination means where I should copy backuped files.

Q3: Yes, I had the same issue, and the answer depends on how you want to do things. Because nautical-backup is quite agnostic, you can do almost whatever you want. So unless you're more precise on what you want to do, I can illustrate what I did on my side, an application (paperless here). Obviously, I'm not running a single app, so I designed my implementation to be able to work on any other apps I run. I let you check the end of this post.

Last point about the documentation, what would you wish to have ? More guides ? Some complete backup scenarios? If you want more doc, please tell what you need, and we can help, otherwise it's quite hard for us to guess what is missing.

Complex backup solution

Folder structure of the application is in /srv/containers/prod/paperless-ngx:

# Root directory of all interesting data
root@host:/srv/containers/prod/paperless-ngx# LANG=C tree -L 2
.
|-- cache  # Webserver container
|   `-- dump.rdb
|-- data # Webserver container
|   |-- celerybeat-schedule.db
|   |-- index
|   |-- log
|   `-- migration_lock
|-- db_data # DB container, mounted on /var/lib/mysql
|   |-- aria_log.00000001
|   |-- aria_log_control
|   |-- ddl_recovery.log
|   |-- ib_buffer_pool
|   |-- ib_logfile0
|   |-- ibdata1
|   |-- ibtmp1
|   |-- mariadb_upgrade_info
|   |-- multi-master.info
|   |-- mysql
|   |-- paperless@002dngx
|   |-- performance_schema
|   |-- sys
|   |-- tc.log
|   |-- undo001
|   |-- undo002
|   `-- undo003
|-- docker-compose.yml
|-- share # Webserver container, 
|   |-- consume
|   |-- export
|   |-- media
|   `-- scripts
`-- state
    `-- nautical-backup

So there are file assets (on webserver) and database to backup. All things that need to be persistant are mapped to those directory, via bind mount. I manage everything with docker-compose.yml in this case. There is no backup yet. How to make this to work? Then I'll add an nautical-backup instance, just for this app, I feel simpler to manage this way, but it's an opinion. Let's create the nautical-backup service:

  nautical-backup:
    environment:
      CRON_SCHEDULE: 0 4 * * *
      CRON_SCHEDULE_ENABLED: 'true'
      DEST_LOCATION: /app/destination/
      SOURCE_LOCATION: /app/source/
      LOG_LEVEL: INFO
      TZ: America/Toronto
    image: minituff/nautical-backup:latest
    restart: unless-stopped
    volumes:
    - /srv/containers/prod/paperless-ngx:/source/   # IMPORTANT BIT HERE, we mount the parent of any different volumes, to make management easier.
    - /srv/backups/prod/paperless-ngx:/destination/ # Destination is on another disk
    - /var/run/docker.sock:/var/run/docker.sock

Ok, so each night at 4 in the morning, nautical will start and lookup for backups. How it does that? It will scan containers labels:

  db:
    image: docker.io/library/mariadb:11
    labels:
      nautical-backup.enable: 'true'
      nautical-backup.group: archives_paperless
      nautical-backup.override-source-dir: db_data # HERE, we want to backup db_data instead of a folder of the container name
      nautical-backup.stop-before-backup: 'true'
  webserver:
    depends_on:
      broker:
        condition: service_started
      db:
        condition: service_started
    labels:
      nautical-backup.enable: 'true'
      nautical-backup.group: archives_paperless
      nautical-backup.override-source-dir: media # HERE, we want to backup media instead of a filder of the container name
      nautical-backup.additional-folders: data,share # HERE, To add more folder to backup, just to illustrate, but you may not want to backup cache folder
      nautical-backup.stop-before-backup: 'true'

Since data are backuped in a the destination directory, I use then borgmatic (borg) in another container to keep track and manage differents backups.

TBH, I did not choose the simplest implementation, but maybe something simpler may be a better fit for you, unfortunately I had no simpler use cases to demonstrate, maybe somebody else could show a simpler setup ?

[mrjk]

Also, there is this thread https://github.com/Minituff/nautical-backup/issues/326#issuecomment-2366885062 . If you enable logging, things should become crystal clear ;)

[Minituff]

Hi, I just waned to check if this is still an active problem? Otherwise I'll close this one out :)