docker-mc-backup icon indicating copy to clipboard operation
docker-mc-backup copied to clipboard

Add Bandwidth limiters to backups

Open LichLord91 opened this issue 1 year ago • 3 comments

I noticed in my Oracle Cloud VPS setup when the backup is ran it sometimes blows up my network latency on the server for a short period of time it'll go from 50ms to up to 300ms for a bit and even disconnect some far away bedrock players because of this spike.

I'm using rclone/restic and found some command to help limit the backup network bandwidth

Restic: https://restic.readthedocs.io/en/latest/manual_rest.html#usage-help

--limit-download rate        limits downloads to a maximum rate in KiB/s. (default: unlimited)
--limit-upload rate          limits uploads to a maximum rate in KiB/s. (default: unlimited)

Rclone: https://rclone.org/docs/#bwlimit-bandwidth-spec

--bwlimit=BANDWIDTH_SPEC

This option controls the bandwidth limit. For example

--bwlimit 10M

would mean limit the upload and download bandwidth to 10 MiB/s. NB this is bytes per second not bits per second. To use a single limit, specify the desired bandwidth in KiB/s, or use a suffix B|K|M|G|T|P. The default is 0 which means to not limit bandwidth.

I'm playing with the backup-loop.sh script right now to get this working so far changed the following bits of code

(this would not be the default values, i'd change them to the unlimited default values and then use a environment variable in the docker compose/dockerfile to set it.)

: "${RESTIC_BW_DL:=20480}"
: "${RESTIC_BW_UP:=5120}"
: "${RCLONE_BW_LIMIT:=10m:20m}" 
  if ! command rclone copy "${outFile}" "${RCLONE_REMOTE}:${RCLONE_DEST_DIR}" --bwlimit "${RCLONE_BW_LIMIT}" ; then
  backup() {
    log INFO "Backing up content in ${SRC_DIR} as host ${RESTIC_HOSTNAME}"
    args=(
      --host "${RESTIC_HOSTNAME}"
      --limit-download "${RESTIC_BW_DL}" #change made
      --limit-upload "${RESTIC_BW_UP}" #change made
    )
    if isDebug || isTrue "$RESTIC_VERBOSE"; then
      args+=(-vv)
    fi
    (cd "$SRC_DIR" &&
          command restic backup "${args[@]}" "${restic_tags_arguments[@]}" "${excludes[@]}" "${includes_patterns[@]}" | log INFO
    )
  }

Just starting to test this but figured I'd put this in as and Idea and may be other backup methods that have this argument as well that may have a similar feature

LichLord91 avatar Nov 10 '24 00:11 LichLord91