Create Mirror

[slackjeff]

Hi! I want create a repository mirror in Brazil. We need. But I saw that the size is around 1TB. I need ways (how to) how to reduce the size of the repository as much as possible. Unfortunately VPS in brazil is expensive

[Vaelatern]

If you wanted to make a second class mirror without the debug packages, for example, the size would be smaller.

[slackjeff]

rsync -avHP --delete
--exclude 'debug'
--exclude 'aarch64'
--exclude '.armv7l.xbps'
--exclude 'armv6l.xbps'
--exclude '.i686.xbps'
rsync://mirror.clarkson.edu/voidlinux/current/
/var/www/html/voidlinux/current/

Would it be enough? Another thing directory is correct for rsync: /var/www/html/voidlinux/current/ Sorry. I couldn't find any article on the internet on how to mirror a mirror.

[karibuTW]

Hi Slackjeff, you have been able to do it in that manner? I'm thinking to mirror the project as well here in Singapore. I guess I will keep all except debug. Not sure the guidelines.

[JonathanxD]

@slackjeff As I commented on your YouTube video about this, if you're going with anything but glibc x86_64 and aarch64, you can do:

rsync --list-only --stats --recursive \
  --exclude "debug" \
  --exclude "musl" \
  --exclude "aarch64/debug" \
  --exclude "*.aarch64-musl.xbps" \
  --exclude "*.aarch64-musl.xbps.sig" \
  --exclude "*.armv7l.xbps" \
  --exclude "*.armv7l.xbps.sig" \
  --exclude "*.armv6l.xbps" \
  --exclude "*.armv6l.xbps.sig" \
  --exclude "*.i686.xbps" \
  --exclude "*.i686.xbps.sig" rsync://mirrors.servercentral.com/voidlinux/current/

This will result in a mirror 101GiB big.

I would prefer musl being excluded from the mirror than aarch64 (but I'm biased, since I have aarch64 machines), you can also go the other way around and keep musl but get rid of aarch64:

rsync --list-only --stats --recursive \
  --exclude "debug" \
  --exclude "musl/debug" \
  --exclude "aarch64" \
  --exclude "*.aarch64-musl.xbps" \
  --exclude "*.aarch64-musl.xbps.sig" \
  --exclude "*.armv7l.xbps" \
  --exclude "*.armv7l.xbps.sig" \
  --exclude "*.armv7l-musl.xbps" \
  --exclude "*.armv7l-musl.xbps.sig" \
  --exclude "*.armv6l.xbps" \
  --exclude "*.armv6l.xbps.sig" \
  --exclude "*.armv6l-musl.xbps" \
  --exclude "*.armv6l-musl.xbps.sig" \
  --exclude "*.i686.xbps" \
  --exclude "*.i686.xbps.sig" rsync://mirrors.servercentral.com/voidlinux/current/

And this will result in 195GiB.

But if you're comfortable with a 222GiB mirror, you can do:

rsync --list-only --stats --recursive \
  --exclude "debug" \
  --exclude "musl/debug" \
  --exclude "aarch64/debug" \
  --exclude "*.armv7l.xbps" \
  --exclude "*.armv7l.xbps.sig" \
  --exclude "*.armv7l-musl.xbps" \
  --exclude "*.armv7l-musl.xbps.sig" \
  --exclude "*.armv6l.xbps" \
  --exclude "*.armv6l.xbps.sig" \
  --exclude "*.armv6l-musl.xbps" \
  --exclude "*.armv6l-musl.xbps.sig" \
  --exclude "*.i686.xbps" \
  --exclude "*.i686.xbps.sig" rsync://mirrors.servercentral.com/voidlinux/current/

This way you keep both aarch64 and musl, only getting rid of armv7, armv6 and i686, but the mirror will be 270GiB big.

Now if this will really work, you need to try and see if xbps is able to handle the mirror this way, which I think it can.

Edit:

If you went with anything but x86_64, you can get to 62GiB:

rsync --list-only --stats --recursive \
  --exclude "debug" \
  --exclude "musl" \
  --exclude "aarch64" \
  --exclude "*.armv7l.xbps" \
  --exclude "*.armv7l.xbps.sig" \
  --exclude "*.armv6l.xbps" \
  --exclude "*.armv6l.xbps.sig" \
  --exclude "*.i686.xbps" \
  --exclude "*.i686.xbps.sig" rsync://mirrors.servercentral.com/voidlinux/current/