[Feature] Add repositories for apt/rpm?

[stone-w4tch3r]

IMHO package repository is better than binary.

I would be glad to implement this feature, but do you think it is needed?

There already exists linux build script, it can be wrapped with packagecloud publish action, for example.

[love-linger]

Thank you very much if you can help. I'm not very familiar with this aspect.

[aikawayataro]

After some research I found https://build.opensuse.org/ and it looks like they even provide repositories for Debian, Fedora, OpenSUSE and CentOS which I see as very complete. This might be worth a look.

[hannesbraun]

FWIW, I packaged SourceGit for Alpine Linux and it is currently available in the testing repository: https://pkgs.alpinelinux.org/package/edge/testing/x86_64/sourcegit

[chidniu]

Considering you are already bulding an .rpm I think it should be relatively simple to provide a Fedora repo via COPR.

[AdrienMatricon]

• I think the most natural way to publish the deb packages would be to create an Ubuntu PPA (Personal Package Archive), which is basically an apt repository that a user can declare (through sudo add-apt-repository XXX) so that the packages it contains are seen by apt (you run apt update and then you can install them through sudo apt install XXX and get the updates with sudo apt upgrade)
• PPAs seem to basically be a directory containing the right files and that can be accessed through http, so there are plenty of ways to create one (to get an idea, you can look at all the replies to this post)
• There is a platform called Launchpad which seems to be the most mainstream way to host PPAs, more info here : https://help.launchpad.net/Packaging/PPA

[AdrienMatricon]

Alternately, if publishing sourcegit as a package for various linux distributions seem like a chore, it might be worth looking it into making it a flatpak package hosted on Flathub (those are cross-distribution)

[aikawayataro]

PPAs seem to basically be a directory containing the right files and that can be accessed through http

You mean deb repository? I believe this is not an option. I see it as a burden to host packages and sign them.

to create an Ubuntu PPA (Personal Package Archive)

PPA's provided repository will only work for deb based distros. I think OBS is better solution (works with more package managers and repositories) and it solves the same issue.

Alternately, if publishing sourcegit as a package for various linux distributions seem like a chore, it might be worth looking it into making it a flatpak package hosted on Flathub (those are cross-distribution)

See #462 To package SourceGit in Flatpak requires major code changes.

I also want to say that software packaging is not a developer's job, in practice software is packaged by distro/repository maintainers. I believe it is impossible for one person to package software for a wide range of distros, their flavors, their release channels, and so on.

[Masgalor]

I am not quiet sure if someone is working on this right now. So I am just here to say that I am already packaging several projects on OBS for all kinds of distributions. Vaultwarden for example. OBS is the best multi-distro build/publishing platform I know and I can definitely help with that.

The downside is, they are very strict about reproducible builds. This means you can not use external repositories like microsofts dotnet repo, you need to package it first on OBS in order to use it. You are also forced to build your binaries without internet connection, you need to package your dependencies first on OBS or bring them in an vendor.tar.xz archive. Those restrictions make especially dotnet applications very hard to build from source.

A good and effortless way forward could be to not only build installable packages with the github workflow but also source packages (SRPM / src.deb) and publish them here on github. The content would be almost the same because the binaries are compiled first and only put in the right place by the packaging recipe, but source packages can be easily picked up by OBS and can be repacked for every distribution you want.

[aikawayataro]

you need to package your dependencies first on OBS or bring them in an vendor.tar.xz archive. Those restrictions make especially dotnet applications very hard to build from source.

I believe vendoring nuget deps is a simple and preferable way. Bringing them all to OBS? Hell no. There's more than 30 of them. You can pass local source to dotnet command, I also explored some solutions Flatpak use, so It is possible to build SourceGit offline.

A good and effortless way forward could be to not only build installable packages with the github workflow but also source packages (SRPM / src.deb) and publish them here on github. The content would be almost the same because the binaries are compiled first and only put in the right place by the packaging recipe, but source packages can be easily picked up by OBS and can be repacked for every distribution you want.

That's interesting. Will source packages include vendored nuget deps?

How harsh is reproducibility requirement is? Should it be reproducible anywhere or only in OBS environment? I will explore reproducible build possibilities on dotnet.

I can handle source packages, but I believe they don't belong to this repo, such packaging should be separated. I look forward to your reply, thank you.

[Masgalor]

You can pass local source to dotnet command, I also explored some solutions Flatpak use, so It is possible to build SourceGit offline.

I do not doubt that this is possible, but it may be more work then you expect right now. The first thing you will notice is, that there is no dotnet command, because it is in no official repository and you can not access microsofts repository. I am packaging several Rust projects with Cargo which is similarly to nuget expecting internet access all the time. Passing local sources to a build-system that was developed with permanent internet connectivity in mind is not always as easy as it sounds.

How harsh is reproducibility requirement is?

As long as we do not want to have our packages in their official repositories it is not important if the result is reproducible. This is more about having a lot of restrictions in their build-system like no internet-connection, no external repositories, no modification of the root env.

For source packages in general. They contain the buildrecipe and every resource that is declared in the recipe. At the moment the sourcegit binary is not built with the packaging recipe it is built in an entirely different CI flow and the packaging recipe only moves the final binary in its place.

You do not need to do anything to generate the source packages they are automatically generated by rpmbuild and additionally to dpkg-deb you can run dpkg-source. So all we need to do is publish those artifacts.

[aikawayataro]

Passing local sources to a build-system that was developed with permanent internet connectivity in mind is not always as easy as it sounds.

Well you have a point here. From my Flatpak experience dotnet offline build with local source works just fine (of course you'll have to specify every nupkg resource in manifest but there's script to automate this). You're right about troubles getting dotnet sdk in the first place though.

So as I understood you want source packages with binary prebuild? What should be done to emit proper source packages?

[Masgalor]

My advice is to use the existing packaging workflow to generate source packages in parallel to those installable packages we already have. Yes they will contain prebuilt binaries, but there is no maintenance at all.

+dpkg-source --root-owner-group --build resources/deb "sourcegit_$VERSION-1_$arch.deb"
 dpkg-deb --root-owner-group --build resources/deb "sourcegit_$VERSION-1_$arch.deb"

-rpmbuild -bb --target="$target" resources/rpm/SPECS/build.spec --define "_topdir $(pwd)/resources/rpm" --define "_version $VERSION"
+rpmbuild -ba --target="$target" resources/rpm/SPECS/build.spec --define "_topdir $(pwd)/resources/rpm" --define "_version $VERSION"

Change the build instructions as above, resolve any errors that may arise , and you are done.

If you want proper source packages, you need to move the complete build process inside your packaging recipe, find a way to provide all dependencies for your build process (vendored/offline), make sure it works without internet connection and then run the same commands as above.

[stone-w4tch3r]

I apologize for not doing anything for many months, although I promised. Shame on me.

The trouble is that packagecloud died during this time. You cannot register a new account. Buildkite bought them, I tried to configure it, but it works disgustingly, due to bugs I can’t use it at all. I wrote to support, if the situation changes for the better, I’ll try to use it.

The only free alternative I found is cloudesmith, but they have too few bandwidth on the free tier (1GB, ~50 downloads). You need to either pay or self-host something.

Things like this(

P.S. OBS is an interesting idea, but too complicated for me. P.P.S. we also can try Launchpad + COPR, but it would be much more difficult, and there may be issues on other distros than fedora/ubuntu

[stone-w4tch3r]

Actually I managed to create a working pipeline! Waiting for a PR to be merged. Also I created a PR to publish AppImage to AppImageHub

[stone-w4tch3r]

sourcegit is now on appimagehub! https://appimage.github.io/SourceGit/

I think all new repositories should be listed in readme. But a release with a new publish pipeline must run first.

When next release is planned?

[love-linger]

every Monday

[aikawayataro]

sourcegit is now on appimagehub!

It has been there since August :thinking:

[stone-w4tch3r]

@aikawayataro

wtf 👀 looks like added it again...