gitit icon indicating copy to clipboard operation
gitit copied to clipboard

Created a workflow for building binaries on release

Open JustusAdam opened this issue 5 years ago • 11 comments

This PR adds a new workflow, which build the project and then packages the compiled binaries and other asset files into standalone archives that users can download and use immediately without the need to install the many libraries gitit depends on and works without having GHC installed.

The workflow only executes when a release is triggered and it creates and uploads tarballs for Linux (latest Ubuntu version available on github actions) and macOS and both with and without plugins respectively. For an example you can visit the releases section of my fork.

This of course does not cater to all users of gitit, but I think it will make it significantly easier for new users to try the software. In particular because gitit depends on pandoc, the build times for a first time build are insane. I think it took over 30 minutes on my modern machine

I've described the way this PR was created in more detail including my handling of data-dir style assets in this blogpost.

JustusAdam avatar Aug 27 '20 20:08 JustusAdam

I think this is a great idea!

Suggestions/questions:

  • it would be good to have the version in the tarball's filename

  • do we really need separate versions without plugins? Why not just provide a full-featured binary, for simplicity?

  • the data file handling issue is ugly; we should really just fix filestore and gitit so that they bake the data files into the binary (as pandoc does with the embed_data_files flag. This isn't too hard to do using the file-embed package (see pandoc for an example). In the case of filestore, we could avoid file-embed and just embed the shell script as a string literal.

jgm avatar Aug 27 '20 22:08 jgm

Thank you, glad you like it.

Here are some comments/answers:

  • That would be good. The easiest way is probably pulling the tag name from the release, but I also could make something that gets it from the cabal file. Which one would you prefer?
  • We could drop the noplugins version, I only included it because those binaries are quite a bit smaller. Though I am not sure anyone cares about that.
  • It is ugly and what you suggest should work fine I think. I just want to remark that we have to make sure we change the documentation for the overridable templates so the user knows where to get them, i.e. probably directly from this repo.

JustusAdam avatar Aug 27 '20 22:08 JustusAdam

The easiest way is probably pulling the tag name from the release, but I also could make something that gets it from the cabal file. Which one would you prefer?

Maybe getting it from the cabal file is more robust, since then this could even be used for non-tagged commits? (Pandoc gets it from the cabal file using sed, if I recall.)

We could drop the noplugins version, I only included it because those binaries are quite a bit smaller. Though I am not sure anyone cares about that.

Let's drop them for now; if people request it, we could consider changing.

It is ugly and what you suggest should work fine I think. I just want to remark that we have to make sure we change the documentation for the overridable templates so the user knows where to get them, i.e. probably directly from this repo.

My suggestion will require changes in both filestore and gitit (including documentation).

jgm avatar Aug 27 '20 22:08 jgm

Actually I changed my mind. Maybe it makes most sense to take the version from the release tag; if the commit is not tagged, we can fall back on the commit hash. That gives a nice visual distinction between actual releases and e.g. nightlies.

jgm avatar Aug 27 '20 22:08 jgm

I have been working on embedding default files into the binary and an issues cropped up that I am not sure how to solve:

When exporting to ODT, the images etc are embedded as absolute URL's. When we embed these in the binary, they no longer have a path on the system. The question is, how should we link them with the export? Should we dump them somewhere on the system?

JustusAdam avatar Sep 15 '20 12:09 JustusAdam

One approach would be to add these to the MediaBag before calling the writer in the export. The MediaBag in pandoc is a container for named resources which will be available to fetchItem in the writer. See insertMedia in Text.Pandoc.Class.

jgm avatar Sep 15 '20 20:09 jgm

@JustusAdam do you want to create a separate issue concerning ODT export and images? (The request would be to embed the images when exporting.) I don't think this is a blocking issue for the release binary workflow.

jgm avatar Dec 21 '20 17:12 jgm

I forgot what the main action item here is: add an embed_data_files flag to filestore and gitit, so we can bake everything into the binary.

jgm avatar Dec 21 '20 17:12 jgm

I see that I've already removed the data files from filestore.0.6.5.

jgm avatar Dec 21 '20 17:12 jgm

I'll see if I can add the needed flag to gitit.

jgm avatar Dec 21 '20 17:12 jgm

Looking into it some more, one complication for embedding the data files is that the string template functions we're using require a directory of templates (falling back to data).

jgm avatar Dec 21 '20 18:12 jgm