AppImage
Consider to bundle appstreamcli in the appimagetool AppImage
It works but only when I remove metadata xml file. Otherwise it fails as below:
Found appimagetool: /tmp/.mount_linuxdCqqmwp/usr/bin/appimagetool Running command: /tmp/.mount_linuxdCqqmwp/usr/bin/appimagetool "AppDir" "-g"
appimagetool, continuous build (commit fef038a), build 2093 built on 2019-07-07 12:07:34 UTC
WARNING: appstreamcli command is missing, please install it if you want to use AppStream metadata
Using architecture x86_64
/dev/shm/rclone-browser-1.6.0-d9e3ebe.AppImage/AppDir should be packaged as Rclone_Browser-1.6.0-d9e3ebe-x86_64.AppImage
Deleting pre-existing .DirIcon
Creating .DirIcon symlink based on information from desktop file
AppStream upstream metadata found in usr/share/metainfo/rclone-browser.appdata.xml
Trying to validate AppStream information with the appstream-util tool
In case of issues, please refer to https://github.com/hughsie/appstream-glib
/dev/shm/rclone-browser-1.6.0-d9e3ebe.AppImage/AppDir/usr/share/metainfo/rclone-browser.appdata.xml: FAILED:
• url-not-found : <screenshot> failed to connect: SSL handshake failed [https://github.com/kapitainsky/RcloneBrowser/wiki/images/IMGdefault.png]
• url-not-found : <screenshot> failed to connect: SSL handshake failed [https://github.com/kapitainsky/RcloneBrowser/wiki/images/IMGtransfer.png]
Validation of files failed
Failed to validate AppStream information with appstream-util
rename: Rclone_Browser*: rename to rclone-browser* failed: No such file or directory
cp: cannot stat ‘*AppImage’: No such file or directory
the same metadata xml file works perfectly on Ubuntu 16.04.
Both urls it complains about are valid.
The issue is in reference to linuxdeploy/linuxdeploy-plugin-appimage#8
As the message says,
In case of issues, please refer to https://github.com/hughsie/appstream-glib
It is not so simple:) what about to bundle working appstreamcli within your tool?
Isn’t all AppImage idea about providing working bundle? Not just telling me that your version of OS cli whatever tool is not what we like. Tough.
Not just telling me that your version of OS cli whatever tool is not what we like. Tough.
Well, then you have a really old version, and even if your file is valid with the bundled appstream-cli version, it might break on newer systems.
AppStream is broken by design...
Yes it is old because I try to follow AppImage guidance and build my app on the oldest still supported distro. And I prefer this way. I want to make my AppIamge app working on as many Linux versions as possible.
I am not criticising here. Just looking for solution.
On 25 Nov 2019, at 21:41, TheAssassin [email protected] wrote:
Not just telling me that your version of OS cli whatever tool is not what we like. Tough.
Well, then you have a really old version, and even if your file is valid with the bundled appstream-cli version, it might break on newer systems.
AppStream is broken by design...
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.
Well, you're not the only one to have problems with this stuff... https://github.com/xournalpp/xournalpp/issues/1618
Agree with @kapitainsky - the tools to validate AppStream are not easy to use because a) there are two different tools which don't always agree, and b) it's hard to get a version that validates AppStream against a recent spec to run on systems that are less than recent.
Something needs to happen indeed. I am considering to build appstreamcli in a way that uses nothing from the target system (e.g., build with musl like the one linked below) and perhaps include it in the appimagetool AppImage?
In the meantime, here is a build that should be able to do the trick: https://github.com/probonopd/static-tools/releases/download/continuous/appstreamcli.tar.gz
Ta. Obviously it is something you should find the way forward. What’s the point of having something which sounds like is not going to work.
In my small case not so important. I just drop all these meta info and will carry on:)
On 25 Nov 2019, at 21:49, TheAssassin [email protected] wrote:
Well, you're not the only one to have problems with this stuff... xournalpp/xournalpp#1618
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.
In the meantime, here is a build that should be able to do the trick: https://github.com/probonopd/static-tools/releases/download/continuous/appstreamcli.tar.gz
Thank you for trying but on CentOS it does not work:
[db@localhost appcli]$ ./appstreamcli
bash: ./appstreamcli: /lib/ld-musl-x86_64.so.1: bad ELF interpreter: No such file or directory
Using the latest version of this tool doesn't make any sense. Neither does bundling. It seems it's possible to bundle an extremely old version, files it validates might be used with newer AppStream tools as well. The other way round is apparently broken. See the Xournal++ issue I linked. That file validates just fine with the latest available version, but broke on the not so old Ubuntu bionic.
Edit: this is just an observation I made recently for some very specific projects. It doesn't mean that it applies globally. But, thinking about it, it makes some sense, right? One should assume that newer versions of the tool remain backwards compatible, i.e., once a file can be validated with a specific version, it'll be able to validate it with any newer version.
AppImage tools should have 'reference' platform they are build to work on. Oldest supported LTS. Now it is either CentOS 7 or Ubuntu 16. I prefer former as appimage generated on CentOS work on Ubuntu not vice versa + CentOS still have many years of support left (till 2024). Now if developers use this platform to compile and package their apps they would expect that it works there. I am not computer expert and frankly even not entirely sure what these metadata is used for but building my first AppImage I wanted to utilize all available functionality (metadata, signing etc.) so step by step trying to follow online AppImage documentation.
Actually, CentOS 6 will continue to receive "maintenance updates" until Nov 2020 (however, we don#t know how well they track security issues and fix them...).
As said, we've had cases when AppStream data which worked on CentOS 6 broke in higher releases, and vice versa. That's the big problem. Sometimes it seems the AppStream people retrospectively fixed bugs in their format, which caused their validator to fail. That means we cannot just say "whatever works on CentOS 6 will work everywhere", that's why everyone has issues with this AppStream stuff.
It is always bad if a format changes retrospectively, but isn't properly versioned, so that newer validators just reject a format that worked in the good old days. Warnings would be okay, but the AppStream validators don't have an option "ignore warnings" or anything.
Check out #603 for more information.
Edit: I haven't checked in a while, the information in this post might be a little outdated. Please correct me if I'm wrong!
from practical perspective reading #603 it looks for me that it is hopeless atm. Chances that Linux world agrees on some standard are close to zero. Simply I package my apps without metadata.
Practically speaking this seems like the best idea, yeah.
It's not really about agreeing on one standard. They have! But the standard doesn't seem to be maintained properly, or the tools aren't properly implemented. I don't know for sure...
Using the latest version of this tool doesn't make any sense. Neither does bundling. It seems it's possible to bundle an extremely old version, files it validates might be used with newer AppStream tools as well. The other way round is apparently broken. See the Xournal++ issue I linked. That file validates just fine with the latest available version, but broke on the not so old Ubuntu bionic.
@TheAssassin I don't understand why you say "Using the latest version of this tool doesn't make any sense. Neither does bundling". Let's assume we can find a good way to bundle their latest version. Why would that not be sufficient? Maybe older base systems can't use the latest AppStream format, but so what - we are not handing AppStream metainfo to the base system, we are just using it internally in AppImage tools.
If I am not mistaken, bundling the most recent appstreamcli would allow the Xournal++ AppStream metainfo file to pass verification.
Thank you for trying but on CentOS it does not work
Ah right, I forgot to explain how to use it:
wget https://github.com/probonopd/static-tools/releases/download/continuous/appstreamcli.tar.gz
tar xfv appstreamcli.tar.gz
./ld-musl-x86_64.so.1 ./appstreamcli
If we decide to bundle it inside the AppImage, of course this would all happen automatically.
on CentOS 7
$ ./ld-musl-x86_64.so.1 ./appstreamcli validate ./rclone-browser.appdata.xml
Segmentation fault (core dumped)
but when I run only ./ld-musl-x86_64.so.1 ./appstreamcli it works listing all possible options
Argh. Can you experiment a bit with the options of this tool and see which ones crash and which don't? I don't know what causes this bug...
sort of all works... but validate correct xml file. If file to be validated is incorrect validation shows error
E: ~:~: xml-markup-invalid Could not parse XML data: Entity: line 1: parser error : Start tag expected, '<' not found
ELF
^
Validation failed: errors: 1
so looks like when parsing XML it crashes
Are you 100% sure your XML validates correctly, e.g., in https://jsonformatter.org/xml-formatter?
The file is here
https://github.com/kapitainsky/RcloneBrowser/blob/master/assets/rclone-browser.appdata.xml
on Ubuntu 16.04 built in appstreamcli does not like my id tag (i don't worry about it now)
$ appstreamcli validate ./rclone-browser.appdata.xml
W - rclone-browser.appdata.xml:org.kapitainsky.rclone-browser
Component id belongs to a desktop-application, but does not resemble the .desktop file name: "org.kapitainsky.rclone-browser"
Validation failed.
but when I remove id it passes:
$ appstreamcli validate ./rclone-browser.appdata.xml
Validation was successful.
Indeed I can reproduce the segfaults. Maybe it doesn't like to be compiled with musl. We should ask @ximion what to do.
Odd. Is there any way to build AppStream with glibc? It isn't really tested with musl, but I also don't see a reason why it wouldn't work - as long as glib can build with musl, at least. First of all though, it looks like the version of AppStream you are using is ancient, or there is something else wrong which doesn't make it produce all validation hints. This is the output I get when validating the file:
I: org.kapitainsky.rclone-browser:8: summary-has-dot-suffix Simple cross platfrom GUI for rclone command line.
I: org.kapitainsky.rclone-browser:4: cid-contains-hyphen org.kapitainsky.rclone-browser
E: org.kapitainsky.rclone-browser:5: metadata-license-invalid Unlicense
Validation failed: errors: 1, infos: 2, pedantic: 2
Maybe update AppStream first, just in case there's an old bug in whatever version you are using? In any case, to say anything for sure I would need a backtrace of this crash - the only thing I can say for now is that the file validates without crash on a glibc-compiled appstreamcli Git master version.
I have corrected the file in my test branch for further testing
https://raw.githubusercontent.com/kapitainsky/RcloneBrowser/kptsky_testing/assets/rclone-browser.appdata.xml
@ximion thanks for chiming in here, do you know a way to link appstreamcli static? The whole musl thing is just a crude workaround.
I never tried doing a static build, but passing --default-library=static -Dstemming=false -Dvapi=false -Dqt=false -Dgir=false to meson and giving it a pkg-config binary that unconditionally sets --static (maybe by creating a dummy shell script that calls the real deal with --static) should do the trick. The pkg-config hack could maybe be replaced by setting static: true on all dependency lines in AppStream's meson.build file as well.
But this hasn't been tried at all yet ;-)
Why does it have to be built statically? I have great doubts that the tool requires any recent glibc features.
we are not handing AppStream metainfo to the base system, we are just using it internally in AppImage tools.
Why bother using AppStream then? The whole idea is that external tools can make use of these AppStream files. appimagetool doesn't need them. Integration tools do.
Please conduct your "AppStream debug session" outside this issue. I had to scroll up two or three pages to find the last comment relevant to "bundling appstreamcli".