devDucks
1.9 update and libcamera problems
Hi, I've updated to the 1.9 and got reports thet the indi camera driver is failing. I tried diagnosing the libcamera, ran the libcamera-hello, and I got the error about the missing libavcodec.so.60. Apparently the avcodec lib was bumped up to 6.1 in the archlinux update and now the libcamera-rpi breaks (and consequently the indi-pylibcamera, which depends on the libcamera-rpi). Any help is much appreciated in getting the camera to work again! Many thanks for your continuous support!
thanks for the report! Having some rest at the moment after a small surgery, if I will find some time, I will recompile the driver in the upcoming week, thanks for the patience in advance!
Mattia
Best wishes for the recovery! Take your time!
I investigated a bit more and there seems to be another issue, the indi-pylibcamera depends on libcamera-rpi, which is identical (at least in version) to the installed libcamera, but it triggers the uninstall of libcamera and install of libcamera-rpi, and in the process an outdated python-libcamera is installed which breaks the indi driver. On top of that, the rpicam-apps does not seem to be installed by default with the stack, together with all dependencies. Just to add a bit to the problem :)
Grazie e buon recupero!
hi @ripkrizbi, I recompiled everything and it seems to be working, whenever you have time to update it woudl be great to get a feedback, thanks!
Thank you Mattia!
I did a quick test with a completely fresh install, followed by update-astroarch, then pacman -S indi-pylibcamera
This prompted me to remove libcamera-* in favour of libcamera-rpi-* and initially it misses to install the rpicam-apps and its dependency (you may wish to add it explicitely to PKGBUILD) libpisp, which I installed manually.
After that, a quick test with libcamera-hello seems to be working.
The INDI part does not work (indiserver -v indi_pylibcamera), as the python-* packages are all compiled for the version 3.11 and the current python version in the new arch is 3.12, so all the installed dependencies are not visible to indiserver's python.
While you were in recovery I tried to recompile the whole stack myself (Arch is not my home territory :) ) and have made it to work, except some of the python depndencies, which I had to brute-force install via pip (could not get the python stuff to compile :( via PKGBUILD :( ). I have saved this install on a separate SD card and I can look up all the dependencies if you want me to, I think I needed to install additional python packages on top of the ones you provide in the stack. If I remember correctly, the very new release of python-libcamera2 has additional stuff it requires.
Additional question - why do you maintain the separate libcamera-rpi set of packages? The ones that come with the libcamera available through extra repo are more up to date (and you can remove one level of complexity by just using the distro packages if I'm correct)...
Thank you for the great support in resolving this, let me know if I can be of any further help!
thanks @ripkrizbi you are absolutely right, I missed some deps in PKGBUILD, I packaged it only once (this week was the second) and the procedure is not perfect, thanks to your inputs it will be :)
I will recompile pylibcamera today against py3.12
to your main question, libcamera (https://git.libcamera.org/libcamera/libcamera.git/) officially supports v4l2 cameras but doesn't support specific rpi cameras, that's why the raspi folks mantain a fork (https://github.com/raspberrypi/libcamera) that adds a bunch of raspi camera specifics.
There was an idea (still on the table) to replace completely (and for good) the normal version of libcamera with the one for the raspi. In my mind it makes sense but I am unsure if this won't break other libcamera "normal" stuff and that's why it's not been done so far
Many many thanks!
All clear about the libcamera-rpi.
Please make sure you recompile all the python-* dependencies for the stack, as I mentioned there were quite a few, and some I was unable to compile as packages :(
Thank you once again for the great support, this really makes a difference!
@ripkrizbi installing now indi-pylibcamera should bring in anything needed.
I checked the python dependencies but as far as I can tell, the latest versions of the ones I have within my repo have all been compiled against 3.12
Thanks Mattia!
unfortunately, some of the deps you maintain are still compiled for 3.11: simplejpeg, av, exifread, kmsxx
I compiled the simplejpeg and bumped it to the most recent version (1.7.4), attached is the PKGBUILD PKGBUILD.simplejpeg.txt (I had to compile it within the venv because there is no arch package for python-cmake, I'm sure there is a simple solution to this but I was unable to find it :) )
I'm struggling to compile the av at the moment as there seem to be some incompatibilities with the new ffmpeg. So I had to brute-force it (pip install av --break-system-packages). Same for kmsxx (pip install rpi-kms --break-system-packages)
after these updates, I'm able to start the indiserver with the indi_pylibcamera module.
Hi i tried installing indi-pylibcamera in astroarch 1.9.0 on a pi4, passed the resolved conflicts with libcamera, but it does not work.
i tested my HQ camera + PI4_4GB with an ASTROBERRY sdcard and it works with the rpicam driver
here's is what i get when i try to launch the server by hand. i tried to fix that by installing the missing python packages one by one but i stumble on the missing "simplejpeg" one that i'm not able to fix.
any (easy) workaround?
Please Help ! regards,
~ » indiserver indi_pylibcamera
2024-09-05T19:39:06: startup: indiserver indi_pylibcamera
2024-09-05T19:39:06: Driver indi_pylibcamera: Traceback (most recent call last):
2024-09-05T19:39:06: Driver indi_pylibcamera: File "/usr/bin/indi_pylibcamera", line 33, in
@albmef as @ripkrizbi made me noticing, there are some deps that are broken. Will try to fix them asap to have the package ready to be working with no additional steps
thanks ! Feel free to ask for any quick/extensive test if needed i own a PI4_4GB fitted with an HQ cam i can't wait to take nice night sky pictures with ;) regards, bob
@ripkrizbi I got all but exif recompiled, are you able to tell me what package requires it and which version? I think we are using exifread but just noticed there is also an exif package and got confused
@MattBlack85 first of all, thank you for all the effort in maintaining the distro and the extra packages! I'm fully aware how much effort it takes to do such a thing, especially on a voluntary basis, and just want to tell you how much I appreciate it!
I quickly checked, and can confirm that the packages I needed to make the indi_pylibcamera driver work were the simplejpeg, av and kmsxx. As I was going through the packages you maintain I noticed that the exifread was still in 3.11 so I included it into the list, thinking that it may be a dependency for something else.
So, at least for my issue, exifread is not required.
On a side note, how frequent are your bleeding-edge libindi/drivers/kstars builds? Do you have an automated process for periodic rebuilds? I'm trying to do a fresh git-based recompile because there may be a fix for my mount in the indilib after the official 2.0.9...
Hi thanks for your kind support today i upgraded arch and the new boost lib version 1.86.0 broke rpicam-apps (see my new issue https://github.com/devDucks/astroarch/issues/105) currently i'm performing some test with HQ camera, for my astro setup, so i needed a quick and dirty fix regards, bob
@ripkrizbi thanks for your words 🙇
For bleeding edge, I don't push it very often, I don't have enough horse power with my building machine to have it running periodically. My build box is a pi4. The best thing would be to have a github actions running daily and in theory, that could work but only for indi. Kstars is a beast and it takes hours to compile on github actions or a raspberrypi 4. I workaround that when I package the new official versions by using distcc but that's it, no fancy automation😋
I am planning to replace my build box with 2/3 Raspi 5 2Gi, that would probably help since the cluster would run 24/7 and have 16 powerful arm cpu to use, but when it is gonna happen I really have no idea 🫠
hi @ripkrizbi Please could you provide me a quick Howto guide for making the indi_pylibcamera to run ? i'm really looking forward this. this morning i tried to compile simplejpeg but no luck. i'm lacking the knowledge and time for fixing that. the pi HQ cam is part of my setup i plan to use it as guiding cam, so a quick and dirty hack is ok for me in the meantime. regards, bob
@albmef you are not supposed to compile anything, all should work just install indi-pylibcamera
sorry but it does not work yet
2024-09-20T09:24:42: startup: indiserver indi_pylibcamera
2024-09-20T09:24:43: Driver indi_pylibcamera: Traceback (most recent call last):
2024-09-20T09:24:43: Driver indi_pylibcamera: File "/usr/lib/python3.12/site-packages/picamera2/previews/drm_preview.py", line 9, in
yes system up to date. i do it everyday i installed the dependencies by hand to be sure sudo pacman -S python-av kmsxx-git python-pillow python-pycamera2 python-lxml python-astropy python-pidng python-v4l2 python-simplejpeg libcamera-rpi rpicam-apps
but no luck
Proceed with installation? [Y/n] :: Retrieving packages... libcamera-rpi-0.3.0-1-aarch64 is up to date python-av-13.0.0-1-aarch64 is up to date rpicam-apps-1.5.0-5-aarch64 is up to date python-simplejpeg-1.7.6-1-any is up to date kmsxx-git-r668.9ae90ce-2-aarch64 is up to date python-pycamera2-0.3.17-2-aarch64 is up to date python-pidng-4.0.9-2-aarch64 is up to date python-v4l2-0.3.4-2-any is up to date (11/11) checking keys in keyring [--------------------------------------------------------------------------] 100% (11/11) checking package integrity [--------------------------------------------------------------------------] 100% (11/11) loading package files [--------------------------------------------------------------------------] 100% (11/11) checking for file conflicts [--------------------------------------------------------------------------] 100% (11/11) checking available disk space [--------------------------------------------------------------------------] 100% :: Processing package changes... ( 1/11) reinstalling python-pillow [--------------------------------------------------------------------------] 100% ( 2/11) reinstalling python-av [--------------------------------------------------------------------------] 100% ( 3/11) reinstalling kmsxx-git [--------------------------------------------------------------------------] 100% ( 4/11) reinstalling libcamera-rpi [--------------------------------------------------------------------------] 100% ( 5/11) reinstalling python-pycamera2 [--------------------------------------------------------------------------] 100% ( 6/11) reinstalling python-lxml [--------------------------------------------------------------------------] 100% ( 7/11) reinstalling python-astropy [--------------------------------------------------------------------------] 100% ( 8/11) reinstalling python-pidng [--------------------------------------------------------------------------] 100% ( 9/11) reinstalling python-v4l2 [--------------------------------------------------------------------------] 100% (10/11) reinstalling python-simplejpeg [--------------------------------------------------------------------------] 100% (11/11) reinstalling rpicam-apps [--------------------------------------------------------------------------] 100% :: Running post-transaction hooks... (1/3) Reloading device manager configuration... (2/3) Arming ConditionNeedsUpdate... (3/3) Refreshing PackageKit...
Interesting thanks!, We got the camera working flawlessly on @FireNWater somehow, will have a look at what we did additionally
@albmef there seems to be a problem with your kmsxx library, can you do a:
sudo pacman -S python-kmsxx
and then
pacman -Ql python-kmsxx just to verify that it is compiled for python 3.12
this should also be a lib maintained by @MattBlack85. If it fails to install/work, until Mattia can recompile it, you can install it manually with pip (pip install rpi-kms --break-system-packages)
thanks but there is no python-kmsxx package in the arch repos https://archlinux.org/packages/?sort=&q=kmsxx&maintainer=&flagged= bob
@albmef as I have clearly written, the said package is maintained by @MattBlack85 and it is located in the astromatto repo. Have you tried running the commands I gave?
i tried force pip install rpi-kms but this does not work either
2024-09-10T23:18:22: startup: indiserver indi_pylibcamera
2024-09-10T23:18:26: Driver indi_pylibcamera: Traceback (most recent call last):
2024-09-10T23:18:26: Driver indi_pylibcamera: File "/usr/lib/python3.12/site-packages/picamera2/previews/drm_preview.py", line 9, in
the problem is i doubt that "pip install rpi-kms --break-system-packages" actually worked
ls /usr/lib/python3.12/site-packages/rpi*
zsh: no matches found: /usr/lib/python3.12/site-packages/rpi*
tree /usr/lib/python3.12/site-packages/ | grep kms
├── pykms
│ └── pykms.so
how do i check if the files were installed properly ?
If you installed without sudo, then the library should be in your ~/.local/lib/python3.12/site-packages directory, check there.
also, start python, then
import pykms
and if you get no errors then it's there. Make sure you start indiserver with the same user privileges as the ones you used to do the pip install
pip with sudo
sudo pip install rpi-kms --break-system-packages
[sudo] password for astronaut:
DEPRECATION: Loading egg at /usr/lib/python3.12/site-packages/urjtag-2021.3-py3.12-linux-aarch64.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
Collecting rpi-kms
Downloading rpi_kms-0.1a1.tar.gz (3.4 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: rpi-kms
Building wheel for rpi-kms (pyproject.toml) ... done
Created wheel for rpi-kms: filename=rpi_kms-0.1a1-cp312-cp312-linux_aarch64.whl size=227855 sha256=f69514854ae9775b487384492ceb2fac1fa70caa3a54402da17e2c89048a8c0b
Stored in directory: /root/.cache/pip/wheels/c7/a9/5b/1c7a9b1f26df2f341d80cdc0bef9a19bf7236f110565832d4c
Successfully built rpi-kms
Installing collected packages: rpi-kms
Successfully installed rpi-kms-0.1a1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
check if installed
~ » tree /usr/lib/python3.12/site-packages/ | grep kms
├── pykms
│ ├── pykms.cpython-312-aarch64-linux-gnu.so
│ └── pykms.so
├── rpi_kms-0.1a1.dist-info
test with python
~ » python
Python 3.12.6 (main, Sep 13 2024, 13:27:21) [GCC 14.1.1 20240507] on linux
Type "help", "copyright", "credits" or "license" for more information.
import kms Traceback (most recent call last): File "
", line 1, in ModuleNotFoundError: No module named 'kms' import pykms Traceback (most recent call last): File " ", line 1, in File "/home/astronaut/.local/lib/python3.12/site-packages/pykms/init.py", line 1, in from .pykms import * ImportError: libfmt.so.10: cannot open shared object file: No such file or directory
test indiserver with sudo
» sudo indiserver indi_pylibcamera
2024-09-20T11:14:38: startup: indiserver indi_pylibcamera
2024-09-20T11:14:39: Driver indi_pylibcamera: Traceback (most recent call last):
2024-09-20T11:14:39: Driver indi_pylibcamera: File "/usr/lib/python3.12/site-packages/picamera2/previews/drm_preview.py", line 9, in
It appears you have installed the rpi-kms package twice, and now you have confused python somehow. In your error when importing pykms within python, you can see that the path is a LOCAL path (ie rpi-kms installed without sudo, File "/home/astronaut/.local/lib/python3.12/site-packages/pykms...).
remove both packages through pip (both local and system), then install only local (as I have written, please read the suggestions thoroughly). Then try again.
as you said i removed both local and system using pip uninstall
so only kmsxx-git remains on the system
~ » tree /usr/lib/python3.12/site-packages/ | grep kms
├── pykms
│ └── pykms.so
~ » tree .local/lib/python3.12/site-packages | grep kms
~ »
i cannot remove the kmsxx-git package because it break the dependency of indi-pylibcamera
~ » sudo pacman -R kmsxx-git
[sudo] password for astronaut:
checking dependencies...
error: failed to prepare transaction (could not satisfy dependencies)
:: removing kmsxx-git breaks dependency 'kmsxx-git' required by indi-pylibcamera
so i leave it installed.
i test with python
~ » python
Python 3.12.6 (main, Sep 13 2024, 13:27:21) [GCC 14.1.1 20240507] on linux
Type "help", "copyright", "credits" or "license" for more information.
import pykms import kms Traceback (most recent call last): File "
", line 1, in ModuleNotFoundError: No module named 'kms'
pykms is found but still no kms module
now i reinstall rpi-kms system wide with SUDO
~ » sudo pip install rpi-kms --break-system-packages
DEPRECATION: Loading egg at /usr/lib/python3.12/site-packages/urjtag-2021.3-py3.12-linux-aarch64.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
Collecting rpi-kms
Using cached rpi_kms-0.1a1-cp312-cp312-linux_aarch64.whl
Installing collected packages: rpi-kms
Successfully installed rpi-kms-0.1a1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
i check it
~ » tree /usr/lib/python3.12/site-packages/ | grep kms
├── pykms
│ ├── pykms.cpython-312-aarch64-linux-gnu.so
│ └── pykms.so
├── rpi_kms-0.1a1.dist-info
~ » python
Python 3.12.6 (main, Sep 13 2024, 13:27:21) [GCC 14.1.1 20240507] on linux
Type "help", "copyright", "credits" or "license" for more information.
import pykms Traceback (most recent call last): File "
", line 1, in File "/usr/lib/python3.12/site-packages/pykms/init.py", line 1, in from .pykms import * ImportError: libfmt.so.10: cannot open shared object file: No such file or directory import kms Traceback (most recent call last): File " ", line 1, in ModuleNotFoundError: No module named 'kms'
it seems that there is still a conflict but i'm not able to resolve it because indi-pylibcamera depends on kmsxx-git and i cannot remove it without removing indi-pylibcamera. sorry but i don't really understand the ins and outs of forcing rpi-kms with pip, and second, which package actually provides the expected kms module (not pykms which is provided by kmsxx-git) any idea ?