Breakthrough
1.6.2 fail to build
Debian unstable amd64 python 3.12.8 and 3.13.1
1.6.1.1 build fine
dpkg-buildpackage: info: source package dvr-scan-dmo
dpkg-buildpackage: info: source version 1.6.2-dmo1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Christian Marillat <[email protected]>
dpkg-source --before-build .
dpkg-buildpackage: info: host architecture amd64
dpkg-source: info: using options from dvr-scan-dmo-1.6.2/debian/source/options: --extend-diff-ignore=^[^/]*[.]egg-info/
debian/rules clean
dh clean -Spybuild
dh_auto_clean -O-Spybuild
debian/rules override_dh_clean
make[1]: Entering directory '/build/dvr-scan-dmo-1.6.2'
dh_clean dvr_scan/docs
make[1]: Leaving directory '/build/dvr-scan-dmo-1.6.2'
debian/rules execute_after_dh_clean
make[1]: Entering directory '/build/dvr-scan-dmo-1.6.2'
rm -f -r dist/installer/Prerequisites/Visual\ C++\ Redistributable\ for\ Visual\ Studio\ 2015-2019
make[1]: Leaving directory '/build/dvr-scan-dmo-1.6.2'
dpkg-source -b .
dpkg-source: info: using options from dvr-scan-dmo-1.6.2/debian/source/options: --extend-diff-ignore=^[^/]*[.]egg-info/
dpkg-source: info: using source format '3.0 (quilt)'
dpkg-source: info: building dvr-scan-dmo using existing ./dvr-scan-dmo_1.6.2.orig.tar.gz
dpkg-source: warning: ignoring deletion of directory dist/installer/Prerequisites/Visual C++ Redistributable for Visual Studio 2015-2019
dpkg-source: warning: ignoring deletion of file dist/installer/Prerequisites/Visual C++ Redistributable for Visual Studio 2015-2019/VC_redist.x64.exe, use --include-removal to override
dpkg-source: info: building dvr-scan-dmo in dvr-scan-dmo_1.6.2-dmo1.debian.tar.xz
dpkg-source: info: building dvr-scan-dmo in dvr-scan-dmo_1.6.2-dmo1.dsc
debian/rules binary
dh binary -Spybuild
dh_update_autotools_config -O-Spybuild
dh_autoreconf -O-Spybuild
dh_auto_configure -O-Spybuild
dh_auto_build -O-Spybuild
I: pybuild plugin_pyproject:129: Building wheel for python3.12 with "build" module
I: pybuild base:311: python3.12 -m build --skip-dependency-check --no-isolation --wheel --outdir /build/dvr-scan-dmo-1.6.2/.pybuild/cpython3_3.12_dvr-scan
* Building wheel...
running bdist_wheel
running build
running build_py
creating build/lib/dvr_scan
copying dvr_scan/video_joiner.py -> build/lib/dvr_scan
copying dvr_scan/overlays.py -> build/lib/dvr_scan
copying dvr_scan/detector.py -> build/lib/dvr_scan
copying dvr_scan/config.py -> build/lib/dvr_scan
copying dvr_scan/__init__.py -> build/lib/dvr_scan
copying dvr_scan/subtractor.py -> build/lib/dvr_scan
copying dvr_scan/platform.py -> build/lib/dvr_scan
copying dvr_scan/scanner.py -> build/lib/dvr_scan
copying dvr_scan/region.py -> build/lib/dvr_scan
copying dvr_scan/opencv_loader.py -> build/lib/dvr_scan
copying dvr_scan/__main__.py -> build/lib/dvr_scan
creating build/lib/dvr_scan/app
copying dvr_scan/app/__init__.py -> build/lib/dvr_scan/app
copying dvr_scan/app/region_editor.py -> build/lib/dvr_scan/app
creating build/lib/dvr_scan/cli
copying dvr_scan/cli/__init__.py -> build/lib/dvr_scan/cli
copying dvr_scan/cli/controller.py -> build/lib/dvr_scan/cli
error: package directory 'dvr_scan/docs' does not exist
ERROR Backend subprocess exited when trying to invoke build_wheel
E: pybuild pybuild:389: build: plugin pyproject failed with: exit code=1: python3.12 -m build --skip-dependency-check --no-isolation --wheel --outdir /build/dvr-scan-dmo-1.6.2/.pybuild/cpython3_3.12_dvr-scan
dh_auto_build: error: pybuild --build -i python{version} -p 3.12 returned exit code 13
make: *** [debian/rules:6: binary] Error 25
Hi @dmo to build DVR-Scan now you should first run python3 dist/pre_release.py to generate the documentation. Could you try running
that first? It needs the deps from docs/requirements.txt to run.
Also note that DVR-Scan requires the python3-tk package to run now. Let me know if you run into any issues, thanks!
On Wednesday, December 18, 2024, dmo @.***> wrote:
Debian unstable amd64 python 3.12.8 and 3.13.1
1.6.1.1 build fine
dpkg-buildpackage: info: source package dvr-scan-dmo dpkg-buildpackage: info: source version 1.6.2-dmo1 dpkg-buildpackage: info: source distribution unstable dpkg-buildpackage: info: source changed by Christian Marillat < @.*> dpkg-source --before-build . dpkg-buildpackage: info: host architecture amd64 dpkg-source: info: using options from dvr-scan-dmo-1.6.2/debian/source/options: --extend-diff-ignore=^[^/][.]egg-info/ debian/rules clean dh clean -Spybuild dh_auto_clean -O-Spybuild debian/rules override_dh_clean make[1]: Entering directory '/build/dvr-scan-dmo-1.6.2' dh_clean dvr_scan/docs make[1]: Leaving directory '/build/dvr-scan-dmo-1.6.2' debian/rules execute_after_dh_clean make[1]: Entering directory '/build/dvr-scan-dmo-1.6.2' rm -f -r dist/installer/Prerequisites/Visual\ C++\ Redistributable\ for
Visual\ Studio\ 2015-2019 make[1]: Leaving directory '/build/dvr-scan-dmo-1.6.2' dpkg-source -b . dpkg-source: info: using options from dvr-scan-dmo-1.6.2/debian/source/options: --extend-diff-ignore=^[^/][.]egg-info/ dpkg-source: info: using source format '3.0 (quilt)' dpkg-source: info: building dvr-scan-dmo using existing ./dvr-scan-dmo_1.6.2.orig.tar.gz dpkg-source: warning: ignoring deletion of directory dist/installer/Prerequisites/Visual C++ Redistributable for Visual Studio 2015-2019 dpkg-source: warning: ignoring deletion of file dist/installer/Prerequisites/Visual C++ Redistributable for Visual Studio 2015-2019/VC_redist.x64.exe, use --include-removal to override dpkg-source: info: building dvr-scan-dmo in dvr-scan-dmo_1.6.2-dmo1.debian.tar.xz dpkg-source: info: building dvr-scan-dmo in dvr-scan-dmo_1.6.2-dmo1.dsc debian/rules binary dh binary -Spybuild dh_update_autotools_config -O-Spybuild dh_autoreconf -O-Spybuild dh_auto_configure -O-Spybuild dh_auto_build -O-Spybuild I: pybuild plugin_pyproject:129: Building wheel for python3.12 with "build" module I: pybuild base:311: python3.12 -m build --skip-dependency-check --no-isolation --wheel --outdir /build/dvr-scan-dmo-1.6.2/.pybuild/cpython3_3.12_dvr-scan
- Building wheel... running bdist_wheel running build running build_py creating build/lib/dvr_scan copying dvr_scan/video_joiner.py -> build/lib/dvr_scan copying dvr_scan/overlays.py -> build/lib/dvr_scan copying dvr_scan/detector.py -> build/lib/dvr_scan copying dvr_scan/config.py -> build/lib/dvr_scan copying dvr_scan/init.py -> build/lib/dvr_scan copying dvr_scan/subtractor.py -> build/lib/dvr_scan copying dvr_scan/platform.py -> build/lib/dvr_scan copying dvr_scan/scanner.py -> build/lib/dvr_scan copying dvr_scan/region.py -> build/lib/dvr_scan copying dvr_scan/opencv_loader.py -> build/lib/dvr_scan copying dvr_scan/main.py -> build/lib/dvr_scan creating build/lib/dvr_scan/app copying dvr_scan/app/init.py -> build/lib/dvr_scan/app copying dvr_scan/app/region_editor.py -> build/lib/dvr_scan/app creating build/lib/dvr_scan/cli copying dvr_scan/cli/init.py -> build/lib/dvr_scan/cli copying dvr_scan/cli/controller.py -> build/lib/dvr_scan/cli error: package directory 'dvr_scan/docs' does not exist
ERROR Backend subprocess exited when trying to invoke build_wheel E: pybuild pybuild:389: build: plugin pyproject failed with: exit code=1: python3.12 -m build --skip-dependency-check --no-isolation --wheel --outdir /build/dvr-scan-dmo-1.6.2/.pybuild/cpython3_3.12_dvr-scan dh_auto_build: error: pybuild --build -i python{version} -p 3.12 returned exit code 13 make: *** [debian/rules:6: binary] Error 25
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.< https://ci3.googleusercontent.com/meips/ADKq_NZeb2xdCleFsEqydCYgxp28lbztU5RDCmT7Ik7LhZ0QRxnXIOkftacfHREtajmuq9WCofGo5MJHYezh6iKa1LRzJk5TRu7dpwzP146ZVAQEm2TJP08RErMty9jLmCUBilXqVWF6CeYve62D3UhiNp5O74whAbNxQmcdJzk7YqCV71nO25WbFzfxVZie19sgAhWICGZ0ubdeMmqeklnefHaJRtsbGagY-656bucE=s0-d-e1-ft#https://github.com/notifications/beacon/AAA6TBBPY7LDK5NR2KGKF732GFDBTA5CNFSM6AAAAABT2K22WCWGG33NNVSW45C7OR4XAZNFJFZXG5LFVJRW63LNMVXHIX3JMTHKHQNZCA.gif>Message ID: @.***>
Not related to DVR-Scan but to PySceneDetect
python3 dist/pre_release.py
Creating .version_info.
Traceback (most recent call last):
File "/build/dvr-scan-dmo-1.6.2/dist/pre_release.py", line 181, in <module>
write_version_info_for_windows_exe()
File "/build/dvr-scan-dmo-1.6.2/dist/pre_release.py", line 24, in write_version_info_for_windows_exe
import dvr_scan
File "/build/dvr-scan-dmo-1.6.2/dvr_scan/__init__.py", line 40, in <module>
from dvr_scan.platform import init_logger
File "/build/dvr-scan-dmo-1.6.2/dvr_scan/platform.py", line 29, in <module>
from scenedetect.platform import get_and_create_path, get_ffmpeg_version
File "/usr/lib/python3/dist-packages/scenedetect/__init__.py", line 54, in <module>
from scenedetect.scene_manager import SceneManager, save_images, SceneList, CutList, Interpolation
File "/usr/lib/python3/dist-packages/scenedetect/scene_manager.py", line 413, in <module>
image: cv2.Mat,
^^^^^^^
AttributeError: module 'cv2' has no attribute 'Mat'
make[1]: *** [debian/rules:9: override_dh_auto_build] Error 1
Thanks for the update.
I will create a new release this evening which doesn't require you to run the pre_release.py script. This step was added so documentation is included with distributions. However I can include them in the repo to avoid this step so the package can be built with minimal dependencies.
If you need something sooner: the python3-opencv package is a bit out of date, it might be better if you could install a more recent version from pip. Running the pre release script requires all the deps in requirements.txt too.
Lastly, note that python3-tk is a Debian/dpkg package and won't show up in requirements.txt. Is there a correct way to tag that as a dependency? Sorry I am not that familiar with building dpkgs - happy to look more into that if you can point me in the right direction.
I would be happy to add a builder on GitHub that builds the package here if you can provide the commands required. Then I could verify that the Debian package is buildable on each commit.
https://github.com/marketplace/actions/build-debian-packages states that the repo should contain a debian/ folder with details on how to build it. If you are willing and have already done that, feel free to upload a PR with it and I can look into integrating it here on GitHub.
Thanks!
On Wednesday, December 18, 2024, dmo @.***> wrote:
Still doesn't build. python3-opencv 4.6.0 is installed
python3 dist/pre_release.py Creating .version_info. Traceback (most recent call last): File "/build/dvr-scan-dmo-1.6.2/dist/pre_release.py", line 181, in
write_version_info_for_windows_exe() File "/build/dvr-scan-dmo-1.6.2/dist/pre_release.py", line 24, in write_version_info_for_windows_exe import dvr_scan File "/build/dvr-scan-dmo-1.6.2/dvr_scan/init.py", line 40, in from dvr_scan.platform import init_logger File "/build/dvr-scan-dmo-1.6.2/dvr_scan/platform.py", line 29, in from scenedetect.platform import get_and_create_path, get_ffmpeg_version File "/usr/lib/python3/dist-packages/scenedetect/init.py", line 54, in from scenedetect.scene_manager import SceneManager, save_images, SceneList, CutList, Interpolation File "/usr/lib/python3/dist-packages/scenedetect/scene_manager.py", line 413, in image: cv2.Mat, ^^^^^^^ AttributeError: module 'cv2' has no attribute 'Mat' make[1]: *** [debian/rules:9: override_dh_auto_build] Error 1 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.< https://ci3.googleusercontent.com/meips/ADKq_NbNNLNcWqAdJCSHo5kdbgoBlB9kQSJxLWh7ouX4R7XrS0z4MqWEGeNfFhg7BuYZNJVnepDwVUhY1QDYjwP9M1Ryq1BArzQQaSOvQ_ynzraB2TCKB6wJYr2A1HXIFCUnUxI9pOt1opmn8BnAvP3xwEE9brRDIr8AE8UekBRwKIiKCsJx8MNqoq9LFEbZgwnmqj4dDLoRnkMh7ge73cpirSDmi6P4fnUe3J0EF8ryVGjZJ69bRAYISe8=s0-d-e1-ft#https://github.com/notifications/beacon/AAA6TBH3ZCHGF7VGNSMQXJ32GGI6RA5CNFSM6AAAAABT2K22WCWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUYC3RQQ.gif>Message ID: @.***>
I'm going to hold off on making a new release until I can understand more about how the Debian packages are being created, and if it's possible for you to run the pre-release script there. You can see how I build the Python distribution here on Github: https://github.com/Breakthrough/DVR-Scan/blob/main/.github/workflows/build.yml
You can also try doing this in a virtual environment if it would clobber your system packages.
In the meantime, you can still build DVR-Scan 1.6.2 if you copy the docs from the pre-built Python package. I zipped these files here so you can grab them quickly (dvr_scan_docs-1.6.2.zip). If you extract them into dvr_scan/docs the build should succeed. This folder and the contents are the result of running the pre-release script and include offline documentation for DVR-Scan.
To download them from a trusted build, you can also download the Python source package from PyPI or from Github Releases. If you extract dvr_scan-1.6.2.tar.gz it should contain an identical copy of the docs I uploaded above. Lastly, if you want to leave the docs out, you can remove these lines from setup.cfg: https://github.com/Breakthrough/DVR-Scan/blob/main/setup.cfg#L50-L53
Items I could use your response on @marillat:
- Depending on the result of the above, please let me know if you are able to integrate
dist/pre_release.pyinto your build process, or if I need to check-in the built documentation into the git repo. Also let me know if I need to make a new tagged release, or if the workarounds are sufficient for you to update the Debian package. - Is the offline documentation supposed to be in a different location?
- Could you advise on how to mark the
python3-tkpackage as a dependency? My understanding is that this is typically a pre-built module not included on PyPI. - Similarily, with OpenCV, do users typically use the
python3-opencvDebian package? I usually test with the latest version, the one on packages.debian.org seems to be quite a bit older, so I may need to make some changes to both DVR-Scan and PySceneDetect to support that (since it seems to cause issues when you ran the pre release script, which means it also might fail when you try to run DVR-Scan) - Let me know if you wish to include any artifacts in the source code to help with building the Debian package. I will leave distribution up to you, but I do not mind including build rules or automation to ensure DVR-Scan continues to build as expected (e.g. I would probably add a Linux builder that builds an amd64 .deb).
I'm going to hold off on making a new release until I can understand more about how the Debian packages are being created, and if it's possible for you to run the pre-release script there. You can see how I build the Python distribution here on Github: https://github.com/Breakthrough/DVR-Scan/blob/main/.github/workflows/build.yml
You can also try doing this in a virtual environment if it would clobber your system packages.
We don't use venv to do python packages. Packages are build with pybuild under chroot.
https://wiki.debian.org/Python/Pybuild
3. Could you advise on how to mark the `python3-tk` package as a dependency? My understanding is that this is typically a pre-built module not included on PyPI.
You must set python install_requires in setup.py and not setup.cfg with versioned dependency when needed.
Then the installation will fail if a package is missing or doesn't math the needed version
https://packaging.python.org/en/latest/discussions/install-requires-vs-requirements/
4. Similarily, with OpenCV, do users typically use the `python3-opencv` Debian package? I usually test with the latest version, the one on packages.debian.org seems to be quite a bit older, so I may need to make some changes to both DVR-Scan and PySceneDetect to support that (since it seems to cause issues when you ran the pre release script, which means it also might fail when you try to run DVR-Scan)
See 3
5. Let me know if you wish to include any artifacts in the source code to help with building the Debian package. I will leave distribution up to you, but I do not mind including build rules or automation to ensure DVR-Scan continues to build as expected (e.g. I would probably add a Linux builder that builds an amd64 .deb).
Probably a bad idea to add a debian directory. Let the Debian maintainer do the job.
Christian
You must set python
install_requiresinsetup.pyand notsetup.cfgwith versioned dependency when needed. Then the installation will fail if a package is missing or doesn't math the needed version https://packaging.python.org/en/latest/discussions/install-requires-vs-requirements/
This does not work for the tkinter module, as it is not managed the same way. Adding it to install_requires makes it impossible to install the package. Surely there must be another way to make the python3-tkinter Debian package a requirement for dvr-scan-dmo?
The same issue should happen for OpenCV, but somehow that is listed as a dependency of the package.
We cannot use install_requires for OpenCV either, since it is distributed as such with some Linux distributions under python3-opencv package, or on PyPI as opencv-python, both of which provide the cv2 module. As neither opencv-python nor cv2 are in the requirements, how does python3-opencv currently become a dependency of dvr-scan package when you create the package?
Can you try running the pre_release script after uninstalling your system's python3-opencv package, and pip installing the requirements from DVR-Scan's requirements.txt and docs/requirements.txt?
Probably a bad idea to add a debian directory. Let the Debian maintainer do the job.
Sounds good. Can you share how to build the package so I can reproduce the issues you are running into? I wasn't able to find the source code for how you are producing the .debs.
According to the Debian Python Policy, the python3-tk package should be listed under Depends: of a debian/control file. Although I do my best to follow the Debian app/library style guide for Python, there's a few things about DVR-Scan which deviate:
- you can install everything in a virtual env, but the python3-tk package must still be available and installed for the Python interpreter backing the venv
- running dist/pre_release.py to generate the required documentation assets - this should be possible to do under a virtualenv if the python3-venv package is listed as a build requirement
For the last point, there are some workarounds I pointed out previously. Although here we use mkdocs instead of sphinx, it seems typical for documentation building to be integrated as part of the pybuild process, by overriding the dh_auto_build command. If python3-venv is a build requirement, is it not possible to run the pre_release script under a venv in a similar manner?
Sorry for the reply spam, I've just updated the v1.6.2 release branch and tag, so you should be able to avoid running the pre-release script and get the Debian package updated now. Will leave this issue open until you can respond to the other open items.
Thank you!
According to the Debian Python Policy, the
python3-tkpackage should be listed underDepends:of a debian/control file. Although I do my best to follow the Debian app/library style guide for Python, there's a few things about DVR-Scan which deviate:* you can install everything in a virtual env, but the python3-tk package must still be available and installed for the Python interpreter backing the venv
As I'm saying before Debian don't use at all venv to build python package.
Christian
As I'm saying before Debian don't use at all venv to build python package.
Gotcha. I think the OpenCV issue you were running into before was caused by Breakthrough/PySceneDetect#468 so that should be fixed now - when you get a chance, could you let me know if you are able to run the pre_release script successfully and integrate it into the Debian packaging workflow?
I am still unsure how to address the Tkinter issue, however I suspect you may need to add it to the control file.
pre_release script still return an error and DeprecationWarning
I still don't understand why the website need to be generated by the package installer.
Otherwise, I'm happy to do a package without calling the pre_release script now that issue on PySceneDetect has bee, fixed.
python3 dist/pre_release.py
INFO - DeprecationWarning: 'materialx.emoji.twemoji' is deprecated.
Material emoji logic has been officially moved into mkdocs-material
version 9.4. Please use Material's 'material.extensions.emoji.twemoji'
instead of 'materialx.emoji.twemoji' in your 'mkdocs.yml' file.
markdown_extensions:
- pymdownx.emoji:
emoji_index: !!python/name:material.extensions.emoji.twemoji
emoji_generator: !!python/name:material.extensions.emoji.to_svg
'mkdocs_material_extensions' is deprecated and will no longer be
supported moving forward. This is the last release.
File "/usr/lib/python3/dist-packages/materialx/emoji.py", line 118, in twemoji
return _patch_index(options)
File "/usr/lib/python3/dist-packages/materialx/emoji.py", line 68, in _deprecated_func
warnings.warn(
WARNING - Material emoji logic has been officially moved into mkdocs-material
version 9.4. Please use Material's 'material.extensions.emoji.twemoji'
instead of 'materialx.emoji.twemoji' in your 'mkdocs.yml' file.
markdown_extensions:
- pymdownx.emoji:
emoji_index: !!python/name:material.extensions.emoji.twemoji
emoji_generator: !!python/name:material.extensions.emoji.to_svg
'mkdocs_material_extensions' is deprecated and will no longer be
supported moving forward. This is the last release.
INFO - Cleaning site directory
INFO - Building documentation to directory: /build/dvr-scan-dmo-1.6.2/dvr_scan/docs
INFO - The following pages exist in the docs directory, but are not included in the "nav" configuration:
- index.md
INFO - Documentation built in 0.29 seconds
Creating .version_info.
Building docs.
Postprocessing docs.
Traceback (most recent call last):
File "/build/dvr-scan-dmo-1.6.2/dist/pre_release.py", line 182, in <module>
build_docs(use_local_images=bool(len(sys.argv) == 2 and sys.argv[1] == "--use-local-images"))
File "/build/dvr-scan-dmo-1.6.2/dist/pre_release.py", line 162, in build_docs
) if to_remove.is_dir() else to_remove.unlink()
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/pathlib.py", line 1344, in unlink
os.unlink(self)
FileNotFoundError: [Errno 2] No such file or directory: 'dvr_scan/docs/assets/javascripts/bundle.220ee61c.min.js.map'
make[1]: *** [debian/rules:9: override_dh_auto_build] Error 1
make[1]: Leaving directory '/build/dvr-scan-dmo-1.6.2'
make: *** [debian/rules:6: binary] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
In my last comment I'm talking about the 1.6.2 source from https://github.com/Breakthrough/DVR-Scan/releases/tag/v1.6.2-release
Thanks for trying, you can proceed to release without running the script. I bundled the pre built docs in there now.
I was hoping to avoid it since they could become stale, but I will put a check for this in the GitHub builders to prevent that. Thanks so much for your help with this.
On Tue, Dec 24, 2024, 9:14 AM dmo @.***> wrote:
In my last comment I'm talking about the 1.6.2 source from https://github.com/Breakthrough/DVR-Scan/releases/tag/v1.6.2-release
— Reply to this email directly, view it on GitHub https://github.com/Breakthrough/DVR-Scan/issues/187#issuecomment-2561172985, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA6TBGFLQJDM3NTVV6PQLT2HFT4TAVCNFSM6AAAAABT2K22WCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRRGE3TEOJYGU . You are receiving this because you commented.Message ID: @.***>