ngscopeclient
Packaging: ngscopeclient packaging and (eventually) CPack and the Windows and MacOS equivalent
Intro
We need to make our linux build outputs CPackable so we can generate distro-specific packages that pull in external dependencies automagically. Afterwards we need to fix Windows MSYS DLLs + installer We also need to do the same for MacOS, not sure how yet.
Linux distros:
Definitely:
- Debian 12
- Ubuntu 22.x LTS
- Ubuntu 24.x LTS
- Fedora 40
- CentOS/AlmaLinux/Rocky Linux (RHEL is used a lot by pro EDA, having compatibility with rhel through an open source distro would likely be useful)
Feedback on which distro of the three would be useful.
Maybe:
- Arch Linux (Current Release: 2024.09.01)
Definition of task/completion
Primary: ngscopeclient needs to be able to be built using the instructions from the documentation and a full build needs to result. We likely need to write out dependency lists per distro Secondary: We need to be able to run CPack to package this build and have dependency lists be part of the package meta-info.
Relevant other issues: #141 #309 Maybe relevant: #497 #611
An easy solution to include Windows DLL dependencies is to use my open source tool mingw-bundledlls at end of the build in the directory of ngscopeclient.exe
export MINGW_BUNDLEDLLS_SEARCH_PATH="C:\msys64\mingw64\bin;."
cd ./
./mingw-bundledlls.exe ./ngscopeclient.exe --copy
After that all DLL dependencies will be copied near ngscopeclient.exe and ngscopeclient could be launched from any Windows computer (like a portable executable) without requiring to have MSYS2/Mingw64 installed ...
Thanks for your input @bvernoux. I'm a little hesitant to pull in external dependencies. What is the least invasive way to use your tool? Export the search path and download the .exe and run it then delete again?
Ideas? Maybe you can release an .exe instead of a compressed file as well. This would be helpful for an automated process, since we can rely on github to host it and always be available. There used to be a link to a download on your personal website in the CMake which felt, well, kind of dirty, haha.
Solution1
- You can install the latest pacman package for mingw64/msys2
mingw-bundledlls-0.2.4-1-x86_64.pkg.tar.zstavailable in https://github.com/bvernoux/mingw-bundledlls/releases- We already use Mingw64 so it is trivial to do that in https://github.com/ngscopeclient/scopehal-apps/blob/master/.github/workflows/build-windows.yml Something like that shall do the trick (to be cross checked/tested) Added step "Install mingw-bundledlls" & step "Bundle DLLs"
name: Build-windows
on:
push:
pull_request:
workflow_dispatch:
env:
VULKAN_SDK_VERSION: 1.3.275.0
jobs:
Windows:
runs-on: windows-latest
defaults:
run:
shell: msys2 {0}
strategy:
matrix:
include:
- { sys: mingw64, env: x86_64 }
- { sys: ucrt64, env: ucrt-x86_64 }
- { sys: clang64, env: clang-x86_64 }
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
fetch-depth: 0
- uses: msys2/setup-msys2@v2
with:
update: true
msystem: ${{matrix.sys}}
install: >-
git
wget
pacboy: >-
cmake:p
toolchain:p
libsigc++:p
cairomm:p
yaml-cpp:p
glfw:p
catch:p
vulkan-headers:p
vulkan-loader:p
shaderc:p
glslang:p
spirv-tools:p
ffts:p
- name: Install mingw-bundledlls
shell: msys2 {0}
run: |
wget https://github.com/bvernoux/mingw-bundledlls/releases/download/v0.2.4/mingw-bundledlls-0.2.4-1-x86_64.pkg.tar.zst
pacman -U mingw-bundledlls-0.2.4-1-x86_64.pkg.tar.zst --noconfirm
- name: Build
run: |
mkdir build
cd build
cmake .. -GNinja
ninja
- name: Bundle DLLs
shell: msys2 {0}
run: |
export MINGW_BUNDLEDLLS_SEARCH_PATH="/mingw64/bin;."
cd build
mingw-bundledlls.exe ./ngscopeclient.exe --copy
- name: Test
if: ${{ false }}
shell: msys2 {0}
run: |
ninja test
- name: Upload Artifacts
uses: actions/upload-artifact@v4
with:
name: glscopeclient-build-windows-${{ runner.os }}-${{ matrix.env }}-${{ github.job }}
path: build
# - name: Upload Artifacts
# uses: actions/upload-artifact@v4
# with:
# name: glscopeclient-windows-${{ runner.os }}-${{ github.job }}
# path: msys2/*.zst
# - name: Upload Artifacts (ngscopeclient portable zip)
# uses: actions/upload-artifact@v4
# with:
# name: ngscopeclient-windows-portable-${{ runner.os }}-${{ github.job }}
# path: build/dist/ngscopeclient*.zip
# - name: Upload Artifacts (ngscopeclient MSI)
# uses: actions/upload-artifact@v4
# with:
# name: ngscopeclient-${{ runner.os }}-${{ github.job }}.msi
# path: build/dist/ngscopeclient*.msi
Solution2
- You can extract the exe from the zip from latest version https://github.com/bvernoux/mingw-bundledlls/releases and execute it ... (there is no any external dependencies)
Solution3
- You can just rebuild the exe from source it is very basic with a Makefile (you can even integrate the source code in the project the license is MIT License so fully compatible ...)
Hi @bvernoux,
Thanks for the extensive input. I see some valuable input indeed, hoewever the main question if it is possible to pull in a mingw-bundledlls.exe from your releases is not answered. Do you have any dependencies that would prevent this?
Thanks John
Solution1
You can install the latest pacman package for mingw64/msys2
mingw-bundledlls-0.2.4-1-x86_64.pkg.tar.zstavailable in https://github.com/bvernoux/mingw-bundledlls/releases
- We already use Mingw64 so it is trivial to do that in https://github.com/ngscopeclient/scopehal-apps/blob/master/.github/workflows/build-windows.yml Something like that shall do the trick (to be cross checked/tested) Added step "Install mingw-bundledlls" & step "Bundle DLLs"
name: Build-windows on: push: pull_request: workflow_dispatch: env: VULKAN_SDK_VERSION: 1.3.275.0 jobs: Windows: runs-on: windows-latest defaults: run: shell: msys2 {0} strategy: matrix: include: - { sys: mingw64, env: x86_64 } - { sys: ucrt64, env: ucrt-x86_64 } - { sys: clang64, env: clang-x86_64 } steps: - uses: actions/checkout@v4 with: submodules: recursive fetch-depth: 0 - uses: msys2/setup-msys2@v2 with: update: true msystem: ${{matrix.sys}} install: >- git wget pacboy: >- cmake:p toolchain:p libsigc++:p cairomm:p yaml-cpp:p glfw:p catch:p vulkan-headers:p vulkan-loader:p shaderc:p glslang:p spirv-tools:p ffts:p - name: Install mingw-bundledlls shell: msys2 {0} run: | wget https://github.com/bvernoux/mingw-bundledlls/releases/download/v0.2.4/mingw-bundledlls-0.2.4-1-x86_64.pkg.tar.zst pacman -U mingw-bundledlls-0.2.4-1-x86_64.pkg.tar.zst --noconfirm - name: Build run: | mkdir build cd build cmake .. -GNinja ninja - name: Bundle DLLs shell: msys2 {0} run: | export MINGW_BUNDLEDLLS_SEARCH_PATH="/mingw64/bin;." cd build mingw-bundledlls.exe ./ngscopeclient.exe --copy - name: Test if: ${{ false }} shell: msys2 {0} run: | ninja test - name: Upload Artifacts uses: actions/upload-artifact@v4 with: name: glscopeclient-build-windows-${{ runner.os }}-${{ matrix.env }}-${{ github.job }} path: build # - name: Upload Artifacts # uses: actions/upload-artifact@v4 # with: # name: glscopeclient-windows-${{ runner.os }}-${{ github.job }} # path: msys2/*.zst # - name: Upload Artifacts (ngscopeclient portable zip) # uses: actions/upload-artifact@v4 # with: # name: ngscopeclient-windows-portable-${{ runner.os }}-${{ github.job }} # path: build/dist/ngscopeclient*.zip # - name: Upload Artifacts (ngscopeclient MSI) # uses: actions/upload-artifact@v4 # with: # name: ngscopeclient-${{ runner.os }}-${{ github.job }}.msi # path: build/dist/ngscopeclient*.msiSolution2
- You can extract the exe from the zip from latest version https://github.com/bvernoux/mingw-bundledlls/releases and execute it ... (there is no any external dependencies)
Solution3
- You can just rebuild the exe from source it is very basic with a Makefile (you can even integrate the source code in the project the license is MIT License so fully compatible ...)
@d235j what do you think, is this acceptable. I'm nudging towards yes, even though I'd prefer just an .exe and worry about cross-platform building. Have not thought about this project for a while though so may be thinking up issues that don't exist
