roc-toolkit Minimal MinGW build for Windows

Last revised: Oct 2023

We have #132 to track efforts related to Windows porting. We've already fixed a few minor issues with SCons on Windows.

This sub-task is the first significant step towards Windows support: make minimal Roc configuration building with MinGW:

scons --host=x86_64-w64-mingw32 --compiler=gcc --disable-lib --disable-tools --disable-doc --disable-openfec --disable-sox --disable-libunwind --disable-pulseaudio

(The exact --host may differ.)

See also Dependencies and SCons options.

To achieve this, we should add a windows platform to SConstruct. As with other platforms, we should auto-detect it from --host. Currently supported platforms are linux, darwin, and android.

The selected platform defines what target directories are enabled.

We can enable the following target directories on Windows / MinGW:

target_pc
target_c11
target_libuv
target_nobacktrace
target_nodemangle

If some code from these directories will fail to build on MinGW, we will have to fix it.

The following target directories, most likely, can't be enabled:

target_posix
target_posix_ext
target_posix_pc

We will have to create target_windows, enable it on Windows, and fill it with Windows implementations of functions and classes from target_posix and target_posix_ext.

We don't have to implement all of them in this task. Some of them are not needed because they are used only on POSIX systems. Some of them are not needed on minimal build. And some of them are needed, but may be no-op on Windows.

A reasonable approach would be to run compilation, see what's missing, and add corresponding implementations.

We will also have to fix any issues with SCons running on Windows, if they appear.

BTW, MinGW toolchain is available on Linux. If it makes sense, we can start from fixing MinGW build on Linux first, and then try to run it on Windows.

After finishing this task, we should also add minimal MinGW build to CI.

Dec 05 '19 09:12 gavv

scons --disable-shared --disable-tools --disable-soversion --disable-openfec \
  --disable-speexdsp --disable-sox --disable-sndfile --disable-openssl --disable-libunwind \
  --host=x86_64-w64-mingw32 --build-3rdparty=libuv roc_core

Jul 12 '24 12:07 gavv

Good morning everybody, and specially gavv.

First, thanks for this excellent library, i'd love to see this protocol and toolkit become an universal cross-platform standard.

I've done some tests, just to see how easy or hard it would be to have a minimalist version of the sender run on Windows and send to a Linux sound server.

Didn't manage to get SCons working on MinGW, always failing to find ar, in the PATH and whatever i tried to add to SConstruct, so i went for a Makefile written from scratch.

But i've managed to compile a minimalist working roc-send.exe (no RS8M support yet), by a quick-n-dirty method of linking all required objects statically, yet still having all source changes into a separate mingw subdirectory, to not mess with original souce, with the goal of porting roc-recv too and eventually building roc-toolkit as a library for MinGW and properly linking from the tools.

Tests were done from Commit 9c65791faec0aab3e24fd30b80194480f9106a4d of 2025-04-29

Changes required to source code were pretty marginal (well, for minimalist sender-only), boils down to: (kept all modified files in the mingw directory and its inc_local subdir)

socket_addr{.h,.cpp},socket_ops.cpp: Adaptations to use Winsock2 for networking

main.cpp, at the beginning of main(): #ifdef WIN32 WSADATA wsaData; WSAStartup(MAKEWORD(2, 2), &wsaData); #endif

Define _USE_MATH_DEFINES (or in the Makefile) else M_PI isn't defined on Windows (fast_random.cpp, builtin_resampler.cpp)

time.cpp: add && !defined(WIN32) to conditionnals at 65 and 93: #if defined(CLOCK_REALTIME) && !defined(APPLE) && !defined(MACH) && !defined(WIN32) #if defined(CLOCK_REALTIME) && defined(TIMER_ABSTIME) && !defined(WIN32)

Else it compiles ok but at runtime panic()s with Error 4294967295 when sleep_until or sleep_for is called. (quick Google serach looks like it's an error with the Windows Subsystem for Linux, might try to see the best recommended way to wait nanoseconds with Windows API, but for my quick tests the existing second implementations already worked flawlessly).

For some reason i can't attach the archive so i've uploaded it at: ** https://www.jmd-tech.com/public/roc-toolkit-mingw_build.tgz **

Glad to have any feedback, specially for either help on having the SCons thing work in MinGW, or some guidelines on how to have those modifications follow roc-toolikt code structure for later clean integration/merge. (obviously still much to do as this port is yet a mere test)

Jun 03 '25 01:06 JMD-tech

Hi, thanks a lot for looking into this!

This experiment looks very promising. Changes in .tgz look fine too.

Glad to have any feedback, specially for either help on having the SCons thing work in MinGW

Sure, I'm happy to help!

I've pushed a few commits to start with:

Allows mingw build with scons, at least on my machine :)

529d7c934bfd50d4cd3b2a3ae0c15df2ea077bf8

Please let me know if it works for you.

I'm runnning this command:
```
scons --disable-shared --disable-tools --disable-soversion --disable-openfec \
  --disable-speexdsp --disable-sox --disable-sndfile --disable-openssl --disable-libunwind \
  --host=x86_64-w64-mingw32 --build-3rdparty=all
```
For now I've tested it on Linux (using mingw-w64 package in Debian). Let me know if you're working from Windows, I try it there too.
Add windows-specific defines and libs

4d47943ff9f89a037c7b19ffeb4d0a25c0bbef40

Ported defines & libs from your Makefile + added a few more.
Move sockets code from target_posix to target_berkley

786dcb87fde6ee9f2ffc711ace5abdf8fac0db53

target_posix is for unix-like systems, but since socket code will be very similar to windows, there is now a separate target dir for OSes with BSD-like sockets
Reimplemented a few files from roc_core/target_posix for windows

d872a86a187ec3c9dfa7f30cef9fa913a601f0a1

(see below)

In your makefile I see that you use mutex/cond/sem/etc from winpthreads from mingw. In the end I'd like to avoid winpthreads and instead disable target_posix and target_posix_ext on windows and craft alternative implementations in target_windows using winapi directly.

This way windows code would be the same regardless whether we use mingw or msvc (in future), and we'd avoid an unnecessary wrapper and risk of additional bugs or performance issues. (pthreads have quite rich API and I guess properly emulating it is hard and has costs).

So I added target_windows to scons and for starters implemented a few files - some macros, mutex, cond, and errno_to_str.

Here is the list of remaining files to reimplement from target_posix:

./src/internal_modules/roc_core/target_posix/roc_core/crash_handler.cpp
./src/internal_modules/roc_core/target_posix/roc_core/die.cpp
./src/internal_modules/roc_core/target_posix/roc_core/singleton.h
./src/internal_modules/roc_core/target_posix/roc_core/thread.cpp
./src/internal_modules/roc_core/target_posix/roc_core/time.cpp
./src/internal_modules/roc_core/target_posix_ext/roc_core/semaphore.cpp
./src/internal_modules/roc_core/target_posix_pc/roc_core/console.cpp
./src/internal_modules/roc_dbgio/target_posix/roc_dbgio/temp_file.cpp

Some notes:

crash_handler may be no-op on windows
I think the best way to implement time.cpp is to use QPC + WaitableTimer APIs (see also related discussion and implementation in Golang: 1, 2)
semaphore can be implemented with windows CreateSemaphore + an atomic for happy path, one good example I know can be found here (we don't need waitMany, so we can make implementation much simpler than the linked one)

WSAStartup(MAKEWORD(2, 2), &wsaData);

I suggest to hide this call behind a function, say, socket_init(), and invoke it in node::Context::Context rather than from main().

This way, it will be called both when we're in a CLI tool and are used as a C library. roc_context is the first thing that the user of the library creates, and I think it's a good place to do initialization.

And I guess we should also add socket_deinit() that calls WSACleanup() and invoke it from Context destructor.

So what sub-tasks I see here:

Merge your updates for socket_addr and socket_ops
Reimplement target_posix files
See what else stops us from successful build :)

I'd appreciate if you can send your changes for (1) in a separate PR.

You're welcome to work on (2) and send pull-request(s) for that stuff. (Maybe semaphore.cpp and time.cpp deserve separate PRs - would be easier to review).

If you'd like to concentrate on (3) instead, you could temporarily copy missing implementations from target_posix to target_windows and rely on winpthreads. Then we can return to these implementation later, e.g. when porting to msvc.

In general, I'd suggest incremental approach when we review and merge PRs one by one until we get working port. In the end we can add it to CI.

Also let me know if you need any further help with the build system.

Jun 09 '25 05:06 gavv

Thanks for all those, perfectly answers.

Obviously having a new target directory feels much more comfortable without risk of breaking things in another target.

I was building on Windows, but to have a closer environment now i've installed mingw-w64 on my Debian 12 VM, first trying to build here and eventually try again on Windows when mingw build is ok on Linux.

About libpthread, used those because it was the quickest way to test and also out of habit with it, but agree that for performance critical code it deserves a native implementation. While i'm relatively familiar with pthread which i already used for production codes on both platforms, with Windows threads API i've only tested some tutorial codes something like 15 years ago but remember it didn't feel specially difficult so i'll give it a try. Though used only mutexes and never semaphores, but learning is cool :)

About MSVC compatibility, went first with MinGW as an 'intermediate step', as my ultimate goal would be a Windows virtual sound card driver to connect to multi-room sound servers, i guess Windows driver development being much easier with MSVC than MinGW so really makes sense not to rely on posix for Windows build.

Was about to ask for a suggestion on where better place to put the WSAStartup, perfect in the node context ctor/dtor as it'll be totally 'transparent' to the library user.

Going for route (1) (3) (2) as it will be much easier if i manage to use the official scons build rather than rely on my various quick temporary hacks when working on (2).

(1) will very probably have some more minor modifications later to solve the -fpermissive issue (see below).

Here's what i've done yet to try to build for MinGW on my Debian VM:

fork roc-toolkit with all branches, then clone then git checkout develop

modifications:

First a trivial update of libatomic_ops URL, was doing a 404, seems like they moved the location of the releases archives, sending PR now.
Then temp copy of singleton+thread+semaphore+console from target_posix, and ifdefs in socket_addr
Temporarily adding -fpermissive to C/CXXFLAGS:

Doesn't build without it yet, and my first Makefile port already outputted tons of warnings related to that. When getting a full build i'll grep and sort all those to confirm what it looks like, mostly casts of (size_t*) and passing of file and socket descriptors. At first was wary of modifying many source files with risk of impacting working tagets, but i believe that now as intended with the new target, it would only concern files in target_windows and berkeley.

fixing some error: ‘errno_to_str’ is not a member of ‘roc::core’ by explicitly including errno_to_str.h in wav_sink and wav_source.cpp Strangely, this wasn't happening with the Linux and other builds, only MinGW one. (edit: looks like MacOS one too, as the CI build was failing before and now succeeds with this).

Going to read some docs on how to include specific commits into a separate PR and sending this one too.

Meanwhile added yet another ifdef in socket_addr.cpp, pushing it into current pending PR as it belongs with (1) already in: https://github.com/roc-streaming/roc-toolkit/pull/817

Now continuing progress on (3)

BTW, before reading your first reply, i managed to have the roc-send.exe work with RS8M/OpenFEC with the Makefile build, but i'll postpone this for after those 3 priority items.

Jun 11 '25 16:06 JMD-tech

Now MinGW build compiles without using -fpermissive, also tested Linux build and a roc-receive+roc-send test still works. To sum up what i've changed:

some console outputs from %lu to %llu and ull casts
payload alignment checks: first changed the cast to unsigned long long, worked but wondered if this would have some impact on performance for some 32bit platforms (older Raspberry pi's, and i suppose calculation of a 64 bit modulo is really overkill for values in the order of magnitude of small buffers and UDP datagrams) so went for a size_t cast instead.
And a bit trickier part with libuv:

According to this: https://github.com/libuv/libuv/issues/200#issuecomment-74080840 Explains why my first tests with -fpermissive were working, but also that it was begging to fail if/when Windows API starts allocating full 64b socket handles. So i needed to get the socket handle from libuv. udp_port.cpp uses libuv's uv_fileno function to retrieve fd, but there's not yet a similar libuv func to get the socket handle (checked latest upstream). So went doing this (according to some other forum source it was a similar way to get the fd before they implemented uv_fileno function):

#ifndef __WIN32__
    const int fd_err = uv_fileno((uv_handle_t*)&handle_, &fd_);

    if (fd_err != 0) {
        roc_panic("udp port: %s: uv_fileno(): [%s] %s", descriptor(), uv_err_name(fd_err),
                  uv_strerror(fd_err));
    }
#else  // __WIN32__
    fd_ = handle_.socket;
#endif // __WIN32__

Works but isn't really satisfying, i think the ugliness of both having to peek a value from the internals of an included library, and having a ifdef WIN32 in target_libuv could be solved by adding some uv_sockno function to libuv, will try to PR them this solution.

Yet i suspect this'd affect only MinGW build, MinGW being this "somewhat in between WIN32 API and posix" and keeping fd's as signed int, while native Windows implementation having 64bit unsigned file handles, thus being compatible with Windows socket handles.

BTW something i noticed when added more CPUs to my D12 VM to speed-up builds, MinGW build fails at something related to libuv with make -j4 (i suppose any>1), but builds ok once i went back to 1 CPU for the VM.

Now going to add the socket_init/deinit in node ctx ctor/dtor, and try to cross-compile roc-recv and roc-send.

Jun 18 '25 22:06 JMD-tech

OpenFEC / MinGW

This isn't yet priority, but i thought might be of interest, as OpenFEC compilation in my early Makefile test was just one single line of code to change, in lib_common/of_types.h, line 66 to compile for MinGW:

#ifndef UINT64
#ifdef WIN32
#define		INT64		__int64
#define		UINT64		__uint64
#else  /* UNIX */
#define		INT64		long long
#define		UINT64		unsigned long long
#endif /* OS */
#endif /* !UINT64 */

to: (Why OpenFEC doesn't use uint64_t ?)

#ifndef UINT64
#if defined( __int64 ) && defined( __uint64 )
#define		INT64		__int64
#define		UINT64		__uint64
#else  /* UNIX */
#define		INT64		long long
#define		UINT64		unsigned long long
#endif /* OS */
#endif /* !UINT64 */

Not yet read about the differences in the local copy of OpenFEC vs upstream but thought as the way it's packaged it would matter to have it fixed in the local package before waiting to be fixed in upstream?

Attaching OpenFEC's build log from trying cross-compile without --disable-openfec, i don't get this explicit error about __int64 not defined but might be linked, BTW with MinGW on Windows, had to do an explicit make openfec in the Makefile, as mere make after OpenFEC's cmake was failing trying to compile some tools/tests.

build.log

Jun 19 '25 01:06 JMD-tech

About MSVC compatibility, went first with MinGW as an 'intermediate step', as my ultimate goal would be a Windows virtual sound card driver to connect to multi-room sound servers

Very interesting, are you thinking about making a miniport driver?

Going for route (1) (3) (2) as it will be much easier if i manage to use the official scons build rather than rely on my various quick temporary hacks when working on (2).

👍

payload alignment checks: first changed the cast to unsigned long long, worked but wondered if this would have some impact on performance for some 32bit platforms (older Raspberry pi's, and i suppose calculation of a 64 bit modulo is really overkill for values in the order of magnitude of small buffers and UDP datagrams) so went for a size_t cast instead.

Agree, size_t fit better there.

BTW something i noticed when added more CPUs to my D12 VM to speed-up builds, MinGW build fails at something related to libuv with make -j4 (i suppose any>1), but builds ok once i went back to 1 CPU for the VM.

Can you please replace ctx.prefer_cmake = bool(args.macos_platform or args.android_platform) with ctx.prefer_cmake = True in scripts/scons_helpers/build-3rdparty.py, do a full clean, and see if this helps?

I think we should set prefer_cmake when building for windows anyway, but haven't tested it yet with mingw.

Not yet read about the differences in the local copy of OpenFEC vs upstream but thought as the way it's packaged it would matter to have it fixed in the local package before waiting to be fixed in upstream?

Unless things have changed, the upstream is dead, and distros use our fork when they package openfec. At least I wasn't able to upstream our patches so far.

Attaching OpenFEC's build log from trying cross-compile without --disable-openfec, i don't get this explicit error about __int64 not defined but might be linked, BTW with MinGW on Windows, had to do an explicit make openfec in the Makefile, as mere make after OpenFEC's cmake was failing trying to compile some tools/tests.

First guess is that cmake needs some help to use mingw. Probably we should add -DCMAKE_SYSTEM_NAME=Windows and maybe other defines. But I didn't test it yet.

Jun 21 '25 06:06 gavv

Thanks for update, looks neat!

I've submitted review in the PR and answers to the question above, let me know if I've missed something.

Jun 21 '25 06:06 gavv

About MSVC compatibility, went first with MinGW as an 'intermediate step', as my ultimate goal would be a Windows virtual sound card driver to connect to multi-room sound servers

Very interesting, are you thinking about making a miniport driver?

I didn't knew miniport drivers could implement sound card, thought they were only for network interfaces, will look into that later when i go back to Windows development.

As i was initially searching for virtual sound card driver what i thought of some code base/samples were:

ESD sound driver: mostly obsolete, was from W2K/XP era
The WDM driver for scream: to what i read WDM API is still supported but not recommended, and IIRC scream implements only one device per driver (and only stereo, no multi-channel): https://github.com/duncanthrax/scream
A Windows sample, still WDM but allowing multiple devices: https://github.com/microsoft/Windows-driver-samples/tree/main/audio/sysvad
A recent ACX sample, seems not multi device but latest API: https://github.com/microsoft/Windows-driver-samples/tree/main/audio/Acx/Samples

The ideal would be a mix of the two last ones, ACX multi-device, and being able to add/remove new virtual sound cards from some IPC calls:

First for testing with a cli tool
Then maybe some end-user GUI
Eventually either some dedicated discovery service, or if possible plugin/tapping into WSD discovery (or some other like zeroconf) for automatic connection to audio server(s) present on the network.

Have to admit i'd only quickly studied those for feasability, was initially going for development of a Windows driver for either pulseaudio or pipewire network protocol, then decided to dig a bit on different available protocols, found roc-toolkit and thought "that's exactly what i want" and here i am now :)

BTW something i noticed when added more CPUs to my D12 VM to speed-up builds, MinGW build fails at something related to libuv with make -j4 (i suppose any>1), but builds ok once i went back to 1 CPU for the VM.

Can you please replace ctx.prefer_cmake = bool(args.macos_platform or args.android_platform) with ctx.prefer_cmake = True in scripts/scons_helpers/build-3rdparty.py, do a full clean, and see if this helps?

I think we should set prefer_cmake when building for windows anyway, but haven't tested it yet with mingw.

My mistake, it was the VM not having enough RAM and the OOM-killer triggering, i suppose cross-compile for MinGW to be more RAM-hungry than native Linux. BTW tested with prefer_cmake = True and it fails even with 1 CPU, so the original build is fine So now it works fine with -j4 without changing anything.

Not yet read about the differences in the local copy of OpenFEC vs upstream but thought as the way it's packaged it would matter to have it fixed in the local package before waiting to be fixed in upstream?

Unless things have changed, the upstream is dead, and distros use our fork when they package openfec. At least I wasn't able to upstream our patches so far.

Ok, for testing i had uploaded a modified copy of the OpenFEC .tar.gz to my webhost and changed the URL in 3rdparty/SConscript to use this one.

But for now i'll focus on the modifications for the PR review first.

Jun 25 '25 20:06 JMD-tech

Hello,

Resuming my works on this subject, with those two new commits i've pushed into the PR:

Solved the last warning on MinGW cross-compile

(Wsign-conversion on IN_MULTICAST in socket_addr.cpp) by doing:

#ifndef ROC_TARGET_WINDOWS
#define IN_MULTICAST_U(i) IN_MULTICAST(i) 
#else
// Defined as (((long)(i) & 0xf0000000) == 0xe0000000) in Windows, causes Wsign-conversion
#define IN_MULTICAST_U(i) (((unsigned long)(i) & 0xf0000000u) == 0xe0000000u)
#endif

Or maybe i should rather do a whole SocketAddr::is_multicast() specific for ROC_TARGET_WINDOWS ?

Added a time.cpp using exclusive WIN32 API

First did some tests here to ensure the accuracy was at least better than what i had previously. ...rather easily, as nanosleep on Windows has terrible accuracy, High-res timers were way better, here are the details of the timing tests: https://github.com/JMD-tech/timetest Yet, windows accuracy for waits is rather disappointing, but as it already worked with MinGW nanosleep, and HR timers are way better... (but what i tested was sender only, maybe the receiver code would be much more sensitive about timings accuracy, for synchronization with output hardware?)

Pushed into the PR a very simple working version, not yet implemented some more stuff like:

Two-phase wait, with first an HR timer with slight undershoot, and completing by a very small amount of busy-wait, at a first glance it obviously gives superb accuracy (on idle machine), but as my time tests shows it crumbles under load to become one of the least accurates, so had very poor consistency and discarded the idea.
Fallback to Normal resolution timer if HR not supported, if we need to support Windows versions prior to Windows 10 1803, just 2 lines of code but not sure if it's a good idea.

Now the easiest one is done, moving to the other necessary files in target_windows.

Oct 04 '25 19:10 JMD-tech