services-flake Darwin: inconsistencies between CI and Runtime

CI uses nix flake check, which calls the process-compose app wrapped inside pkgs.runCommand. The apps that run inside runCommand gets a few packages by default, for example readlink from coreutils. This behaviour could make the tests pass in CI but fail on runtime (when ran using nix run), if the service doesn’t pass those packages in its environment.

For more detailed analysis, see: https://github.com/juspay/services-flake/pull/164#issuecomment-2053876700

Apr 14 '24 04:04 shivaraj-bh

Is it just darwin? wouldn’t it also affect linux?

Edit: So far, there hasn’t been any issue reported about the inconsistencies happening on Linux but stdenv in Linux could also have packages which the host machine will not during runtime

Apr 14 '24 04:04 shivaraj-bh

Is this something to do with sandbox being disabled by default on darwin?

Apr 14 '24 08:04 srid

Is this something to do with sandbox being disabled by default on darwin?

No, it is to do with some packages (belonging to stdenv-darwin) being present in the runCommand’s environment (making the CI pass), while not being present in the environment while nix runing the service. Leading to regressions like:

https://github.com/juspay/services-flake/pull/163 https://github.com/juspay/services-flake/pull/164

Apr 14 '24 09:04 shivaraj-bh