ocaml
repeat 'dune exec/build's are removing build artefacts (.cmt files)
using v2.9.1 running dune build x/y.exe twice, back to back and without altering the source, results in some .cmt files being removed but not all. other types of build artefact remain.
as I'm developing a tool that relies on the presence of .cmt files this behaviour is bad news. worse, it is confusing. assuming it isn't a bug, why is this happening?
why is this happening?
It is happening because of dune likes to delete stale artifacts. This is important to make sure that rules don't accidentally read files they're not supposed to. For example, some leftover .cmi file from a previous build should not interfere with the current build.
There's no way to disable this feature and it's unlikely we'll add such an option. What is producing these .cmt files that are being deleted? Usually this is an issue of a rule not declaring all the targets that it's producing. However, in the case of .cmt files, dune indeed declares .cmt files to be targets. So they shouldn't be deleted.
What I'm seeing is, after a successful build, another 'dune build ...' will result in a .cmt being removed, even though there is nothing to be done as such.
Given that dune knows about .cmt files for rules purposes does the behaviour I'm describing not look odd? When nothing needs rebuilding what house-keeping is necessary? Build artefacts should rightly be cleared away in preparation for rules being run but what rules would need running after a clean build?
I was previously unable to anticipate when deletions would occur, but have noticed one invocation that does reliably demonstrate this is [dune build @install] You stated that .cmt are covered by the rules so disappearing .cmt files is a bug , no?
The behavior your describe indeed seems odd. Dune should not delete any cmt files because it knows they are targets.
Do you have a test case we could look at?
could you first try this on any project you have at hand: o build afresh some library within the project directly dune build x/y/z.cma o find _build/default -name *.cmt o run dune build @install o re-run find and compare
it is happening generally unpredictably for non-install development (at least I haven't noticed the pattern yet) but the build @install always seems to cause removals.
I just tried reproducing this and wasn't successful. You need to give us a minimal example that reproduces the issue you're observing before we can make progress.
a minimal example codeberg.org/progman/dunebug
dune build ./ok.cmxa find _build -name *.cm* _build/default/ok.cmxa _build/default/.ok.objs/native/ok.cmt _build/default/.ok.objs/native/ok.cmx _build/default/.ok.objs/byte/ok.cmo _build/default/.ok.objs/byte/ok.cmi _build/default/.ok.objs/byte/ok.cmt dune build ./ok.cmxa find _build -name *.cm* _build/default/ok.cmxa _build/default/.ok.objs/native/ok.cmx _build/default/.ok.objs/byte/ok.cmo _build/default/.ok.objs/byte/ok.cmi _build/default/.ok.objs/byte/ok.cmt
the native/ok.cmt file has disappeared
I am unable to reproduce the issue. The first dune build never gives me the first native/ok.cmt. What version of Dune/OCaml are you using?
Here is my test case:
https://github.com/ocaml/dune/pull/8813
I'm reopening this until we get to the bottom of the issue.
can this be re-opened now a minimal example has been stumbled across?
I'm unable to reproduce your example. The first command does not produce a native/ok.cmt. In fact, dune always produces bin annot files only with the bytecode
Did you muck about with OCAMLPARAM perhaps?
yes! I have OCAMLPARAM=_,bin-annot=1 in my .bashrc. I didn't know dune was doing it anyway, let alone specifically for bytecode only. I suppose on the 2nd build cmd dune sees the native/ok.cmt as an unknown artefact and deletes it.
perhap dune ought to become aware of the various envars influencing the compilers. it already parses ocamlc_flags and friends, no?
I suppose we could. It's quite hard to do a thorough job here as quite a bit can be configured via OCAMLPARAM.
copy the ocaml source? in any case thanks for sorting my problem. I'll remove my global OCAMLPARAM and rely on dune.