es-shell icon indicating copy to clipboard operation
es-shell copied to clipboard

`make -j` explodes, sometimes

Open jpco opened this issue 2 years ago • 5 comments

es is quick enough to build that this doesn't seem terribly material, but it's likely a sign of a missing dependency link somewhere in the Makefile that should be fixed up. It seems more likely if optimization is off, like -O0.

The errors seem to all be around y.tab.c.

jpco avatar Feb 05 '24 23:02 jpco

Okay, as soon as I filed this it stopped happening. I'm gonna leave this open for now to see if I can figure out how to get it to start happening again. It really was just as simple as

; make clean; make -j

and about one in four builds would fail catastrophically while trying to compile y.tab.c.

jpco avatar Feb 06 '24 03:02 jpco

Aha, I'm getting repros again! Yay!

The failures all happen while trying to produce y.tab.o. I notice in the Makefile there is no config line referring to dependencies of y.tab.o -- I suspect it may be trying to start compiling y.tab.c too early due to a missing dependency statement.

Experimentally, it seems like adding the line

y.tab.o : y.tab.c y.tab.h 

stops the errors happening. But this is all weird and probabilistic behavior so I'd love to find some documentation that actually helps me know what the Right Thing to do here would be.

jpco avatar Feb 07 '24 17:02 jpco

Here's my best guess right now as to what's going on:

make doesn't know that a single yacc invocation produces both y.tab.c and y.tab.h. So what happens is, if make tries to get at y.tab.c it'll run yacc, and then if it also wants y.tab.h before yacc is done the first time, it'll run yacc again, re-generating both. And then, if make starts trying to build y.tab.o while y.tab.c is being re-generated, it'll blow up due to whatever weird write race going on.

This would explain both why yacc runs twice when make is invoked with -j but not without, and why adding y.tab.h as a dependency for y.tab.o fixes the crashes despite y.tab.h not having anything to do (directly) with actually compiling y.tab.c.

Turns out GNU make 4.3 added a feature called "grouped targets" for just this scenario! That makes me more confident this is what's happening here.

Unfortunately that feature isn't very portable, so unless we expect everybody to use a recent-ish GNU make to build es, it might be better to just specify y.tab.o : y.tab.c y.tab.h and allow yacc to get run twice, sometimes.

jpco avatar Feb 08 '24 06:02 jpco