aros-development-team
revert PR #710, fixes issue #709
LLVM toolchain build ok, problems later with duplicate symbols SysBase/DOSBase
Unfortunately I cannot accept this.
It changes the llvm build in ways that break the ability to deploy on nightly build servers - namely it reverts to needing > 10GB of space to build everything in one go.
looking at the logs, the current LLVM toolchain build is failing for the same reason: No space left on device
/home/vsts/work/1/b/builds/pc-x86_64-smp/bin/linux-x86_64/gen/host/tools/crosstools/llvm/tree/llvm/utils/TableGen/Common/CodeGenDAGPatterns.cpp:4834:1: fatal error: error writing to /tmp/ccGH2UUD.s: No space left on device
looking at the logs, the current LLVM toolchain build is failing for the same reason: No space left on device
/home/vsts/work/1/b/builds/pc-x86_64-smp/bin/linux-x86_64/gen/host/tools/crosstools/llvm/tree/llvm/utils/TableGen/Common/CodeGenDAGPatterns.cpp:4834:1: fatal error: error writing to /tmp/ccGH2UUD.s: No space left on device
Yes & No. It is a tight fit to get it to build, hence the need for that way of building it in the first place, however the buikd 'is' broken just now due to it being so tight.
This patch however doesnt fix this - it makes the situation worse and unfixable.
just to add some info, here is the disk usage after install-LLVM-stripped:
14G bin/linux-x86_64/gen/host/tools/crosstools/llvm/llvm
1.6G bin/linux-x86_64/gen/host/tools/crosstools/llvm/tree
and after clean:
162M bin/linux-x86_64/gen/host/tools/crosstools/llvm/llvm
1.6G bin/linux-x86_64/gen/host/tools/crosstools/llvm/tree
this is for nightly / MinSizeRel build
just to add some info, here is the disk usage after
install-LLVM-stripped:14G bin/linux-x86_64/gen/host/tools/crosstools/llvm/llvm
This is the problem. There is only 10G available on the build servers - so building LLVM in one pass is a no-go. I have spent many weeks of long builds tweaking this to make it fit on the servers, so undoing it is not an option.
The current failure is due to boost pushing it over 10G again and needs work to squeeze it on still - but that is a separate issue that this patch will not resolve (and as explained, instead this worsens the situation).
more info on this issue: I made a log of all the objects that are built after those 3 steps
- install-LLVM-stripped
- install-lld-stripped
- install-clang-resource-headers-stripped
seems that the same ~2000 objects are rebuilt over and over again. llvm-objs-list.zip
#MM
tools-crosstools-llvm-toolchain : $(LLLVM_EXTRA_DEPS)
ifneq (,$(findstring AROS_BUILD_TYPE_NIGHTLY,$(CONFIG_BASE_CPPFLAGS)))
$(Q)$(IF) ! $(TEST) -f $(llvm-installflag); then \
$(ECHO) "Building llvm/LLVM.so" && \
$(MAKE) -j$(JOBS) -C $(HOSTGENDIR)/$(CURDIR)/llvm LLVM $(crosstools-llvm-make-env) && \
$(MAKE) -j1 -C $(HOSTGENDIR)/$(CURDIR)/llvm install-LLVM-stripped && \
$(MAKE) -C $(HOSTGENDIR)/$(CURDIR)/llvm clean && \
$(ECHO) "Building llvm/lld" && \
$(MAKE) -j$(JOBS) -C $(HOSTGENDIR)/$(CURDIR)/llvm lld $(crosstools-llvm-make-env) && \
$(MAKE) -j1 -C $(HOSTGENDIR)/$(CURDIR)/llvm install-lld-stripped $(crosstools-llvm-install-env) && \
$(MAKE) -C $(HOSTGENDIR)/$(CURDIR)/llvm clean && \
$(ECHO) "Building llvm/Clang" && \
$(MAKE) -j$(JOBS) -C $(HOSTGENDIR)/$(CURDIR)/llvm clang $(crosstools-llvm-make-env) && \
$(MAKE) -j1 -C $(HOSTGENDIR)/$(CURDIR)/llvm install-clang-stripped $(crosstools-llvm-install-env) && \
$(MAKE) -j1 -C $(HOSTGENDIR)/$(CURDIR)/llvm install-clang-libraries-stripped $(crosstools-llvm-install-env) && \
$(MAKE) -j1 -C $(HOSTGENDIR)/$(CURDIR)/llvm install-clang-resource-headers-stripped $(crosstools-llvm-install-env) && \
$(MAKE) -C $(HOSTGENDIR)/$(CURDIR)/llvm clean && \
$(ECHO) "Building llvm tools" && \
$(MAKE) -j$(JOBS) -C $(HOSTGENDIR)/$(CURDIR)/llvm $(LLVM_TOOL_TARGETS) $(crosstools-llvm-make-env) && \
$(MAKE) -j1 -C $(HOSTGENDIR)/$(CURDIR)/llvm $(LLVM_TOOL_INSTTARGETS) $(crosstools-llvm-install-env) && \
$(ECHO) "Installing llvm-as" && \
cp $(HOSTGENDIR)/$(CURDIR)/llvm/bin/llvm-as $(LLVM_BUILD_BINDIR)/ && \
$(HOST_STRIP) $(LLVM_BUILD_BINDIR)/llvm-as && \
$(ECHO) "Installing llvm-config" && \
cp $(HOSTGENDIR)/$(CURDIR)/llvm/bin/llvm-config $(LLVM_BUILD_BINDIR)/ && \
$(HOST_STRIP) $(LLVM_BUILD_BINDIR)/llvm-config && \
$(ECHO) "Installing llvm cmake files" && \
$(MKDIR) $(CROSSTOOLSDIR)/lib/cmake/llvm && \
cp $(HOSTGENDIR)/$(CURDIR)/llvm/lib/cmake/llvm/*.cmake $(CROSSTOOLSDIR)/lib/cmake/llvm/ && \
$(MAKE) -C $(HOSTGENDIR)/$(CURDIR)/llvm clean && \
$(TOUCH) $(llvm-installflag); \
fi
else
$(Q)$(IF) ! $(TEST) -f $(llvm-installflag) ; then \
$(MAKE) -j$(JOBS) -C $(HOSTGENDIR)/$(CURDIR)/llvm clang lld $(crosstools-llvm-make-env) \
&& $(MAKE) -j1 -C $(HOSTGENDIR)/$(CURDIR)/llvm install/strip $(crosstools-llvm-install-env) \
&& $(MAKE) -C $(HOSTGENDIR)/$(CURDIR)/llvm clean \
&& $(TOUCH) $(llvm-installflag) ; \
fi
endif
Firstly, the first set of rules that build it in stages are only for nightly builds - targeted specifically at building on the nightly build servers. You have removed that which breaks the build completely on the nightly servers for the reasons given.
Secondly, this has been tested - it successfully builds the toolchain on the servers when done this way, but does not when done in a single pass. It only works with the default llvm toolchain, and needs more work for the newer toolchain - but regardless it needs done in stages on the servers or it cannot be built.