Polygeist icon indicating copy to clipboard operation
Polygeist copied to clipboard

Memory run out problem

Open jhbi826c opened this issue 3 years ago • 7 comments

When I tried to build this repo as the README file, it always use out all of my memory. Even When I allocate another 100GB swapfile when ninja is using all the cores to run the build. For this kind of issue, I highly recommand here for anyone who wants to build this to speicify clang as the compiler using -DCMAKE_C_COMPILER and -DCMAKE_CXX_COMPILER and lld as the linker. Hope this will help.

jhbi826c avatar Jan 10 '23 11:01 jhbi826c

Yes, maybe we can submit a pull...

hushanjushi avatar Jan 11 '23 10:01 hushanjushi

Well, it's better using these with any project which takes llvm-project as a sub-module. Also if one wants to build llvm itself.

jhbi826c avatar Jan 11 '23 10:01 jhbi826c

Well, it's better using these with any project which takes llvm-project as a sub-module. Also if one wants to build llvm itself.

There maybe someone without Clang wants to build clang. So I think use comments just like mlir build process is fine...

hushanjushi avatar Jan 11 '23 10:01 hushanjushi

Well, it's better using these with any project which takes llvm-project as a sub-module. Also if one wants to build llvm itself.

There maybe someone without Clang wants to build clang. So I think use comments just like mlir build process is fine...

Actually it's ok, anyone can use a pre-built clang binary provided by llvm under lots of linux distribution to build the clang inside the llvm-project. Say you want to build the latest version of clang, you can use a pre-built clang-14 or whatever. It's kinda weird but it works haha;)

jhbi826c avatar Jan 11 '23 10:01 jhbi826c

Most of the memory is consumed by the linker. Changing the compiler is unlikely to have big impact, but changing the linker will. lld has a lower memory footprint, but there are two major cmake flags that help overall:

  • -DCMAKE_BUILD_TYPE=Release or -DCMAKE_BUILD_TYPE=RelWithDebInfo if you need debug symbols, debug builds consume a lot more when linking
  • -DLLVM_PARALLEL_LINK_JOBS=1 prevents the build system from running multiple linkers in parallel, each of which consumes memory.

ftynse avatar Jan 31 '23 17:01 ftynse

Most of the memory is consumed by the linker. Changing the compiler is unlikely to have big impact, but changing the linker will. lld has a lower memory footprint, but there are two major cmake flags that help overall:

  • -DCMAKE_BUILD_TYPE=Release or -DCMAKE_BUILD_TYPE=RelWithDebInfo if you need debug symbols, debug builds consume a lot more when linking
  • -DLLVM_PARALLEL_LINK_JOBS=1 prevents the build system from running multiple linkers in parallel, each of which consumes memory.

Thanks! That's why I'm using lld wherever there is llvm ;)

jhbi826c avatar Jan 31 '23 18:01 jhbi826c

Thanks, I will try. Building it is so slow and my memory is exhausted.

hm123450 avatar Dec 16 '23 08:12 hm123450