How to use fastpm with nbodykit

[fire-bot]

Sent by Andreas Finke (@andipotal). Created by fire.

---

Hi!

I'm a new PhD student in cosmology at the Univerity of Geneva. As your software makes a very professional impression, I would like to use it.

My goal now is to modify fastpm for the cosmology I'm interested in, and for tests I would like to run it in the nbodykit framework.

I installed nbodykit, and the simple examples run. Further, I managed to compile fastpm on my macbook pro

(I had to "fix": 1) stop clang complaining about converting the value for .K_LINEAR = 0.04 to int in basically all tests 2) installing openmpi and gsl with homebrew, pointing the path to this mpicc instead of the anaconda one)

But now I simply don't know how to include the result into nbodykit such that the example from the paper can be run, i.e.

from fastpm.nbkit import *

matter = FastPMCatalogSource(linear, Nsteps=10)

in a jupyter notebook - not for any standard precompiled fastpm and even less so for my own compilation.

Does this have anything to do with this "Docker" thing?

Thanks for your help and with best regards

Andreas Finke

[andipotal]

(can I (or someone else) edit this to strip off the personal details? Wasn't aware that sending an email would create a public post on github)

[nickhand]

Thanks for raising this issue, @andipotal...I've been meaning to add a more complete explainer on FastPM + nbodykit to the docs. How did you install fastpm? Using the C library or the Python version? If you have an Anaconda environment set-up, you should be able to do

conda install -c bccp nbodykit fastpm

which will install nbodykit as well as the Python version of FastPM from https://github.com/rainwoodman/fastpm-python. From there you should be able to follow along with the example from the paper.

P.S. I think you should be able to edit these posts to remove any personal details...right now it lists your name and GitHub username. Let us know if you want us to edit any of those details!

[andipotal]

Thanks a lot for the quick help! Indeed, I stared at the C implementation of fastpm. I was not aware of the python one. Installing the latter makes things work immediately!

Am I seeing this correctly that I have the source code now in

anaconda3/pkgs/fastpm-0.0.6-py36h50f55b0_1/lib/python3.6/site-packages/fastpm

so that I could just make modifications right there and would expect them to become active immediately?

BTW, is there a noticable performance drop for smaller simulations (e.g. 512^3) going from C to python for the nbody solver? I suppose most time is spent in the fft's anyway? EDIT: memory-wise, I'd also be interested if C provides an advantage. I currently see a 512^3 sim taking >30 Gb in python)

(Regarding the "personal details", I meant in particular the first sentence of my mailed message, which could indeed be deleted. I cannot edit that first message here, it seems. However, I leave it to you to decide this :) )

[rainwoodman]

We were usually running FastPM on many many cores, such that the memory was not an issue. The memory usage of the C version was once understood very well (when the paper was written), but I suspect over the time it has been slightly going up.

I can look into the memory issue in either versions of FastPM; if the 30GB is a bottleneck for your application. I think there is some space for quick improvements:

-- In the Python version, the default boost factor of 2 requires at least 3 meshes of 1024^3 in memory at the same time, taking about 12GB, then 512^3 particles each about 40 ~ 60 bytes (velocity, position, initial position, and perhaps ID) which is 5~7G. This only adds up to about 20GB. So we can probably squeeze about 30% ~ 40% without doing too much work.

[rainwoodman]

We never did a head to head benchmark over the C and the Python version. The two codes share not only the FFT library, but also the the particle mesh resampling code. Therefore I do expect them to be very similar.

for development, you can get the source code by cloning the fastpm-python github repo (rainwoodman/fastpm-python). From there you can modify the code and use run-tests.py to check if the modification breaks it. You can also install from the source tree in development mode with pip install -e.

[andipotal]

Thanks again - awesome support on a Sunday! :)

I can look into the memory issue in either versions of FastPM; if the 30GB is a bottleneck for your application.

As 256^3 sims run in only 5 minutes on the macbook, I was hoping to get results for 512^3 there as well within an hour or so. Now, 20 GB is still more than the 16 GB available this is perhaps not the best approach. Let's say it would be interesting to try, but it is not pressing.

I do have access to a small cluster. If memory is turning out to be an issue there, I'll get back to you.