2 identical module installations produces different files

[ronhanson]

Hi, I have an issue with the fact the lib thread_aio.cpython-312-darwin.so file differs for each installation made. I explain :

mkdir -p ./requirements1 ./requirements2
pip install --disable-pip-version-check --break-system-package --no-python-version-warning \
    --isolated --root-user-action ignore --upgrade \
    caio==0.9.24 -t ./requirements1/

pip install --disable-pip-version-check --break-system-package --no-python-version-warning \
    --isolated --root-user-action ignore --upgrade \
    caio==0.9.24 -t ./requirements2/

Both calls being the same, it should produce the same result. Except that it does not.

diff -rq requirements1 requirements2    

Files requirements1/caio/__pycache__/__init__.cpython-310.pyc and requirements2/caio/__pycache__/__init__.cpython-310.pyc differ
Files requirements1/caio/__pycache__/abstract.cpython-310.pyc and requirements2/caio/__pycache__/abstract.cpython-310.pyc differ
Files requirements1/caio/__pycache__/asyncio_base.cpython-310.pyc and requirements2/caio/__pycache__/asyncio_base.cpython-310.pyc differ
Files requirements1/caio/__pycache__/linux_aio_asyncio.cpython-310.pyc and requirements2/caio/__pycache__/linux_aio_asyncio.cpython-310.pyc differ
Files requirements1/caio/__pycache__/python_aio.cpython-310.pyc and requirements2/caio/__pycache__/python_aio.cpython-310.pyc differ
Files requirements1/caio/__pycache__/python_aio_asyncio.cpython-310.pyc and requirements2/caio/__pycache__/python_aio_asyncio.cpython-310.pyc differ
Files requirements1/caio/__pycache__/thread_aio_asyncio.cpython-310.pyc and requirements2/caio/__pycache__/thread_aio_asyncio.cpython-310.pyc differ
Files requirements1/caio/__pycache__/version.cpython-310.pyc and requirements2/caio/__pycache__/version.cpython-310.pyc differ

Is there a reason for that behavior? On all my python libs, this is the sole one that differs, and it breaks hash mechanisms, making it hard to determine when a code has changed or not.

Thanks in advance. Ronan

[mosquito]

The differing files occur because Python .pyc bytecode files are timestamped during compilation, and the thread_aio library includes platform-specific compiled binaries that may not produce deterministic builds.

This happens even when installing the same version because:

• *.pyc files contain metadata timestamps from exact compilation moment
• The .so binaries (C extensions) are compiled during installation, often including non-deterministic elements like build paths or compiler-specific artifacts

You can either use pre-compiled wheels pip install --only-binary

From my side I can add something like CFLAGS="-frandom-seed=0 -Wno-builtin-macro-redefined" but IMHO it is useless and I will have to support it without reasonable aim.

If you know any recipe how can I handle it without adding compiller flags fell free to write me.

[mosquito]

LLM suggest me about adding something like this before compiling: export SOURCE_DATE_EPOCH=946681200

But I newer test it myself :-D

[ronhanson]

Hi @mosquito, Thanks for the answer. I tried with --only-binary ":all:" :

export SOURCE_DATE_EPOCH=946681200

mkdir -p ./requirements1 ./requirements2
pip install --disable-pip-version-check --break-system-package --isolated --root-user-action ignore --only-binary ":all:" --upgrade caio==0.9.24 -t ./requirements1/

pip install --disable-pip-version-check --break-system-package --isolated --root-user-action ignore --only-binary ":all:" --upgrade caio==0.9.24 -t ./requirements2/

But it gives exactly the same result :

diff -rq requirements1 requirements2    
Files requirements1/caio/__pycache__/__init__.cpython-310.pyc and requirements2/caio/__pycache__/__init__.cpython-310.pyc differ
Files requirements1/caio/__pycache__/abstract.cpython-310.pyc and requirements2/caio/__pycache__/abstract.cpython-310.pyc differ
Files requirements1/caio/__pycache__/asyncio_base.cpython-310.pyc and requirements2/caio/__pycache__/asyncio_base.cpython-310.pyc differ
Files requirements1/caio/__pycache__/linux_aio_asyncio.cpython-310.pyc and requirements2/caio/__pycache__/linux_aio_asyncio.cpython-310.pyc differ
Files requirements1/caio/__pycache__/python_aio.cpython-310.pyc and requirements2/caio/__pycache__/python_aio.cpython-310.pyc differ
Files requirements1/caio/__pycache__/python_aio_asyncio.cpython-310.pyc and requirements2/caio/__pycache__/python_aio_asyncio.cpython-310.pyc differ
Files requirements1/caio/__pycache__/thread_aio_asyncio.cpython-310.pyc and requirements2/caio/__pycache__/thread_aio_asyncio.cpython-310.pyc differ
Files requirements1/caio/__pycache__/version.cpython-310.pyc and requirements2/caio/__pycache__/version.cpython-310.pyc differ

Looking at a file binary comparison :

 vimdiff <(xxd requirements1/caio/__pycache__/__init__.cpython-310.pyc) <(xxd requirements2/caio/__pycache__/__init__.cpython-310.pyc)

It shows the [...]pip-target-zzq33qy/lib/python/caio/__init__.py // [...]pip-target-b_1uze1c/lib/python/caio/__init__.py, so yes, it is about a remporary file being written out with a random string. If you find any easy way to change that behavior, by downloading precompiled binaries or by using a hash instead of a random, that would be nice indeed.

Thanks again. Ronan