[Bug]: 0.98.x tar.gz file size increased?

[brandonburkett]

Bug report

With version 0.97 and below, the tar.gz file size was around 135mb, but with 0.98 release, I see it is up to 400mb+. Our elastic beanstalk default /tmp only has 924M without attempting to give /tmp more space.

In short, php datadog-setup.php --php-bin=all is failing because /tmp is out of disk space while trying to untar, which was not a problem with 0.97 and below.

PHP version

8.2.15

Tracer or profiler version

0.98.x

Installed extensions

No response

Output of phpinfo()

No response

Upgrading from

No response

[bwoebi]

Hey @brandonburkett,

We've decided to keep full debug symbols of the rust code to enhance debuggability in case there are issues. However, this comes, as you noticed at the cost of a size increase.

To avoid the particular issue with /tmp space being exceeded, you can specify a different temp directory: php -d sys_temp_dir=/other/path datadog-setup.php --php-bin=all.

I'll keep this issue open so that we invest into splitting the .tar.gz up or using other strategies.

[morrisonlevi]

I slightly improved the situation in v1.1.0. Here are the sizes for the aarch64-linux-gnu artifacts over time, but there are similarly sized improvements for x86_64-linux-gnu:

dd-library-php-0.98.0-aarch64-linux-gnu.tar.gz  482 MB
dd-library-php-0.99.0-aarch64-linux-gnu.tar.gz  485 MB
dd-library-php-1.0.0-aarch64-linux-gnu.tar.gz   495 MB

dd-library-php-1.1.0-aarch64-linux-gnu.tar.gz   425 MB

It's still over 400 MiB, so we plan to continue working on this.

[awons]

We worked around it by copying only the necessary files and using them in our final Docker image with this script:

cd $(mktemp -d)
curl -Lo datadog-php-tracer.apk https://github.com/DataDog/dd-trace-php/releases/download/${DATADOG_TRACER_VERSION}/datadog-php-tracer_${DATADOG_TRACER_VERSION}_$(uname -m).apk
apk add datadog-php-tracer.apk --allow-untrusted
PHP_API=$(php -i | grep "PHP API" | awk '{split($0,a,"=>"); print a[2]}' | xargs)
DD_TRACER_EXTENSION="/opt/datadog-php/extensions/ddtrace-${PHP_API}-alpine.so"
for filename in /opt/datadog-php/extensions/*; do
  [ -e "${filename}" ] || continue
  if [ "${filename}" = "${DD_TRACER_EXTENSION}" ]; then continue; fi
  rm -f "${filename}"
done

mkdir /tmp/ddtracer
for filename in $(apk info -L datadog-php-tracer); do
  [ -e "/${filename}" ] || continue
  cp -p --parents "/${filename}" /tmp/ddtracer
done
cp -p --parents /usr/local/etc/php/conf.d/98-ddtrace.ini /tmp/ddtracer

We use in combination with COPY --from=ddtracer /tmp/ddtracer / but this could easily be adapted for use cases outside of Docker environment.

Our images are down to <70MB.

[Liphtier]

the Alpine apk installation leaves all unneded bundled extension binaries in place:

du -had1 /opt/datadog-php/extensions/
15.7M   /opt/datadog-php/extensions/ddtrace-20180731-alpine.so
16.5M   /opt/datadog-php/extensions/ddtrace-20230831.so
16.3M   /opt/datadog-php/extensions/ddtrace-20180731.so
15.8M   /opt/datadog-php/extensions/ddtrace-20230831-alpine.so
15.8M   /opt/datadog-php/extensions/ddtrace-20220829-alpine.so
16.0M   /opt/datadog-php/extensions/ddtrace-20180731-debug.so
1.6M    /opt/datadog-php/extensions/ddtrace-20121113-debug.so
1.6M    /opt/datadog-php/extensions/ddtrace-20100412-debug.so
16.4M   /opt/datadog-php/extensions/ddtrace-20200930.so
16.2M   /opt/datadog-php/extensions/ddtrace-20180731-zts.so
2.0M    /opt/datadog-php/extensions/ddtrace-20121113-zts.so
16.4M   /opt/datadog-php/extensions/ddtrace-20170718.so
2.1M    /opt/datadog-php/extensions/ddtrace-20121113.so
16.5M   /opt/datadog-php/extensions/ddtrace-20220829.so
16.0M   /opt/datadog-php/extensions/ddtrace-20160303-debug.so
15.7M   /opt/datadog-php/extensions/ddtrace-20160303-alpine.so
16.1M   /opt/datadog-php/extensions/ddtrace-20220829-debug.so
15.7M   /opt/datadog-php/extensions/ddtrace-20190902-alpine.so
16.4M   /opt/datadog-php/extensions/ddtrace-20210902-zts.so
15.8M   /opt/datadog-php/extensions/ddtrace-20210902-alpine.so
2.0M    /opt/datadog-php/extensions/ddtrace-20100412.so
15.7M   /opt/datadog-php/extensions/ddtrace-20170718-alpine.so
2.0M    /opt/datadog-php/extensions/ddtrace-20131106-alpine.so
16.2M   /opt/datadog-php/extensions/ddtrace-20170718-zts.so
2.0M    /opt/datadog-php/extensions/ddtrace-20121113-alpine.so
15.8M   /opt/datadog-php/extensions/ddtrace-20200930-alpine.so
16.1M   /opt/datadog-php/extensions/ddtrace-20190902-debug.so
16.4M   /opt/datadog-php/extensions/ddtrace-20230831-zts.so
16.0M   /opt/datadog-php/extensions/ddtrace-20170718-debug.so
2.1M    /opt/datadog-php/extensions/ddtrace-20131106.so
2.0M    /opt/datadog-php/extensions/ddtrace-20131106-zts.so
16.2M   /opt/datadog-php/extensions/ddtrace-20160303-zts.so
16.3M   /opt/datadog-php/extensions/ddtrace-20200930-zts.so
16.3M   /opt/datadog-php/extensions/ddtrace-20151012.so
16.4M   /opt/datadog-php/extensions/ddtrace-20220829-zts.so
16.5M   /opt/datadog-php/extensions/ddtrace-20210902.so
16.1M   /opt/datadog-php/extensions/ddtrace-20200930-debug.so
16.2M   /opt/datadog-php/extensions/ddtrace-20151012-zts.so
1.7M    /opt/datadog-php/extensions/ddtrace-20131106-debug.so
16.2M   /opt/datadog-php/extensions/ddtrace-20190902-zts.so
16.2M   /opt/datadog-php/extensions/ddtrace-20210902-debug.so
15.7M   /opt/datadog-php/extensions/ddtrace-20151012-alpine.so
16.4M   /opt/datadog-php/extensions/ddtrace-20190902.so
16.0M   /opt/datadog-php/extensions/ddtrace-20151012-debug.so
16.2M   /opt/datadog-php/extensions/ddtrace-20230831-debug.so
2.0M    /opt/datadog-php/extensions/ddtrace-20100412-zts.so
2.0M    /opt/datadog-php/extensions/ddtrace-20100412-alpine.so
16.3M   /opt/datadog-php/extensions/ddtrace-20160303.so
603.7M  /opt/datadog-php/extensions/

[razvanphp]

This is really unacceptable, our docker image with 0.89.0 is 455 Mb, 0.99.1 is 907 Mb, 1.2.0 is 1076 Mb, keeping everything else unchanged. We use .deb files from github releases.

BTW: maybe using https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php is a good alternative.

[bwoebi]

@razvanphp If you use the docker image, you probably should download, use dpkg and remove the .deb file in the same dockerfile RUN line, to avoids storing the whole .deb file in the image itself.

[razvanphp]

I do that already, still the sizes above. I think you should publish separate -dbg.deb packages instead.

the Alpine apk installation leaves all unneded bundled extension binaries in place:

BTW, .deb does the same:

root@server:/web# du -sh /opt/datadog-php/extensions/
231M	/opt/datadog-php/extensions/

[bwoebi]

Oh, you are right, sorry. This is the .deb. I suppose we need to strip the extensions we don't install somehow.

The problem with separate -dbg.deb packages is that it's going to defeat the point of crashtracking which #2763 will introduce: we want to automatically collect fully symbolicated dumps (meaning we need at least line tables). The primary problem here lies with the linker - rust depends on the linker to strip away unused code, but debug info of unused code stays in the binaries and that's blowing up the binary sizes :-(

[razvanphp]

While I totally understand the reasoning why you would need this, please imagine what this does for an infrastructure where we deploy thousands of containers, each of them with a size of 1 GB; not counting the repo traffic cost, the time it takes for one container to come up is exponential with the size.

[bwoebi]

datadog-setup.php is now split in individual small packages, downloading less than 50 MB for single version installs. We probably need to set this up for deb and rpm and apk too, but for these we'll likely have to move to proper repositories instead.