Celery tracing has stopped reporting to new relic console after version 9.9

[fernandoaguilar]

Description After upgrading to agent version 9.13.0 celery monitoring has stopped reporting and showing up in new relic console. I noticed it stopped working after version 9.9.0.

Expected Behavior celery metrics should report to new relic console.

Troubleshooting or NR Diag results tried using nrdiag but its not detecting the virtual environement, therefore it fails to detect newrelic is even installed.

Steps to Reproduce CELERY_BIN="NEW_RELIC_CONFIG_FILE=/path/to/newrelic.ini /path/to/newrelic-admin run-program /path/to/celery" service file

[Unit]
Description=Celery Service
After=network.target

[Service]
Type=forking
User=ubuntu
Group=ubuntu
EnvironmentFile=/etc/default/celeryd
WorkingDirectory=/srv/plugshare
ExecStart=/bin/sh -c '${CELERY_BIN} -A ${CELERY_APP} multi start ${CELERYD_NODES} \
  --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} \
  --pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} -A ${CELERY_APP} multi restart ${CELERYD_NODES} \
  --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'

[Install]
WantedBy=multi-user.target

Your Environment new relic agent: 9.13.0 celery version 5.4.0 python 3.11

[workato-integration[bot]]

https://new-relic.atlassian.net/browse/NR-316900

[lrafeei]

Hello--sorry about the delay on this one. Are you using a custom base task for any of your tasks, perhaps? We had a similar issue with this that involved New Relic missing some instrumentation for custom base tasks in subsequent versions. If not, would it be possible to get a basic repro of your app?

[workato-integration[bot]]

All attempts at reproducing this issue failed, or not enough information was available to reproduce the issue. Reading the code produces no clues as to why this behavior would occur. If more information appears later, please reopen the issue.

[amcgavin]

Hi @lrafeei, we're also experiencing the same issue on 10.6.0 - we are using custom base tasks for some tasks which aren't working.

Are there any known workarounds?

[fernandoaguilar]

Hello--sorry about the delay on this one. Are you using a custom base task for any of your tasks, perhaps? We had a similar issue with this that involved New Relic missing some instrumentation for custom base tasks in subsequent versions. If not, would it be possible to get a basic repro of your app?

Yes we are using custom base task for all tasks. I no longer am working on that code, so I won't be able to provide a repro.

[lrafeei]

Hello--

Long story short, we need to do some longer term fixes to get this to work properly, but in the meantime, could you try this workaround?

https://github.com/newrelic/newrelic-python-agent/issues/1243#issuecomment-2536839894

[workato-integration[bot]]

The problem is a duplicate of an existing issue.

[lrafeei]

For some reason, the github ticket was closed, but the work is being tracked here: https://new-relic.atlassian.net/browse/NR-373685

[amcgavin]

@lrafeei I'm unable to view that link. Is there somewhere public where I can track the work for the root cause?

[lrafeei]

Hello--

We have a branch up that should solve this issue. If anyone wants to test it out before we release it, the branch is celery-custom-task-instrumentation