Skip to content

Commit

Permalink
temp: Enable Celery distributed tracing in stage edxapp
Browse files Browse the repository at this point in the history
Datadog Support suggested that we try enabling this to see if it changes
anything about the orphaned spans on the edxapp workers (unexpected
top-level spans that are not `operation_name:celery.run`, but instead are
other spans that have lost their parent association).

Just going to enable this on stage so we don't mess with traces in prod,
for now.

See edx/edx-arch-experiments#822
  • Loading branch information
timmc-edx committed Jan 9, 2025
1 parent ad70187 commit 531211d
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 1 deletion.
10 changes: 9 additions & 1 deletion playbooks/roles/edxapp/templates/edx/app/edxapp/cms.sh.j2
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,14 @@ export DD_TRACE_LOG_STREAM_HANDLER=false
# Suppress middleware spans because there are about 100 for each request.
# Remove (or set to true) for debugging.
export DD_DJANGO_INSTRUMENT_MIDDLEWARE=false

{% if COMMON_ENVIRONMENT == "stage" %}
# Temporary 2025-01-09: Enable Celery distributed tracing to see if it
# has an effect on the remaining orphaned spans.
# https://github.com/edx/edx-arch-experiments/issues/822
export DD_CELERY_DISTRIBUTED_TRACING=true
{% endif %}

{% endif -%}

# We want to be able to toggle this on separately from DD in general.
Expand Down Expand Up @@ -54,4 +62,4 @@ source {{ edxapp_app_dir }}/edxapp_env
exec {{ executable }} -c {{ edxapp_app_dir }}/cms_gunicorn.py {{ EDXAPP_CMS_GUNICORN_EXTRA }} cms.wsgi --log-file {{ edxapp_gunicorn_log_dir[1] }}/edx.log
{% else %}
exec {{ executable }} -c {{ edxapp_app_dir }}/cms_gunicorn.py {{ EDXAPP_CMS_GUNICORN_EXTRA }} cms.wsgi
{% endif %}
{% endif %}
7 changes: 7 additions & 0 deletions playbooks/roles/edxapp/templates/edx/app/edxapp/lms.sh.j2
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ export DD_TRACE_LOG_STREAM_HANDLER=false
# Remove (or set to true) for debugging.
export DD_DJANGO_INSTRUMENT_MIDDLEWARE=false

{% if COMMON_ENVIRONMENT == "stage" %}
# Temporary 2025-01-09: Enable Celery distributed tracing to see if it
# has an effect on the remaining orphaned spans.
# https://github.com/edx/edx-arch-experiments/issues/822
export DD_CELERY_DISTRIBUTED_TRACING=true
{% endif %}

{% endif -%}

# We want to be able to toggle this on separately from DD in general.
Expand Down
8 changes: 8 additions & 0 deletions playbooks/roles/edxapp/templates/edx/app/edxapp/worker.sh.j2
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,14 @@ export DD_TRACE_LOG_STREAM_HANDLER=false
# Suppress middleware spans because there are about 100 for each request.
# Remove (or set to true) for debugging.
export DD_DJANGO_INSTRUMENT_MIDDLEWARE=false

{% if COMMON_ENVIRONMENT == "stage" %}
# Temporary 2025-01-09: Enable Celery distributed tracing to see if it
# has an effect on the remaining orphaned spans.
# https://github.com/edx/edx-arch-experiments/issues/822
export DD_CELERY_DISTRIBUTED_TRACING=true
{% endif %}

{% endif -%}

# We want to be able to toggle this on separately from DD in general.
Expand Down

0 comments on commit 531211d

Please sign in to comment.