[BUG]: Enabling Code origin for spans spikes CPU resources

[awilson1801]

Tracer Version(s)

3.7.0

Python Version(s)

3.12.7

Pip Version(s)

23.0.1

Bug Report

We recently turned on "Code Origin for Spans" via environment variable and noticed that it cause CPU to spike in our Django applications:

We had to turn the feature off for all applications while we investigate. One thing that stood out to us is the massive increase in time spent in the django cache layer:

Reproduction Code

No response

Error Logs

No response

Libraries in Use

Django 4.2

Operating System

No response

[P403n1x87]

@awilson1801 thanks for reporting this. Do the spike last well beyond the application boot or the first call to an endpoint? These are the moments when the Code Origin feature needs to build some internal mappings/caches to infer what's user code and what's third-party. Subsequent invocations of the endpoint handled by the same worker process should make use of the computed mappings and be much quicker.

[awilson1801]

@awilson1801 thanks for reporting this. Do the spike last well beyond the application boot or the first call to an endpoint? These are the moments when the Code Origin feature needs to build some internal mappings/caches to infer what's user code and what's third-party. Subsequent invocations of the endpoint handled by the same worker process should make use of the computed mappings and be much quicker.

Yes, unfortunately the resource spike lasted for hours while we tracked down where the spike was coming from. We observed the spike for about 7 hours until we finally figured it out and turned off the feature.

[P403n1x87]

@awilson1801 thanks for the extra details. We do have some work in-flight to address performance in general. Could you please reach out to support with this issue? We can then look more closely at your case 🙏