How to handle UINT64 counter rollover in OpenTelemetry C++? Can counters be reset after UINT64_MAX? #3752
abhinavkumar9962
started this conversation in
General
Replies: 1 comment
-
|
You should switch to doubles. Yes eventually you are going to lose precision, but if you are dealing with large numbers this might be okay. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi team,
I am using OpenTelemetry C++ SDK to emit UInt64Counter metrics to VictoriaMetrics (via vmagent).
My counters are monotonic and long-running, and in a high-throughput real production workload they eventually reach UINT64_MAX.
After the counter reaches the limit and overflows, the exported values become negative (2's complement). This breaks the behavior of:
increase(<counter>[range])in VictoriaMetrics — it returns 0 after the rollover because the counter appears to "go backwards".
❗ Additional Context
I cannot use Delta Aggregation Temporality because VictoriaMetrics does not support delta temporality, so I must rely on cumulative counters. This means overflow becomes a real problem, as backends cannot compute the correct increase() when wraparound happens.
❓ Question
Is there a recommended or official way in OpenTelemetry C++ to handle counter rollover?
Specifically:
✔️ Can we reset a UInt64Counter after it reaches UINT64_MAX?
✔️ Is there any mechanism in OTEL SDK that tracks epochs or wraps counters safely?
Any guidance on best practice for extremely large counters would be appreciated.
🔧 Reproduction Code
Below is a minimal C++ program that reproduces the rollover behaviour:
`
`
I have a collection interval of 60s.
📤 Behavior Observed in VictoriaMetrics Export
When I export using:
{"metric":{"__name__":"test_counter","id":"99","service.name":"unknown_service", "telemetry.sdk.language":"cpp","telemetry.sdk.name":"opentelemetry", "telemetry.sdk.version":"1.14.2"}, "values":[1,479,957,1436,1914,2392, 9223372036854774784, -9223372036854774784, -9223372036854773760, -9223372036854773760, -9223372036854771712, -9223372036854771712, -9223372036854771712, -9223372036854771712], "timestamps":[1763378911222,1763378970222,1763379029225,1763379088230, 1763379147231,1763379206235,1763379265238,1763379324242, 1763379383245,1763379442248,1763379501253,1763379560255, 1763379619260,1763379678263]}As soon as the counter crosses the signed range, the values flip into negative space.
Going into -ve is fine increase would handle it but it appears to same or undefined behaviour after roll over.
Request
Please advise:
What is the recommended strategy for preventing counter overflow?
Thanks for your guidance!
Beta Was this translation helpful? Give feedback.
All reactions