Service metrics duplicated when service member belongs to multiple servicegroups

[iwanmikhajlov]

Describe the bug
The get_svc_grp_services_stats() function in exporter.py creates duplicate metrics when a service member belongs to multiple servicegroups. The function iterates through each servicegroup and adds all service members to the result list without checking if a service was already added from a previous servicegroup. This results in duplicate metrics being exported to Prometheus.

Slack Channel
N/A

To Reproduce
Steps to reproduce the behavior:

1. Configure a NetScaler with a service that is a member of multiple servicegroups (e.g., service 10.42.5.10 in both puppet_8140_sgp and puppet_4000_sgp)
2. Version of the metrics exporter: 1.5.0 (from quay.io/netscaler/netscaler-adc-metrics-exporter:1.5.0)
3. Version of the Citrix ADC VPX: NS13.1: Build 55.34.nc
4. Observe metrics appearing multiple times with count > 1

Result in our environment

Total metrics: 70
Unique metrics: 36
Duplicates: 34 (nearly 50% duplication rate)

Logs from the metrics exporter show no errors, but Prometheus logs show warnings about duplicate samples with identical timestamps.

Expected behavior
Each service should appear only once in the metrics output, regardless of how many servicegroups it belongs to. The citrixadc_services_response_bytes_rate and other service metrics should have unique combinations of labels.

Additional context
Root cause: In exporter.py line ~460-490, the get_svc_grp_services_stats() function does:

for servicegroups_ds in servicegroup_list_ds['servicegroup']:
    for individual_svc_binding_data in data_tmp['servicegroup'][0]['servicegroupmember']:
        servicegroup_data.append(individual_svc_binding_data)  # No deduplication

Proposed fix: Add deduplication using a set:

servicegroup_data = []
seen_services = set()

for servicegroups_ds in servicegroup_list_ds['servicegroup']:
    for individual_svc_binding_data in data_tmp['servicegroup'][0]['servicegroupmember']:
        service_key = (servername, individual_svc_binding_data.get('primaryipaddress', ''), _servicegroup_name)
        if service_key not in seen_services:
            seen_services.add(service_key)
            servicegroup_data.append(individual_svc_binding_data)

I have created a fork of this repository where I've applied this fix along with updated dependencies (Alpine 3.19, updated pip packages), tested it, and haven't seen any duplicates so far.
While I understand that NetScaler can expose metrics through its native capabilities, the metrics exporter is still a production tool that is used by many companies, including ours. I would be glad if you take a look at this issue, and fix it in some convinient way.

Thanks in advance!

[subashd]

hi @iwanmikhajlov
Thank you for raising the request. We are reviewing this request and will incorporate the necessary changes.
Meanwhile, shall you send an email to subash.dangol@citrix.com, and we can discuss if you require any other Cloud Native solution that NetScaler provides.