Refactor Prometheus configuration to use flexible label filters

- Replaced prometheus_label_type_instance with two new options:
  - vm_uuid_label_name: configurable name of the UUID label (default: "uuid")
  - labels: list of key:value pairs for label filters (default: ["type_instance:scaph_process_power_microwatts"])
- Updated _energy_consumed_wh to parse label list and build label dictionary
- Added support for multiple label filters in any combination
- Updated all tests to use new configuration structure
- Regenerated etc/caso/caso.conf.sample with oslo-config-generator
- Updated prometheus-extractor.rst documentation:
  - New configuration parameters explained
  - Added examples for multiple label filters
  - Updated troubleshooting section
- All tests pass (4/4 prometheus tests)

Co-authored-by: alvarolopez <468751+alvarolopez@users.noreply.github.com>

Author: Copilot

caso/extract/prometheus.py

@@ -39,9 +39,16 @@
         help="Name of the Prometheus metric to query for energy consumption.",
     ),
     cfg.StrOpt(
-        "prometheus_label_type_instance",
-        default="scaph_process_power_microwatts",
-        help="Value for the type_instance label in Prometheus queries.",
+        "vm_uuid_label_name",
+        default="uuid",
+        help="Name of the label that matches the VM UUID in Prometheus metrics.",
+    ),
+    cfg.ListOpt(
+        "labels",
+        default=["type_instance:scaph_process_power_microwatts"],
+        help="List of label filters as key:value pairs to filter the Prometheus "
+        "metric (e.g., 'type_instance:scaph_process_power_microwatts'). "
+        "The VM UUID label will be added automatically based on vm_uuid_label_name.",
     ),
     cfg.IntOpt(
         "prometheus_step_seconds",
@@ -99,6 +106,8 @@ def _energy_consumed_wh(self, vm_uuid):
         step_seconds = CONF.prometheus.prometheus_step_seconds
         query_range = CONF.prometheus.prometheus_query_range
         verify_ssl = CONF.prometheus.prometheus_verify_ssl
+        vm_uuid_label_name = CONF.prometheus.vm_uuid_label_name
+        label_filters = CONF.prometheus.labels
 
         prom = prometheus_api_client.PrometheusConnect(
             url=prom_url, disable_ssl=not verify_ssl
@@ -107,11 +116,20 @@ def _energy_consumed_wh(self, vm_uuid):
         # factor = step_seconds / 3600 converts µW·s to µWh
         factor = step_seconds / 3600
 
-        # Build labels for this VM
-        labels = {
-            "type_instance": CONF.prometheus.prometheus_label_type_instance,
-            "uuid": vm_uuid,
-        }
+        # Build labels dictionary from the list of "key:value" strings
+        labels = {}
+        for label_filter in label_filters:
+            if ":" in label_filter:
+                key, value = label_filter.split(":", 1)
+                labels[key.strip()] = value.strip()
+            else:
+                LOG.warning(
+                    f"Invalid label filter format '{label_filter}', "
+                    "expected 'key:value'. Skipping."
+                )
+
+        # Add the VM UUID label
+        labels[vm_uuid_label_name] = vm_uuid
 
         # Build label string: {key="value", ...}
         label_selector = ",".join(f'{k}="{v}"' for k, v in labels.items())

caso/tests/extract/test_prometheus.py

@@ -60,6 +60,17 @@ def configured_extractor(mock_flavors):
     # Configure CONF
     CONF.set_override("site_name", "TEST-Site")
     CONF.set_override("service_name", "TEST-Service")
+    CONF.set_override("prometheus_endpoint", "http://localhost:9090", group="prometheus")
+    CONF.set_override("prometheus_metric_name", "prometheus_value", group="prometheus")
+    CONF.set_override("vm_uuid_label_name", "uuid", group="prometheus")
+    CONF.set_override(
+        "labels",
+        ["type_instance:scaph_process_power_microwatts"],
+        group="prometheus",
+    )
+    CONF.set_override("prometheus_step_seconds", 30, group="prometheus")
+    CONF.set_override("prometheus_query_range", "1h", group="prometheus")
+    CONF.set_override("prometheus_verify_ssl", True, group="prometheus")
 
     with mock.patch(
         "caso.extract.openstack.base.BaseOpenStackExtractor.__init__",

doc/source/prometheus-extractor.rst

@@ -24,8 +24,11 @@ prometheus_endpoint = http://localhost:9090
 # Name of the Prometheus metric to query
 prometheus_metric_name = prometheus_value
 
-# Value for the type_instance label
-prometheus_label_type_instance = scaph_process_power_microwatts
+# Name of the label that matches the VM UUID
+vm_uuid_label_name = uuid
+
+# List of label filters as key:value pairs
+labels = type_instance:scaph_process_power_microwatts
 
 # Frequency between samples in seconds
 prometheus_step_seconds = 30
@@ -43,14 +46,16 @@ The Prometheus extractor:
 
 1. **Scans VMs**: Retrieves the list of VMs from Nova for each configured project
 2. **Queries Per VM**: For each VM, executes a Prometheus query using the configured metric name and labels
-3. **Calculates Energy**: Uses the formula `sum_over_time(metric_name{type_instance="value", uuid="vm-uuid"}[query_range]) * (step_seconds/3600) / 1000000` to convert microwatt power samples to Watt-hours
-4. **Creates Records**: Generates an `EnergyRecord` for each VM with energy consumption data and execution metrics
+3. **Builds Labels**: Combines the configured label filters with the VM UUID label (e.g., `{type_instance="scaph_process_power_microwatts", uuid="vm-uuid"}`)
+4. **Calculates Energy**: Uses the formula `sum_over_time(metric_name{labels}[query_range]) * (step_seconds/3600) / 1000000` to convert microwatt power samples to Watt-hours
+5. **Creates Records**: Generates an `EnergyRecord` for each VM with energy consumption data and execution metrics
 
 ## Configuration Parameters
 
 - **prometheus_endpoint**: URL of the Prometheus server (default: `http://localhost:9090`)
 - **prometheus_metric_name**: Name of the metric to query (default: `prometheus_value`)
-- **prometheus_label_type_instance**: Value for the `type_instance` label used to filter metrics (default: `scaph_process_power_microwatts`)
+- **vm_uuid_label_name**: Name of the label that matches the VM UUID in Prometheus metrics (default: `uuid`)
+- **labels**: List of label filters as `key:value` pairs to filter the Prometheus metric. The VM UUID label will be added automatically (default: `["type_instance:scaph_process_power_microwatts"]`)
 - **prometheus_step_seconds**: Frequency between samples in the time series, in seconds (default: `30`)
 - **prometheus_query_range**: Time range for the query (default: `1h`). Examples: `1h`, `6h`, `24h`
 - **prometheus_verify_ssl**: Whether to verify SSL certificates when connecting to Prometheus (default: `true`)
@@ -65,7 +70,8 @@ Scaphandre exports energy metrics in microwatts:
 [prometheus]
 prometheus_endpoint = http://prometheus.example.com:9090
 prometheus_metric_name = prometheus_value
-prometheus_label_type_instance = scaph_process_power_microwatts
+vm_uuid_label_name = uuid
+labels = type_instance:scaph_process_power_microwatts
 prometheus_step_seconds = 30
 prometheus_query_range = 6h
 prometheus_verify_ssl = false
@@ -79,12 +85,28 @@ If you have custom energy metrics with different labels:
 [prometheus]
 prometheus_endpoint = http://prometheus.example.com:9090
 prometheus_metric_name = my_custom_power_metric
-prometheus_label_type_instance = my_power_label_value
+vm_uuid_label_name = instance_id
+labels = environment:production,datacenter:dc1
 prometheus_step_seconds = 60
 prometheus_query_range = 1h
 prometheus_verify_ssl = true
 ```
 
+### For Multiple Label Filters
+
+You can specify multiple label filters:
+
+```ini
+[prometheus]
+prometheus_endpoint = http://prometheus.example.com:9090
+prometheus_metric_name = node_power_watts
+vm_uuid_label_name = vm_uuid
+labels = type_instance:power_consumption,source:ipmi,rack:rack42
+prometheus_step_seconds = 30
+prometheus_query_range = 1h
+prometheus_verify_ssl = true
+```
+
 ## Energy Record Format
 
 The Prometheus extractor generates `EnergyRecord` objects with the following fields:
@@ -140,7 +162,8 @@ messengers = ssm
 [prometheus]
 prometheus_endpoint = http://prometheus.example.com:9090
 prometheus_metric_name = prometheus_value
-prometheus_label_type_instance = scaph_process_power_microwatts
+vm_uuid_label_name = uuid
+labels = type_instance:scaph_process_power_microwatts
 prometheus_step_seconds = 30
 prometheus_query_range = 6h
 prometheus_verify_ssl = false
@@ -156,7 +179,7 @@ output_path = /var/spool/apel/outgoing/openstack
 - Check that your metric exists in Prometheus UI
 - Ensure the metric has data for the configured time range
 - Verify VMs exist in the configured projects
-- Check that the metric has the required labels (`type_instance` and `uuid`)
+- Check that the metric has the required labels (those specified in `labels` config and the UUID label specified by `vm_uuid_label_name`)
 
 **Connection timeout:**
 - Check network connectivity to Prometheus
@@ -165,8 +188,8 @@ output_path = /var/spool/apel/outgoing/openstack
 
 **Invalid query results:**
 - Ensure your metric contains instantaneous power values in microwatts
-- Check that the metric has the `uuid` label matching VM UUIDs
-- Verify the `type_instance` label matches your configuration
+- Check that the metric has the UUID label matching VM UUIDs (configured via `vm_uuid_label_name`)
+- Verify the label filters match your configuration
 - Test the query in Prometheus UI: `sum_over_time(prometheus_value{type_instance="scaph_process_power_microwatts", uuid="<vm-uuid>"}[1h])`
 
 **No VMs found:**

etc/caso/caso.conf.sample

@@ -410,8 +410,14 @@
 # Name of the Prometheus metric to query for energy consumption. (string value)
 #prometheus_metric_name = prometheus_value
 
-# Value for the type_instance label in Prometheus queries. (string value)
-#prometheus_label_type_instance = scaph_process_power_microwatts
+# Name of the label that matches the VM UUID in Prometheus metrics. (string
+# value)
+#vm_uuid_label_name = uuid
+
+# List of label filters as key:value pairs to filter the Prometheus metric
+# (e.g., 'type_instance:scaph_process_power_microwatts'). The VM UUID label
+# will be added automatically based on vm_uuid_label_name. (list value)
+#labels = type_instance:scaph_process_power_microwatts
 
 # Frequency between samples in the time series (in seconds). (integer value)
 #prometheus_step_seconds = 30