Update documentation and remove requests dependency

- Removed requests dependency from pyproject.toml (no longer needed)
- Updated prometheus-extractor.rst with new configuration parameters:
  - Documented prometheus_metric_name, prometheus_label_type_instance
  - Documented prometheus_step_seconds, prometheus_query_range, prometheus_verify_ssl
  - Removed old prometheus_query and prometheus_timeout references
  - Added detailed explanation of energy calculation formula
  - Updated examples for Scaphandre and custom metrics
  - Updated troubleshooting section
- Updated configuration.rst with new prometheus section:
  - Documented all new configuration options
  - Added reference to prometheus-extractor.rst
  - Explained energy calculation from microwatt samples
- Updated etc/caso/caso.conf.sample with new prometheus configuration options
- Updated poetry.lock to reflect dependency changes
- All tests pass (6/6 energy-related tests)

Co-authored-by: alvarolopez <468751+alvarolopez@users.noreply.github.com>

Author: Copilot

doc/source/configuration.rst

@@ -212,20 +212,28 @@ messenger. Available options:
 ------------------------
 
 Options defined here configure the Prometheus extractor for gathering energy
-consumption metrics. This extractor queries a Prometheus instance to retrieve
-energy usage data. Available options:
+consumption metrics. This extractor uses the ``prometheus-api-client`` library
+to query a Prometheus instance and calculate energy consumption from 
+instantaneous power samples. Available options:
 
 * ``prometheus_endpoint`` (default: ``http://localhost:9090``), Prometheus
   server endpoint URL.
-* ``prometheus_query`` (default: 
-  ``sum(rate(node_energy_joules_total[5m])) * 300 / 3600000``), PromQL query
-  to retrieve energy consumption in kWh. This query should return energy 
-  consumption metrics that will be converted to accounting records.
-* ``prometheus_timeout`` (default: ``30``), Timeout for Prometheus API
-  requests in seconds.
+* ``prometheus_metric_name`` (default: ``prometheus_value``), Name of the 
+  Prometheus metric to query for energy consumption data.
+* ``prometheus_label_type_instance`` (default: ``scaph_process_power_microwatts``),
+  Value for the ``type_instance`` label used to filter metrics in Prometheus.
+* ``prometheus_step_seconds`` (default: ``30``), Frequency between samples in
+  the time series, in seconds. This is used to calculate energy from power samples.
+* ``prometheus_query_range`` (default: ``1h``), Time range for the Prometheus
+  query (e.g., ``1h``, ``6h``, ``24h``).
+* ``prometheus_verify_ssl`` (default: ``true``), Whether to verify SSL 
+  certificates when connecting to Prometheus.
+
+The extractor calculates energy in Watt-hours (Wh) from microwatt power samples
+using the formula: ``sum_over_time(metric{labels}[range]) * (step_seconds/3600) / 1000000``.
 
 To use the Prometheus extractor, add ``prometheus`` to the ``extractor`` option
-in the main configuration.
+in the main configuration. For more details, see :doc:`prometheus-extractor`.
 
 Other cASO configuration options
 --------------------------------

doc/source/prometheus-extractor.rst

@@ -6,6 +6,8 @@ This document provides information on using the Prometheus extractor to gather e
 
 The Prometheus extractor queries a Prometheus instance to retrieve energy consumption metrics for each VM in the configured projects and generates `EnergyRecord` objects that can be published through cASO's messenger system.
 
+The extractor uses the `prometheus-api-client` library to connect to Prometheus and calculate energy consumption in Watt-hours (Wh) from instantaneous power samples stored in Prometheus.
+
 ## Configuration
 
 To use the Prometheus extractor, add the following configuration to your `caso.conf` file:
@@ -19,63 +21,90 @@ extractor = nova,cinder,prometheus
 # Prometheus server endpoint URL
 prometheus_endpoint = http://localhost:9090
 
-# PromQL query to retrieve energy consumption in kWh
-# Use {{uuid}} as a template variable for the VM UUID
-prometheus_query = sum(rate(libvirt_domain_info_energy_consumption_joules_total{uuid=~"{{uuid}}"}[5m])) * 300 / 3600000
+# Name of the Prometheus metric to query
+prometheus_metric_name = prometheus_value
+
+# Value for the type_instance label
+prometheus_label_type_instance = scaph_process_power_microwatts
+
+# Frequency between samples in seconds
+prometheus_step_seconds = 30
 
-# Timeout for Prometheus API requests (in seconds)
-prometheus_timeout = 30
+# Query time range (e.g., '1h', '6h', '24h')
+prometheus_query_range = 1h
+
+# Whether to verify SSL when connecting to Prometheus
+prometheus_verify_ssl = true
 ```
 
 ## How It Works
 
 The Prometheus extractor:
 
 1. **Scans VMs**: Retrieves the list of VMs from Nova for each configured project
-2. **Queries Per VM**: For each VM, executes a customizable Prometheus query
-3. **Template Variables**: Replaces `{{uuid}}` in the query with the actual VM UUID
-4. **Creates Records**: Generates an `EnergyRecord` for each VM with energy consumption data
+2. **Queries Per VM**: For each VM, executes a Prometheus query using the configured metric name and labels
+3. **Calculates Energy**: Uses the formula `sum_over_time(metric_name{type_instance="value", uuid="vm-uuid"}[query_range]) * (step_seconds/3600) / 1000000` to convert microwatt power samples to Watt-hours
+4. **Creates Records**: Generates an `EnergyRecord` for each VM with energy consumption data and execution metrics
 
-## Customizing the PromQL Query
+## Configuration Parameters
 
-The query template can use `{{uuid}}` as a placeholder for the VM UUID. The default query assumes you have energy consumption metrics labeled with the VM UUID.
+- **prometheus_endpoint**: URL of the Prometheus server (default: `http://localhost:9090`)
+- **prometheus_metric_name**: Name of the metric to query (default: `prometheus_value`)
+- **prometheus_label_type_instance**: Value for the `type_instance` label used to filter metrics (default: `scaph_process_power_microwatts`)
+- **prometheus_step_seconds**: Frequency between samples in the time series, in seconds (default: `30`)
+- **prometheus_query_range**: Time range for the query (default: `1h`). Examples: `1h`, `6h`, `24h`
+- **prometheus_verify_ssl**: Whether to verify SSL certificates when connecting to Prometheus (default: `true`)
 
-### Example Queries
+## Example Configurations
 
-**For libvirt domain energy metrics:**
-```promql
-sum(rate(libvirt_domain_info_energy_consumption_joules_total{uuid=~"{{uuid}}"}[5m])) * 300 / 3600000
-```
+### For Scaphandre Energy Metrics
 
-**For per-VM IPMI power metrics:**
-```promql
-avg_over_time(ipmi_power_watts{instance=~".*{{uuid}}.*"}[5m]) * 5 * 60 / 1000 / 3600
-```
+Scaphandre exports energy metrics in microwatts:
 
-**For VM-specific RAPL energy metrics:**
-```promql
-sum(rate(node_rapl_package_joules_total{vm_uuid="{{uuid}}"}[5m])) * 300 / 3600000
+```ini
+[prometheus]
+prometheus_endpoint = http://prometheus.example.com:9090
+prometheus_metric_name = prometheus_value
+prometheus_label_type_instance = scaph_process_power_microwatts
+prometheus_step_seconds = 30
+prometheus_query_range = 6h
+prometheus_verify_ssl = false
 ```
 
-**For Scaphandre per-process metrics:**
-```promql
-sum(rate(scaph_process_power_consumption_microwatts{exe=~".*qemu.*",cmdline=~".*{{uuid}}.*"}[5m])) * 300 / 1000000 / 3600
+### For Custom Energy Metrics
+
+If you have custom energy metrics with different labels:
+
+```ini
+[prometheus]
+prometheus_endpoint = http://prometheus.example.com:9090
+prometheus_metric_name = my_custom_power_metric
+prometheus_label_type_instance = my_power_label_value
+prometheus_step_seconds = 60
+prometheus_query_range = 1h
+prometheus_verify_ssl = true
 ```
 
 ## Energy Record Format
 
 The Prometheus extractor generates `EnergyRecord` objects with the following fields:
 
-- `uuid`: Unique identifier for the record
-- `measurement_time`: Timestamp when the measurement was taken
-- `site_name`: Name of the site (from configuration)
-- `user_id`: User identifier (optional for energy records)
-- `group_id`: Project/group identifier
-- `user_dn`: User Distinguished Name (optional)
-- `fqan`: Fully Qualified Attribute Name (VO mapping)
-- `energy_consumption`: Energy consumption value (in kWh)
-- `energy_unit`: Unit of measurement (default: "kWh")
-- `compute_service`: Service name (from configuration)
+- `ExecUnitID`: VM UUID
+- `StartExecTime`: Start time of the measurement period (ISO 8601 format)
+- `EndExecTime`: End time of the measurement period (ISO 8601 format)
+- `EnergyWh`: Energy consumption in Watt-hours
+- `Work`: CPU hours (CPU duration in hours)
+- `Efficiency`: Efficiency factor (placeholder value)
+- `WallClockTime_s`: Wall clock time in seconds
+- `CpuDuration_s`: CPU duration in seconds (wall time × vCPUs)
+- `SuspendDuration_s`: Suspend duration in seconds
+- `CPUNormalizationFactor`: CPU normalization factor
+- `ExecUnitFinished`: 0 if running, 1 if stopped
+- `Status`: VM status (active, stopped, etc.)
+- `Owner`: VO/project owner
+- `SiteName`: Site name (from configuration)
+- `CloudComputeService`: Service name (from configuration)
+- `CloudType`: Cloud type (e.g., "openstack")
 
 ## Integration with Messengers
 
@@ -93,13 +122,13 @@ The records will be serialized as JSON with field mapping according to the accou
 To test your Prometheus extractor configuration:
 
 1. Verify Prometheus is accessible from cASO
-2. Test your PromQL query directly in Prometheus UI with a sample UUID
+2. Test your metric exists in Prometheus UI with a sample VM UUID
 3. Run cASO with the `--dry-run` option to preview records without publishing
 4. Check the logs for any errors or warnings
 
 ## Example
 
-Here's a complete example configuration:
+Here's a complete example configuration for Scaphandre metrics:
 
 ```ini
 [DEFAULT]
@@ -110,8 +139,11 @@ messengers = ssm
 
 [prometheus]
 prometheus_endpoint = http://prometheus.example.com:9090
-prometheus_query = sum(rate(libvirt_domain_info_energy_consumption_joules_total{uuid=~"{{uuid}}"}[5m])) * 300 / 3600000
-prometheus_timeout = 30
+prometheus_metric_name = prometheus_value
+prometheus_label_type_instance = scaph_process_power_microwatts
+prometheus_step_seconds = 30
+prometheus_query_range = 6h
+prometheus_verify_ssl = false
 
 [ssm]
 output_path = /var/spool/apel/outgoing/openstack
@@ -121,22 +153,37 @@ output_path = /var/spool/apel/outgoing/openstack
 
 **No records extracted:**
 - Verify Prometheus is accessible
-- Check that your query returns results in Prometheus UI (replace {{uuid}} with an actual VM UUID)
-- Ensure the time range (extract_from/extract_to) covers periods with data
+- Check that your metric exists in Prometheus UI
+- Ensure the metric has data for the configured time range
 - Verify VMs exist in the configured projects
+- Check that the metric has the required labels (`type_instance` and `uuid`)
 
 **Connection timeout:**
-- Increase `prometheus_timeout` value
 - Check network connectivity to Prometheus
 - Verify Prometheus is not overloaded
+- If using SSL, ensure certificates are valid or set `prometheus_verify_ssl = false`
 
 **Invalid query results:**
-- Ensure your query returns numeric values
-- Check the query format matches PromQL syntax
-- Verify the metrics exist in your Prometheus instance for the VMs
-- Test the query with a real VM UUID in Prometheus UI
+- Ensure your metric contains instantaneous power values in microwatts
+- Check that the metric has the `uuid` label matching VM UUIDs
+- Verify the `type_instance` label matches your configuration
+- Test the query in Prometheus UI: `sum_over_time(prometheus_value{type_instance="scaph_process_power_microwatts", uuid="<vm-uuid>"}[1h])`
 
 **No VMs found:**
 - Verify the projects are correctly configured in cASO
 - Check that VMs exist in the OpenStack environment
 - Ensure cASO has proper credentials to query Nova
+
+## Technical Details
+
+The energy calculation uses the following formula:
+
+```
+energy_wh = sum_over_time(metric{labels}[range]) * (step_seconds / 3600) / 1000000
+```
+
+Where:
+- `step_seconds / 3600` converts µW·s to µWh
+- Division by `1000000` converts µWh to Wh
+
+This approach works with metrics that export instantaneous power consumption in microwatts, sampled at the configured frequency.