Skip to content

Commit 5ce5f9a

Browse files
pvaneckpvaneck
authored
[Monitor] Add TypeSpec-based azure-monitor-querymetrics package (#42174)
Signed-off-by: Paul Van Eck <paulvaneck@microsoft.com>
1 parent 7576eed commit 5ce5f9a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+6296
-1
lines changed

‎.vscode/cspell.json

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1071,7 +1071,10 @@
10711071
"percentilesw",
10721072
"plotly",
10731073
"nbformat",
1074-
"nbconvert"
1074+
"nbconvert",
1075+
"rollupby",
1076+
"milli",
1077+
"resourceid"
10751078
]
10761079
},
10771080
{

‎sdk/monitor/azure-monitor-querymetrics/CHANGELOG.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Release History
2+
3+
## 1.0.0 (Unreleased)
4+
5+
### Features Added
6+
7+
- Initial release. This version includes the core functionality for querying metrics from Azure Monitor using the `MetricsClient`, which was originally introduced in the `azure-monitor-query` package.
8+
9+
### Breaking Changes
10+
11+
### Bugs Fixed
12+
13+
### Other Changes

‎sdk/monitor/azure-monitor-querymetrics/LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
Copyright (c) Microsoft Corporation.
2+
3+
MIT License
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎sdk/monitor/azure-monitor-querymetrics/MANIFEST.in

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
include *.md
2+
include LICENSE
3+
include azure/monitor/querymetrics/py.typed
4+
recursive-include tests *.py
5+
recursive-include samples *.py *.md
6+
include azure/__init__.py
7+
include azure/monitor/__init__.py

‎sdk/monitor/azure-monitor-querymetrics/README.md

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
# Azure Monitor Query Metrics client library for Python
2+
3+
The Azure Monitor Query Metrics client library is used to execute read-only queries against [Azure Monitor][azure_monitor_overview]'s metrics data platform:
4+
5+
- [Metrics](https://learn.microsoft.com/azure/azure-monitor/essentials/data-platform-metrics) - Collects numeric data from monitored resources into a time series database. Metrics are numerical values that are collected at regular intervals and describe some aspect of a system at a particular time. Metrics are lightweight and capable of supporting near real-time scenarios, making them useful for alerting and fast detection of issues.
6+
7+
**Resources:**
8+
9+
<!-- TODO: Add PyPI, Conda, Ref Docs, Samples links-->
10+
- [Source code][source]
11+
- [Service documentation][azure_monitor_overview]
12+
- [Change log][changelog]
13+
14+
## Getting started
15+
16+
### Prerequisites
17+
18+
- Python 3.9 or later
19+
- An [Azure subscription][azure_subscription]
20+
- Authorization to read metrics data at the Azure subscription level. For example, the [Monitoring Reader role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/monitor#monitoring-reader) on the subscription containing the resources to be queried.
21+
- An Azure resource of any kind (Storage Account, Key Vault, Cosmos DB, etc.).
22+
23+
### Install the package
24+
25+
Install the Azure Monitor Query Metrics client library for Python with [pip][pip]:
26+
27+
```bash
28+
pip install azure-monitor-querymetrics
29+
```
30+
31+
### Create the client
32+
33+
An authenticated client is required to query Metrics. The library includes both synchronous and asynchronous forms of the client. To authenticate, create an instance of a token credential. Use that instance when creating a `MetricsClient`. The following examples use `DefaultAzureCredential` from the [azure-identity](https://pypi.org/project/azure-identity/) package.
34+
35+
#### Synchronous client
36+
37+
Consider the following example, which creates a synchronous client for Metrics querying:
38+
39+
```python
40+
from azure.identity import DefaultAzureCredential
41+
from azure.monitor.querymetrics import MetricsClient
42+
43+
credential = DefaultAzureCredential()
44+
metrics_client = MetricsClient("https://<regional endpoint>", credential)
45+
```
46+
47+
#### Asynchronous client
48+
49+
The asynchronous form of the client API is found in the `.aio`-suffixed namespace. For example:
50+
51+
```python
52+
from azure.identity.aio import DefaultAzureCredential
53+
from azure.monitor.querymetrics.aio import MetricsClient
54+
55+
credential = DefaultAzureCredential()
56+
async_metrics_client = MetricsClient("https://<regional endpoint>", credential)
57+
```
58+
59+
To use the asynchronous clients, you must also install an async transport, such as [aiohttp](https://pypi.org/project/aiohttp/).
60+
61+
```sh
62+
pip install aiohttp
63+
```
64+
65+
#### Configure client for Azure sovereign cloud
66+
67+
By default, the client is configured to use the Azure public cloud. To use a sovereign cloud, provide the correct `audience` argument when creating the `MetricsClient`. For example:
68+
69+
```python
70+
from azure.identity import AzureAuthorityHosts, DefaultAzureCredential
71+
from azure.monitor.querymetrics import MetricsClient
72+
73+
# Authority can also be set via the AZURE_AUTHORITY_HOST environment variable.
74+
credential = DefaultAzureCredential(authority=AzureAuthorityHosts.AZURE_GOVERNMENT)
75+
76+
metrics_client = MetricsClient(
77+
"https://usgovvirginia.metrics.monitor.azure.us", credential, audience="https://metrics.monitor.azure.us"
78+
)
79+
```
80+
81+
### Execute the query
82+
83+
For examples of Metrics queries, see the [Examples](#examples) section.
84+
85+
## Key concepts
86+
87+
### Metrics data structure
88+
89+
Each set of metric values is a time series with the following characteristics:
90+
91+
- The time the value was collected
92+
- The resource associated with the value
93+
- A namespace that acts like a category for the metric
94+
- A metric name
95+
- The value itself
96+
- Some metrics have multiple dimensions as described in multi-dimensional metrics.
97+
98+
## Examples
99+
100+
- [Metrics query](#metrics-query)
101+
- [Handle metrics query response](#handle-metrics-query-response)
102+
103+
### Metrics query
104+
105+
To query metrics for one or more Azure resources, use the `query_resources` method of `MetricsClient`. This method requires a regional endpoint when creating the client. For example, "https://westus3.metrics.monitor.azure.com".
106+
107+
Each Azure resource must reside in:
108+
109+
- The same region as the endpoint specified when creating the client.
110+
- The same Azure subscription.
111+
112+
The resource IDs must be that of the resources for which metrics are being queried. It's normally of the format `/subscriptions/<id>/resourceGroups/<rg-name>/providers/<source>/topics/<resource-name>`.
113+
114+
To find the resource ID/URI:
115+
116+
1. Navigate to your resource's page in the Azure portal.
117+
1. Select the **JSON View** link in the **Overview** section.
118+
1. Copy the value in the **Resource ID** text box at the top of the JSON view.
119+
120+
Furthermore:
121+
122+
- The user must be authorized to read monitoring data at the Azure subscription level. For example, the [Monitoring Reader role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/monitor#monitoring-reader) on the subscription to be queried.
123+
- The metric namespace containing the metrics to be queried must be provided. For a list of metric namespaces, see [Supported metrics and log categories by resource type][metric_namespaces].
124+
125+
```python
126+
from datetime import timedelta
127+
import os
128+
129+
from azure.core.exceptions import HttpResponseError
130+
from azure.identity import DefaultAzureCredential
131+
from azure.monitor.querymetrics import MetricsClient, MetricAggregationType
132+
133+
endpoint = "https://westus3.metrics.monitor.azure.com"
134+
credential = DefaultAzureCredential()
135+
client = MetricsClient(endpoint, credential)
136+
137+
resource_ids = [
138+
"/subscriptions/<id>/resourceGroups/<rg-name>/providers/<source>/storageAccounts/<resource-name-1>",
139+
"/subscriptions/<id>/resourceGroups/<rg-name>/providers/<source>/storageAccounts/<resource-name-2>"
140+
]
141+
142+
response = client.query_resources(
143+
resource_ids=resource_ids,
144+
metric_namespace="Microsoft.Storage/storageAccounts",
145+
metric_names=["UsedCapacity"],
146+
timespan=timedelta(hours=2),
147+
granularity=timedelta(minutes=5),
148+
aggregations=[MetricAggregationType.AVERAGE],
149+
)
150+
151+
for metrics_query_result in response:
152+
for metric in metrics_query_result.metrics:
153+
print(f"Metric: {metric.name}")
154+
for time_series in metric.timeseries:
155+
for metric_value in time_series.data:
156+
if metric_value.average is not None:
157+
print(f"Average: {metric_value.average}")
158+
```
159+
160+
#### Handle metrics query response
161+
162+
The metrics query API returns a list of `MetricsQueryResult` objects. The `MetricsQueryResult` object contains properties such as a list of `Metric`-typed objects, `granularity`, `namespace`, and `timespan`. The `Metric` objects list can be accessed using the `metrics` param. Each `Metric` object in this list contains a list of `TimeSeriesElement` objects. Each `TimeSeriesElement` object contains `data` and `metadata_values` properties. In visual form, the object hierarchy of the response resembles the following structure:
163+
164+
```
165+
MetricsQueryResult
166+
|---granularity
167+
|---timespan
168+
|---cost
169+
|---namespace
170+
|---resource_region
171+
|---metrics (list of `Metric` objects)
172+
|---id
173+
|---type
174+
|---name
175+
|---unit
176+
|---timeseries (list of `TimeSeriesElement` objects)
177+
|---metadata_values
178+
|---data (list of data points)
179+
```
180+
181+
**Note:** Each `MetricsQueryResult` is returned in the same order as the corresponding resource in the `resource_ids` parameter. If multiple different metrics are queried, the metrics are returned in the order of the `metric_names` sent.
182+
183+
**Example of handling response**
184+
185+
```python
186+
import os
187+
from azure.monitor.querymetrics import MetricsClient, MetricAggregationType
188+
from azure.identity import DefaultAzureCredential
189+
190+
credential = DefaultAzureCredential()
191+
client = MetricsClient("https://<regional endpoint>", credential)
192+
193+
metrics_uri = os.environ['METRICS_RESOURCE_URI']
194+
response = client.query_resource(
195+
metrics_uri,
196+
metric_names=["PublishSuccessCount"],
197+
aggregations=[MetricAggregationType.AVERAGE]
198+
)
199+
200+
for metrics_query_result in response:
201+
for metric in metrics_query_result.metrics:
202+
print(f"Metric: {metric.name}")
203+
for time_series in metric.timeseries:
204+
for metric_value in time_series.data:
205+
if metric_value.average is not None:
206+
print(f"Average: {metric_value.average}")
207+
```
208+
209+
## Troubleshooting
210+
211+
See our [troubleshooting guide][troubleshooting_guide] for details on how to diagnose various failure scenarios.
212+
213+
## Next steps
214+
215+
To learn more about Azure Monitor, see the [Azure Monitor service documentation][azure_monitor_overview].
216+
217+
### Samples
218+
219+
The following code samples show common scenarios with the Azure Monitor Query Metrics client library.
220+
221+
#### Metrics query samples
222+
223+
To be added.
224+
225+
## Contributing
226+
227+
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit [cla.microsoft.com][cla].
228+
229+
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.
230+
231+
This project has adopted the [Microsoft Open Source Code of Conduct][code_of_conduct]. For more information see the [Code of Conduct FAQ][coc_faq] or contact [opencode@microsoft.com][coc_contact] with any additional questions or comments.
232+
233+
<!-- LINKS -->
234+
235+
[azure_core_exceptions]: https://aka.ms/azsdk/python/core/docs#module-azure.core.exceptions
236+
[azure_core_ref_docs]: https://aka.ms/azsdk/python/core/docs
237+
[azure_monitor_overview]: https://learn.microsoft.com/azure/azure-monitor/
238+
[azure_subscription]: https://azure.microsoft.com/free/python/
239+
[changelog]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-querymetrics/CHANGELOG.md
240+
[metric_namespaces]: https://learn.microsoft.com/azure/azure-monitor/reference/supported-metrics/metrics-index#supported-metrics-and-log-categories-by-resource-type
241+
[pip]: https://pypi.org/project/pip/
242+
[python_logging]: https://docs.python.org/3/library/logging.html
243+
[samples]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-querymetrics/samples
244+
[source]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-querymetrics/
245+
[troubleshooting_guide]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-querymetrics/TROUBLESHOOTING.md
246+
247+
[cla]: https://cla.microsoft.com
248+
[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/
249+
[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
250+
[coc_contact]: mailto:opencode@microsoft.com

‎sdk/monitor/azure-monitor-querymetrics/TROUBLESHOOTING.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Troubleshooting Azure Monitor Query Metrics client library issues
2+
3+
This troubleshooting guide contains instructions to diagnose frequently encountered issues while using the Azure Monitor Query Metrics client library for Python.
4+
5+
## Table of contents
6+
7+
* [General Troubleshooting](#general-troubleshooting)
8+
* [Enable client logging](#enable-client-logging)
9+
* [Troubleshooting authentication issues with metrics query requests](#authentication-errors)
10+
* [Troubleshooting running async APIs](#errors-with-running-async-apis)
11+
* [Troubleshooting Metrics Query](#troubleshooting-metrics-query)
12+
* [Troubleshooting insufficient access error](#troubleshooting-insufficient-access-error-for-metrics-query)
13+
* [Troubleshooting unsupported granularity for metrics query](#troubleshooting-unsupported-granularity-for-metrics-query)
14+
* [Additional azure-core configurations](#additional-azure-core-configurations)
15+
16+
17+
## General Troubleshooting
18+
19+
Monitor query raises exceptions described in [`azure-core`](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md)
20+
21+
### Enable client logging
22+
23+
To troubleshoot issues with Azure Monitor Query Metrics library, it is important to first enable logging to monitor the behavior of the application. The errors and warnings in the logs generally provide useful insights into what went wrong and sometimes include corrective actions to fix issues.
24+
25+
This library uses the standard [logging](https://docs.python.org/3/library/logging.html) library for logging. Basic information about HTTP sessions, such as URLs and headers, is logged at the INFO level.
26+
Detailed DEBUG level logging, including request/response bodies and unredacted headers, can be enabled on a client with the logging_enable argument:
27+
28+
```python
29+
import logging
30+
from azure.monitor.querymetrics import MetricsClient
31+
32+
# Create a logger for the 'azure.monitor.querymetrics' SDK
33+
logger = logging.getLogger('azure.monitor.querymetrics')
34+
logger.setLevel(logging.DEBUG)
35+
36+
# Configure a console output
37+
handler = logging.StreamHandler(stream=sys.stdout)
38+
logger.addHandler(handler)
39+
40+
client = MetricsClient(credential, logging_enable=True)
41+
```
42+
43+
Similarly, logging_enable can enable detailed logging for a single operation, even when it isn't enabled for the client:
44+
45+
```python
46+
client.query_workspace(logging_enable=True)
47+
```
48+
49+
### Authentication errors
50+
51+
Azure Monitor Query Metrics supports Azure Active Directory authentication. The MetricsClient has methods to set the `credential`. To provide a valid credential, you can use
52+
`azure-identity` dependency. For more details on getting started, refer to
53+
the [README](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-querymetrics#create-the-client)
54+
of Azure Monitor Query Metrics library. You can also refer to
55+
the [Azure Identity documentation](https://learn.microsoft.com/python/api/overview/azure/identity-readme)
56+
for more details on the various types of credential supported in `azure-identity`.
57+
58+
For more help on troubleshooting authentication errors please see the Azure Identity client library [troubleshooting guide](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/azure-identity/TROUBLESHOOTING.md).
59+
60+
### Errors with running async APIs
61+
62+
The async transport is designed to be opt-in. [AioHttp](https://pypi.org/project/aiohttp/) is one of the supported implementations of async transport. It is not installed by default. You need to install it separately as follows:
63+
64+
```
65+
pip install aiohttp
66+
```
67+
68+
## Troubleshooting Metrics Query
69+
70+
### Troubleshooting insufficient access error for metrics query
71+
72+
If you encounter 401 authorization errors while querying metrics using `MetricsClient`, please ensure you are authorized to read monitoring data at the Azure subscription level. For example, the [Monitoring Reader role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/monitor#monitoring-reader) on the subscription to be queried.
73+
74+
### Troubleshooting unsupported granularity for metrics query
75+
76+
If you notice the following exception, this is due to an invalid time granularity in the metrics query request. Your
77+
query might have set the `granularity` keyword argument to an unsupported duration.
78+
79+
```text
80+
"{"code":"BadRequest","message":"Invalid time grain duration: PT10M, supported ones are: 00:01:00,00:05:00,00:15:00,00:30:00,01:00:00,06:00:00,12:00:00,1.00:00:00"}"
81+
```
82+
83+
As documented in the error message, the supported granularity for metrics queries are 1 minute, 5 minutes, 15 minutes,
84+
30 minutes, 1 hour, 6 hours, 12 hours and 1 day.
85+
86+
## Additional azure-core configurations
87+
88+
When calling the methods, some properties including `retry_mode`, `timeout`, `connection_verify` can be configured by passing in as keyword arguments. See
89+
[configurations](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md#configurations) for list of all such properties.

0 commit comments

Comments
 (0)