Release/1.0.2 (#9)

* Refactoring base classes; new parser and parser tests.

* Refactoring base classes; New base classes for resources and objects.

* Base class refactoring #3.

* Proper session handling; added unit test cases.

* Fixed handling of empty results for get/post/put.

* Improved header preparation/handling.

* Added general integration testing fixture; Added .env file support.

* Base class refactoring; new structure for SimpleObject and ComplexObject and their respective parsers; Added base classes unit tests.

* Model base class improvements; Updated Measurement object model and tests.

* Model base class improvements.

* Module structure improvements; Model base class improvements - generic CRUD functions support additional resource spec; Fixed parser: False boolean fields were skipped always; User/Users implementation cleanup + base class adjustments; Adding integration tests for User/Users; Adding unit tests for User class; Added common test utils.

* Updated measurements API; Fixed select by series; Added integration/unit tests; Fixed on-creation definition of fragments for Devices/ManagedObjects.

* Default accept header can now be removed; Base API improvements + documentation.

* Added application key header tests.

* Base model/parsing improvements; Consolidation of ManagedObject, Device, Measurements.

* Doc fixes.

* Fixed device registry logging/initialization and added integration test cases.

* Parsing improvements to prevent not updatable fields from being part of the full json.

* Improved base class typing and handling of paths in derived classes; Cleansed GlobalRole model implementation; Adoption or User to new principles; Fixed factories for integration tests.

* Added generic object factory for broader use in integration tests.

* Added functionality to add/remove users and permissions from a global role.

* Updated/improved Events API.

* Fixed apply_to functionality.

* Long due refactoring of json parse/format; updated documentation strings.

* Updated/improved alarms API.

* Added apply_by and count functions to Alarms API.

* Documentation and cleanup.

* Moved Identity API to separate file; Added test cases.

* Fixed Binaries API; added tests.

* Documentation update.

* Added body/params to delete.

* Fixed imports; Added devicegroups API.

* Removed last_updated properties as it is already covered with update_time.

* Added safe_executor.

* Fixed handling of revert parameter.

* Fixed devicegroups; added integration tests.

* Fixed protected access, more consistent update handling.

* Added documentation.

* Moved ManagedObject classes to separate file.

* Improved documentation.

* Introducing Invoke; improved tag-based versioning.

* Removed obsolete utils.py.

* Fixed logging; fixed circular import from Identiy API.

* Removed outdated samples.

* Fixed applications.

* Added encoding to file open statements.

* Added argument hints.

* Fixed documentation.

* Fixed documentation.

* Fixed inventory roles implementation.

* Small documentation fixes; Added update_password functionality; Fixed inventory assignments.

* Linting fixes.

* Removed samples folder from linting; ignoring import errors.

* Prepare release.

* Issue/0007/cache bootstrap (#8)

* Improved caching and user experience of CumulocityApp class; added corresponding unit tests.

* Added changelog.

* Added function to resolve the tenant ID from the authorization header.

* Update CHANGELOG.md

Author: chsou

CHANGELOG.md

@@ -0,0 +1,35 @@
+# Changelog
+
+
+## Work in progress
+
+## Version 1.0.2
+
+### Changed
+
+* Added this changelog :-)
+
+* Fixed [Issue #7](https://github.com/SoftwareAG/cumulocity-python-api/issues/7):
+  Improved caching and user experience when creating CumulocityApp instances. Added unit tests.
+
+* Added possibility to resolve the tenant ID from authorization headers (both `Basic` and `Bearer`).
+
+
+## Version 1.0.1
+
+### Changed
+
+* The cumulocity-pyton-api library is now available on [PyPI](https://pypi.org) under the name `c8y_api` (see https://pypi.org/project/c8y-api/) 
+* Updated README to reflect installation from PyPI 
+
+
+## Version 1.0
+
+Major refactoring of beta version:
+* Unified user experience
+* Complete documentation
+* Performance improvements
+* Introduced `CumulocityApp` to avoid mix-up with `CumulocityApi`
+* Complete unit tests
+* Structured integration tests
+* Removed samples (sorry, need to be re-organized)

c8y_api/_base_api.py

@@ -7,12 +7,21 @@
 from __future__ import annotations
 
 import json as json_lib
-from typing import Union, Dict, BinaryIO
+import os
+from typing import Union, Dict, BinaryIO, Set
 
 import requests
 import collections
 
 
+def c8y_keys() -> Set[str]:
+    """Provide the names of defined Cumulocity environment variables.
+
+    Returns: A set of environment variable names, starting with 'C8Y_'
+    """
+    return set(filter(lambda x: 'C8Y_' in x, os.environ.keys()))
+
+
 class CumulocityRestApi:
     """Cumulocity base REST API.
 

c8y_api/app/__init__.py

@@ -3,12 +3,15 @@
 # and/or its subsidiaries and/or its affiliates and/or their licensors.
 # Use, reproduction, transfer, publication or disclosure is prohibited except
 # as specifically provided for in your License Agreement with Software AG.
-
+import base64
 import dataclasses
+import json
 import logging
 import os
-import requests
 
+from functools import lru_cache
+
+from c8y_api._base_api import c8y_keys
 from c8y_api._main_api import CumulocityApi
 
 
@@ -27,72 +30,94 @@ class CumulocityApp(CumulocityApi):
 
     If the application is executed in PER_TENANT mode, all necessary
     authentication information is provided directly by Cumulocity as
-    environment variables injected into the Docker container.
+    environment variables injected into the Docker container. A corresponding
+    instance can be created by invoking `CumulocityApp()` (without any parameter).
 
     If the application is executed in MULTITENANT mode, only the so-called
     bootstrap user's authentication information is injected into the Docker
     container. An CumulocityApi instances providing access to a specific
     tenant can be obtained buy the application using the
-    `get_tenant_instance` function.
+    `get_tenant_instance` function or by invoking `CumulocityApp(id)` (with the
+    ID of the tenant that should handle the requests).
     """
     @dataclasses.dataclass
     class Auth:
         """Bundles authentication information."""
         username: str
         password: str
 
-    __auth_by_tenant = {}
-    __bootstrap_instance = None
-    __tenant_instances = {}
+    _auth_by_tenant = {}
+    _bootstrap_instance = None
+    _tenant_instances = {}
+
+    _log = logging.getLogger(__name__)
+
+    def __init__(self, tenant_id: str = None, application_key: str = None):
+        """Create a new tenant specific instance.
 
-    __log = logging.getLogger(__name__)
+        Args:
+            tenant_id (str|None):  If None, it is assumed that the application
+                is running in an PER_TENANT environment and the instance is
+                created for the injected tenant information. Otherwise it is
+                assumed that the application is running in a MULTITENANT
+                environment and the provided ID reflects the ID of a
+                subscribed tenant.
+            application_key (str|None): An application key to include in
+                all requests for tracking purposes.
 
-    def __init__(self, tenant_id=None, application_key=None):
-        self.baseurl = self.__get_env('C8Y_BASEURL')
+        Returns:
+            A new CumulocityApp instance
+        """
         if tenant_id:
             self.tenant_id = tenant_id
-            bootstrap_tenant_id = self.__get_env('C8Y_BOOTSTRAP_TENANT')
-            bootstrap_username = self.__get_env('C8Y_BOOTSTRAP_USER')
-            bootstrap_password = self.__get_env('C8Y_BOOTSTRAP_PASSWORD')
-            self.__bootstrap_auth = self.__build_auth(bootstrap_tenant_id, bootstrap_username, bootstrap_password)
-            auth = self.__get_auth(tenant_id)
-            self.username = auth.username
-            self.__password = auth.password
-            super().__init__(self.baseurl, self.tenant_id, auth.username, auth.password,
+            auth = self._get_tenant_auth(tenant_id)
+            baseurl = self._get_env('C8Y_BASEURL')
+            super().__init__(baseurl, tenant_id, auth.username, auth.password,
                              tfa_token=None, application_key=application_key)
         else:
-            self.tenant_id = self.__get_env('C8Y_TENANT')
-            self.username = self.__get_env('C8Y_USER')
-            self.__password = self.__get_env('C8Y_PASSWORD')
-            super().__init__(self.baseurl, self.tenant_id, self.username, self.__password,
+            baseurl = self._get_env('C8Y_BASEURL')
+            tenant_id = self._get_env('C8Y_TENANT')
+            username = self._get_env('C8Y_USER')
+            password = self._get_env('C8Y_PASSWORD')
+            super().__init__(baseurl, tenant_id, username, password,
                              tfa_token=None, application_key=application_key)
 
     @staticmethod
-    def __get_env(name):
-        val = os.getenv(name)
-        assert val, "Missing environment variable: " + name
-        return val
+    def _get_env(name: str) -> str:
+        try:
+            return os.environ[name]
+        except KeyError as e:
+            raise ValueError(f"Missing environment variable: {name}. Found {', '.join(c8y_keys())}.") from e
 
-    @staticmethod
-    def __build_auth(tenant_id, username, password):
-        return f'{tenant_id}/{username}', password
-
-    def __get_auth(self, tenant_id):
-        if tenant_id not in self.__auth_by_tenant:
-            self.__update_auth_cache()
-        return self.__auth_by_tenant[tenant_id]
-
-    def __update_auth_cache(self):
-        r = requests.get(self.baseurl + '/application/currentApplication/subscriptions', auth=self.__bootstrap_auth)
-        if r.status_code != 200:
-            self.__log.error("Unable to perform GET request. Status: {}, Response: {}", r.status_code, r.text)
-        self.__log.info("get subscriptions: {}", r.json())
-        self.__auth_by_tenant.clear()
-        for subscription in r.json()['users']:
+    @classmethod
+    def _get_tenant_auth(cls, tenant_id: str) -> Auth:
+        if tenant_id not in cls._auth_by_tenant:
+            cls._auth_by_tenant = cls._read_subscriptions()
+        return cls._auth_by_tenant[tenant_id]
+
+    @classmethod
+    def _read_subscriptions(cls):
+        """Read subscribed tenant's auth information.
+
+        Returns:
+            A dict of tenant auth information by ID
+        """
+        subscriptions = cls.get_bootstrap_instance().get('/application/currentApplication/subscriptions')
+        cache = {}
+        for subscription in subscriptions['users']:
             tenant = subscription['tenant']
             username = subscription['name']
             password = subscription['password']
-            self.__auth_by_tenant[tenant] = CumulocityApp.Auth(username, password)
+            cache[tenant] = CumulocityApp.Auth(username, password)
+        return cache
+
+    @classmethod
+    def _create_bootstrap_instance(cls) -> CumulocityApi:
+        baseurl = cls._get_env('C8Y_BASEURL')
+        tenant_id = cls._get_env('C8Y_BOOTSTRAP_TENANT')
+        username = cls._get_env('C8Y_BOOTSTRAP_USER')
+        password = cls._get_env('C8Y_BOOTSTRAP_PASSWORD')
+        return CumulocityApi(baseurl, tenant_id, username, password)
 
     @classmethod
     def get_bootstrap_instance(cls) -> CumulocityApi:
@@ -102,21 +127,70 @@ def get_bootstrap_instance(cls) -> CumulocityApi:
         Returns:
             A CumulocityApi instance authorized for the bootstrap user
         """
-        if not cls.__bootstrap_instance:
-            cls.__bootstrap_instance = CumulocityApp()
-        return cls.__bootstrap_instance
+        if not cls._bootstrap_instance:
+            cls._bootstrap_instance = cls._create_bootstrap_instance()
+        return cls._bootstrap_instance
 
     @classmethod
-    def get_tenant_instance(cls, tenant_id) -> CumulocityApi:
+    def get_tenant_instance(cls, tenant_id: str = None, headers: dict = None) -> CumulocityApi:
         """Provide access to a tenant-specific instance in a multi-tenant
         application setup.
 
         Args:
             tenant_id (str):  ID of the tenant to get access to
+            headers (dict):  Inbound request headers, the tenant ID
+                is resolved from the Authorization header
 
         Returns:
             A CumulocityApi instance authorized for a tenant user
         """
-        if tenant_id not in cls.__tenant_instances:
-            cls.__tenant_instances[tenant_id] = CumulocityApp(tenant_id)
-        return cls.__tenant_instances[tenant_id]
+        # (1) if the tenant ID is specified we just
+        if tenant_id:
+            return cls._get_tenant_instance(tenant_id)
+
+        # (2) otherwise, look for the Authorization header
+        if not headers:
+            raise RuntimeError("At least one of 'tenant_id' or 'headers' must be specified.")
+
+        auth_header = headers[next(filter(lambda k: 'Authorization'.upper() == k.upper(), headers.keys()))]
+        if not auth_header:
+            raise ValueError("Missing Authentication header. Unable to resolve tenant ID.")
+
+        return cls._get_tenant_instance(cls._resolve_tenant_id_from_auth_header(auth_header))
+
+    @classmethod
+    def _get_tenant_instance(cls, tenant_id: str) -> CumulocityApi:
+        if tenant_id not in cls._tenant_instances:
+            cls._tenant_instances[tenant_id] = CumulocityApp(tenant_id)
+        return cls._tenant_instances[tenant_id]
+
+    @classmethod
+    @lru_cache(maxsize=128, typed=False)
+    def _resolve_tenant_id_from_auth_header(cls, auth_header: str) -> str:
+        auth_type, auth_value = auth_header.split(' ')
+
+        if auth_type.upper() == 'BASIC':
+            return cls._resolve_tenant_id_basic(auth_value)
+        if auth_type.upper() == 'BEARER':
+            return cls._resolve_tenant_id_token(auth_value)
+
+        raise ValueError(f"Unexpected authorization header type: {auth_type}")
+
+    @staticmethod
+    def _resolve_tenant_id_basic(auth_string: str) -> str:
+        decoded = base64.b64decode(bytes(auth_string, 'utf-8'))
+        username = decoded.split(b':', 1)[0]
+        if b'/' not in username:
+            raise ValueError(f"nable to resolve tenant ID. Username '{username}' does not appear to include it.")
+        return username.split(b'/', 1)[0].decode('utf-8')
+
+    @classmethod
+    def _resolve_tenant_id_token(cls, auth_token: str) -> str:
+        # we assume that the token is an JWT token
+        jwt_parts = auth_token.split('.')
+        if len(jwt_parts) != 3:
+            raise ValueError("Unexpected token format (not an JWT?). Unable to resolve tenant ID.")
+        jwt_body = json.loads(base64.b64decode(jwt_parts[1].encode('utf-8')))
+        if 'ten' not in jwt_body:
+            raise ValueError("Unexpected token format (missing 'ten' claim). Unable to resolve tenant ID.")
+        return jwt_body['ten']

integration_tests/conftest.py

@@ -12,6 +12,7 @@
 from dotenv import load_dotenv
 import pytest
 
+from c8y_api._base_api import c8y_keys
 from c8y_api.app import CumulocityApp
 from c8y_api.model import Device
 
@@ -49,9 +50,6 @@ def logger():
 def test_environment(logger):
     """Prepare the environment, i.e. read a .env file if found."""
 
-    def c8y_keys():
-        return filter(lambda x: 'C8Y_' in x, os.environ.keys())
-
     # check if there is a .env file
     if os.path.exists('.env'):
         logger.info("Environment file (.env) exists and will be considered.")

requirements.txt

@@ -7,4 +7,5 @@ responses~=0.13
 python-dotenv~=0.19.0
 setuptools_scm~=6.2
 invoke~=1.6.0
-pylint=2.11.1
+pylint=2.11.1
+PyJWT~=2.3.0