'401 Client Error: Unauthorized for url: https://tenant.sharepoint.com/sites/site/_api/contextInfo' when uploading a file to SharePoint with cert auth

[nicklockhart-fullfibre]

Hi all,

Not sure if this is an issue with my environment/setup or a bug with the library. I'm currently trying to migrate a script using SAML auth (see #981) to certificate-based app-only auth. I have a cert which was generated as per the Microsoft documentation, which results in a PFX-format private key. I can see that this library needs a PEM-format private key.

I converted the PFX-format key to PEM-format using openssl:

openssl pkcs12 -in myapp.pfx -out myapp.pem -nodes

If I then check the fingerprint of the PEM-format key, it matches the PFK-format key:

openssl x509 -fingerprint -noout -in myapp.pem
SHA1 Fingerprint=8F:42:8E:60[...]

However, if I then try to upload a file to Sharepoint using certificate authentication...

    login_cert = {
        'tenant': 'tenant.onmicrosoft.com',
        'client_id': 'bc4b906b[...]',
        'thumbprint': '8f428e60[...]',
        'cert_path': '/usr/local/apps/move_to_sharepoint/myapp.pem',
    }
    ctx = ClientContext(site_url).with_client_certificate(**login_cert)

    # Specify destination folder
    folder = ctx.web.get_folder_by_server_relative_url("[path]")

    # Upload file
    local_file_path = args.filename
    with open(local_file_path, "rb") as f:
        file = folder.files.upload(f).execute_query()

...the following error is thrown:

Traceback (most recent call last):
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/office365/runtime/client_request.py", line 37, in execute_query
    response = self.execute_request_direct(request)
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/office365/runtime/client_request.py", line 46, in execute_request_direct
    self.beforeExecute.notify(request)
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/office365/runtime/types/event_handler.py", line 41, in notify
    listener(*args, **kwargs)
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/office365/sharepoint/client_context.py", line 278, in _build_modification_query
    self._ensure_form_digest(request)
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/office365/sharepoint/client_context.py", line 216, in _ensure_form_digest
    self._ctx_web_info = self._get_context_web_information()
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/office365/sharepoint/client_context.py", line 228, in _get_context_web_information
    response = client.execute_request_direct(request)
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/office365/runtime/client_request.py", line 101, in execute_request_direct
    response.raise_for_status()
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://tenant.sharepoint.com/sites/site/_api/contextInfo

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/apps/move_to_sharepoint/move_to_sharepoint.py", line 37, in <module>
    file = folder.files.upload(f).execute_query()
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/office365/runtime/client_object.py", line 55, in execute_query
    self.context.execute_query()
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/office365/runtime/client_runtime_context.py", line 173, in execute_query
    self.pending_request().execute_query(qry)
  File "/usr/local/apps/move_to_sharepoint/venv/lib/python3.10/site-packages/office365/runtime/client_request.py", line 41, in execute_query
    raise ClientRequestException(*e.args, response=e.response)
office365.runtime.client_request_exception.ClientRequestException: (None, None, '401 Client Error: Unauthorized for url: https://tenant.sharepoint.com/sites/site/_api/contextInfo')

The app has been granted Sites.FullControl.All as per the library's documentation, so I don't think this is a permissions issue. The file upload code should be correct and previously worked OK when using username/password auth (sadly now deprecated).

Is this an issue with my environment (eg PKCS key converted to X509 via OpenSSL), a bug in my code (eg wrong params passed to with_client_certificate) or have I hit a bug in the library?

I'm using the latest version of the library (2.6.2) with Python 3.10.12 on Ubuntu 22.04 LTS, if that affects anything.

[sengatang]

From our communication with Microsoft, using Azure AD Certified Authentication requires the following steps:
Step 1: Register an App in Azure AD

1. Go to https://portal.azure.com/.
2. Navigate to Azure Active Directory > App registrations > New registration.
3. Enter a name (e.g., SharePointAccessScript).
4. Set Supported account types to "Accounts in this organizational directory only".
5. Click Register.

Step 2:Configure API Permissions

1. Grant Application Permission (Sites.Read.All or Sites.FullControl.All)
2. Grant admin consent for those permissions

Step 3: Create and configure a Client Secret

Step 4: Update Python Script

Maybe you can check your permissions?

And just a thought for the pfx and pem, from what GPT told me"office365-rest-python-client requires the private key to be in the standard PKCS#8 format"

You can use the following command to convert the private key to PKCS#8 format:

openssl pkcs12 -in your_cert.pfx -nocerts -nodes | openssl pkcs8 -topk8 -nocrypt -out private_key.pem

[nicklockhart-fullfibre]

I have already created an application in our Azure portal and configured it correctly. The application has Sites.FullControl.All permissions, as suggested by Microsoft.

from what GPT told me

Please don't use ChatGPT to respond to other people's tickets. If I wanted to hear AI's best guess at a solution, I would've already asked it myself.

I did try converting the cert to PKCS8 format using the snippet provided, but this did not resolve the issue. I receive the same 401 error from the same contextInfo endpoint as I did with my original cert (converted from PFX to PEM using the snippet in my original comment - openssl pkcs12 -in myapp.pfx -out myapp.pem -nodes)

[sengatang]

Fine, I did not just quote GPT.
This is what it responded to me and worked for me.