Automate AzureMFA metadata retrieval and update

[phavekes]

Currently, the institution.yaml (definition of accepted endpoints) is generated manually, based upon the Federation Metadata url. The certificate in this configuration expires. This does not cause an issue, because the certificate validity date is not validated.

However, the expiration of the certificate does cause institutions to generate and apply a new certificate, and this causes authentications to fail, because then the validation of XML signature on the SAML assertion / response will fail.

To solve this, we want to load and cache the information from the Federation Metadata url. There are two places where metadata refresh can be triggered

1. Before sending an AuthnRequest - we need tot sso_location and entity_id to send and AuthnRequest to the Azure IdP.
2. We also need to use an unusual "just-in-time" trigger for refreshing the metadata of an entity – we refresh the metadata when we receive a SAML Response that was signed with a different certificate than that we have in our cache for the entity / institution. See Automate AzureMFA metadata retrieval and update #194 (comment) for why.

• Add global config parameters for federation_metadata_cache_location

• Be backwards compatible. If no metadata_url is configured for an entry in institution.yml, use the current behaviour

• If :

  • a SAML Response is received, AND
  • metadata_url is present for an entry in institutions.yml, AND
  • no cache entry is present when sending the AutnRequest OR the certificate in the Signature in the SAML Response does not match one of the certificates from the cache then:
    • fetch the new metadata for the institution from the metadata_url
    • cache the information from the metadata (entityid, ssolocation, certificate) in a configurable file path.
    • use a separate cache file for each key in institutions.yml, using the key in the filename of the cash file.

• An entry in institution.yml must at least hold:

  • identifier
  • email_domains
  • and either
    • metadata_location
  • or
    • entity_id
    • sso_location
    • certificates

• log:

  • metadata cache-misses
  • signature validation failures
  • refreshing of metadata (start, success, failure),
  • the certificate that triggered a cache refresh.
  • on cache write all the info that was written to the cache:
    • certificates
    • entity_id
    • sso_location

[phavekes]

Will resolve #156 and OpenConext/Stepup-Project#444

[phavekes]

Test metatadata location : https://login.microsoftonline.com/8cd24984-0aa3-4fc5-b09b-4d6ecfaa58fb/federationmetadata/2007-06/federationmetadata.xml?appid=dafeea3f-2071-4151-9287-b90a899a6987

The relevant information to extract from the metadata:

• From the root element (EntityDescriptor):
  • entity_id – The value of the entityID attribute

<EntityDescriptor xmlns="urn:oasis:names:tc:SAML:2.0:metadata" ID="_b036d81b-01ef-4341-898c-4631c44c7f76" entityID="https://sts.windows.net/8cd24984-0aa3-4fc5-b09b-4d6ecfaa58fb/">

• From the IDPSSODescriptor element (at the and of the XML)
  • certificates – all the contents of each X509Certificate element from the KeyDescriptor with use = signing. (there is probably only going to be one, but if the Azure metadata is going to become more standard and include both the active and new certificate, we're ready if we read them all.
  • The Location from the first SingleSignOnService with Binding urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST. (Having multiple SingleSignOnService elements with the same Binding is not allowed by the spec. So there should never be more than one)

<IDPSSODescriptor protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
  <KeyDescriptor use="signing">
    <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
      <X509Data>
        <X509Certificate>MIIC8DCCAdigAwIBAgIQSq .... tjZgKWwmiR</X509Certificate>
      </X509Data>
    </KeyInfo>
  </KeyDescriptor>
  <SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://login.microsoftonline.com/8cd24984-0aa3-4fc5-b09b-4d6ecfaa58fb/saml2"/>
  <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://login.microsoftonline.com/8cd24984-0aa3-4fc5-b09b-4d6ecfaa58fb/saml2"/>
  <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://login.microsoftonline.com/8cd24984-0aa3-4fc5-b09b-4d6ecfaa58fb/saml2"/>
</IDPSSODescriptor>

It is an error if:

• the root element is not EntityDescriptor
• the entityID attribute is not present
• there are no signing certificates found
• there is no SingleSignOnService with Binding urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST

You do not have to validate the signature on the metadata, the fact that it was downloaded over https from a trusted location is sufficient.

[pmeulen]

We found that the Azure metadata behaves differently than expected from a SAML IdP during a key rollover. The IdP metadata only ever contains one certificate, never more. Under normal circumstances this is the certificate that is used for signing SAML as expected.

However, when a new certificate is generated this new certificate is published in the metadata but this certificate is not used for signing, and the certificate that is used for signing is not present in the metadata. Only after the Azure admin activates the certificate, it is used for signing.

This in between "generated" state makes it impossible to do a "cold start" of the Azure IdP connection based on the metadata, and means that the SP must keep state.

It is not possible to see in which "state" the metadata is by looking at the metadata alone. I.e. "generated" and "activated" states look exactly the same.

We discussed how to implement key rollover. We do not like the "dashboard" solution where admins have to manually coordinate the certificate rollover process. Reasons: overhead (dashboard implementation) and it is a manual process.

We chose a solution where we automatically detect when we should refresh the metadata by looking at the certificate in the SAML response in the Azure IdP response. If the certificate does not match the certificate that we have in the cash, this triggers a metadata refresh of the entity. We perform this refresh-cache update inline, when receiving the response. This way we update the cache just-in-time.

[pmeulen]

Metadata cache format proposal:

{
  "updated": "2025-06-10T11:23:04Z",
  "entity_id": "https://sts.windows.net/8cd24984-0aa3-4fc5-b09b-4d6ecfaa58fb/",
  "sso_location": "https://login.microsoftonline.com/8cd24984-0aa3-4fc5-b09b-4d6ecfaa58fb/saml2",
  "certificates": [
    "MIIC8DCCAdigAwIBAgIQSqj0oiRjb .... Se8ut81Lhv5tjZgKWwmiR"
  ]
}

The "key" in the cash is the institution id. That works well when sending the AuthnRequest.
When receiving the SAML Response, we need to retrieve the metadata, this means that we need to store the cache key (institution id) in the HTTP session.

[pablothedude]

This feature is merged and can be tested:
https://github.com/OpenConext/Stepup-AzureMFA/releases/tag/2.2.0-beta2