Inconsistent encoding/decoding of identifiers with special characters

[mrkvon]

Summary

Special characters in {identifier} don't get decoded, leading to inconsistent behaviour, including not being decoded in callbacks, and being double URI-encoded in collections.

Background

I use fedify to add a layer of ActivityPub on top of Solid Pod. This is achieved by hosting actor and its webfinger on a Solid Pod, and providing ActivityPub endpoints by the agent layer, on a different domain; with {identifier} being the URI of the actor. (and yes, it works)

Expected Behavior

Special characters in {identifier} get decoded in callbacks, and then encoded only once in collection id.

Actual Behavior

When my identifiers have URI-encoded special characters, they don't get decoded. Initially I assumed it was not an issue, as I decodeURIComponent'd the identifier manually.

e.g. template /users/{identifier}/inbox, together with URI https://agent.example/users/https%3A%2F%2Fsolid.example%2Factivitypub%2Factor/inbox yield identifier https%3A%2F%2Fsolid.example%2Factivitypub%2Factor instead of https://solid.example/activitypub/actor

However, when trying to produce a collection (e.g. followers), its {identifier} becomes double-URI-encoded, instead of being just encoded once, e.g.

federation
    .setFollowersDispatcher(
      `/users/{identifier}/followers`,
      async (ctx, identifier, cursor) => {
       // implementation
      },
    )

result collection id is

• https://agent.example/users/https%253A%252F%252Fsolid.example%252Factivitypub%252Factor/followers
  instead of
• https://agent.example/users/https%3A%2F%2Fsolid.example%2Factivitypub%2Factor/followers

Please note the double encoding.

Environment

OS: Arch Linux
Runtime: Node.js
Runtime version: 22.17.1

Logs / Screenshots

I have already narrowed the issue down to the two libraries that are being used for parsing and generating URIs:

• uri-template-router does not decode special characters, which I have reported in Inconsistent handling of URI-encoded special characters awwright/uri-template-router#11.
• url-template encodes URI components correctly, leading to double encoding of some special characters (additional encoding of %) in case of collections.

If this observation is not sufficient to identify the issue, or is misguided, I will provide logs or screenshots.

Until the above issue is addressed, a possible mitigation could be to swap {identifier} in templates to {+identifier} and decode the results with decodeURIComponent. It is not sufficient to just do the decoding, because the uri-template-router library with {identifier} decodes the URIs inconsistently (see the linked issue for details).

Steps to Reproduce

1. set up a basic fedify server that accepts identifiers with URI-encoded special characters in URLs, e.g. /users/${encodeURIComponent(':|')}/some-endpoint
2. set up followers collection endpoint with federation.setFollowersDispatcher
3. request the followers of the special-character actor, e.g. GET https://base.url/users/%3A%7C/followers
4. log the identifier in dispatcher callbacks, and see that it's partially decoded: %3A|
5. see collection id and observe that one of the special characters in the URI is double-encoded: %253A%7C

[dahlia]

Thank you for the detailed bug report and investigation! You've correctly identified the core issue with URI identifier handling.

Analysis

1. Correct usage for URI identifiers: For your use case where identifiers contain URIs, you should use {+identifier} (reserved string expansion) instead of {identifier} (simple string expansion). This is the correct approach according to RFC 6570 specification that Fedify follows (as documented in our actor dispatcher and collections manual sections).

2. Documentation enhancement needed: While Fedify correctly implements RFC 6570, we should improve our documentation and examples to guide users toward {+identifier} when dealing with URI-containing identifiers to avoid this confusion.

Root cause: library asymmetry

The real issue you've encountered stems from the asymmetric behavior between url-template (expansion) and uri-template-router (parsing):

• Your uri-template-router issue highlights inconsistent decoding behavior
• However, this isn't necessarily a “bug” from uri-template-router's perspective, since RFC 6570 only defines expansion behavior, not matching/parsing
• Pattern matching using URI templates goes beyond the original RFC 6570 specification and is essentially unstandardized territory

Short-term solution

Please switch to {+identifier} in your templates:

federation.setFollowersDispatcher(
  `/users/{+identifier}/followers`,
  async (ctx, identifier, cursor) => {
    // identifier will now be properly decoded: "https://solid.example/activitypub/actor"
  },
)

Long-term plan

If the uri-template-router maintainer doesn't address the asymmetry issue (which would be understandable given the lack of standardization), Fedify may need to implement its own RFC 6570 expansion and symmetric pattern matching to ensure consistent behavior.

We'll also update our documentation and examples to recommend {+identifier} for URI-containing identifiers to prevent this confusion for future users.

Thanks again for the thorough analysis and for reporting the upstream issue!