Update MCP protocol expert agent to spec 2025-11-25 (#2911)

* Update MCP protocol expert agent to spec 2025-11-25

Updates the MCP protocol expert agent documentation to reflect the
latest MCP specification (2025-11-25). Key changes include:

- Update spec version references from 2025-06-18 to 2025-11-25
- Add Tasks section documenting the new durable state machine feature
- Update Elicitation section with new URL mode for sensitive data
- Note HTTP+SSE transport deprecation in favor of Streamable HTTP
- Add MCP-Protocol-Version and Mcp-Session-Id header requirements
- Fix key file paths to match actual codebase structure
- Add new spec URLs for Tasks, Transports, and Elicitation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add MCP 2025-11-25 authorization updates to agents

Updates both MCP protocol expert and OAuth expert agents with the
authorization model changes from MCP spec 2025-11-25:

mcp-protocol-expert.md:
- Add brief Authorization section with key points
- Defer to oauth-expert for implementation details
- Add Authorization and Security Best Practices URLs

oauth-expert.md:
- Add MCP Authorization resources (RFC 9728, 8707, Client ID Metadata)
- Add comprehensive MCP Authorization Model section covering:
  - Client Registration Priority order
  - Client ID Metadata Documents (new preferred approach)
  - Protected Resource Metadata (RFC 9728)
  - Resource Indicators (RFC 8707)
  - MCP-specific security requirements
- Note ToolHive implementation status for each feature

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: JAORMX

.claude/agents/mcp-protocol-expert.md

@@ -40,13 +40,14 @@ Do NOT invoke for:
 The MCP specification is versioned. To find and use the latest version:
 
 1. **Start here**: Fetch https://modelcontextprotocol.io to find links to current documentation
-2. **Known working URLs** (as of 2025-06-18 spec version):
-   - Main overview: `https://modelcontextprotocol.io/specification/2025-06-18`
-   - Basic spec: `https://modelcontextprotocol.io/specification/2025-06-18/basic`
-   - Transports: `https://modelcontextprotocol.io/specification/2025-06-18/basic/transports`
-   - Lifecycle: `https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle`
+2. **Known working URLs** (as of 2025-11-25 spec version):
+   - Main overview: `https://modelcontextprotocol.io/specification/2025-11-25`
+   - Transports: `https://modelcontextprotocol.io/specification/2025-11-25/basic/transports`
+   - Lifecycle: `https://modelcontextprotocol.io/specification/2025-11-25/basic/lifecycle`
+   - Tasks: `https://modelcontextprotocol.io/specification/2025-11-25/basic/utilities/tasks`
+   - Elicitation: `https://modelcontextprotocol.io/specification/2025-11-25/client/elicitation`
 
-3. **Check for newer versions**: The date in the URL (2025-06-18) indicates spec version. Look for newer dates.
+3. **Check for newer versions**: The date in the URL (2025-11-25) indicates spec version. Look for newer dates.
 
 4. **Always verify URLs**: If a URL gives 404, try alternative paths or check the main site.
 
@@ -71,10 +72,11 @@ When fetching specifications during a session:
 
 ### Transport Types
 
-Per the 2025-06-18 specification, MCP defines these transports:
-- **stdio**: Standard input/output (preferred, clients SHOULD support)
-- **Streamable HTTP**: HTTP-based transport for multiple connections
-- **Custom transports**: Implementations MAY add custom transports
+Per the 2025-11-25 specification, MCP defines two standard transports:
+- **stdio**: Standard input/output. Server reads JSON-RPC from stdin, writes to stdout. Messages delimited by newlines. Preferred for single-process architectures.
+- **Streamable HTTP**: HTTP-based transport for multiple connections. Uses POST/GET on single endpoint. Supports optional SSE for streaming. Includes session management via `Mcp-Session-Id` header. Requires `MCP-Protocol-Version` header on subsequent requests.
+
+**Note**: HTTP+SSE transport from 2024-11-05 is deprecated; use Streamable HTTP instead.
 
 **Always fetch the latest transport documentation** as this evolves over time.
 
@@ -85,8 +87,9 @@ Per the 2025-06-18 specification, MCP defines these transports:
 - **Clients**: Connectors within host applications
 - **Servers**: Services providing context and capabilities
 
-**Server capabilities**: Resources, Prompts, Tools
+**Server capabilities**: Resources, Prompts, Tools, Utilities (Completion, Logging, Pagination)
 **Client capabilities**: Sampling, Roots, Elicitation
+**Protocol utilities**: Cancellation, Ping, Progress, Tasks
 
 ### Lifecycle Phases
 
@@ -102,6 +105,39 @@ Per the 2025-06-18 specification, MCP defines these transports:
 
 All messages MUST follow JSON-RPC 2.0 specification.
 
+### Tasks (New in 2025-11-25)
+
+Tasks enable durable state machines for tracking long-running operations:
+- **Methods**: `tasks/list`, `tasks/get`, `tasks/cancel`, `tasks/result`
+- **Notification**: `notifications/tasks/status`
+- **Status lifecycle**: working → input_required | completed | failed | cancelled
+- **Key fields**: taskId, status, statusMessage, createdAt, lastUpdatedAt, ttl, pollInterval
+- **Capability**: Declared via `tasks.requests.*` during initialization
+
+ToolHive implementation: Parser support in `pkg/mcp/parser.go`
+
+### Elicitation
+
+Elicitation enables servers to request information from users:
+- **Method**: `elicitation/create`
+- **Form Mode**: Direct in-band data collection with JSON schema validation
+- **URL Mode** (New in 2025-11-25): Out-of-band for sensitive data (credentials, OAuth flows)
+- **Security**: Form mode MUST NOT be used for sensitive information
+
+ToolHive implementation: Full support in `pkg/vmcp/composer/elicitation_handler.go`
+
+### Authorization (Updated in 2025-11-25)
+
+MCP 2025-11-25 significantly updated the authorization model for HTTP transports:
+- Based on OAuth 2.1, RFC 9728 (Protected Resource Metadata), RFC 8707 (Resource Indicators)
+- **Client ID Metadata Documents** is the NEW preferred client registration (over DCR)
+- Token audience validation and PKCE are REQUIRED
+- Security best practices: `https://modelcontextprotocol.io/specification/2025-11-25/basic/security_best_practices`
+
+**For OAuth/auth implementation details, defer to the oauth-expert agent.**
+
+ToolHive auth implementation: `pkg/auth/` (discovery, OAuth flows, token handling)
+
 ## ToolHive Implementation
 
 ### Package Structure
@@ -111,11 +147,13 @@ All messages MUST follow JSON-RPC 2.0 specification.
 - `pkg/api/`: REST API server wrapping MCP servers
 
 ### Key Files
-- `pkg/transport/types.go`: Transport interface definitions
-- `pkg/transport/stdio/`: stdio transport implementation
-- `pkg/transport/http/`: HTTP transport implementation
-- `pkg/transport/sse/`: SSE transport (check deprecation status)
-- `pkg/transport/streamable/`: Streamable HTTP implementation
+- `pkg/transport/types/transport.go`: Transport interface definitions
+- `pkg/transport/stdio.go`: stdio transport implementation
+- `pkg/transport/http.go`: HTTP transport implementation
+- `pkg/transport/proxy/streamable/`: Streamable HTTP proxy implementation
+- `pkg/transport/proxy/httpsse/`: HTTP+SSE proxy (legacy, prefer streamable)
+- `pkg/transport/session/`: Session management for transports
+- `pkg/mcp/parser.go`: MCP JSON-RPC message parsing middleware
 
 ### Design Patterns
 - Factory pattern for transport creation
@@ -163,6 +201,9 @@ When working on MCP-related code:
 - Proxy functionality enables networked MCP servers
 - Cedar policies control access to MCP resources
 - Stdio is the preferred transport (clients SHOULD support it)
+- Streamable HTTP requires `MCP-Protocol-Version` header on requests after initialize
+- Session management uses `Mcp-Session-Id` header for Streamable HTTP
+- Tasks provide durable state tracking for long-running operations (2025-11-25+)
 
 ## Security Considerations
 
@@ -176,12 +217,19 @@ Per MCP spec, always consider:
 
 **Primary (Always Check First)**:
 - MCP Main Site: https://modelcontextprotocol.io
-- Specification: Check main site for latest version
+- Latest Spec (2025-11-25): https://modelcontextprotocol.io/specification/2025-11-25
+- Tasks: https://modelcontextprotocol.io/specification/2025-11-25/basic/utilities/tasks
+- Transports: https://modelcontextprotocol.io/specification/2025-11-25/basic/transports
+- Elicitation: https://modelcontextprotocol.io/specification/2025-11-25/client/elicitation
+- Authorization: https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization
+- Security Best Practices: https://modelcontextprotocol.io/specification/2025-11-25/basic/security_best_practices
 
 **ToolHive Implementation**:
 - Transport code: `pkg/transport/`
+- MCP parsing: `pkg/mcp/`
 - Runner code: `pkg/runner/`
 - API code: `pkg/api/`
+- vMCP (Virtual MCP): `pkg/vmcp/`
 
 ## Remember
 

.claude/agents/oauth-expert.md

@@ -30,7 +30,7 @@ Do NOT invoke for:
 - **Token Standards**: JWT, token introspection, token revocation, token exchange
 - **Discovery**: OAuth/OIDC discovery documents, metadata endpoints
 - **Security**: PKCE, state parameters, nonce, token binding
-- **RFC Compliance**: OAuth 2.0 (RFC 6749), OIDC Core, JWT (RFC 7519), Token Exchange (RFC 8693)
+- **RFC Compliance**: OAuth 2.0/2.1, OIDC Core, JWT (RFC 7519), Token Exchange (RFC 8693), Protected Resource Metadata (RFC 9728)
 
 ## Critical: Always Use Latest Standards
 
@@ -53,6 +53,13 @@ Do NOT invoke for:
 - OAuth 2.0 Security BCP (RFC 8252, RFC 8628): https://datatracker.ietf.org/doc/html/draft-ietf-oauth-security-topics
 - PKCE (RFC 7636): https://datatracker.ietf.org/doc/html/rfc7636
 
+**MCP Authorization (for MCP server authentication)**:
+- MCP Auth Spec: https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization
+- MCP Security Best Practices: https://modelcontextprotocol.io/specification/2025-11-25/basic/security_best_practices
+- Protected Resource Metadata (RFC 9728): https://datatracker.ietf.org/doc/html/rfc9728
+- Resource Indicators (RFC 8707): https://datatracker.ietf.org/doc/html/rfc8707
+- Client ID Metadata Documents: https://datatracker.ietf.org/doc/html/draft-ietf-oauth-client-id-metadata-document-00
+
 ## ToolHive Authentication Architecture
 
 ### Package Structure
@@ -199,6 +206,78 @@ grant_type=urn:ietf:params:oauth:grant-type:token-exchange
 &actor_token_type=...       // optional
 ```
 
+## MCP Authorization Model (2025-11-25)
+
+MCP 2025-11-25 defines specific OAuth requirements for HTTP transports. This section covers MCP-specific OAuth patterns.
+
+### Client Registration Priority
+
+MCP defines a specific priority order for client registration:
+
+1. **Pre-registration**: When client and server have existing relationship
+2. **Client ID Metadata Documents** (PREFERRED): Clients host metadata at HTTPS URLs
+3. **Dynamic Client Registration (DCR)**: Fallback for backwards compatibility
+4. **User prompt**: Last resort when no other option available
+
+### Client ID Metadata Documents (NEW)
+
+The preferred approach for MCP client registration when no prior relationship exists:
+
+**Client Requirements**:
+- Host metadata at HTTPS URL with path component (e.g., `https://example.com/client.json`)
+- `client_id` MUST match the document URL exactly
+- Required fields: `client_id`, `client_name`, `redirect_uris`
+- MAY use `private_key_jwt` for client authentication
+
+**Server Discovery**:
+- Check `client_id_metadata_document_supported: true` in auth server metadata
+- Fall back to DCR if not supported
+
+**Example Metadata Document**:
+```json
+{
+  "client_id": "https://app.example.com/oauth/client-metadata.json",
+  "client_name": "Example MCP Client",
+  "redirect_uris": ["http://localhost:3000/callback"],
+  "grant_types": ["authorization_code"],
+  "token_endpoint_auth_method": "none"
+}
+```
+
+**ToolHive Status**: NOT YET IMPLEMENTED - DCR is currently used as primary approach
+
+### Protected Resource Metadata (RFC 9728)
+
+MCP servers MUST implement RFC 9728 for authorization server discovery:
+
+**Discovery Methods** (in order):
+1. `WWW-Authenticate` header with `resource_metadata` parameter on 401 response
+2. Well-known URI: `/.well-known/oauth-protected-resource/<path>` (endpoint-specific)
+3. Well-known URI: `/.well-known/oauth-protected-resource` (root-level)
+
+**ToolHive Implementation**: `pkg/auth/discovery/` - Full RFC 9728 support
+
+### Resource Indicators (RFC 8707)
+
+MCP clients MUST include `resource` parameter in auth and token requests:
+- Identifies the specific MCP server the token is for
+- Prevents token reuse across different services
+- Uses canonical URI format (lowercase scheme and host)
+
+**ToolHive Implementation**: `OAuthFlowConfig.Resource` in `pkg/auth/discovery/`
+
+### MCP-Specific Security Requirements
+
+**REQUIRED**:
+- PKCE with S256 code challenge method
+- Token audience validation (reject tokens not issued for you)
+- Token passthrough FORBIDDEN (never forward client tokens upstream)
+
+**Scope Handling**:
+1. Use `scope` from initial 401 `WWW-Authenticate` header if provided
+2. Fall back to `scopes_supported` from Protected Resource Metadata
+3. Handle 403 `insufficient_scope` for step-up authorization
+
 ## Security Best Practices
 
 ### Token Validation