OAuth updates for mcp-agent

TESTING_GUIDE.md

@@ -0,0 +1,180 @@
+# OAuth Flow Validation Guide
+
+This checklist lets you flip between the original `feat/mcp_agent_oauth` branch and the current branch, exercise every scenario we discussed, and confirm the token flows (including caching) behave as intended. All commands use `uv` for Python execution.
+
+---
+
+## 0. One-Time Setup
+
+### 0.1 Create a Virtual Environment
+
+```bash
+uv venv
+source .venv/bin/activate
+```
+
+### 0.2 Install Project (with optional extras)
+
+```bash
+uv pip install -e .[oauth,cli,redis]
+```
+
+### 0.3 Start Temporal (required by the workflow examples)
+
+```bash
+temporal server start-dev
+```
+
+### 0.4 Set Secrets You'll Reuse
+
+```bash
+export GITHUB_CLIENT_ID=your_client_id
+export GITHUB_CLIENT_SECRET=your_client_secret
+export GITHUB_ACCESS_TOKEN=ghp_your_pat   # for pre-auth / pre-seeded flows
+```
+
+### 0.5 (Optional) Start Redis for Redis-backed tests
+
+```bash
+docker run --rm -p 6379:6379 redis:7-alpine
+export OAUTH_REDIS_URL=redis://127.0.0.1:6379
+```
+
+---
+
+## 1. Original Branch (`feat/mcp_agent_oauth`)
+
+```bash
+git checkout feat/mcp_agent_oauth
+```
+
+### 1.1 Unit Tests
+
+```bash
+uv run python -m pytest tests/test_oauth_utils.py tests/test_audience_validation.py
+# add tests/test_token_manager.py if it exists in that branch
+```
+
+### 1.2 Examples
+
+#### 1.2.1 Preconfigured (static token cache)
+
+- Terminal 1:
+  ```bash
+  uv run python examples/oauth/preconfigured/main.py
+  ```
+
+#### 1.2.2 Workflow Pre-Auth
+
+- Terminal 1:
+  ```bash
+  uv run python examples/oauth/workflow_pre_auth/worker.py
+  ```
+- Terminal 2:
+  ```bash
+  uv run python examples/oauth/workflow_pre_auth/main.py
+  ```
+- Terminal 3:
+  ```bash
+  uv run python examples/oauth/workflow_pre_auth/client.py
+  ```
+  Optional: rerun the client with `--skip-pre-auth` to confirm cached token reuse.
+
+#### 1.2.3 Dynamic Interactive Flow
+
+- Terminal 1:
+  ```bash
+  uv run python examples/oauth/dynamic_auth/worker.py
+  ```
+- Terminal 2:
+  ```bash
+  uv run python examples/oauth/dynamic_auth/main.py
+  ```
+- Terminal 3:
+  ```bash
+  uv run python examples/oauth/dynamic_auth/client.py
+  ```
+
+#### 1.2.4 Standalone OAuth Helper
+
+```bash
+uv run python examples/oauth/preconfigured/oauth_demo.py
+```
+
+---
+
+## 2. Current Branch (`your current working branch`)
+
+```bash
+git checkout <current-branch>
+```
+
+### 2.1 Unit Tests
+
+```bash
+uv run python -m pytest tests/test_oauth_utils.py tests/test_audience_validation.py tests/test_token_manager.py
+```
+
+_(Fix pytest/pytest_asyncio integration if you still have the autoload error before running.)_
+
+### 2.2 Examples
+
+#### 2.2.1 Interactive Tool Flow (dynamic auth)
+
+- Terminal 1:
+  ```bash
+  uv run python examples/oauth/interactive_tool/server.py
+  ```
+- Terminal 2:
+  ```bash
+  uv run python examples/oauth/interactive_tool/client.py
+  ```
+  After finishing the first run, run the client **again** (with the server still running). The second invocation should return immediately with no additional auth prompt—confirming token caching.
+
+#### 2.2.1b Client-only loopback (basic agent)
+
+- In `examples/basic/oauth_basic_agent/`, copy the secrets template and add your credentials:
+  ```bash
+  cd examples/basic/oauth_basic_agent
+  cp mcp_agent.secrets.yaml.example mcp_agent.secrets.yaml  # fill in keys/client details
+  uv pip install -r requirements.txt
+  export GITHUB_CLIENT_ID=your_client_id   # skip if secrets file already has these
+  export GITHUB_CLIENT_SECRET=your_client_secret
+  uv run python main.py
+  ```
+  On first run the browser opens to GitHub; authorize and the agent completes. Run the same command again and it should reuse the cached token without prompting. To choose different loopback ports, set `oauth.loopback_ports` in `mcp_agent.config.yaml`.
+
+#### 2.2.2 Workflow Pre-Auth
+
+- Terminal 1:
+  ```bash
+  uv run python examples/oauth/workflow_pre_auth/worker.py
+  ```
+- Terminal 2:
+  ```bash
+  uv run python examples/oauth/workflow_pre_auth/main.py
+  ```
+- Terminal 3:
+  ```bash
+  uv run python examples/oauth/workflow_pre_auth/client.py
+  ```
+  Repeat with `--skip-pre-auth` to verify cached token reuse after the workflow has been seeded once.
+
+#### 2.2.3 Redis-backed Token Cache (optional)
+
+- Make sure Redis is running (step 0.5 above).
+- With `OAUTH_REDIS_URL` exported and the redis extra installed, re-run either example. Tokens will persist across server restarts, so you can stop and restart the server and rerun the client to confirm the Redis cache is used.
+
+---
+
+## 3. Manual Validation Matrix
+
+| Scenario                             | Terminal(s)                               | Expected outcome                                                            |
+| ------------------------------------ | ----------------------------------------- | --------------------------------------------------------------------------- |
+| Static pre-auth (both branches)      | Worker + server + client                  | First run seeds token, subsequent run with `--skip-pre-auth` reuses it.     |
+| Interactive flow (both branches)     | Server + client                           | First run asks for auth; immediate re-run uses cached token without prompt. |
+| Redis token caching (current branch) | Same as above but with Redis env vars set | Tokens survive server restart thanks to Redis-backed store.                 |
+
+---
+
+Following this checklist will let you validate the complete OAuth functionality on both branches, including the multiple-user token caching behavior.

docs/configuration.mdx

@@ -144,8 +144,10 @@ oauth:
   callback_base_url: https://agent.example.com
   flow_timeout_seconds: 180
   token_store:
-    backend: memory   # use "redis" for multi-instance deployments
+    backend: memory   # set to "redis" for multi-instance deployments
     refresh_leeway_seconds: 90
+    redis_url: redis://localhost:6379
+    redis_prefix: mcp_agent:oauth_tokens
 
 mcp:
   servers:
@@ -164,6 +166,7 @@ mcp:
 
 - When `authorization.enabled` is true the MCP server advertises `/.well-known/oauth-protected-resource` and enforces bearer tokens using the provided introspection or JWKS configuration.
 - `oauth` enables delegated authorization flows; the default in-memory token store is ideal for local development while Redis is recommended for production clusters.
+- To use Redis for token storage, configure `token_store.backend: redis` and supply `redis_url` (see optional dependency `mcp-agent[redis]`).
 - Downstream servers opt into OAuth via `mcp.servers.<name>.auth.oauth`. Supplying a `client_id`/`client_secret` allows immediate usage; support for dynamic client registration is planned as a follow-up.
 
 ## Configuration Reference

docs/oauth_support_design.md

@@ -13,11 +13,14 @@
 2. **Protected Resource Metadata** – Serve `/.well-known/oauth-protected-resource` using FastMCP hooks so clients can discover the auth server.
 3. **Access Token Validation** – Enforce bearer tokens on every inbound MCP request via `RequireAuthMiddleware`, populating the request context with the authenticated user.
 4. **OAuth Token Service** – New `mcp_agent.oauth` package with:
-   - `TokenStore`/`TokenRecord` abstractions
-   - `InMemoryTokenStore` and Redis-backed implementation (second pass)
+  - `TokenStore`/`TokenRecord` abstractions
+  - `InMemoryTokenStore` and Redis-backed implementation (optional for multi-instance)
    - `TokenManager` orchestration (acquire, refresh, revoke)
    - `OAuthHttpxAuth` for attaching tokens to downstream HTTP transports
-   - `AuthorizationFlowCoordinator` that interacts with the user via MCP `auth/request`
+   - `AuthorizationFlowCoordinator` that interacts with the user via MCP `auth/request`.
+     When no upstream client session is available, a client-only loopback flow starts a
+     temporary local callback listener on 127.0.0.1 using a configurable fixed port list
+     (default: 33418, 33419, 33420), opens the browser, and completes the PKCE code flow.
 5. **Delegated Authorization UI Flow** – Extend the gateway/session relay so servers can send `auth/request` messages to MCP clients, capturing authorization codes via either:
    - Client-returned callback URL (preferred, works with SEP-capable clients)
    - MCP Agent hosted callback endpoint (`/internal/oauth/callback/{flow_id}`) as a fallback / native-app style loopback.
@@ -59,7 +62,7 @@ src/mcp_agent/oauth/
 
 Integration touchpoints:
 - `mcp_agent/config.py` – add OAuth settings models.
-- `mcp_agent/core/context.py` – add `current_user`, `token_manager`, `token_store`, `oauth_config` fields.
+- `mcp_agent/core/context.py` – add `token_manager`, `token_store`, `oauth_config` fields.
 - `mcp_agent/app.py` – initialize token store/manager based on settings.
 - `mcp_agent/server/app_server.py` – configure FastMCP auth settings, register callback route, surface user identity, extend relay to handle `auth/request`.
 - `mcp_agent/mcp/mcp_server_registry.py` & `mcp_agent/mcp/mcp_connection_manager.py` – wire `OAuthHttpxAuth` into HTTP transports and expose helper for manual token teardown.
@@ -95,7 +98,7 @@ Integration touchpoints:
    - Provide method to revoke tokens via authorization server when supported.
 
 ## Open Questions / Follow-ups
-- Redis-backed `TokenStore` (requires deployment secrets) – planned second phase.
+- Additional operational hardening (token rotation policies, rate limits).
 - How LastMile auth server exposes token introspection + JWKS; need concrete endpoint specs to finalize `MCPAgentTokenVerifier`.
 - MCP client adoption of `auth/request` SEP – need capability detection; until widely supported we rely on hosted callback fallback & manual instructions.
 - Access control DSL (include/exclude by email/domain) – to be evaluated once token identity payload finalized.
@@ -105,4 +108,3 @@ Integration touchpoints:
 - Metadata discovery + PKCE generation (pure python tests).
 - Integration-style test for delegated flow using mocked HTTP server + fake MCP client (ensures `auth/request` plumbing works end-to-end).
 - Tests around server 401 enforcement + WWW-Authenticate header.
-

examples/basic/oauth_basic_agent/README.md

@@ -0,0 +1,29 @@
+# OAuth Basic MCP Agent example (client-only loopback)
+
+This example mirrors `mcp_basic_agent` but adds GitHub MCP with OAuth using the client-only loopback flow.
+
+## Setup
+
+1. Register a GitHub OAuth App and add redirect URIs (at least one of):
+
+   - `http://127.0.0.1:33418/callback`
+   - `http://127.0.0.1:33419/callback`
+   - `http://localhost:33418/callback`
+
+2. Copy the secrets template and fill in your API keys / OAuth client (or export the env vars manually):
+
+```bash
+cp mcp_agent.secrets.yaml.example mcp_agent.secrets.yaml
+```
+
+3. Install deps and run:
+
+```bash
+uv pip install -r requirements.txt
+# If you populated the secrets file you can skip these exports.
+export GITHUB_CLIENT_ID=...
+export GITHUB_CLIENT_SECRET=...
+uv run main.py
+```
+
+On first run, a browser window opens to authorize GitHub; subsequent runs reuse the cached token.