chore: upgrade PostgreSQL 17 and Supabase CLI 2.54.11 #300

jumski · 2025-11-02T22:04:28Z

Summary

This PR upgrades the pgflow project to PostgreSQL 17 and Supabase CLI 2.54.11, removing strict version pins for Docker images and the PGMQ extension to use the latest compatible versions.

Changes

PostgreSQL Version Upgrade

PostgreSQL 15 → PostgreSQL 17 across all environments:
- Atlas migration Docker image (pkgs/core/atlas/Dockerfile)
- Edge worker test database (pkgs/edge-worker/tests/db/compose.yaml)
- Supabase config files (all config.toml files)
- Atlas configuration (pkgs/core/atlas/atlas.hcl)
- Build and deployment scripts

PGMQ Version Management

Updated Atlas Docker image to use PGMQ v1.5.1 (from v1.4.4)
Removed strict version pins to allow using latest compatible PGMQ versions in production environments
This prepares for the next PR which requires PGMQ 1.5.0+ features

Configuration Updates

Updated PostgreSQL version references in:

examples/playground/supabase/config.toml
pkgs/client/supabase/config.toml
pkgs/core/supabase/config.toml
pkgs/edge-worker/supabase/config.toml
pkgs/core/atlas/atlas.hcl

Database Types

Regenerated TypeScript database types (pkgs/core/src/database-types.ts) to reflect PostgreSQL 17 schema
Updated package dependencies in pnpm-lock.yaml

Cleanup

Removed temporary Supabase version tracking files:
- pkgs/core/supabase/.temp/pgmeta-version
- pkgs/core/supabase/.temp/postgres-version

Documentation

Added documentation update task to pkgs/website/src/content/docs/deploy/update-pgflow.mdx for upgrade procedures

Testing

All existing tests pass with PostgreSQL 17. No functional changes to pgflow behavior.

Impact

Users: No breaking changes. Existing deployments will continue to work.
New Deployments: Will use PostgreSQL 17 by default
Self-hosted: Upgrading PostgreSQL version may require database migration depending on your hosting setup

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

merge:queue - adds this PR to the back of the merge queue
hotfix:queue - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

_{An organization admin has enabled the Graphite Merge Queue in this repository.} _{Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.}

This stack of pull requests is managed by Graphite. Learn more about stacking.

nx-cloud · 2025-11-02T22:05:41Z

🤖 Nx Cloud AI Fix Eligible

An automatically generated fix could have helped fix failing tasks for this run, but Self-healing CI is disabled for this workspace. Visit workspace settings to enable it and get automatic fixes in future runs.

_{To disable these notifications, a workspace admin can disable them in workspace settings.}

View your CI Pipeline Execution ↗ for commit 8cf6285

Command	Status	Duration	Result
`nx affected -t lint typecheck test --parallel -...`	❌ Failed	5m 47s	View ↗
`nx affected -t test:e2e --parallel --base=origi...`	✅ Succeeded	7m 49s	View ↗

☁️ Nx Cloud last updated this comment at 2025-11-19 04:19:48 UTC

graphite-app · 2025-11-12T23:01:04Z

PLAN.md

+
+**File**: `pkgs/core/schemas/0080_function_read_with_poll.sql` (entire file)


The PR description and PLAN.md mention deleting schemas/0080_function_read_with_poll.sql as part of Task 2, but this file deletion isn't included in the actual PR diff. This backport function should be removed since PGMQ 1.5.1 includes the fixed version natively.

Consider adding this deletion to complete the migration from the custom backport to the native implementation, which aligns with the comment in the file that states it "will be removed once Supabase upgrades to 1.5.0 or higher."

Spotted by Graphite Agent

Is this helpful? React 👍 or 👎 to let us know.

…kerfiles, and scripts - Changed major_version from 15 to 17 in multiple config files for various environments - Updated Dockerfile to use postgres:17 and related build scripts for new image - Modified Atlas setup to rely on custom postgres:17-pgmq image - Adjusted database type definitions for compatibility with PostgreSQL 17 - Updated compose files to reference the new PostgreSQL 17 image - Added note in documentation about PostgreSQL 17 support and migration guidance

graphite-app · 2025-11-19T08:24:17Z

Merge activity

Nov 19, 8:24 AM UTC: jumski added this pull request to the Graphite merge queue.
Nov 19, 8:24 AM UTC: CI is running for this pull request on a draft pull request (#398) due to your merge queue CI optimization settings.
Nov 19, 8:25 AM UTC: Merged by the Graphite merge queue via draft PR: #398.

## Summary This PR upgrades the pgflow project to PostgreSQL 17 and Supabase CLI 2.54.11, removing strict version pins for Docker images and the PGMQ extension to use the latest compatible versions. ## Changes ### PostgreSQL Version Upgrade - **PostgreSQL 15 → PostgreSQL 17** across all environments: - Atlas migration Docker image (`pkgs/core/atlas/Dockerfile`) - Edge worker test database (`pkgs/edge-worker/tests/db/compose.yaml`) - Supabase config files (all `config.toml` files) - Atlas configuration (`pkgs/core/atlas/atlas.hcl`) - Build and deployment scripts ### PGMQ Version Management - Updated Atlas Docker image to use PGMQ v1.5.1 (from v1.4.4) - Removed strict version pins to allow using latest compatible PGMQ versions in production environments - This prepares for the next PR which requires PGMQ 1.5.0+ features ### Configuration Updates Updated PostgreSQL version references in: - `examples/playground/supabase/config.toml` - `pkgs/client/supabase/config.toml` - `pkgs/core/supabase/config.toml` - `pkgs/edge-worker/supabase/config.toml` - `pkgs/core/atlas/atlas.hcl` ### Database Types - Regenerated TypeScript database types (`pkgs/core/src/database-types.ts`) to reflect PostgreSQL 17 schema - Updated package dependencies in `pnpm-lock.yaml` ### Cleanup - Removed temporary Supabase version tracking files: - `pkgs/core/supabase/.temp/pgmeta-version` - `pkgs/core/supabase/.temp/postgres-version` ### Documentation - Added documentation update task to `pkgs/website/src/content/docs/deploy/update-pgflow.mdx` for upgrade procedures ## Testing All existing tests pass with PostgreSQL 17. No functional changes to pgflow behavior. ## Impact - **Users:** No breaking changes. Existing deployments will continue to work. - **New Deployments:** Will use PostgreSQL 17 by default - **Self-hosted:** Upgrading PostgreSQL version may require database migration depending on your hosting setup ## Related - Prepares infrastructure for PR #301 which requires PGMQ 1.5.0+ features - Part of keeping pgflow aligned with latest Supabase platform versions

## Summary This PR removes the pgmq 1.4.x compatibility layer and requires pgmq 1.5.1 or higher. It eliminates deprecated functions, removes backported SQL code, and leverages new PGMQ features including message headers support. **⚠️ BREAKING CHANGE:** This version requires pgmq 1.5.0 or higher and will NOT work with pgmq 1.4.x. ## Changes ### PGMQ Version Requirement - **Minimum version:** pgmq 1.5.0+ (tested with 1.5.1) - **Migration guard:** Added compatibility check in migration that fails early with clear error message if pgmq < 1.5.0 is detected - **Affected packages:** `@pgflow/core`, `@pgflow/edge-worker` ### Removed Compatibility Layer #### Backported Functions Removed - **`pgflow.read_with_poll`** (68 lines) - Removed backport, now uses native `pgmq.read_with_poll` directly - **`pgflow.create_realtime_partition`** - Removed deprecated partitioning function - **Test file removed:** `supabase/tests/realtime/create_realtime_partition.test.sql` (83 lines) #### Function Updates - **`pgflow.set_vt_batch`** - Updated to return `headers` column from pgmq 1.5.0+: - Changed return type from `SETOF` to `TABLE` with explicit column definitions - Now returns: `msg_id`, `read_ct`, `enqueued_at`, `vt`, `message`, **`headers`** - Added comprehensive pgTAP test for headers handling ### Migration **File:** `pkgs/core/supabase/migrations/20251102201302_pgflow_upgrade_pgmq_1_5_1.sql` The migration includes: 1. **Compatibility check** - Verifies `pgmq.message_record` has `headers` column 2. **Clear error message** - Guides users to upgrade pgmq if version is incompatible 3. **Function replacement** - Drops and recreates `set_vt_batch` with new signature ### Code Updates #### TypeScript - **`PgflowSqlClient.ts`** - Changed queue polling to use `pgmq.read_with_poll` instead of `pgflow.read_with_poll` - **`Queue.ts`** - Updated queue reading logic - **Database types** - Regenerated to reflect new function signatures and removed deprecated functions - **Test fixtures** - Added `headers` field to message record types #### SQL Tests Updated 11 test files to remove calls to deprecated functions: - Removed `pgflow.create_realtime_partition()` calls from all realtime tests - Simplified test setup by using native PGMQ functions - Added new test: `supabase/tests/set_vt_batch/headers_handling.test.sql` ### Documentation - [**README.md**](http://README.md) - Enhanced documentation with: - Clarified root vs dependent map step behavior - Improved formatting and examples - Added edge case documentation - Better explanation of map step input handling - **BREAKING CHANGE notice** in changeset explaining: - Version requirement - Migration path for Supabase users - Self-hosting upgrade requirements ### Migration Management Skill - Updated `.claude/skills/migration-management/SKILL.md` with additional troubleshooting guidance ## Migration Guide ### For Supabase Users Recent Supabase versions include pgmq 1.5.0+ by default. Simply upgrade pgflow: ```bash # Upgrade will apply the migration automatically pnpm nx migrate @pgflow/core ``` The migration will verify pgmq compatibility and fail with a clear message if your pgmq version is too old. ### For Self-Hosted Users **Before upgrading pgflow:** 1. Verify your pgmq version: 2. If `extversion < 1.5.0`, upgrade pgmq first: 3. Then upgrade pgflow: ## Benefits - **Simplified codebase:** Removed 200+ lines of compatibility code - **Native PGMQ features:** Direct access to new pgmq 1.5.0+ capabilities - **Message headers:** Support for metadata propagation (foundation for future features) - **Cleaner tests:** Removed deprecated function calls across test suite - **Better error messages:** Migration fails early with actionable guidance ## Verified Fixes This PR verifies that upstream issues in Supabase Realtime have been resolved: - supabase/realtime#1369 - The Janitor's 10-minute startup delay that caused silent failures with `realtime.send()` after database resets has been fixed in recent Supabase versions - With the fix in place, pgflow no longer needs the `create_realtime_partition` workaround function that manually created partitions before tests - Removal of `pgflow.create_realtime_partition` and its test file confirms the upstream fix is stable and working as expected ## Testing - All pgTAP tests updated and passing - New test added for headers handling - Migration tested on both compatible (1.5.1) and incompatible (1.4.4) versions - Edge worker integration tests updated with headers support ## Dependencies - **Requires:** PR #300 (PostgreSQL 17 and Supabase CLI 2.54.11 upgrade) - **Breaking:** Incompatible with pgmq < 1.5.0 ## Related Files **Changed:** - 37 files changed, 629 insertions(+), 926 deletions(-) - Net reduction of ~300 lines of code **Key files:** - Migration: `pkgs/core/supabase/migrations/20251102201302_pgflow_upgrade_pgmq_1_5_1.sql` - Schema: `pkgs/core/schemas/0110_function_set_vt_batch.sql` - Client: `pkgs/core/src/PgflowSqlClient.ts` - Worker: `pkgs/edge-worker/src/queue/Queue.ts`

jumski mentioned this pull request Nov 2, 2025

feat: require pgmq 1.5.1+ and remove compatibility layer #301

Closed

jumski force-pushed the chore-update-supabase-versions branch 4 times, most recently from 8f99711 to b05ba56 Compare November 9, 2025 16:06

jumski force-pushed the chore-update-supabase-versions branch 6 times, most recently from eba7284 to f8447bd Compare November 12, 2025 22:58

graphite-app bot reviewed Nov 12, 2025

View reviewed changes

jumski force-pushed the chore-update-supabase-versions branch 7 times, most recently from 229a71e to 211fd84 Compare November 17, 2025 17:39

This was referenced Nov 17, 2025

docs: add a 0.8.0 release post and update README.md #390

Closed

chore: update deno to 2.1.4 #391

Closed

jumski force-pushed the chore-update-supabase-versions branch from 211fd84 to 8cf6285 Compare November 19, 2025 04:10

jumski mentioned this pull request Nov 19, 2025

enable demo preview deployment #397

Open

graphite-app bot closed this Nov 19, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore: upgrade PostgreSQL 17 and Supabase CLI 2.54.11 #300

chore: upgrade PostgreSQL 17 and Supabase CLI 2.54.11 #300

Uh oh!

jumski commented Nov 2, 2025 •

edited

Loading

Uh oh!

changeset-bot bot commented Nov 2, 2025 •

edited

Loading

Uh oh!

jumski commented Nov 2, 2025 •

edited

Loading

Uh oh!

nx-cloud bot commented Nov 2, 2025 •

edited

Loading

Uh oh!

graphite-app bot Nov 12, 2025

Uh oh!

graphite-app bot commented Nov 19, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		File: `pkgs/core/schemas/0080_function_read_with_poll.sql` (entire file)

chore: upgrade PostgreSQL 17 and Supabase CLI 2.54.11 #300

chore: upgrade PostgreSQL 17 and Supabase CLI 2.54.11 #300

Uh oh!

Conversation

jumski commented Nov 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

PostgreSQL Version Upgrade

PGMQ Version Management

Configuration Updates

Database Types

Cleanup

Documentation

Testing

Impact

Related

Uh oh!

changeset-bot bot commented Nov 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

jumski commented Nov 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How to use the Graphite Merge Queue

Uh oh!

nx-cloud bot commented Nov 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 Nx Cloud AI Fix Eligible

Uh oh!

graphite-app bot Nov 12, 2025

Choose a reason for hiding this comment

Uh oh!

graphite-app bot commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jumski commented Nov 2, 2025 •

edited

Loading

changeset-bot bot commented Nov 2, 2025 •

edited

Loading

jumski commented Nov 2, 2025 •

edited

Loading

nx-cloud bot commented Nov 2, 2025 •

edited

Loading

graphite-app bot commented Nov 19, 2025 •

edited

Loading