[Local Catalog] Integrate catalog API behind a feature flag #16301

jaclync · 2025-10-31T07:15:56Z

Just one review is required.

⚠️ Based on #16298, please ensure it's approved without significant change requests before reviewing this PR

Description

This PR integrates the catalog API endpoints into the POS catalog sync flow behind a feature flag. The implementation adds support for forcing catalog regeneration and handles edge cases where the API returns variations without their parent products.

Key changes:

Feature flag integration: Added pointOfSaleCatalogAPI feature flag to control whether to use the new catalog API or the existing paginated endpoints. Default to false, as the catalog API is still not fully ready in core
Catalog regeneration: Implemented regenerateCatalog parameter to force catalog generation from merchant-initiated actions (e.g., manual refresh in catalog settings)
Full sync with catalog API: Integrated full sync with catalog API pcShBQ-3wD-p2#comment-5426
Orphaned variation filtering: Added logic to filter out variations whose parent products are not included in the catalog (occurs when variations have public status but parent products don't - right now, the API fetches simple, variable, variation product types with publish status. As a future improvement, we can update the product DB arguments later to include all status cases to avoid this workaround)

Test Steps

Prerequisites:

Web: install the WooCommerce and AI Product Feed plugins (zips at a Google Drive folder at secret store 13636) on a test site where WooCommerce isn't automatically managed by the host.
- Ensure you have products with variations in the store, including some edge cases:
  - Variable products with public variations
  - Products with different statuses (some public, some not)
- The self-managed-fun-shop (small store, < 30s), largefuntesting (~5 minutes), and largerfuntesting (~9 minutes) Pressable test sites already have this setup, please DM me if you'd like access.
iOS local changes:
- If you're testing on a store with large catalog, update POSLocalCatalogEligibilityService's defaultCatalogSizeLimit from 1000 to a large number like 10000000.
- Enable pointOfSaleCatalogAPI feature flag in DefaultFeatureFlagService.

In the iOS app, wait for the initial POS full sync if it hasn't been synced before
Navigate to POS settings > Catalog
Manually trigger a catalog refresh using the refresh button
Verify the catalog downloads successfully and variations are properly filtered

I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

…rate` parameter.

…-flag # Conflicts: # Modules/Sources/Experiments/DefaultFeatureFlagService.swift # Modules/Sources/Experiments/FeatureFlag.swift

…er-initiated action, currently from the CTA in catalog settings.

…talog ready immediately or not).

…WOOMOB-1609-catalog-api-behind-feature-flag # Conflicts: # Modules/Sources/Networking/Remote/POSCatalogSyncRemote.swift # Modules/Sources/Yosemite/Tools/POS/POSCatalogSyncCoordinator.swift # WooCommerce/Classes/Tools/BackgroundTasks/ForegroundPOSCatalogSyncDispatcher.swift

… currently returns all products and variations with public status.

… catalog remote.

wpmobilebot · 2025-10-31T07:25:36Z

📲 You can test the changes from this Pull Request in WooCommerce iOS Prototype by scanning the QR code below to install the corresponding build.

	App Name	WooCommerce iOS Prototype
	Build Number	`pr16301-e1990ed`
	Version	`23.6`
	Bundle ID	`com.automattic.alpha.woocommerce`
	Commit	`e1990ed`
	Installation URL	67jgi5g3dutgg

Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

joshheald

Works really well, thanks for integrating it! A few small points inline.

joshheald · 2025-10-31T10:32:40Z

Modules/Sources/Yosemite/Tools/POS/POSCatalogFullSyncService.swift

+    }
+
+    func pollForCatalogCompletion(siteID: Int64, syncRemote: POSCatalogSyncRemoteProtocol) async throws -> String {
+        // Each attempt is made 1 second after the last one completes.


Perhaps we should have some backoff here – e.g. after a minute of checking every second, maybe every 5 seconds is fine even if it means someone's waiting 4 seconds longer than they would with 1s polling.

I also wonder whether we might have any rate limiting issues doing this with some hosts, WDYT?

Perhaps we should have some backoff here

Great idea, some progressive backoff (1s → 5s after 1 minute like you suggested) would be good here. Polling every second for potentially long-running catalog generations unnecessarily consumes server resources. I also plan to add some basic retry mechanism for the polling, so that one failing request doesn't end the full sync. I will implement both in a future PR.

I also wonder whether we might have any rate limiting issues doing this with some hosts, WDYT?

I did some search about rate limiting in WP/WC, there doesn't seem to be a built-in rate limiting system for the plugins and WC API. The WC Store API does have rate limiting support, with default maximum of 25 requests per 10 seconds. Some plugins like WordFence offer the rate limiting feature. Other findings:

Major hosting providers (WP Engine, Kinsta, SiteGround, Bluehost) don't seem to implement explicit REST API rate limiting but use resource-based limits (CPU, bandwidth, visits) instead.

WordPress VIP has global rate-limits (REST APIs aren't explicitly mentioned like XML-RPC and login endpoints, but I assume rate limiting applies to API requests too) of 10 requests per second.

Right now, the polling frequency is a bit under 1 request per second due to the overhead of the request (one request is made after the end of the previous one), and likely fine for the common default settings. However, the backoff implementation will further help reduce the risk of encountering server resource issues.

Thanks for the research!

joshheald · 2025-10-31T10:36:53Z

Modules/Sources/Yosemite/Tools/POS/POSCatalogPersistenceService.swift

+        let variations = catalog.variations.filter { variation in
+            let parentExists = productIDs.contains { $0 == variation.productID }
+            if !parentExists {
+                DDLogWarn("Variation \(variation.productVariationID) references missing product \(variation.productID)")


Good thinking logging this.

Suggested change

DDLogWarn("Variation \(variation.productVariationID) references missing product \(variation.productID)")

DDLogWarn("Variation \(variation.productVariationID) references missing product \(variation.productID) - it will not be available in POS.")

Updated in 786a89a.

joshheald · 2025-10-31T10:55:26Z

Modules/Sources/Yosemite/Tools/POS/POSCatalogFullSyncService.swift

+                }
+                return downloadURL
+            case .pending, .processing:
+                DDLogInfo("🟣 Catalog request \(response.status)... (attempt \(attempts + 1)/\(maxAttempts))")


This will flood the logs, for large catalogs...

Currently we log with .verbose even in prod. Perhaps we should change that to .info in AppDelegate, and then update these to be Debug logs?

Good catch! As an alternative to making global changes for the whole app, I reduced the logging noise by only logging every 10th polling attempt in e1990ed. This keeps the logs useful for debugging (every ~10 seconds with 1s polling or ~50 seconds with 5s backoff polling later) without flooding them for long-running catalog generations. WDYT?

That's fine, yeah... but I do think we should change this separately anyway.

IMO, the right way to do this would be to make this a .debug or even .verbose level log. Then we keep debug builds log output set to verbose and set production logging to info. We'd have to check there aren't other places where we're using debug log levels that we want for debugging customer issues, so it makes sense not to do that in this PR.

…-flag

… that the product is not available in POS.

…arge catalogs.

jaclync · 2025-11-04T01:37:45Z

Merging now, will work on the follow-up tasks in #16301 (comment). Feel free to comment on this PR still.

jaclync added 16 commits October 21, 2025 13:52

Add back catalog endpoints with full sync integration.

bc688c5

Update catalog remote based on the latest API behavior.

cf11514

Update catalog remote to call one endpoint.

87489b4

Add feature flag for catalog API.

7acf2bb

DI feature flag value to use catalog API. Add support for `force_gene…

b5925ac

…rate` parameter.

Merge branch 'trunk' into feat/WOOMOB-1609-catalog-api-behind-feature…

572ab92

…-flag # Conflicts: # Modules/Sources/Experiments/DefaultFeatureFlagService.swift # Modules/Sources/Experiments/FeatureFlag.swift

Revert testing code for large catalog.

e12af58

Add regenerateCatalog parameter to force catalog generation from us…

6235afe

…er-initiated action, currently from the CTA in catalog settings.

Move download URL logging to when the URL is ready for both cases (ca…

8748b80

…talog ready immediately or not).

Remove periphery comments as the code is used.

8694085

Nit: update feature flag comment and remove unused constant.

d912408

Fix unit tests build failures.

ed9b1a8

Filter variations without a parent product in the catalog, as the API…

527ad51

… currently returns all products and variations with public status.

Add test cases for POSCatalogFullSyncService changes.

4f183c1

Add a test case for regenerateCatalog parameter being passed to the…

1166f87

… catalog remote.

jaclync added status: feature-flagged Behind a feature flag. Milestone is not strongly held. feature: POS labels Oct 31, 2025

jaclync added this to the 23.7 milestone Oct 31, 2025

jaclync changed the title ~~POS: Integrate catalog API with feature flag~~ [Local Catalog] Integrate catalog API behind a feature flag Oct 31, 2025

jaclync requested a review from joshheald October 31, 2025 07:45

joshheald self-assigned this Oct 31, 2025

joshheald approved these changes Oct 31, 2025

View reviewed changes

Base automatically changed from feat/WOOMOB-1609-catalog-api-networking-layer to trunk November 3, 2025 07:07

jaclync added 3 commits November 3, 2025 15:40

Merge branch 'trunk' into feat/WOOMOB-1609-catalog-api-behind-feature…

77e49ae

…-flag

Nit: update variation missing parent product comment to make it clear…

786a89a

… that the product is not available in POS.

Only log polling info every 10th attempt to avoid flooding logs for l…

e1990ed

…arge catalogs.

jaclync merged commit df2e5eb into trunk Nov 4, 2025
14 checks passed

jaclync deleted the feat/WOOMOB-1609-catalog-api-behind-feature-flag branch November 4, 2025 01:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Local Catalog] Integrate catalog API behind a feature flag #16301

[Local Catalog] Integrate catalog API behind a feature flag #16301

Uh oh!

jaclync commented Oct 31, 2025 •

edited

Loading

Uh oh!

wpmobilebot commented Oct 31, 2025 •

edited

Loading

Uh oh!

joshheald left a comment

Uh oh!

joshheald Oct 31, 2025

Uh oh!

jaclync Nov 3, 2025

Uh oh!

joshheald Nov 4, 2025

Uh oh!

joshheald Oct 31, 2025

Uh oh!

jaclync Nov 3, 2025

Uh oh!

joshheald Oct 31, 2025

Uh oh!

jaclync Nov 3, 2025

Uh oh!

joshheald Nov 4, 2025

Uh oh!

jaclync commented Nov 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	DDLogWarn("Variation \(variation.productVariationID) references missing product \(variation.productID)")
	DDLogWarn("Variation \(variation.productVariationID) references missing product \(variation.productID) - it will not be available in POS.")

[Local Catalog] Integrate catalog API behind a feature flag #16301

[Local Catalog] Integrate catalog API behind a feature flag #16301

Uh oh!

Conversation

jaclync commented Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Key changes:

Test Steps

Uh oh!

wpmobilebot commented Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joshheald left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jaclync commented Nov 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jaclync commented Oct 31, 2025 •

edited

Loading

wpmobilebot commented Oct 31, 2025 •

edited

Loading