MRT should default to `true` for CPS searches #138105

pawankartik-elastic · 2025-11-14T14:25:18Z

For Cross Project Search, we:

Do not want any endpoints to set ccs_minimize_roundtrips. Should they attempt to do so, its value should be dropped, and,
_async_search should default ccs_minimize_roundtrips to true. All other searches, anyway, at present, default to true since they do not call setCcsMinimizeRoundtrips() and SearchRequest sets it to true upon its instantiation.

Currently, we do accept values for ccs_minimize_roundtrips for Serverless, even though there's no Cross Cluster Search functionality. However, it's ineffective since the search is effectively a "local" search (see TransportSearchAction#executeLocalSearch()) and the values get ignored. Since this parameter would have no effect in Serverless, moving forward, we plan on notifying the same to the user via a Kibana banner.

At some point in the future, instead of ignoring this parameter, we'll error out (only for Serverless).

Check if Kibana is capable of generating a warning by itself or if I need to explicitly use HeaderWarning#addWarning().

pawankartik-elastic · 2025-11-14T16:06:19Z

server/src/main/java/org/elasticsearch/rest/action/search/RestSearchAction.java

+            clusterSupportsFeature,
+            setSize,
+            searchUsageHolder,
+            Optional.empty()


Previously, this overload passed in false for crossProjectEnabled. Now it passes in Optional.empty() to be compatible. Same theory, different syntax.

elasticsearchmachine · 2025-11-14T16:39:54Z

Pinging @elastic/es-search-foundations (Team:Search Foundations)

elasticsearchmachine · 2025-11-14T16:40:44Z

Hi @pawankartik-elastic, I've created a changelog YAML for you.

quux00

Left a few comments for feedback.

quux00 · 2025-11-14T17:17:55Z

.../lang-mustache/src/main/java/org/elasticsearch/script/mustache/RestSearchTemplateAction.java

    public RestSearchTemplateAction(Predicate<NodeFeature> clusterSupportsFeature, Settings settings) {
        this.clusterSupportsFeature = clusterSupportsFeature;
-        this.settings = settings;
+        this.crossProjectEnabled = settings != null && settings.getAsBoolean("serverless.cross_project.enabled", false);


I think we want to use the CrossProjectModeDecider for this, not hardcode the specific setting here.

Oh, yes! Thanks for pointing this one out.

quux00 · 2025-11-14T17:50:01Z

.../lang-mustache/src/main/java/org/elasticsearch/script/mustache/RestSearchTemplateAction.java

-            size -> searchRequest.source().size(size)
+            size -> searchRequest.source().size(size),
+            null,
+            Optional.of(crossProjectEnabled)


It worries me that the only other RestAction (besides the obvious ones of RestSearchAction and the async RestSearchAction) you had to change was this one.

But there are other several Rest actions that converge on _search that are (or will be) also CPS enabled:

_msearch/template _eql/search _sql _mvt _<index>/_count _cat/count

We should track down whether any of those allow setting ccs_minimize_roundtrips and if so, if additional code changes are needed.

The endpoints you've listed have their own dedicated parsing strategies that are lenient wrt the query parameters being passed, and there's some inconsistency between our public-facing docs and how the endpoints work. For example, for some of them, you can pass MRT, but it's not consumed. Here's why I only had to touch only a few Rest*Action-s:

Both _eql and _msearch/template accept MRT and have their own dedicated parser. I can accommodate these changes there.

_sql does not appear to accept MRT and has its own parser.

_mvt does not appear to accept MRT and has its own parser.

{index}/_count does not appear to accept MRT and has its own simple parser.

_cat/count does not appear to accept MRT and has its own simple parser.

Endpoints that aren't explicitly consuming MRT are defaulting to true.

Own parser = the parsing happens within the respective Rest*Action with info manually being extracted via RestRequest#param() and passed to the respective *Request-s.

Awesome. Thanks. Excellent analysis. 🥇

quux00

Nice work. This looks good. Down to the last few comments/changes.

quux00 · 2025-11-17T17:34:29Z

.../lang-mustache/src/main/java/org/elasticsearch/script/mustache/RestSearchTemplateAction.java

-            size -> searchRequest.source().size(size)
+            size -> searchRequest.source().size(size),
+            null,
+            Optional.of(crossProjectEnabled)


Awesome. Thanks. Excellent analysis. 🥇

quux00 · 2025-11-17T17:51:07Z

server/src/main/java/org/elasticsearch/action/search/MultiSearchRequest.java

            if (searchType != null) {
                searchRequest.searchType(searchType);
            }
+            // When crossProjectEnabled is true, ccsMinimizeRoundtrips is guaranteed to be true.


This comment seems out of place? Cross-project enabled part is handled down below, not here, so I'm not sure what this comment is getting at.

Also - be aware that a year from now this statement may no longer be true. We might go and change the default to be false for CPS everywhere if batched execution lives up to its promises, so comments like this will need to be tracked down and reversed/removed if/when that happens, so I would only add comments like this if there are really helpful to understand code flow.

I have a followup question on this: are we guaranteed that for serverless/CPS, we will keep same MRT mode for everywhere and not give a choice like we do for stateful CCS? The reason I'm asking is that right now telemetry explicitly does not track MRT state for CPS, but if we would in the future allow the user to choose then we may need to change that and start tracking it again like we do for CCS.

This comment is true but slightly misleading. MultiSearchRequest#readMultiLineFormat() is called from RestMultiSearchAction#parseMultiLineRequest(), and that's the method that passes down the ccsMinimizeRoundtrips you're seeing here — it refers to the MRT query parameter and is null when not specified. However, when provided an explicit value, we flip it to true anyway for CPS and pass it down here. That's what I meant by "ccsMinimizeRoundtrips is guaranteed to be true".

The misleading part: it only applies to CPS; should've mentioned this. I'll rewrite it in the next push.

quux00 · 2025-11-17T17:53:36Z

server/src/main/java/org/elasticsearch/action/search/MultiSearchRequest.java

        Boolean ccsMinimizeRoundtrips,
-        boolean allowExplicitIndex
+        boolean allowExplicitIndex,
+        Optional<Boolean> crossProjectEnabled


Optional suggestion: Add javadoc here what Optional.TRUE, Optional.FALSE and Optional.empty mean. That's not going to be obvious. Or if you feel like you have to document that 20 times, put a note here on where to see the javadoc for that?

quux00 · 2025-11-17T19:09:59Z

server/src/main/java/org/elasticsearch/rest/action/search/SearchParamsParser.java

+        if (crossProjectEnabled.orElse(false)) {
+            if (request.hasParam("ccs_minimize_roundtrips")) {
+                request.param("ccs_minimize_roundtrips");
+                HeaderWarning.addWarning(MRT_SET_IN_CPS_WARN);


This would be the place to check that we don't set this warning more than once (if that's possible).

There's only 1 place where we don't want to set this warning multiple times: when parsing multiline requests. I could use a simple boolean value to see if we've already fired a warning in the respective multiline request parser itself, rather than bloating up this method just to accommodate one scenario. Wdyt?

Does that one place (multiline requests) use this method?

One option here is to add a new method to HeaderWarning: HeaderWarning.addWarningIfUnique(), but if that would take a while to build and test and you don't need it here, then consider this an optional suggestion.

We could also file a follow-on ticket to handle it later. It's not essential for CPS Tech Preview. (If you choose this path, let me know so I can create the GA scope ticket for it.)

MRT should default to true for CPS searches

9bfc687

elasticsearchmachine added v9.3.0 serverless-linked Added by automation, don't add manually labels Nov 14, 2025

Merge branch 'main' into pkar/cps-search-mrt

4714df1

pawankartik-elastic commented Nov 14, 2025

View reviewed changes

pawankartik-elastic added Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch :Search Foundations/CCS labels Nov 14, 2025

pawankartik-elastic marked this pull request as ready for review November 14, 2025 16:39

pawankartik-elastic requested a review from a team as a code owner November 14, 2025 16:39

pawankartik-elastic requested a review from quux00 November 14, 2025 16:39

pawankartik-elastic added the >enhancement label Nov 14, 2025

Update docs/changelog/138105.yaml

ba24a24

Add header warning

687e383

quux00 reviewed Nov 14, 2025

View reviewed changes

pawankartik-elastic removed the request for review from a team November 17, 2025 10:00

pawankartik-elastic added 3 commits November 17, 2025 10:59

Lock MRT to true for eql and msearch

e01c115

Merge branch 'main' into pkar/cps-search-mrt

9ef6131

CI

06b880b

pawankartik-elastic requested a review from a team as a code owner November 17, 2025 11:24

pawankartik-elastic removed the request for review from a team November 17, 2025 11:37

pawankartik-elastic added 4 commits November 17, 2025 13:31

Move MRT parsing to a util method

b3ff223

Drop unused warn

77807c4

Add overload that takes in a default value

e540ac0

Switch overload

c7a1ad4

quux00 reviewed Nov 17, 2025

View reviewed changes

MRT should default to true for CPS searches #138105

Are you sure you want to change the base?

MRT should default to true for CPS searches #138105

Conversation

pawankartik-elastic commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Nov 14, 2025

Uh oh!

elasticsearchmachine commented Nov 14, 2025

Uh oh!

quux00 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pawankartik-elastic Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

quux00 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pawankartik-elastic Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

MRT should default to `true` for CPS searches #138105

MRT should default to `true` for CPS searches #138105

pawankartik-elastic commented Nov 14, 2025 •

edited

Loading

pawankartik-elastic Nov 14, 2025 •

edited

Loading

pawankartik-elastic Nov 18, 2025 •

edited

Loading