Fix Cache::flexible() to work with AWS ElastiCache Serverless (Valkey)

[yhayase]

Description

This PR fixes Cache::flexible() to work with AWS ElastiCache Serverless (Valkey) and similar environments that have cluster_enabled: 1 but provide a single endpoint.

Problem

AWS ElastiCache Serverless (Valkey) has a unique configuration:

• Server-side: cluster_enabled: 1 (cluster mode enabled)
• Connection: Single endpoint

This creates a problem where neither connection mode works:

phpredis driver

Single-node connection mode (default):

TypeError: array_map(): Argument #2 ($array) must be of type array, bool given
File: vendor/illuminate/redis/Connections/PhpRedisConnection.php:67
Redis last error: CROSSSLOT Keys in request don't hash to the same slot

The flexible() method stores two keys:

• Value key: illuminate:cache:{key}
• Timestamp key: illuminate:cache:flexible:created:{key}

These keys hash to different slots in Redis Cluster. Since the server has cluster mode enabled, MGET returns a CROSSSLOT error.

Cluster connection mode:

RedisClusterException: Error processing response from Redis node!

phpredis RedisCluster fails to process responses from Valkey Serverless (Valkey Serverless-specific issue).

predis driver

Single-node connection mode (default):

Same CROSSSLOT error as phpredis.

Cluster connection mode:

Predis\NotSupportedException: Cannot use 'MGET' with redis-cluster.

predis checks hash tags before executing MGET. Without consistent hash tags, it rejects the operation.

Solution

Add Redis hash tags to ensure all related keys are stored in the same slot:

$hashKey = substr(md5($key), 0, 4);  // 16-bit hash
$valueKey = "{{$hashKey}}:{$key}";
$createdKey = "{{$hashKey}}:illuminate:cache:flexible:created:{$key}";
$lockKey = "{{$hashKey}}:illuminate:cache:flexible:lock:{$key}";

How it works

• Redis Cluster uses the content within {...} to determine the slot
• All three keys share the same hash tag {$hashKey}, ensuring they're in the same slot
• The 4-character MD5 hash (16 bits) provides sufficient uniqueness for slot distribution
• The original $key is preserved outside the hash tag for uniqueness
• Works with standalone Redis (hash tags are ignored)

Why this fixes the issue

For phpredis + default (single-node mode):

• Keys now hash to the same slot
• CROSSSLOT error no longer occurs
• Works with Valkey Serverless ✅

For predis + default (single-node mode):

• Keys now hash to the same slot
• CROSSSLOT error no longer occurs ✅

For predis + clusters (cluster mode):

• predis checks that all keys have the same hash tag {$hashKey}
• Allows MGET to proceed since keys are in the same slot
• Works with Valkey Serverless ✅

Testing

Verified on AWS ElastiCache Serverless (Valkey) with cluster_enabled: 1:

• ✅ phpredis + default (single-node connection): SUCCESS (0.145s)
• ✅ predis + default (single-node connection): SUCCESS (0.008s)
• ✅ predis + clusters (cluster mode connection): SUCCESS (0.095s)

Complete reproduction case: https://github.com/yhayase/laravel-cache-flexible-valkey-serverless-issue

Compatibility

• ✅ Backward compatible with standalone Redis (hash tags are ignored)
• ✅ Works with phpredis and predis drivers
• ✅ Works with all cache backends (hash tags are added to all drivers, but only meaningful for Redis)
• ⚠️ Breaking change: Cache keys will change after upgrading
  • Affects all cache drivers (Redis, Memcached, Database, File, DynamoDB, etc.)
  • Existing flexible() cache entries will be missed on first access after upgrade
  • Callbacks will be executed to regenerate values
  • Old cache entries will be automatically deleted when their TTL expires
  • No manual cache clearing required, but temporary storage increase during TTL period

Related Issue

Fixes CROSSSLOT errors on AWS ElastiCache Serverless (Valkey) and similar environments where the server has cluster mode enabled but provides a single endpoint.

[vadimonus]

@yhayase , your PR is backward incompatible, as will change all logic of setting and retreiving values from cache. It's very dangerous.

Also you do not need to use md5 hash for key, as redis already calculates slot with more quick crc16 hash (see https://redis.io/docs/latest/operate/oss_and_stack/management/scaling/#redis-cluster-data-sharding).

So, all you need are only changes in \Illuminate\Cache\Repository::flexible method.

"illuminate:cache:flexible:created:{$key}" -> "illuminate:cache:flexible:created:\{$key\}"
"illuminate:cache:flexible:lock:{$key}" -> "illuminate:cache:flexible:lock:\{$key\}"

Maybe it was just a mistake not to escape curly braces.

This will lead that slot for key 'xxx' and 'illuminate:cache:flexible:created:{xxx}' would be the same (see how redis calculate slots).

There are integration tests with Redis Cluster in .github/workflows/queues.yml. You may add run of tests/Integration/Cache/RepositoryTest.php there to show if it fails before fix and not fails after it.
You will need CACHE_STORE=redis and REDIS_CACHE_CONNECTION=default for tests with redis cluster

[yhayase]

@vadimonus Thank you so much for your detailed review and valuable suggestions! I really appreciate you taking the time to point out the backward compatibility issue and propose a simpler solution.

You're absolutely right that my original approach (adding hash tags to all keys) was backward incompatible and overly complex. I've been reconsidering the implementation based on your feedback.

Issue with the hash tag escape approach

I initially considered your suggestion to use escaped braces:

"illuminate:cache:flexible:created:{{$key}}"

This would produce a string like illuminate:cache:flexible:created:{mykey} when $key = "mykey", which looks correct for Redis hash tags.

However, I found a critical issue with this approach. When a user provides a key like "a}keyname", the result would be:

illuminate:cache:flexible:created:{a}keyname}

Redis Cluster would only use the content between the first { and } (i.e., a) for slot calculation. This means:

• User key a}keyname → slot based on full string a}keyname
• Created key illuminate:cache:flexible:created:{a}keyname} → slot based on a only

These would hash to different slots, causing the same CROSSSLOT error we're trying to fix.

Alternative approach: Configuration-based sequential operations

I've explored a different approach that maintains backward compatibility while fixing the Redis Cluster issue:

Add a flexible_cluster_mode configuration option that switches flexible() to use sequential operations instead of bulk operations (MGET/MSET):

Configuration (config/cache.php):

'stores' => [
    'redis' => [
        'driver' => 'redis',
        'flexible_cluster_mode' => true,
    ],
],

Implementation:

• When flexible_cluster_mode is enabled, flexible() uses individual get()/put() calls instead of many()/putMany()
• This avoids CROSSSLOT errors entirely since each operation targets a single key
• Default is false, maintaining existing behavior
• No key name changes, so fully backward compatible
• Works consistently with Cache::get(), Cache::forget(), and Cache::tags()

Trade-offs:

• 2 network round-trips instead of 1 (minimal performance impact since callback execution typically dominates)
• Only needed for environments like AWS ElastiCache Serverless where the server has cluster mode enabled but provides a single endpoint
• For proper Redis Cluster connections (predis clusters mode), this configuration is not needed as putMany() already handles it correctly

I've tested this approach successfully on AWS ElastiCache Serverless (Valkey) with all patterns passing.

I've already updated the branch to implement this new approach. I'll update the PR description once we reach a consensus on the direction.

What do you think about this approach? I'm open to any suggestions or alternative solutions you might have.

Thanks again for your guidance!

[vadimonus]

@yhayase You give such a polite and detailed answer that I don't always understand whether I'm communicating with you or with a gpt bot.

Cache keys like a}keyname can appear only in rare specific cases. In such cases you should think about some sort of escaping of keys. They can containt { as well, leading to unpredictable behaviour in many cases. Such cases rare cases can be eliminated on application and not on framework side. Please note, Redis Cluster itself does not provide any way to escape curly braces, or other way to specify, what part of key to use as hash key is. So with any other framework or without it you will experience some difficulties if working with Redis Cluster and keys with curly braces.
In this situation, moving solution of such rare cases from framework to application level is good tradeoff between framework simplicity, backward compatibility and making it workable in most of cases.

If you choose use tho puts instead of one putMany, you do not need special config settings. It's enough to check something like $this->store->connection instanceof PhpRedisCluster. See similar branches in RedisBroadcaster class.

[yhayase]

Hi @vadimonus,

Thank you so much for your thoughtful feedback on this PR! Your suggestion about using INFO cluster to detect server-side cluster mode was the key insight that led to a much better solution.

After our discussion, I realized that instead of trying to make keys hash to the same slot or adding configuration options, the right approach was to:

1. Detect cluster mode automatically via INFO cluster command
2. Leverage the existing fallback logic from PR Avoid writing multiple keys when using redis in cluster mode #53940
3. Apply it not just to flexible(), but to all multi-key operations

I've implemented this approach and tested it successfully on AWS ElastiCache Valkey Serverless. The test-first development process is documented here:

• Development PR: [WIP/Test Record] Redis single-endpoint cluster support with test-first approach yhayase/laravel-framework#3
• Problem reproduction: https://github.com/yhayase/laravel-cache-flexible-valkey-serverless-issue

I'll be submitting a new PR to laravel/framework soon as a follow-up to PR #53940.

You give such a polite and detailed answer that I don't always understand whether I'm communicating with you or with a gpt bot.

Haha, you're actually half right! Since English isn't my native language, I work together with Claude Code (Anthropic's AI coding assistant) when writing PRs and replies. Claude helps me organize technical details and express them clearly in English, though it sometimes makes my messages longer and perhaps more formal than necessary. That's partly the AI's influence, but honestly also just my communication style - I tend to include a lot of detail and try to be thorough (maybe too much so!), especially since I'm still getting used to PR discussions.

Thanks again for pushing me in the right direction. Your feedback was invaluable!

[vadimonus]

You're referencing #53940, where problem of multiple put with Redis was already solved.

Your problem occures on call to $this->many? In such case you should rollback all you have done, and just repeat solution from #53940 for RedisStore::many, and do nothing more.

I said nothing about calling INFO. It's used in test just to wait for all cluster nodes to be up and healthy, without it tests fails randomly.

Your do not need to call cluster INFO, to decide, if it's cluster or not connection. Check of instanceof is enough. Framework not expected to work correctly, if you defined cluster connection, but attach to single node redis, or vice versa.
When connection to single-endpoint cluster, you should provide config as for any other cluster connection, but just with one node in nodes array, that's would be enough.

Also it would be good, if you find a way to add cache integration tests on redis cluster environment, same as for queues, to prevent other developers adding new code that will work incorrectly with cluster. When i found some problems with batches in Redis Cluster, i firstly bring integration tests, showing that problem exists.

[yhayase]

Hi @vadimonus,

Thank you for the detailed feedback!

Correction on INFO cluster: You're absolutely right that you didn't suggest using INFO cluster. Your feedback and discussion helped me think through the problem, which eventually led me to this approach. I apologize for the confusion in my previous message.

Your main point: instanceof check is enough if users configure correctly.

Why this doesn't work: phpredis cluster mode fails on Valkey Serverless.

We tested this and documented it here: https://github.com/yhayase/laravel-cache-flexible-valkey-serverless-issue

Pattern 2 (phpredis + clusters):

'clusters' => [
    'default' => [
        ['host' => 'xxxxx.serverless.apne1.cache.amazonaws.com', 'port' => 6379]
    ]
]

Result:

• Connection initializes successfully
• _masters() returns node information
• Any command (even basic SET) fails with: RedisClusterException: Error processing response from Redis node!

Why this likely happens:

According to phpredis documentation, when connecting in cluster mode, it runs CLUSTER SLOTS to map the cluster topology. For Valkey Serverless:

• External endpoint: xxxxx.serverless.apne1.cache.amazonaws.com:6379
• CLUSTER SLOTS likely returns internal cluster node addresses (e.g., 10.x.x.x:6379)
• When phpredis routes commands to specific slots, it would attempt to connect to these internal addresses
• These internal addresses would not be accessible from outside, potentially causing the connection/response error

This is why users must use regular connection mode, but then instanceof returns false while the server is still in cluster mode.

[vadimonus]

The problem you're talking is not Laravel problem. Redis Cluster and Valkey as its successor do not operate with domain names. It stores only ip address for nodes. When some slot is not found on some node, this node returns redirect answer, which contains IP Address of node, containing slot.
On first connection domain name is resolved to ip address of one of nodes, and this node returns addresses of other nodes to client. After that domain name is not used for communication.
So, for correct work, application and all redis nodes should be able to communicate with each others by ip addresses, so you need to setup routing in proper way.
Take a look at https://laracasts.com/discuss/channels/laravel/aws-serverless-valkey , #49501 , phpredis/phpredis#2083 for AWS specific issues.
Laravel supports Redis and Redis Cluster, not the AWS. So if some issue cannot be reproduced on local redis cluster, it should not be treated as Laravel problem.