FEAT: add support for shard key in ketama hashing #1025

singsangssong · 2025-11-26T09:34:42Z

🔗 Related Issue

⌨️ What I did

원하는 key string을 하나의 노드에 저장할 수 있도록 그룹화하는 Shard Key 기능을 구현합니다.
shard 결정을 위해서는 key의 일부만을 사용하며, 해당 조건은 아래와 같습니다.

키에 {...} 패턴이 포함될 경우, 전체 키 대신 {와 } 사이의 문자열만을 해싱하여 저장할 노드를 결정합니다.
- 처음 등장하는 {와 처음 등장하는 } 사이의 하나 이상의 문자열을 해싱
이를 통해 동일한 Shard Key를 가진 키들은 항상 같은 노드에 배치됨을 보장하도록 합니다.
테스트코드는 새로운 ArcusShardKeyTest클래스를 추가했습니다.
- 키 전체가 서로 다르더라도 하나의 노드에 모두 저장되는지 확인
- shard key값이 다르다면 서로 다른 노드로 분산되는지 확인
- 키에 Shard key 규칙에 관한 패턴이 없다면 전체 키를 통해 해싱하는지 확인

src/main/java/net/spy/memcached/ArcusReplKetamaNodeLocator.java

src/test/java/net/spy/memcached/ArcusShardKeyTest.java

singsangssong · 2025-12-09T09:26:54Z

https://github.com/jam2in/arcus-works/issues/790#issuecomment-3629954750

위에서 코멘트 주신대로 useShardKey필드를 추가하고, shard key 사용 여부를 설정할 수 있도록 했습니다.
생성자 인자를 수정하면서, 관련된 부분도 수정했습니다. 참고부탁드립니다.

src/main/java/net/spy/memcached/ArcusReplKetamaNodeLocator.java

src/test/java/net/spy/memcached/ArcusShardKeyTest.java

src/main/java/net/spy/memcached/ConnectionFactoryBuilder.java

oliviarla · 2025-12-10T09:10:33Z

MemcachedClient#validateKey 에서 {, } 가 prefix에 포함되어도 유효한 키로 간주해야 합니다.

oliviarla · 2025-12-10T11:28:52Z

리뷰 가능한 상태가 되면 요청 주세요.

singsangssong · 2025-12-11T04:15:16Z

@oliviarla @uhm0311
말씀해주신대로 validateKey()에서 {, }가 포함되어도 유효한 키가 되도록 수정했고, 연관된 다른 코드도 확인해본 결과 리뷰받아도 될 것 같습니다. 참고로 ArcusKetamaNodeLocator의 생성자가 변경되면서, 클래스를 사용하는 연관 코드들도 함께 수정되었습니다. 참고 부탁드립니다.

src/main/java/net/spy/memcached/ArcusKetamaNodeLocator.java

src/test/java/net/spy/memcached/ArcusShardKeyTest.java

jhpark816

리뷰 완료

src/main/java/net/spy/memcached/ArcusKetamaNodeLocator.java

jhpark816 · 2025-12-12T08:08:54Z

src/main/java/net/spy/memcached/ArcusReplKetamaNodeLocator.java

  private ArcusReplKetamaNodeLocator(TreeMap<Long, SortedSet<MemcachedReplicaGroup>> kg,
                                     ConcurrentHashMap<String, MemcachedReplicaGroup> ag,
-                                     Collection<MemcachedNode> an) {
+                                     Collection<MemcachedNode> an, boolean shardKey) {


ArcusReplKetamaNodeLocator 기존 생성자 구현에서는
config 인자를 가지도록 먼저 변경하는 것이 좋겠습니다.

singsangssong · 2025-12-16T05:01:14Z

testLibKetamaCompatTwo()의 테스트코드를 수정하면서 빌드 환경에서 "code too large'현상이 발생했습니다. (Java 메서드 바이트코드 64KB 제한 초과 에러) 이에 따라 테스트케이스 개수를 줄인점 참고부탁드립니다.

uhm0311

질문입니다.

src/main/java/net/spy/memcached/ArcusKetamaNodeLocator.java

src/test/java/net/spy/memcached/ArcusKetamaNodeLocatorTest.java

singsangssong · 2025-12-18T01:18:46Z

@oliviarla
재리뷰 요청드립니다.

oliviarla · 2025-12-18T01:32:20Z

commit message의 첫 문자를 대문자로 변경해주세요.

singsangssong requested review from oliviarla and uhm0311 November 26, 2025 09:34

uhm0311 requested changes Nov 26, 2025

View reviewed changes

singsangssong force-pushed the task/shard branch from 3c2d05c to 0b1d784 Compare November 27, 2025 01:59

This comment was marked as resolved.

Sign in to view

singsangssong force-pushed the task/shard branch from 0b1d784 to 609d705 Compare November 27, 2025 06:12

uhm0311 previously approved these changes Nov 27, 2025

View reviewed changes

This comment was marked as resolved.

Sign in to view

singsangssong dismissed uhm0311’s stale review via 05d6ef7 December 5, 2025 07:09

singsangssong force-pushed the task/shard branch from 609d705 to 05d6ef7 Compare December 5, 2025 07:09

This comment was marked as resolved.

Sign in to view

singsangssong force-pushed the task/shard branch from 05d6ef7 to 0930062 Compare December 5, 2025 09:27

oliviarla previously approved these changes Dec 5, 2025

View reviewed changes

singsangssong dismissed oliviarla’s stale review via 3e147e9 December 5, 2025 09:40

singsangssong force-pushed the task/shard branch from 0930062 to 3e147e9 Compare December 5, 2025 09:40

oliviarla previously approved these changes Dec 5, 2025

View reviewed changes

uhm0311 previously approved these changes Dec 5, 2025

View reviewed changes

jhpark816 added the merged next time PR will be merged next time. label Dec 5, 2025

singsangssong dismissed stale reviews from uhm0311 and oliviarla via e4c2caf December 9, 2025 09:24

singsangssong force-pushed the task/shard branch from 3e147e9 to e4c2caf Compare December 9, 2025 09:24

uhm0311 requested changes Dec 9, 2025

View reviewed changes

src/main/java/net/spy/memcached/ArcusReplKetamaNodeLocator.java Show resolved Hide resolved

singsangssong force-pushed the task/shard branch from e4c2caf to 7aa0868 Compare December 10, 2025 05:36

uhm0311 previously approved these changes Dec 10, 2025

View reviewed changes

oliviarla reviewed Dec 10, 2025

View reviewed changes

src/test/java/net/spy/memcached/ArcusShardKeyTest.java Outdated Show resolved Hide resolved

src/main/java/net/spy/memcached/ConnectionFactoryBuilder.java Outdated Show resolved Hide resolved

singsangssong dismissed uhm0311’s stale review via 67e1908 December 10, 2025 07:40

singsangssong force-pushed the task/shard branch from 7aa0868 to 67e1908 Compare December 10, 2025 07:40

singsangssong force-pushed the task/shard branch from 67e1908 to 24f0d77 Compare December 10, 2025 09:31

oliviarla reviewed Dec 11, 2025

View reviewed changes

src/main/java/net/spy/memcached/ArcusKetamaNodeLocator.java Outdated Show resolved Hide resolved

src/test/java/net/spy/memcached/ArcusShardKeyTest.java Outdated Show resolved Hide resolved

singsangssong force-pushed the task/shard branch from 24f0d77 to 5060c02 Compare December 11, 2025 09:07

oliviarla previously approved these changes Dec 12, 2025

View reviewed changes

oliviarla requested a review from jhpark816 December 12, 2025 07:12

jhpark816 reviewed Dec 12, 2025

View reviewed changes

singsangssong dismissed oliviarla’s stale review via 2c6110d December 16, 2025 02:24

singsangssong force-pushed the task/shard branch 2 times, most recently from 2c6110d to 4231505 Compare December 16, 2025 02:42

uhm0311 reviewed Dec 16, 2025

View reviewed changes

src/main/java/net/spy/memcached/ArcusKetamaNodeLocator.java Outdated Show resolved Hide resolved

src/test/java/net/spy/memcached/ArcusKetamaNodeLocatorTest.java Outdated Show resolved Hide resolved

singsangssong force-pushed the task/shard branch from 4231505 to 326f307 Compare December 17, 2025 06:22

uhm0311 approved these changes Dec 17, 2025

View reviewed changes

oliviarla approved these changes Dec 18, 2025

View reviewed changes

oliviarla requested a review from jhpark816 December 18, 2025 01:31

FEAT: Add support for shard key in ketama hashing

1e15d08

singsangssong force-pushed the task/shard branch from 326f307 to 1e15d08 Compare December 18, 2025 01:43

jhpark816 merged commit 7140254 into naver:develop Dec 18, 2025
2 checks passed

FEAT: add support for shard key in ketama hashing #1025

FEAT: add support for shard key in ketama hashing #1025

Uh oh!

Conversation

singsangssong commented Nov 26, 2025

🔗 Related Issue

⌨️ What I did

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

Uh oh!

singsangssong commented Dec 9, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

oliviarla commented Dec 10, 2025

Uh oh!

oliviarla commented Dec 10, 2025

Uh oh!

singsangssong commented Dec 11, 2025

Uh oh!

Uh oh!

Uh oh!

jhpark816 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jhpark816 Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

singsangssong commented Dec 16, 2025

Uh oh!

uhm0311 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

singsangssong commented Dec 18, 2025

Uh oh!

oliviarla commented Dec 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants