Support fulltext ngrams #28082
Support fulltext ngrams #28082
Conversation
|
🟢 |
There was a problem hiding this comment.
Pull Request Overview
This PR implements support for n-gram and edge n-gram filtering in the fulltext indexing system. Previously, these settings were rejected as unsupported; now they are validated and applied during text analysis.
- Added
BuildNgramsfunction to generate n-gram tokens from input strings - Updated validation logic to accept and validate n-gram configuration parameters
- Integrated n-gram filtering into the
Analyzefunction pipeline
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| ydb/core/base/fulltext.cpp | Implements n-gram generation logic, updates validation to accept n-gram settings, and applies n-gram filtering in the Analyze function |
| ydb/core/base/ut/fulltext_ut.cpp | Adds unit test for n-gram filtering with Russian text |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
⚪ ⚪ Ya make output | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ ⚪ Ya make output | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
2 participants
Changelog entry
...
Changelog category
Description for reviewers
...