Feature score eviction backend and frontend support #3287
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
meta-cla
bot
added
the
CLA Signed
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D78138679 |
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Aug 15, 2025
Summary: X-link: pytorch/FBGEMM#4681 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Aug 15, 2025
Summary: X-link: meta-pytorch/torchrec#3287 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Aug 15, 2025
Summary: X-link: meta-pytorch/torchrec#3287 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Aug 15, 2025
Summary: X-link: pytorch/FBGEMM#4681 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
EddyLXJ
force-pushed
the
export-D78138679
branch
from
c37d4c1 to
ca96b54
Compare
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Aug 15, 2025
Summary: X-link: meta-pytorch/torchrec#3287 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Aug 15, 2025
Summary: X-link: meta-pytorch/torchrec#3287 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Aug 15, 2025
Summary: X-link: meta-pytorch/torchrec#3287 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Aug 15, 2025
Summary: X-link: pytorch/FBGEMM#4681 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
EddyLXJ
force-pushed
the
export-D78138679
branch
from
ca96b54 to
faaef87
Compare
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Aug 15, 2025
Summary: X-link: meta-pytorch/torchrec#3287 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D78138679 |
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Aug 15, 2025
Summary: Pull Request resolved: meta-pytorch#3287 X-link: pytorch/FBGEMM#4681 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
EddyLXJ
force-pushed
the
export-D78138679
branch
from
faaef87 to
a96a883
Compare
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Aug 15, 2025
Summary: X-link: meta-pytorch/torchrec#3287 Pull Request resolved: pytorch#4681 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Aug 15, 2025
Summary: X-link: meta-pytorch/torchrec#3287 Pull Request resolved: pytorch#4681 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
Summary: Pull Request resolved: meta-pytorch#3287 X-link: pytorch/FBGEMM#4681 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D78138679 |
EddyLXJ
force-pushed
the
export-D78138679
branch
from
a96a883 to
45d6de2
Compare
facebook-github-bot
pushed a commit
to pytorch/FBGEMM
that referenced
this pull request
Aug 15, 2025
Summary: X-link: meta-pytorch/torchrec#3287 Pull Request resolved: #4681 X-link: facebookresearch/FBGEMM#1707 ## Context We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction. ## Detail * New Eviction Strategy: BY_FEATURE_SCORE Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic. This strategy uses feature scores derived from engagement rates to decide which IDs to evict. * FeatureScoreBasedEvict Class Implements the feature score based eviction logic. Maintains buckets of feature scores per shard and table to compute eviction thresholds. * Supports a dry-run mode to calculate thresholds before actual eviction. Eviction decisions are based on thresholds computed from feature score distributions. Supports decay of feature score statistics over time. * Async Metadata Update API Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store. This method shards the input indices and engagement rates and updates the feature score statistics in parallel. * Dry Run Eviction Mode Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting. Dry run results are used to finalize thresholds for real eviction rounds. Reviewed By: emlin Differential Revision: D78138679 fbshipit-source-id: 6196c3676abf94b690f1ac776ca8f5c739cae1ea
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
X-link: pytorch/FBGEMM#4681
X-link: https://github.com/facebookresearch/FBGEMM/pull/1707
Context
We need a new eviction policy for large embedding which has high id growth rate. The feature score eviction is based on engagement rate of id instead of only time or counter. This will help model to keep all relatively important ids during eviction.
Detail
Added a new eviction trigger strategy BY_FEATURE_SCORE in the eviction config and logic.
This strategy uses feature scores derived from engagement rates to decide which IDs to evict.
Implements the feature score based eviction logic.
Maintains buckets of feature scores per shard and table to compute eviction thresholds.
Eviction decisions are based on thresholds computed from feature score distributions.
Supports decay of feature score statistics over time.
Added set_kv_zch_eviction_metadata_async method to update feature score metadata asynchronously in the KV store.
This method shards the input indices and engagement rates and updates the feature score statistics in parallel.
Introduced a dry run mode to simulate eviction rounds to compute thresholds without actually evicting.
Dry run results are used to finalize thresholds for real eviction rounds.
Reviewed By: emlin
Differential Revision: D78138679