[Improvement] Implement manifest-driven data expiration

[xxubai]

Why are the changes needed?

Close #3749 .

Brief change log

• Reduce I/O frequency by tuning manifest operations
• Limit the maximum number of cached data/delete files to 10k to avoid OOM
• Expire files partition by partition (starting with older partitions first). This makes physical cleanup during snapshot expiration more readable and effective.

How was this patch tested?

• Add some test cases that check the changes thoroughly including negative and positive cases if possible

• Add screenshots for manual tests if appropriate

• Run test locally before making a pull request

Documentation

• Does this pull request introduce a new feature? (no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[github-actions]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@amoro.apache.org list. Thank you for your contributions.