Why you need it?

I would like to suggest introducing an automated flaky test tracking mechanism, similar to what OpenSearch has adopted.

How it could be?

Proposal

Introduce a similar system to automatically report flaky tests in Seata. This would help:

• Systematically track flaky behavior
• Prioritize test stability improvements
• Avoid merging regressions masked by inconsistent test results

Implementation Ideas

• Hook into existing CI test result processing
• Identify tests that fail inconsistently across retries
• Auto-generate GitHub issues using a bot or GitHub Actions
• Apply consistent labels (e.g., flaky-test, test-failure, component:)

Benefits

• Easier triage of test failures
• More transparent test health history
• Helps contributors focus on test reliability

Thanks 🙏

Other related information

• https://learn.microsoft.com/en-us/azure/devops/report/powerbi/sample-test-analytics-flaky-tests?view=azure-devops&tabs=powerbi