-
Notifications
You must be signed in to change notification settings - Fork 2.6k
fix: skipped failing bailedout tests in CI #12462
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
moonming
merged 10 commits into
apache:master
from
Revolyssup:revolyssup/fix-failing-bailout
Jul 28, 2025
Merged
fix: skipped failing bailedout tests in CI #12462
moonming
merged 10 commits into
apache:master
from
Revolyssup:revolyssup/fix-failing-bailout
Jul 28, 2025
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Revolyssup
commented
Jul 25, 2025
"time_window": 60 | ||
} | ||
] | ||
"ai-rate-limiting": {"instances": [{"name": "openai-gpt3","limit": 50,"time_window": 60},{"name": "openai-gpt4","limit": 20,"time_window": 60}] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://github.com/apache/apisix/actions/runs/16519816286/job/46718888247?pr=12462
to fix inlined code too long err
membphis
approved these changes
Jul 26, 2025
nic-6443
approved these changes
Jul 26, 2025
moonming
approved these changes
Jul 28, 2025
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The test here https://github.com/apache/apisix/actions/runs/16511586539/job/46694453675
failed due to invalid configuration but in case of Bailout, the tests is considered successful. This change asserts and finds for the bailed out test and fails CI if any test was bailed out.
The following tests fail after the changes in script in this pr https://github.com/apache/apisix/actions/runs/16519080359/job/46716551717
https://github.com/apache/apisix/actions/runs/16519816286/job/46718888226?pr=12462#step:16:236
I have modified the above tests to finally pass.
Further enquiries
The doc says
"By default the test scaffold treats NGINX server startup failures as fatal errors in running the tests. The must_die section, however, turns such a failure into a normal test checkup."
If you have just one test then it holds true. But when even one test passes and subsequent test has invalid config then TEST IS CONSIDERED PASS.
WHen invalid configuration was passed in Test 1 (As expected the Bailout is considered total test failure)
When invalid configuration was passed in Test 2(or any other test after one passed test): (The test pass even after bailout)
Checklist