-
Notifications
You must be signed in to change notification settings - Fork 470
fix: resolve critical race conditions in WebSocket proxy #530
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: resolve critical race conditions in WebSocket proxy #530
Conversation
- Add subscription_lock to prevent concurrent subscribe/unsubscribe races - Add adapter_lock to prevent duplicate broker adapter initialization - Add user_lock to protect user_mapping during auth and cleanup - Implement global subscription tracking with reference counting - Add pre-registration before broker subscribe with rollback on failure - Move success log to after broker subscribe completes - Protect ZMQ listener with subscription snapshots - Add existence validation before unsubscribe operations - Create immutable copies before dictionary iteration - Implement proper error handling with state rollback Fixes multiple race conditions that could cause: - Duplicate broker subscriptions - Duplicate adapter connections - Dictionary changed size during iteration errors - Misleading subscription logs - Resource leaks on failed operations All changes maintain backward compatibility with existing clients.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
5 issues found across 6 files
Prompt for AI agents (all 5 issues)
Understand the root cause of the following 5 issues and fix them.
<file name="websocket_proxy/server.py">
<violation number="1" location="websocket_proxy/server.py:671">
Indexing mode_to_str with the raw client mode will raise a KeyError when the value isn’t 1/2/3 (e.g., clients sending the string "3" worked before but now blows up). Please fall back gracefully when the key is missing so subscriptions keep working.</violation>
</file>
<file name="docs/zmq_new_audit_report.md">
<violation number="1" location="docs/zmq_new_audit_report.md:1191">
The audit report claims the added file is `ZMQ_Audit_Report.md`, but the repository file is named `docs/zmq_new_audit_report.md`, so this reference is inaccurate.</violation>
<violation number="2" location="docs/zmq_new_audit_report.md:1192">
Listing `COMMIT_MESSAGE.md` as an added file is inaccurate because no file with that name is present in the repository.</violation>
</file>
<file name="test/test_improved_zmq_server.py">
<violation number="1" location="test/test_improved_zmq_server.py:984">
This test only counts messages containing "ERROR", but the helper methods record unauthenticated request failures as "...FAILED", so error_count never increases and the test always reports failure even when behavior is correct.</violation>
</file>
<file name="test/test_multi_client_fix.py">
<violation number="1" location="test/test_multi_client_fix.py:146">
The test only logs message counts instead of asserting that other clients continue receiving data after one unsubscribes, so it will silently succeed even when the regression reappears. Please add explicit assertions that detect regressions.</violation>
</file>
Since this is your first cubic review, here's how it works:
- cubic automatically reviews your code and comments on bugs and improvements
- Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
- Ask questions if you need clarification on any suggestion
React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.
websocket_proxy/server.py
Outdated
| }) | ||
| continue | ||
| else: | ||
| logger.info(f"First client subscribed to {symbol}.{exchange}.{mode_to_str[mode]}, broker subscribe successful") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indexing mode_to_str with the raw client mode will raise a KeyError when the value isn’t 1/2/3 (e.g., clients sending the string "3" worked before but now blows up). Please fall back gracefully when the key is missing so subscriptions keep working.
Prompt for AI agents
Address the following comment on websocket_proxy/server.py at line 671:
<comment>Indexing mode_to_str with the raw client mode will raise a KeyError when the value isn’t 1/2/3 (e.g., clients sending the string "3" worked before but now blows up). Please fall back gracefully when the key is missing so subscriptions keep working.</comment>
<file context>
@@ -614,22 +610,82 @@ async def subscribe_client(self, client_id, data):
+ })
+ continue
+ else:
+ logger.info(f"First client subscribed to {symbol}.{exchange}.{mode_to_str[mode]}, broker subscribe successful")
+ except Exception as e:
+ self._remove_global_subscription(client_id, user_id, symbol, exchange, mode)
</file context>
| logger.info(f"First client subscribed to {symbol}.{exchange}.{mode_to_str[mode]}, broker subscribe successful") | |
| logger.info(f"First client subscribed to {symbol}.{exchange}.{mode_to_str.get(mode, mode)}, broker subscribe successful") |
✅ Addressed in 963e9de
docs/zmq_new_audit_report.md
Outdated
|
|
||
| ### Files Added | ||
| - `ZMQ_Audit_Report.md` - This comprehensive audit report | ||
| - `COMMIT_MESSAGE.md` - Detailed commit message |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Listing COMMIT_MESSAGE.md as an added file is inaccurate because no file with that name is present in the repository.
Prompt for AI agents
Address the following comment on docs/zmq_new_audit_report.md at line 1192:
<comment>Listing `COMMIT_MESSAGE.md` as an added file is inaccurate because no file with that name is present in the repository.</comment>
<file context>
@@ -0,0 +1,1328 @@
+
+### Files Added
+- `ZMQ_Audit_Report.md` - This comprehensive audit report
+- `COMMIT_MESSAGE.md` - Detailed commit message
+
+### Dependencies
</file context>
✅ Addressed in 963e9de
docs/zmq_new_audit_report.md
Outdated
| - `server.py` - Core WebSocket proxy server (128 lines changed) | ||
|
|
||
| ### Files Added | ||
| - `ZMQ_Audit_Report.md` - This comprehensive audit report |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The audit report claims the added file is ZMQ_Audit_Report.md, but the repository file is named docs/zmq_new_audit_report.md, so this reference is inaccurate.
Prompt for AI agents
Address the following comment on docs/zmq_new_audit_report.md at line 1191:
<comment>The audit report claims the added file is `ZMQ_Audit_Report.md`, but the repository file is named `docs/zmq_new_audit_report.md`, so this reference is inaccurate.</comment>
<file context>
@@ -0,0 +1,1328 @@
+- `server.py` - Core WebSocket proxy server (128 lines changed)
+
+### Files Added
+- `ZMQ_Audit_Report.md` - This comprehensive audit report
+- `COMMIT_MESSAGE.md` - Detailed commit message
+
</file context>
| - `ZMQ_Audit_Report.md` - This comprehensive audit report | |
| - `docs/zmq_new_audit_report.md` - This comprehensive audit report |
✅ Addressed in 963e9de
test/test_improved_zmq_server.py
Outdated
|
|
||
| time.sleep(1) | ||
| results = client.get_test_results() | ||
| error_count = sum(1 for result in results if "ERROR" in result) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test only counts messages containing "ERROR", but the helper methods record unauthenticated request failures as "...FAILED", so error_count never increases and the test always reports failure even when behavior is correct.
Prompt for AI agents
Address the following comment on test/test_improved_zmq_server.py at line 984:
<comment>This test only counts messages containing "ERROR", but the helper methods record unauthenticated request failures as "...FAILED", so error_count never increases and the test always reports failure even when behavior is correct.</comment>
<file context>
@@ -0,0 +1,1364 @@
+
+ time.sleep(1)
+ results = client.get_test_results()
+ error_count = sum(1 for result in results if "ERROR" in result)
+
+ if error_count > 0:
</file context>
| error_count = sum(1 for result in results if "ERROR" in result) | |
| error_count = sum(1 for result in results if "ERROR" in result or "FAILED" in result) |
✅ Addressed in 963e9de
|
|
||
| # Check that all clients are receiving data | ||
| for i, client in enumerate(clients): | ||
| market_data_count = len([msg for msg in client.received_messages if msg.get('type') == 'market_data']) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test only logs message counts instead of asserting that other clients continue receiving data after one unsubscribes, so it will silently succeed even when the regression reappears. Please add explicit assertions that detect regressions.
Prompt for AI agents
Address the following comment on test/test_multi_client_fix.py at line 146:
<comment>The test only logs message counts instead of asserting that other clients continue receiving data after one unsubscribes, so it will silently succeed even when the regression reappears. Please add explicit assertions that detect regressions.</comment>
<file context>
@@ -0,0 +1,198 @@
+
+ # Check that all clients are receiving data
+ for i, client in enumerate(clients):
+ market_data_count = len([msg for msg in client.received_messages if msg.get('type') == 'market_data'])
+ logger.info(f"Client {client.client_id} received {market_data_count} market data messages")
+
</file context>
✅ Addressed in 963e9de
- Fix KeyError in websocket proxy when mode_to_str receives invalid keys - Improve test error detection to catch both ERROR and FAILED messages - Add assertions to multi-client test to detect subscription regressions - Correct inaccurate file references in ZMQ audit report documentation - Remove unnecessary comment in Angel broker adapter Resolves issues identified in zmq_new_audit_report.md audit findings.
Summary by cubic
Fixes critical race conditions in the WebSocket proxy using locks and reference counting to stabilize multi-client streaming. Also corrects Angel OHLC fields and returns structured API errors; adds tests and an audit report.
Bug Fixes
Refactors