Binance Live Strategy cannot handle GTE_GTC order

[graceyangfan]

Bug Report

Binance Live Strategy cannot handle GTE_GTC order

Expected Behavior

Strategy should handler manual closed GTE_GTC order

Actual Behavior

Strategy cannot process position closed event feedback info.

Steps to Reproduce the Problem

1.Use binance future strategy to open a position and manually close it on phone or web with market stop loss order.
2.look at the logs.

Specifications

• OS platform:Ubuntu 20.04
• Python version:python 3.10
• nautilus_trader version: latest develop version

here is the log

�[1m2023-02-23T08:55:52.325183918Z�[0m �[1;31m[ERR] TESTER-001.ExecClient-BINANCE: Error on handling b'{"stream":********************************************,"data":{"e":"ORDER_TRADE_UPDATE","T":1677142552316,"E":1677142552321,"o":{"s":"ETHBUSD","c":"android_VpsL6tMAZkC7KNDV5mdE","S":"BUY","o":"TAKE_PROFIT_MARKET","f":"GTE_GTC","q":"0","p":"0","ap":"0","sp":"1664","x":"NEW","X":"NEW","i":23348852026,"l":"0","z":"0","L":"0","T":1677142552316,"t":0,"b":"0","a":"0","m":false,"R":true,"wt":"MARK_PRICE","ot":"TAKE_PROFIT_MARKET","ps":"BOTH","cp":true,"rp":"0","pP":true,"si":0,"ss":0}}}'
ValidationError(Invalid enum value 'GTE_GTC' - at `$.data.o.f`)
Traceback (most recent call last):
  File "/home/ubuntu/nautilus_trader/nautilus_trader/adapters/binance/futures/execution.py", line 248, in _handle_user_ws_message
    self._futures_user_ws_handlers[wrapper.data.e](raw)
  File "/home/ubuntu/nautilus_trader/nautilus_trader/adapters/binance/futures/execution.py", line 257, in _handle_order_trade_update
    order_update = self._decoder_futures_order_update_wrapper.decode(raw)
�[0m
�[1m2023-02-23T08:58:31.354549449Z�[0m �[1;31m[ERR] TESTER-001.ExecClient-BINANCE: Error on handling b'{"stream":********************************************,"data":{"e":"ORDER_TRADE_UPDATE","T":1677142711347,"E":1677142711351,"o":{"s":"ETHBUSD","c":"android_VpsL6tMAZkC7KNDV5mdE","S":"BUY","o":"TAKE_PROFIT_MARKET","f":"GTE_GTC","q":"0","p":"0","ap":"0","sp":"1664","x":"CANCELED","X":"CANCELED","i":23348852026,"l":"0","z":"0","L":"0","T":1677142711347,"t":0,"b":"0","a":"0","m":false,"R":true,"wt":"MARK_PRICE","ot":"TAKE_PROFIT_MARKET","ps":"BOTH","cp":true,"rp":"0","pP":true,"si":0,"ss":0}}}'
ValidationError(Invalid enum value 'GTE_GTC' - at `$.data.o.f`)
Traceback (most recent call last):
  File "/home/ubuntu/nautilus_trader/nautilus_trader/adapters/binance/futures/execution.py", line 248, in _handle_user_ws_message
    self._futures_user_ws_handlers[wrapper.data.e](raw)
  File "/home/ubuntu/nautilus_trader/nautilus_trader/adapters/binance/futures/execution.py", line 257, in _handle_order_trade_update
    order_update = self._decoder_futures_order_update_wrapper.decode(raw)
�[0m
�[1m2023-02-23T09:05:58.913686877Z�[0m �[1;31m[ERR] TESTER-001.ExecClient-BINANCE: Error on handling b'{"stream":********************************************,"data":{"e":"ORDER_TRADE_UPDATE","T":1677143158906,"E":1677143158910,"o":{"s":"ETHBUSD","c":"android_xwg43aufYauUhC22oNDf","S":"BUY","o":"TAKE_PROFIT_MARKET","f":"GTE_GTC","q":"0","p":"0","ap":"0","sp":"1665","x":"NEW","X":"NEW","i":23349213443,"l":"0","z":"0","L":"0","T":1677143158906,"t":0,"b":"0","a":"0","m":false,"R":true,"wt":"MARK_PRICE","ot":"TAKE_PROFIT_MARKET","ps":"BOTH","cp":true,"rp":"0","pP":true,"si":0,"ss":0}}}'
ValidationError(Invalid enum value 'GTE_GTC' - at `$.data.o.f`)
Traceback (most recent call last):
  File "/home/ubuntu/nautilus_trader/nautilus_trader/adapters/binance/futures/execution.py", line 248, in _handle_user_ws_message
    self._futures_user_ws_handlers[wrapper.data.e](raw)
  File "/home/ubuntu/nautilus_trader/nautilus_trader/adapters/binance/futures/execution.py", line 257, in _handle_order_trade_update
    order_update = self._decoder_futures_order_update_wrapper.decode(raw)
�[0m
�[1m2023-02-23T09:12:17.508810603Z�[0m [INF] TESTER-001.BinanceFuturesInstrumentProvider: Loading all instruments...�[0m
�[1m2023-02-23T09:12:17.529163563Z�[0m �[1;33m[WRN] TESTER-001.BinanceFuturesInstrumentProvider: Currently not requesting actual trade fees. All instruments will have zero fees.�[0m
�[1m2023-02-23T09:24:03.101204618Z�[0m �[1;31m[ERR] TESTER-001.ExecClient-BINANCE: Error on handling b'{"stream":********************************************,"data":{"e":"ORDER_TRADE_UPDATE","T":1677144243094,"E":1677144243098,"o":{"s":"ETHBUSD","c":"android_xwg43aufYauUhC22oNDf","S":"BUY","o":"TAKE_PROFIT_MARKET","f":"GTE_GTC","q":"0","p":"0","ap":"0","sp":"1665","x":"CANCELED","X":"CANCELED","i":23349213443,"l":"0","z":"0","L":"0","T":1677144243094,"t":0,"b":"0","a":"0","m":false,"R":true,"wt":"MARK_PRICE","ot":"TAKE_PROFIT_MARKET","ps":"BOTH","cp":true,"rp":"0","pP":true,"si":0,"ss":0}}}'
ValidationError(Invalid enum value 'GTE_GTC' - at `$.data.o.f`)
Traceback (most recent call last):
  File "/home/ubuntu/nautilus_trader/nautilus_trader/adapters/binance/futures/execution.py", line 248, in _handle_user_ws_message
    self._futures_user_ws_handlers[wrapper.data.e](raw)
  File "/home/ubuntu/nautilus_trader/nautilus_trader/adapters/binance/futures/execution.py", line 257, in _handle_order_trade_update
    order_update = self._decoder_futures_order_update_wrapper.decode(raw)
�[0m
�[1m2023-02-23T09:26:23.662876687Z�[0m �[36m[INF] TESTER-001.TestStrategy-000: TestBar(bar_type=ETHBUSD-PERP.BINANCE-250-VALUE-LAST-EXTERNAL, open=1669.59, high=1673.29, low=1666.00, close=1666.31, volume=10396.068, small_value=133938.71237000002, big_value=8962078.989519976, small_value2=130579.64594999974, big_value2=8130489.000450003, ts_event=1677144383648000000, ts_init=1677144383648000000)�[0m
�[1m2023-02-23T09:26:23.669777962Z�[0m [INF] TESTER-001.TestStrategy-000: meta_signal is [0.45841897 0.541581  ]�[0m
�[1m2023-02-23T09:26:23.669878894Z�[0m [INF] TESTER-001.TestStrategy-000: predict_signal is [0.515124 0.484876]�[0m
�[1m2023-02-23T09:26:23.670636197Z�[0m �[1;31m[ERR] TESTER-001.TestStrategy-000: Error on handling TradeTick(ETHBUSD-PERP.BINANCE,1666.31,67.654,BUYER,356086485,1677144383648000000)
AttributeError('NoneType' object has no attribute 'close')
Traceback (most recent call last):
  File "nautilus_trader/trading/strategy.pyx", line 1039, in nautilus_trader.trading.strategy.Strategy.handle_trade_tick
  File "nautilus_trader/common/actor.pyx", line 421, in nautilus_trader.common.actor.Actor.on_trade_tick
  File "/home/ubuntu/nautilus_trader/nautilus_trader/examples/strategies/live_imbalance_strategy.py", line 177, in on_trade_tick
    self.bar_builder.handle_trade_tick(tick)
  File "nautilus_trader/model/data/aggregate_imbalance_bar.pyx", line 177, in nautilus_trader.model.data.aggregate_imbalance_bar.TestBuilder.handle_trade_tick
  File "nautilus_trader/model/data/aggregate_imbalance_bar.pyx", line 187, in nautilus_trader.model.data.aggregate_imbalance_bar.TestBuilder.handle_trade_tick
  File "nautilus_trader/model/data/aggregate_imbalance_bar.pyx", line 225, in nautilus_trader.model.data.aggregate_imbalance_bar.TestBuilder._apply_update
  File "/home/ubuntu/nautilus_trader/nautilus_trader/examples/strategies/live_imbalance_strategy.py", line 311, in on_bar
    self.profitloss_model.atr_stop(self,data)
  File "/home/ubuntu/nautilus_trader/nautilus_trader/examples/strategies/profitloss_model.py", line 143, in atr_stop
    if strategy.portfolio.is_net_short(strategy.instrument_id) and bar.close.as_double()> prev_bar.close.as_double() and \
�[0m
�[1m2023-02-23T09:26:33.702202962Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Timed out (10.0s) waiting for node to stop.
Status
------
DataEngine.check_disconnected() == False
ExecEngine.check_disconnected() == False�[0m
�[1m2023-02-23T09:26:33.702225173Z�[0m [INF] TESTER-001.TradingNode: DISPOSING...�[0m
�[1m2023-02-23T09:26:33.702468169Z�[0m [INF] TESTER-001.TESTER-001: STOPPING...�[0m
�[1m2023-02-23T09:26:33.702477921Z�[0m [INF] TESTER-001.TestStrategy-000: STOPPING...�[0m
�[1m2023-02-23T09:26:33.702519911Z�[0m [INF] TESTER-001.TestStrategy-000: No open or emulated ETHBUSD-PERP.BINANCE orders to cancel.�[0m
�[1m2023-02-23T09:26:33.702531031Z�[0m [INF] TESTER-001.TestStrategy-000: Closing 1 open position...�[0m
�[1m2023-02-23T09:26:33.702648395Z�[0m [INF] TESTER-001.TestStrategy-000: <--[EVT] OrderInitialized(instrument_id=ETHBUSD-PERP.BINANCE, client_order_id=O-20230223-001-000-2, side=BUY, type=MARKET, quantity=0.021, time_in_force=GTC, post_only=False, reduce_only=True, options={}, emulation_trigger=NO_TRIGGER, contingency_type=NO_CONTINGENCY, order_list_id=None, linked_order_ids=None, parent_order_id=None, tags=None).�[0m
�[1m2023-02-23T09:26:33.702679516Z�[0m [INF] TESTER-001.TestStrategy-000: [CMD]--> SubmitOrder(order=MarketOrder(BUY 0.021 ETHBUSD-PERP.BINANCE MARKET GTC, status=INITIALIZED, client_order_id=O-20230223-001-000-2, venue_order_id=None, tags=None), position_id=ETHBUSD-PERP.BINANCE-BOTH, exec_algorithm_spec=None).�[0m
�[1m2023-02-23T09:26:33.702773578Z�[0m [INF] TESTER-001.TestStrategy-000: [CMD]--> Unsubscribe(TradeTick{'instrument_id': InstrumentId('ETHBUSD-PERP.BINANCE')}).�[0m
�[1m2023-02-23T09:26:33.702794090Z�[0m [INF] TESTER-001.TestStrategy-000: STOPPED.�[0m
�[1m2023-02-23T09:26:33.702807439Z�[0m [INF] TESTER-001.TESTER-001: STOPPED.�[0m
�[1m2023-02-23T09:26:33.702818291Z�[0m [INF] TESTER-001.DataEngine: STOPPING...�[0m
�[1m2023-02-23T09:26:33.702823191Z�[0m [INF] TESTER-001.DataClient-BINANCE: STOPPING...�[0m
�[1m2023-02-23T09:26:33.702829760Z�[0m [INF] TESTER-001.DataClient-BINANCE: STOPPED.�[0m
�[1m2023-02-23T09:26:33.702844321Z�[0m [INF] TESTER-001.DataEngine: STOPPED.�[0m
�[1m2023-02-23T09:26:33.702853761Z�[0m [INF] TESTER-001.RiskEngine: STOPPING...�[0m
�[1m2023-02-23T09:26:33.702861581Z�[0m [INF] TESTER-001.RiskEngine: STOPPED.�[0m
�[1m2023-02-23T09:26:33.702871141Z�[0m [INF] TESTER-001.ExecEngine: STOPPING...�[0m
�[1m2023-02-23T09:26:33.702875072Z�[0m [INF] TESTER-001.ExecClient-BINANCE: STOPPING...�[0m
�[1m2023-02-23T09:26:33.702880362Z�[0m [INF] TESTER-001.ExecClient-BINANCE: STOPPED.�[0m
�[1m2023-02-23T09:26:33.702908762Z�[0m [INF] TESTER-001.ExecEngine: STOPPED.�[0m
�[1m2023-02-23T09:26:33.702924783Z�[0m [INF] TESTER-001.OrderEmulator: STOPPING...�[0m
�[1m2023-02-23T09:26:33.702935903Z�[0m [INF] TESTER-001.OrderEmulator: STOPPED.�[0m
�[1m2023-02-23T09:26:33.702943575Z�[0m [INF] TESTER-001.TESTER-001: DISPOSING...�[0m
�[1m2023-02-23T09:26:33.702950675Z�[0m [INF] TESTER-001.TestStrategy-000: DISPOSING...�[0m
�[1m2023-02-23T09:26:33.702958824Z�[0m [INF] TESTER-001.TestStrategy-000: DISPOSED.�[0m
�[1m2023-02-23T09:26:33.702969724Z�[0m [INF] TESTER-001.TESTER-001: DISPOSED.�[0m
�[1m2023-02-23T09:26:33.702980276Z�[0m [INF] TESTER-001.DataEngine: DISPOSING...�[0m
�[1m2023-02-23T09:26:33.702983645Z�[0m [INF] TESTER-001.DataClient-BINANCE: DISPOSING...�[0m
�[1m2023-02-23T09:26:33.702989145Z�[0m [INF] TESTER-001.DataClient-BINANCE: DISPOSED.�[0m
�[1m2023-02-23T09:26:33.703002636Z�[0m [INF] TESTER-001.DataEngine: DISPOSED.�[0m
�[1m2023-02-23T09:26:33.703022366Z�[0m [INF] TESTER-001.RiskEngine: DISPOSING...�[0m
�[1m2023-02-23T09:26:33.703032466Z�[0m [INF] TESTER-001.RiskEngine: DISPOSED.�[0m
�[1m2023-02-23T09:26:33.703041906Z�[0m [INF] TESTER-001.ExecEngine: DISPOSING...�[0m
�[1m2023-02-23T09:26:33.703047458Z�[0m [INF] TESTER-001.ExecClient-BINANCE: DISPOSING...�[0m
�[1m2023-02-23T09:26:33.703053107Z�[0m [INF] TESTER-001.ExecClient-BINANCE: DISPOSED.�[0m
�[1m2023-02-23T09:26:33.703064227Z�[0m [INF] TESTER-001.ExecEngine: DISPOSED.�[0m
�[1m2023-02-23T09:26:33.703075898Z�[0m [INF] TESTER-001.OrderEmulator: DISPOSING...�[0m
�[1m2023-02-23T09:26:33.703081539Z�[0m [INF] TESTER-001.OrderEmulator: DISPOSED.�[0m
�[1m2023-02-23T09:26:33.703093239Z�[0m [INF] TESTER-001.TradingNode: Shutting down executor...�[0m
�[1m2023-02-23T09:26:33.703112328Z�[0m [INF] TESTER-001.TradingNode: Stopping event loop...�[0m
�[1m2023-02-23T09:26:33.703184751Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='res_queue' coro=<LiveDataEngine._run_res_queue() running at nautilus_trader/live/data_engine.pyx:411> cb=[gather.<locals>._done_callback() at /usr/local/lib/python3.10/asyncio/tasks.py:720]>�[0m
�[1m2023-02-23T09:26:33.703211482Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='evt_queue' coro=<LiveExecutionEngine._run_evt_queue() running at nautilus_trader/live/execution_engine.pyx:339> cb=[gather.<locals>._done_callback() at /usr/local/lib/python3.10/asyncio/tasks.py:720]>�[0m
�[1m2023-02-23T09:26:33.703243272Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='Task-37' coro=<WebSocketClient.start() running at nautilus_trader/network/websocket.pyx:261> wait_for=<Future pending cb=[Task.task_wakeup()]>>�[0m
�[1m2023-02-23T09:26:33.703269953Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='cmd_queue' coro=<LiveDataEngine._run_cmd_queue() running at nautilus_trader/live/data_engine.pyx:371> cb=[gather.<locals>._done_callback() at /usr/local/lib/python3.10/asyncio/tasks.py:720]>�[0m
�[1m2023-02-23T09:26:33.703291254Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='evt_queue' coro=<LiveRiskEngine._run_evt_queue() running at nautilus_trader/live/risk_engine.pyx:253> cb=[gather.<locals>._done_callback() at /usr/local/lib/python3.10/asyncio/tasks.py:720]>�[0m
�[1m2023-02-23T09:26:33.703314705Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='Task-23' coro=<WebSocketClient.start() running at nautilus_trader/network/websocket.pyx:261> wait_for=<Future pending cb=[Task.task_wakeup()]>>�[0m
�[1m2023-02-23T09:26:33.703335655Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='Task-10' coro=<LiveExecutionEngine._inflight_check_loop() running at nautilus_trader/live/execution_engine.pyx:359> wait_for=<Future cancelled>>�[0m
�[1m2023-02-23T09:26:33.703356847Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='req_queue' coro=<LiveDataEngine._run_req_queue() running at nautilus_trader/live/data_engine.pyx:391> cb=[gather.<locals>._done_callback() at /usr/local/lib/python3.10/asyncio/tasks.py:720]>�[0m
�[1m2023-02-23T09:26:33.703384397Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='cmd_queue' coro=<LiveExecutionEngine._run_cmd_queue() running at nautilus_trader/live/execution_engine.pyx:319> cb=[gather.<locals>._done_callback() at /usr/local/lib/python3.10/asyncio/tasks.py:720]>�[0m
�[1m2023-02-23T09:26:33.703440769Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='_update_instruments' coro=<BinanceCommonDataClient._update_instruments() running at /home/ubuntu/nautilus_trader/nautilus_trader/adapters/binance/common/data.py:217> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[LiveMarketDataClient._on_task_completed(None, None)()]>�[0m
�[1m2023-02-23T09:26:33.703464430Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='cmd_queue' coro=<LiveRiskEngine._run_cmd_queue() running at nautilus_trader/live/risk_engine.pyx:235> cb=[gather.<locals>._done_callback() at /usr/local/lib/python3.10/asyncio/tasks.py:720]>�[0m
�[1m2023-02-23T09:26:33.703508200Z�[0m �[1;33m[WRN] TESTER-001.TradingNode: Canceling pending task <Task pending name='_ping_listen_keys' coro=<BinanceCommonExecutionClient._ping_listen_keys() running at /home/ubuntu/nautilus_trader/nautilus_trader/adapters/binance/common/execution.py:242> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[LiveExecutionClient._on_task_completed(None, None)()]>�[0m
�[1m2023-02-23T09:26:33.703676485Z�[0m �[1;33m[WRN] TESTER-001.DataEngine: DataCommand message queue processing stopped with 2 message(s) on queue.�[0m
�[1m2023-02-23T09:26:33.703701717Z�[0m �[1;33m[WRN] TESTER-001.DataEngine: DataRequest message queue processing stopped with 1 message(s) on queue.�[0m
�[1m2023-02-23T09:26:33.703714097Z�[0m �[1;33m[WRN] TESTER-001.DataEngine: DataResponse message queue processing stopped with 1 message(s) on queue.�[0m
�[1m2023-02-23T09:26:33.703723627Z�[0m �[1;33m[WRN] TESTER-001.RiskEngine: Running canceled with 2 message(s) on queue.�[0m
�[1m2023-02-23T09:26:33.703733378Z�[0m �[1;33m[WRN] TESTER-001.RiskEngine: Running canceled with 1 message(s) on queue.�[0m
�[1m2023-02-23T09:26:33.703743818Z�[0m �[1;33m[WRN] TESTER-001.ExecEngine: Command message queue processing canceled with 1 message(s) on queue.�[0m
�[1m2023-02-23T09:26:33.703756888Z�[0m �[1;33m[WRN] TESTER-001.ExecEngine: Event message queue processing canceled with 1 message(s) on queue.�[0m
�[1m2023-02-23T09:26:33.704806059Z�[0m �[1;31m[ERR] TESTER-001.TradingNode: Error on dispose

[cjdsellers]

Looks like Binance returned a weird and undocumented (at least I can't find it) time in force value:

ValidationError(Invalid enum value 'GTE_GTC' - at `$.data.o.f`)

Do you regularly manually close positions, and it normally works?

[graceyangfan]

@cjdsellers not so frequently，current I have not find a good stop condition，so sometimes I leave open position with strategy and close position manually

[cjdsellers]

Understood, just trying to establish that Nautilus can normally handle this, when Binance sends a proper time in force? Last QA testing I did it was handling it well.

[cjdsellers]

What were the original order params you used (especially the time in force)?

[graceyangfan]

    def buy(self, trade_size, price, limit = False):
        """
        Users simple buy method (example).
        """
        if limit:
            order: LimitOrder = self.order_factory.limit(
                instrument_id=self.instrument_id,
                order_side=OrderSide.BUY,
                quantity=self.instrument.make_qty(trade_size),
                price=self.instrument.make_price(price),
            )
        else:
            order: MarketOrder = self.order_factory.market(
                instrument_id=self.instrument_id,
                order_side=OrderSide.BUY,
                quantity=self.instrument.make_qty(trade_size),
            )

        self.entry = order 
        self.submit_order(order)

current using MarketOrder

[cjdsellers]

Thanks for the feedback, could you please let me know if you see this again when manually closing a position 🙏 .

[graceyangfan]

GTE_GTC//GoodTillExpiredOrCanceled NOT for Futures,this is not includ in binance document,but I find some form github code,maybe we need to add this to TimeInForce.
https://stackoverflow.com/questions/70447698/how-to-cancel-stop-loss-and-take-profit-order-when-position-close-on-binance-fut
https://github.com/ccxt/binance-api-node/pull/573

[cjdsellers]

@poshcoe How do you think we should handle this?

[poshcoe]

A GTE_GTC value will need to be added to the BinanceTimeInForce enum, and maybe we just parse it back to the internal TimeInForce.GTC in parse_binance_time_in_force?

[cjdsellers]

A GTE_GTC value will need to be added to the BinanceTimeInForce enum, and maybe we just parse it back to the internal TimeInForce.GTC in parse_binance_time_in_force?

Agreed.

[cjdsellers]

Now fixed from 914d3cf on develop.