bartosh
Broker vs feed performance
Has anyone experienced slower performance when using CCXT feed with a broker vs using CCXT feed without a broker?
Can anyone explain the difference?
I am definitively getting the same experience during prenext. 11sec per bar to compute few indicators when it run instantaneously without the broker.
I tried to explained my problem here : https://github.com/bartosh/backtrader/issues/17#issuecomment-418694471
and here: https://community.backtrader.com/topic/623/anyone-use-backtrader-to-do-live-trading-on-bitcoin-exchange/274
I have found functions which are taking too much time. Here is my ouput :
time is 2018-10-02 17:07:10.848704 ## start
CCXTBroker getposition 4.0 Seconds
0.01
init pausetrade = True
***** DATA NOTIF: DELAYED
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is 2018-10-02 17:07:25.269656 ## prenext
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is 2018-10-02 17:07:35.509687 ## prenext
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is 2018-10-02 17:07:45.745484 ## prenext
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is 2018-10-02 17:07:55.980936 ## prenext
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is 2018-10-02 17:08:06.219192 ## prenext
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getcash 2.0 Seconds
***** DATA NOTIF: LIVE
CCXTBroker getcash 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
CCXTBroker getvalue 2.0 Seconds
time is 2018-10-02 17:08:20.552561 ## next
***** NEXT: 2018-10-02 15:08:00 6551.0 6551.5 6550.0 6551.5 78848.0 Minute 6
2018-10-02 15:08:00, Close, 6551.50
CCXTBroker getposition 4.0 Seconds
CCXTBroker getposition 4.0 Seconds
It seems that getvalue and getcash functions are taking 2 sec and getposition 4sec. So I have found my 10-11 sec per bar during prenext.
With profiler, we see that retry_method in CCXTStore take all.
def retry(method):
@wraps(method)
def retry_method(self, *args, **kwargs):
for i in range(self.retries):
time.sleep(self.exchange.rateLimit / 1000)
try:
return method(self, *args, **kwargs)
except (NetworkError, ExchangeError):
if i == self.retries - 1:
raise
return retry_method
self.exchange.rateLimit = 2000 so it wait time.sleep(2.0 sec)
Does that make sense to modify this code like that ?
def retry(method):
@wraps(method)
def retry_method(self, *args, **kwargs):
for i in range(self.retries):
try:
return method(self, *args, **kwargs)
except (NetworkError, ExchangeError):
if i == self.retries - 1:
raise
time.sleep(self.exchange.rateLimit / 1000)
return retry_method
Does that make sense to modify this code like that ?
Yes, it makes sense. Please, submit PR. I'll be happy to accept it.
May be a wrong PR.
If I load 100 bar and, in prenext, BT request 5 times (2 getcash 3 getvalue) the broker per bar. We will reach this rolling limit of 300 per 5min.
Ideally, we should use a smart solution like this: https://github.com/tomasbasham/ratelimit
from ratelimit import limits, sleep_and_retry
import datetime
import time
import random
class Test:
TEN_SEC = 10
counter = 0
@sleep_and_retry
@limits(calls=5, period=TEN_SEC )
def ratelimiter(self):
pass
def test(self):
self.ratelimiter()
self.counter +=1
print('test ' + str(self.counter) + ' ' + str(datetime.datetime.now()))
def test2(self):
self.ratelimiter()
self.counter +=1
print('test2 ' + str(self.counter) + ' ' + str(datetime.datetime.now()))
a=Test()
for i in range(1,20) :
a.test()
time.sleep(random.randint(0,2))
a.test2()
Or what about an option to totally remove getcash/getvalue requests to the broker. In that case of Bitmex, returned data does not seem relevant as Bitmex permits to trade Futures and not currency pairs. And it requested 5 times for nothing, and getposition call 2 times retry decorator.
I have never done a PR yet but it will come one day 👍
I don't think threading or adjusting the rate limiting will solve the main performance issue which is caused by a lack of caching in the broker/store.
For my own testing I have resolved to only updating the balance (cash/value/position) once for each candle, and no more than once per ten seconds. This eliminates the slowness when running on historical data and greatly reduces issues with rate limiting.
My changes can be seen in this commit: https://github.com/VegarS/backtrader/commit/179f6841a80ba2b5edff38cf4c1a8b840ee9b6fd
It is not a complete solution, but it is a step in the right direction. A better solution is maybe to periodically update the balances and orders in a separate thread, which will also allow for proper order status notifications. @bartosh: Do periodic updates sound reasonable?
