ta-lib-python icon indicating copy to clipboard operation
ta-lib-python copied to clipboard

Slowdown after PCA

Open DaxMao opened this issue 4 years ago • 2 comments

Talib function runs less efficiently after a PCA transform from sklearn is done:

import numpy as np
import talib
import time
import sklearn
from sklearn.decomposition import PCA

import sys
print(sys.version)
print(talib.__version__)
print(sklearn.__version__)

print('without pca')

for i in range(10):
    T1 = time.process_time()
    talib.MOM(np.random.rand(10000000))
    T2 = time.process_time()
    print(T2-T1)
    
print('with pca')

pca = PCA(n_components=10)
pca.fit(np.random.rand(10000,50))

for i in range(10):
    pca.transform(np.random.rand(10000,50))
    T1 = time.process_time()
    talib.MOM(np.random.rand(10000000))
    T2 = time.process_time()
    print(T2-T1)    
    

Result:

3.8.3 (default, Jul 19 2021, 00:04:17) 
[GCC 8.4.1 20200928 (Red Hat 8.4.1-1)]
0.4.18
0.24.2
without pca
0.15120640800000018
0.1477869940000005
0.1483331020000005
0.14869244900000034
0.1482859940000001
0.14942492199999968
0.14725573899999844
0.14845959500000028
0.14640222299999905
0.1488888760000009
with pca
0.48027800200000037
0.49989518399999966
0.49738425399999997
0.4995920690000002
0.4974063589999993
0.49985235000000117
0.4973194620000001
0.500063376
0.4975280720000015
0.5006387300000004

DaxMao avatar Jul 18 '21 16:07 DaxMao

That's weird, removing the pca.transform() call, or lifting it out of the loop appears to speed it up again.

mrjbq7 avatar Jul 22 '21 19:07 mrjbq7

Using time.time() instead of time.process_time() does not show a slowdown.

mrjbq7 avatar Jul 22 '21 19:07 mrjbq7

Closing old issue.

mrjbq7 avatar Jan 20 '24 19:01 mrjbq7