Write transactions and recorded data during algorithm run

[kooomix]

Hi,

I have set the "record" function to log information and have configured the Analyze function to output recorded and perf data into a CSV file.

As far as I could found, the Analyze function can only be executed (by default) at the end of the algorithm run.

My question is - how can I output recorded and perf data during algorithm run, say, every 30 minutes?

Thanks.

[AvishaiW]

Hi, I'm not sure I've understood your needs here, but if i've understood correctly, you can access the recorded values at any stage, locally at: ~/.catalyst/data/live_algos/{algo_namespace}/custom_signals_stats_{mode_name}.csv if this is not what you are referring to, please try to rephrase.

[kooomix]

Hi Avishai,

Sorry for not being clear.. What you have suggested is good for the custom recorded data. There's also the "perf" parameter in the analyze function from which I extract transactions. This function, "analyze", is being executed when the algo ends. My question is - can I somehow access / write the perf transactions data during the algorithm run, and not only at the end?

Thanks.

[AvishaiW]

so what about using context.custom_signals_stats?

[kooomix]

I'm not familiar with context.custom_signals_stats and didn't find much information about it. Can you refer me to a documentation or alternatively explain further how exactly to work with it?

[AvishaiW]

Sorry for being unclear. Currently there is no appropriate method to perform what you have expected therefore, this is not documented. However, if you are interested in extracting the perf info during the run of the algorithm, you can look at the following: context.custom_signals_stats argument which contains the recorded variables. In order to extract the transactions you can call context.perf_tracker.todays_performance.processed_transactions. perf_tracker contains all the performance info and specifically todays_performance contains the portfolio info.

[sawyercade]

@kooomix I haven't tested it myself, but catalyst.api.run_algorithm() has an analyze_live parameter that you can pass in the same way as your analyze function, and it's called after every tick in live mode:

analyze_live: callable[(context, pd.DataFrame) -> None], optional
        The interactive analyze function to be used with
        the live graph clock in every tick.```

[kooomix]

@AvishaiW - Thanks for the tracker info. I have tried to experiments with the "tracker" parameter. The "context.perf_tracker.todays_performance.processed_transactions" indeed helps a bit as it enables me to extract the transactions during algo run time. Is there a variable which holds all accumulated transactions info or must I manage the data on a daily basis?

@sawyercade Analyze_live is a nice tool I might use, thanks! Note that if using the Analyze_live together with Analyze - "perf" variables turned out to be empty in Analyze(context, perf). A bug maybe?

[kooomix]

@AvishaiW - I have been using context.perf_tracker.todays_performance.processed_transactions and it serves me very well. I have though a question of how it behaves - I have noticed that sometimes when I "kill" the algorithm actively and then run it again (under the same namespace), some of the transactions appear to be missing, the transactions I just made before killing the algo. It also seems like this info is forever lost and can't be recovered. So my questions:

1. How exactly transactions are being synchronized with the exchange? On which point I can know for sure transactions are well synced?
2. In case like the one described above, does it make sense part of the transactions are lost completely?

Thanks!