Kuan Butts
Kuan Butts
Addressed (but still slow) via https://github.com/kuanb/peartree/pull/14
Used snakeviz with cProfile and this is what the breakdown on performance of the operation looks like at present:  `generate_edge_and_wait_values` is the real hog here. It is primarily comprised...
Parallelization with performant pickling enabled via https://github.com/kuanb/peartree/issues/12
Noticing the unaccounted for stop id management step is taking quite a while: ``` Some unaccounted for stop ids. Resolving 2457... ``` ^ Example from LA Metro GTFS zip file.
On smaller feeds (or even mid-sized feeds, like AC Transit), MP is slower. I need to figure out how to intelligently navigate away from using MP in these situations. Sigh,...
Huge performance gain found right here: https://github.com/kuanb/peartree/issues/87 (Thank you @yiyange)
Updated performance, with the last few updates incorporates (see all commits from Wed to today): Without MP: 87.5s (63.3% faster) With MP: 93.97s (67% faster) cc @yiyange
There is a higher initialization cost to using multiprocessing. The gains can be seen primarily on larger datasets, such as LA Metro. I should bench mark that.
Whoops sorry didn't mean to close.