fairseq2 icon indicating copy to clipboard operation
fairseq2 copied to clipboard
verified publisher icon facebookresearch

DataPipeline execution profiling

Open artemru opened this issue 1 year ago • 0 comments

How could one to see the overall performance iterations of a complex DataPipeline and identify potential bottlenecks ? Ideally we would like to have several time series informations (like CPU/Memory usage) per any single stage in DataPipeline. Maybe something similar to https://www.tensorflow.org/guide/data_performance_analysis .

artemru avatar Feb 28 '24 10:02 artemru