pfio icon indicating copy to clipboard operation
pfio copied to clipboard
verified publisher icon pfnet

Support PPE profiling

Open kuenishi opened this issue 4 years ago • 1 comments

One of requirements in our systems is to be suitable with other tools. As PPE has now Profiling feature, it would be nice if PFIO rides on its use cases, rather than inventing its own.

c.f. https://github.com/pfnet/pytorch-pfn-extras/pull/144

Update: PPE and PyTorch are both not ready for 3.10. https://github.com/pfnet/pytorch-pfn-extras/issues/386

Target tasks:

  • [ ] Local #326
  • [ ] S3
  • [ ] Zip
  • [ ] CachedHttpFs
  • [ ] FileCache / multiprocess FileCache
  • [ ] HttpCache
  • [ ] Add test suite with/without PPE

kuenishi avatar Jan 11 '22 08:01 kuenishi

Approach:

  • Put PPE's decorator onto PFIO
  • Check if we can get stats by adding current decorator
  • Check the chrome tracing in outdir once PPE emit happened: c.f. https://github.com/pfnet/pytorch-pfn-extras/issues/729
  • Save the start/end timing as trace with PPE

kuenishi avatar Aug 31 '23 10:08 kuenishi