whisper Integrate Pytorch's TorchDynamo as a passed in callable [1.375x faster on A100] [Needs more benchmarking]

Sep 25 '22 02:09 voznesenskym

Do you want benchmarks on an rtx 2060?

Oct 08 '22 16:10 oscardssmith

Can someone provide any info in how to actually use this change? My guess is that you have to set TorchDynamo to Whisper somehow. but nor sure how.

TorchDynamo documentation writes about decorating a method with @dynamo.optimize("inductor") but that's not used in this change.

I could provide benchmark on a RTX 3090 or GTX 1080.

Oct 25 '22 08:10 Sharrnah

Can someone provide any info in how to actually use this change?

My guess is that you have to set TorchDynamo to Whisper somehow. but nor sure how.

TorchDynamo documentation writes about decorating a method with @dynamo.optimize("inductor") but that's not used in this change.

I could provide benchmark on a RTX 3090 or GTX 1080.

Hey! I can take a look today. We ported dynamo into pytorch and it underwent some renaming. My apologies for falling behind on this.

Oct 25 '22 15:10 voznesenskym

Thanks, I'll close this for now, since it doesn't quite yet work "out of the box" and relying on nightly versions makes things difficult for me to maintain. I'm hoping to get an easier integration with the stable PyTorch 2 interface once it's out.

Dec 04 '22 23:12 jongwook