CTranslate2 icon indicating copy to clipboard operation
CTranslate2 copied to clipboard
verified publisher icon OpenNMT

DirectML Support

Open gdiaz384 opened this issue 2 years ago • 3 comments

Assume that I already followed Microsoft's instructions to Enable PyTorch with DirectML on Windows and the DirectML library loads correctly according to MS's example code.

If I wanted to use PyTorch + DirectML + Ctranslate2 + an AMD GPU, how would I do it? Does Ctranslate2 need any major updates? Shouldn't I just need to pass directml instead of cuda to device= as found in the quickstart?

What is the bottleneck to this not working right now?

Aside: MS's instructions for DirectML's Python library install pytorch==2.0 as of early 2024. MS's documentation suggests it only works with PyTorch 1.3, which seems to be incorrect? So, it should not be a matter of supporting PyTorch 1.x since DirectML runs on 2.x in practice.

gdiaz384 avatar Feb 04 '24 23:02 gdiaz384

I need DirectML too.

faker2048 avatar Mar 02 '24 14:03 faker2048

Currently, Ctranslate2 do not support DirectML. To support this, the new implementation for this backend is required.

minhthuc2502 avatar Mar 04 '24 17:03 minhthuc2502