TorchSharp Force GPU memory limit

As mentioned in this question, PyTorch now supports limiting GPU memory usage, which helps better management and planning on it.

Mar 07 '22 09:03 QingtaoLi1

I'm searching for the C++ code that implements it. Haven't found it yet.

Mar 07 '22 18:03 NiklasGustafsson

@NiklasGustafsson I found method implementation of set_per_process_memory_fraction is exists on torch/csrc/cuda/Module.cpp::_cuda_setMemoryFraction I don't know about how pytorch implementing native method to python, but this might be help :D

Oct 09 '22 17:10 dayo05

Maybe this method is doing that: c10::cuda::CUDACachingAllocator::setMemoryFraction(fraction, device);

Oct 09 '22 17:10 dayo05

CUDACachingAllocator

Which header file is that declared in? I don't find it in torch/cuda.h

Oct 10 '22 17:10 NiklasGustafsson

CUDACachingAllocator

Which header file is that declared in? I don't find it in torch/cuda.h

c10/cuda/CUDACachingAllocator.cpp here in 927 line

c10/cuda/CUDACachingAllocator.h this is header

Oct 12 '22 00:10 dayo05

@NiklasGustafsson I'm trying to implement this method but, this method is only available on CUDA-backend api. Is there any place to locate cuda-only-apis?

Oct 22 '22 06:10 dayo05

No, because the interop layer is backend-independent, so it doesn't link to anything that isn't available in both.

Oct 24 '22 19:10 NiklasGustafsson