InvokeAI [bug]: invokeai 5.3.2 rc5 - segmentation fault on AMD GPU

Is there an existing issue for this?

[X] I have searched the existing issues

OS

Linux

GPU

amd

VRAM

8GB

What happened?

I receive a segmentation fault when starting to load the models in InvokeAI 5.3 rc-5

I included python's faulthandler around line 317 of ldm.invoke.model_manager This is the output I get.

$ invokeai
>> Initialization file /home/user/invokeai/invokeai.init found. Loading...
/home/user/invokeai/.venv/lib/python3.10/site-packages/torch/cuda/__init__.py:497: UserWarning: Can't initialize NVML
  warnings.warn("Can't initialize NVML")
Fatal Python error: Segmentation fault

Current thread 0x00007f83d52a0440 (most recent call first):
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 987 in convert
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 664 in _apply
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 641 in _apply
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 641 in _apply
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 641 in _apply
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 641 in _apply
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 989 in to
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/pytorch_lightning/core/mixins/device_dtype_mixin.py", line 113 in to
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/ldm/invoke/model_manager.py", line 429 in _load_ckpt_model
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/ldm/invoke/model_manager.py", line 318 in _load_model
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/ldm/invoke/model_manager.py", line 97 in get_model
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/ldm/generate.py", line 869 in set_model
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/ldm/generate.py", line 839 in load_model
  File "/home/user/invokeai/.venv/lib/python3.10/site-packages/ldm/invoke/CLI.py", line 139 in main
  File "/home/user/invokeai/.venv/bin/invokeai", line 8 in <module>

Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, yaml._yaml, PIL._imaging, torch._C, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, google.protobuf.pyext._message, greenlet._greenlet, __original_module__thread, __original_module_select, __original_module_time, psutil._psutil_linux, psutil._psutil_posix, scipy._lib._ccallback_c, scipy.special._ufuncs_cxx, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg._cythonized_array_utils, scipy.linalg._flinalg, scipy.linalg._solve_toeplitz, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_lapack, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.sparse.linalg._isolve._iterative, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, scipy.special._ellip_harm_2, scipy.integrate._odepack, scipy.integrate._quadpack, scipy.integrate._vode, scipy.integrate._dop, scipy.integrate._lsoda, scipy.optimize._minpack2, scipy.optimize._group_columns, scipy._lib.messagestream, scipy.optimize._trlib._trlib, numpy.linalg.lapack_lite, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._cobyla, scipy.optimize._slsqp, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy.optimize.__nnls, scipy.optimize._highs.cython.src._highs_wrapper, scipy.optimize._highs._highs_wrapper, scipy.optimize._highs.cython.src._highs_constants, scipy.optimize._highs._highs_constants, scipy.linalg._interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._distance_wrap, scipy.spatial._hausdorff, scipy.spatial.transform._rotation, scipy.optimize._direct, regex._regex, skimage._shared.geometry, skimage.draw._draw, skimage.transform._hough_transform, scipy.interpolate._fitpack, scipy.interpolate.dfitpack, scipy.interpolate._bspl, scipy.interpolate._ppoly, scipy.interpolate.interpnd, scipy.interpolate._rbfinterp_pythran, scipy.interpolate._rgi_cython, scipy._lib._uarray._uarray, scipy.ndimage._nd_image, _ni_label, scipy.ndimage._ni_label, skimage.transform._warps_cy, skimage.measure._find_contours_cy, skimage.measure._marching_cubes_lewiner_cy, skimage.measure._marching_cubes_classic_cy, skimage.measure._moments_cy, scipy.signal._sigtools, scipy.signal._max_len_seq_inner, scipy.signal._upfirdn_apply, scipy.signal._spline, scipy.signal._sosfilt, scipy.signal._spectral, scipy.special.cython_special, scipy.stats._stats, scipy.stats.beta_ufunc, scipy.stats._boost.beta_ufunc, scipy.stats.binom_ufunc, scipy.stats._boost.binom_ufunc, scipy.stats.nbinom_ufunc, scipy.stats._boost.nbinom_ufunc, scipy.stats.hypergeom_ufunc, scipy.stats._boost.hypergeom_ufunc, scipy.stats.ncf_ufunc, scipy.stats._boost.ncf_ufunc, scipy.stats.ncx2_ufunc, scipy.stats._boost.ncx2_ufunc, scipy.stats.nct_ufunc, scipy.stats._boost.nct_ufunc, scipy.stats.skewnorm_ufunc, scipy.stats._boost.skewnorm_ufunc, scipy.stats.invgauss_ufunc, scipy.stats._boost.invgauss_ufunc, scipy.stats._biasedurn, scipy.stats._levy_stable.levyst, scipy.stats._stats_pythran, scipy.stats._statlib, scipy.stats._mvn, scipy.stats._sobol, scipy.stats._qmc_cy, scipy.stats._rcont.rcont, scipy.signal._peak_finding_utils, skimage.measure._pnpoly, skimage.measure._ccomp, skimage.transform._radon_transform, PIL._imagingft (total: 150)

Some details:

(.venv) ~/invokeai ᐅ python --version
Python 3.10.0
(.venv) ~/invokeai ᐅ rocminfo | grep 'Name:'        
  Name:                    gfx1030                            
  Marketing Name:          Radeon RX 580 Series               
  Vendor Name:             AMD                                
      Name:                    amdgcn-amd-amdhsa--gfx1030         
(.venv) ~/invokeai ᐅ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"
(.venv) ~/invokeai ᐅ cat .env
export MIOPEN_DEBUG_CONV_DIRECT_NAIVE_CONV_FWD=0
export MIOPEN_DEBUG_CONV_DIRECT_NAIVE_CONV_BWD=0
export MIOPEN_DEBUG_CONV_DIRECT_NAIVE_CONV_WRW=0
export DRI_PRIME=0
export HSA_OVERRIDE_GFX_VERSION=10.3.0
export AMDGPU_TARGETS="gfx1030"
export ROCM_VERSION=5.4.3
export ROCM_HOME=/opt/rocm-${ROCM_VERSION}
export LD_LIBRARY_PATH=${ROCM_HOME}/lib
(.venv) ~/invokeai ᐅ python -c 'import torch; print(torch.cuda.is_available())'
True

Relevant portion of config.yml:

stable-diffusion-1.5:
  description: The newest Stable Diffusion version 1.5 weight file (4.27 GB)
  weights: models/ldm/stable-diffusion-v1/vae-ft-mse-840000-ema-pruned.ckpt
  config: configs/stable-diffusion/v1-inference.yaml
  width: 512
  height: 512
  default: true

This might be a torch issue? I seem to remember torch with ROCM only works with python 3.8 (and invokeai works with 3.10+)

Screenshots

No response

Additional context

No response

Contact Details

No response

Feb 08 '23 18:02 src-r-r

Might be related to pytorch. Here's the GDB output:

GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python3...
[?2004h(gdb) python3 ./.v[K[Kbin[K[K[Kv[K.ven[K[K[K[K[Kvenb[K[K[K[K[K./.ve[K[K[K[K/.venv/bin/invokeai
[?2004l
Undefined command: "python3".  Try "help".
[?2004h(gdb) python3 ./.venv/bin/invokeai
[C[C[C[C[C[C[1@.[1@/[1P[1P[1@.[1@/[1@.[1P[1P[1P[1P[1P[1P[1P[1P[1P[1P ./.venv/bin/invokeai
[C[C[C[C[C[Cr ./.venv/bin/invokeai
[C[C[C[C[C[C[C
[?2004l
Starting program: /home/user/invokeai/.venv/bin/python3 ./.venv/bin/invokeai
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffe44ba8640 (LWP 859178)]
[Detaching after vfork from child process 859184]
[Detaching after vfork from child process 859185]
[Detaching after vfork from child process 859186]
>> Initialization file /home/user/invokeai/invokeai.init found. Loading...
/home/user/invokeai/.venv/lib/python3.10/site-packages/torch/cuda/__init__.py:497: UserWarning: Can't initialize NVML
  warnings.warn("Can't initialize NVML")
[New Thread 0x7ffe248be640 (LWP 859490)]
[New Thread 0x7ffe240bd640 (LWP 859491)]
[New Thread 0x7ffe238bc640 (LWP 859492)]

Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00007fff546ce116 in hsaKmtDestroyQueue () from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libhsa-runtime64.so
[?2004h[?2004l
[?2004h(gdb) where
[?2004l
#0  0x00007fff546ce116 in hsaKmtDestroyQueue ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libhsa-runtime64.so
#1  0x00007fff546ce5f0 in hsaKmtCreateQueue ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libhsa-runtime64.so
#2  0x00007fff54632c3c in rocr::AMD::AqlQueue::AqlQueue(rocr::AMD::GpuAgent*, unsigned long, unsigned int, rocr::AMD::ScratchCache::ScratchInfo&, void (*)(hsa_status_t, hsa_queue_s*, void*), void*, bool)
    () from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libhsa-runtime64.so
#3  0x00007fff54626946 in rocr::AMD::GpuAgent::QueueCreate(unsigned long, unsigned int, void (*)(hsa_status_t, hsa_queue_s*, void*), void*, unsigned int, unsigned int, rocr::core::Queue**) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libhsa-runtime64.so
#4  0x00007fff54627a6f in rocr::AMD::GpuAgent::CreateInterceptibleQueue(void (*)(hsa_status_t, hsa_queue_s*, void*), void*) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libhsa-runtime64.so
#5  0x00007fff54627ab7 in std::_Function_handler<rocr::core::Queue* (), rocr::AMD::GpuAgent::InitDma()::{lambda()#1}>::_M_invoke(std::_Any_data const&) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libhsa-runtime64.so
#6  0x00007fff54626b2e in rocr::AMD::GpuAgent::QueueCreate(unsigned long, unsigned int, void (*)(hsa_status_t, hsa_queue_s*, void*), void*, unsigned int, unsigned int, rocr::core::Queue**) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libhsa-runtime64.so
#7  0x00007fff5463deae in rocr::HSA::hsa_queue_create(hsa_agent_s, unsigned int, unsigned int, void (*)(hsa_status_t, hsa_queue_s*, void*), void*, unsigned int, unsigned int, hsa_queue_s**) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libhsa-runtime64.so
#8  0x00007fffa606d937 in roc::Device::acquireQueue(unsigned int, bool, std::vector<unsigned int, std::allocator<unsigned int> > const&, amd::CommandQueue::Priority) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libamdhip64.so
#9  0x00007fffa6083659 in roc::VirtualGPU::create() ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libamdhip64.so
#10 0x00007fffa6069673 in roc::Device::createVirtualDevice(amd::CommandQueue*) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libamdhip64.so
#11 0x00007fffa6058c40 in amd::HostQueue::HostQueue(amd::Context&, amd::Device&, unsigned long, unsigned int, amd::CommandQueue::Priority, std::vector<unsigned int, std::allocator<unsigned int> > const&)
    () from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libamdhip64.so
#12 0x00007fffa5fc4731 in hip::Stream::Create() ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libamdhip64.so
#13 0x00007fffa5fc4a40 in hip::Stream::asHostQueue(bool) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libamdhip64.so
#14 0x00007fffa5e953ce in hip::Device::NullStream(bool) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libamdhip64.so
#15 0x00007fffa5f566b1 in hipMemcpyWithStream ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libamdhip64.so
#16 0x00007fffa7b60d9a in c10::hip::memcpy_and_sync(void*, void*, long, hipMemcpyKind, ihipStream_t*)
    () from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_hip.so
#17 0x00007fffa7b4d17a in at::native::copy_kernel_cuda(at::TensorIterator&, bool) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_hip.so
#18 0x00007fffce7be67a in at::native::copy_impl(at::Tensor&, at::Tensor const&, bool) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#19 0x00007fffce7bf9f1 in at::native::copy_(at::Tensor&, at::Tensor const&, bool) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#20 0x00007fffcf2e8826 in at::_ops::copy_::call(at::Tensor&, at::Tensor const&, bool) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#21 0x00007fffceaa68a9 in at::native::_to_copy(at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) () from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#22 0x00007fffcf5f590a in c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>), &at::(anonymous namespace)::(anonymous namespace)::wrapper___to_copy>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> > >, at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#23 0x00007fffceef66bd in at::_ops::_to_copy::redispatch(c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#24 0x00007fffcf45cd05 in c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>), &at::(anonymous namespace)::_to_copy>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> > >, at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#25 0x00007fffceef66bd in at::_ops::_to_copy::redispatch(c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#26 0x00007fffd06477e2 in torch::autograd::VariableType::(anonymous namespace)::_to_copy(c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#27 0x00007fffd0647d3d in c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>), &torch::autograd::VariableType::(anonymous namespace)::_to_copy>, at::Tensor, c10::guts::typelist::typelist<c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> > >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#28 0x00007fffcef49c32 in at::_ops::_to_copy::call(at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#29 0x00007fffcea9fd62 in at::native::to(at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, bool, c10::optional<c10::MemoryFormat>) () from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#30 0x00007fffcf7876b0 in c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, bool, c10::optional<c10::MemoryFormat>), &at::(anonymous namespace)::(anonymous namespace)::wrapper_dtype_layout_to>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, bool, c10::optional<c10::MemoryFormat> > >, at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, bool, c10::optional<c10::MemoryFormat>)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, bool, c10::optional<c10::MemoryFormat>) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#31 0x00007fffcf094086 in at::_ops::to_dtype_layout::call(at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, bool, c10::optional<c10::MemoryFormat>) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so
#32 0x00007fffe8121bbd in torch::autograd::dispatch_to(at::Tensor const&, c10::Device, bool, bool, c10::optional<c10::MemoryFormat>) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_python.so
#33 0x00007fffe81220b3 in torch::autograd::THPVariable_to(_object*, _object*, _object*) ()
   from /home/user/invokeai/.venv/lib/python3.10/site-packages/torch/lib/libtorch_python.so
#34 0x0000555555732da0 in method_vectorcall_VARARGS_KEYWORDS (func=<optimized out>, 
    args=0x7ffe25ffa398, nargsf=<optimized out>, kwnames=<optimized out>)
    at Objects/descrobject.c:346
#35 0x00005555556e4930 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x7ffe25ffa398, callable=0x7fffe9d3bba0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#36 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7ffe25ffa398, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#37 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffa660, tstate=<optimized out>) at Python/ceval.c:5888
#38 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4206
#39 0x00005555556e32ac in _PyEval_EvalFrame (throwflag=0, f=0x7ffe25ffa200, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#40 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=1, kwnames=<optimized out>) at Python/ceval.c:5073
#41 0x00005555556e45ed in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x555573d36830, callable=0x7ffe25fd23b0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#42 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x555573d36830, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#43 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffa8a0, tstate=<optimized out>) at Python/ceval.c:5888
#44 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4221
#45 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x555573d36660, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#46 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=2, kwnames=0x0) at Python/ceval.c:5073
#47 0x000055555572fe9f in _PyObject_VectorcallTstate (kwnames=<optimized out>, 
    nargsf=<optimized out>, args=0x55555c2e7308, callable=0x7fffe7de80d0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#48 method_vectorcall (method=<optimized out>, args=0x55555c2e7310, nargsf=<optimized out>, 
    kwnames=<optimized out>) at Objects/classobject.c:53
#49 0x00005555556e92c0 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x55555c2e7310, callable=0x7ffe25fe2f80, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#50 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x55555c2e7310, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#51 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffab70, tstate=<optimized out>) at Python/ceval.c:5888
#52 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4189
#53 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x55555c2e7140, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#54 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=2, kwnames=0x0) at Python/ceval.c:5073
#55 0x000055555572fe9f in _PyObject_VectorcallTstate (kwnames=<optimized out>, 
    nargsf=<optimized out>, args=0x55557430e7b8, callable=0x7fffe7de80d0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#56 method_vectorcall (method=<optimized out>, args=0x55557430e7c0, nargsf=<optimized out>, 
    kwnames=<optimized out>) at Objects/classobject.c:53
#57 0x00005555556e92c0 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x55557430e7c0, callable=0x7ffe25fe1740, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#58 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x55557430e7c0, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#59 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffae40, tstate=<optimized out>) at Python/ceval.c:5888
#60 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4189
#61 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x55557430e5f0, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#62 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=2, kwnames=0x0) at Python/ceval.c:5073
#63 0x000055555572fe9f in _PyObject_VectorcallTstate (kwnames=<optimized out>, 
    nargsf=<optimized out>, args=0x555571c59c28, callable=0x7fffe7de80d0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#64 method_vectorcall (method=<optimized out>, args=0x555571c59c30, nargsf=<optimized out>, 
    kwnames=<optimized out>) at Objects/classobject.c:53
#65 0x00005555556e92c0 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x555571c59c30, callable=0x7ffe25fe2d00, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#66 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x555571c59c30, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#67 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffb110, tstate=<optimized out>) at Python/ceval.c:5888
#68 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4189
#69 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x555571c59a60, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#70 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=2, kwnames=0x0) at Python/ceval.c:5073
#71 0x000055555572fe9f in _PyObject_VectorcallTstate (kwnames=<optimized out>, 
    nargsf=<optimized out>, args=0x555573d68f28, callable=0x7fffe7de80d0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#72 method_vectorcall (method=<optimized out>, args=0x555573d68f30, nargsf=<optimized out>, 
    kwnames=<optimized out>) at Objects/classobject.c:53
#73 0x00005555556e92c0 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x555573d68f30, callable=0x7ffe26863340, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#74 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x555573d68f30, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#75 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffb3e0, tstate=<optimized out>) at Python/ceval.c:5888
#76 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4189
#77 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x555573d68d60, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#78 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=2, kwnames=0x0) at Python/ceval.c:5073
#79 0x000055555572fe9f in _PyObject_VectorcallTstate (kwnames=<optimized out>, 
    nargsf=<optimized out>, args=0x55557430e9e8, callable=0x7fffe7de80d0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#80 method_vectorcall (method=<optimized out>, args=0x55557430e9f0, nargsf=<optimized out>, 
    kwnames=<optimized out>) at Objects/classobject.c:53
#81 0x00005555556e92c0 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x55557430e9f0, callable=0x7ffe265ff380, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#82 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x55557430e9f0, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#83 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffb6b0, tstate=<optimized out>) at Python/ceval.c:5888
#84 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4189
#85 0x00005555556e32ac in _PyEval_EvalFrame (throwflag=0, f=0x55557430e840, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#86 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=2, kwnames=<optimized out>) at Python/ceval.c:5073
#87 0x0000555555730093 in _PyObject_VectorcallTstate (kwnames=<optimized out>, 
    nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>, tstate=<optimized out>)
    at ./Include/cpython/abstract.h:114
#88 method_vectorcall (method=<optimized out>, args=<optimized out>, nargsf=<optimized out>, 
    kwnames=<optimized out>) at Objects/classobject.c:61
#89 0x000055555567e947 in PyObject_Call ()
#90 0x00005555556e6ea1 in do_call_core (kwdict=0x7ffe2673c440, callargs=0x7ffe20228430, 
    func=0x7ffe265e4840, trace_info=0x7fffffffb9e0, tstate=<optimized out>) at Python/ceval.c:5940
#91 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4285
#92 0x00005555556e3113 in _PyEval_EvalFrame (throwflag=0, f=0x55555e223fa0, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#93 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=2, kwnames=<optimized out>) at Python/ceval.c:5073
#94 0x000055555572fe9f in _PyObject_VectorcallTstate (kwnames=<optimized out>, 
    nargsf=<optimized out>, args=0x55555de1da98, callable=0x7ffe26eb2b00, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#95 method_vectorcall (method=<optimized out>, args=0x55555de1daa0, nargsf=<optimized out>, 
    kwnames=<optimized out>) at Objects/classobject.c:53
#96 0x00005555556e92c0 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x55555de1daa0, callable=0x7ffe263a04c0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#97 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x55555de1daa0, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#98 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffbcb0, tstate=<optimized out>) at Python/ceval.c:5888
#99 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4189
#100 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x55555de1d880, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#101 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=3, kwnames=0x0) at Python/ceval.c:5073
#102 0x00005555556e4930 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x55555de0d848, callable=0x7ffe26164550, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#103 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x55555de0d848, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#104 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffbef0, tstate=<optimized out>) at Python/ceval.c:5888
#105 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4206
#106 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x55555de0d680, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#107 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=2, kwnames=0x0) at Python/ceval.c:5073
#108 0x00005555556e4930 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x7ffe261bbda8, callable=0x7ffe261644c0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#109 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7ffe261bbda8, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#110 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffc130, tstate=<optimized out>) at Python/ceval.c:5888
#111 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4206
#112 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x7ffe261bbc10, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#113 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=2, kwnames=0x0) at Python/ceval.c:5073
#114 0x00005555556e4930 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x55555de0d610, callable=0x7ffe2614feb0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#115 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x55555de0d610, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#116 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffc370, tstate=<optimized out>) at Python/ceval.c:5888
#117 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4206
#118 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x55555de0d450, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#119 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=2, kwnames=0x0) at Python/ceval.c:5073
#120 0x00005555556e4930 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x7ffe261bb810, callable=0x7ffe26187ac0, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#121 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7ffe261bb810, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#122 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffc5b0, tstate=<optimized out>) at Python/ceval.c:5888
#123 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4206
#124 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x7ffe261bb6a0, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#125 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=1, kwnames=0x0) at Python/ceval.c:5073
#126 0x00005555556e4930 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x55555e1895e8, callable=0x7ffe26187a30, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#127 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x55555e1895e8, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#128 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffc7f0, tstate=<optimized out>) at Python/ceval.c:5888
#129 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4206
#130 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x55555e189420, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#131 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=0, kwnames=0x0) at Python/ceval.c:5073
#132 0x00005555556e45ed in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, 
    args=0x7ffff7621bb8, callable=0x7ffff7431d80, tstate=0x555555934c20)
    at ./Include/cpython/abstract.h:114
#133 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7ffff7621bb8, 
    callable=<optimized out>) at ./Include/cpython/abstract.h:123
#134 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, 
    trace_info=0x7fffffffca30, tstate=<optimized out>) at Python/ceval.c:5888
#135 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>)
    at Python/ceval.c:4221
#136 0x00005555556e2bf4 in _PyEval_EvalFrame (throwflag=0, f=0x7ffff7621a40, tstate=0x555555934c20)
    at ./Include/internal/pycore_ceval.h:46
#137 _PyEval_Vector (tstate=<optimized out>, con=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=0, kwnames=0x0) at Python/ceval.c:5073
#138 0x000055555577a01f in PyEval_EvalCode (co=co@entry=0x7ffff7537470, 
    globals=globals@entry=0x7ffff753a640, locals=locals@entry=0x7ffff753a640) at Python/ceval.c:1134
#139 0x000055555578ebb9 in run_eval_code_obj (tstate=0x555555934c20, co=0x7ffff7537470, 
    globals=0x7ffff753a640, locals=0x7ffff753a640) at Python/pythonrun.c:1289
#140 0x000055555578eb44 in run_mod (mod=<optimized out>, filename=<optimized out>, 
    globals=0x7ffff753a640, locals=0x7ffff753a640, flags=<optimized out>, arena=<optimized out>)
    at Python/pythonrun.c:1310
#141 0x0000555555609cc3 in pyrun_file (fp=fp@entry=0x555555939810, 
    filename=filename@entry=0x7ffff7405230, start=start@entry=257, 
    globals=globals@entry=0x7ffff753a640, locals=locals@entry=0x7ffff753a640, 
    closeit=closeit@entry=1, flags=0x7fffffffcd98) at Python/pythonrun.c:1206
#142 0x0000555555609aa5 in _PyRun_SimpleFileObject (fp=fp@entry=0x555555939810, 
    filename=filename@entry=0x7ffff7405230, closeit=closeit@entry=1, 
    flags=flags@entry=0x7fffffffcd98) at Python/pythonrun.c:455
#143 0x0000555555609d6b in _PyRun_AnyFileObject (fp=fp@entry=0x555555939810, 
    filename=filename@entry=0x7ffff7405230, closeit=closeit@entry=1, 
    flags=flags@entry=0x7fffffffcd98) at Python/pythonrun.c:89
#144 0x0000555555752e68 in pymain_run_file_obj (skip_source_first_line=<optimized out>, 
    filename=0x7ffff7405230, program_name=0x7ffff7405290) at Modules/main.c:353
#145 pymain_run_file (config=0x555555919180) at Modules/main.c:372
#146 pymain_run_python (exitcode=0x7fffffffcd90) at Modules/main.c:587
#147 Py_RunMain () at Modules/main.c:666
#148 0x000055555575291d in Py_BytesMain (argc=<optimized out>, argv=<optimized out>)
    at Modules/main.c:720
#149 0x00007ffff7c96d90 in __libc_start_call_main (main=main@entry=0x555555679150 <main>, 
    argc=argc@entry=2, argv=argv@entry=0x7fffffffcfc8) at ../sysdeps/nptl/libc_start_call_main.h:58
#150 0x00007ffff7c96e40 in __libc_start_main_impl (main=0x555555679150 <main>, argc=2, 
    argv=0x7fffffffcfc8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fffffffcfb8) at ../csu/libc-start.c:392
#151 0x000055555575281e in _start ()
[?2004h(gdb) q
[?2004l
[?2004hA debugging session is active.

	Inferior 1 [process 859066] will be killed.

Quit anyway? (y or n) y
[?2004l

I'll see if installing the nightly pytorch will fix the issue.

Feb 08 '23 19:02 src-r-r

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.

Feb 28 '23 06:02 github-actions[bot]

Yes, I'm still experiencing it on v2.3.1-post2 release

Mar 03 '23 13:03 src-r-r

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.

Mar 18 '23 06:03 github-actions[bot]

Yeah, gave up on AMD. Too much effort.

Mar 21 '23 15:03 src-r-r