Weiguang Zhao comments

Results 7 comments of


                                            Weiguang Zhao

Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1809861 milliseconds before timing out

Same with you. Have you solved it?

Nan

> when I use I meet > ![image](https://user-images.githubusercontent.com/73769530/113287935-d19e9680-9320-11eb-91af-140ecfcd5d52.png) @cymdhx @songwaimai @whf9527 我解决了我遇到的nan问题，附上我的解决方法，不知道是否适用于你们的：问题描述： Unet + resnet 改为 unet + rednet50时出现 nan，inf 解决方案：把程序中的以下代码去掉，不要人为初始化 weight and bias ``` def set_bn_init(m):...

RuntimeError: invalid argument at /tmp/pip-req-build-24zubhoq/src/spmm.cu:590

Try GCC 9.3.0

RuntimeError: expected scalar type Half but found Float

add this one `with torch.autocast("cuda"):`

release code

Hi， any plan to release the code of ceco？

Request for Custom Dataset Training Guidelines

Thanks for your attention. We will provide custom data training guidance as soon as possible. In general, we may need approximately **one month** for debugging.

Request for Custom Dataset Training Guidelines

There are three points you may need to pay attention to: 1.Convert all data to the npy format, including XYZ [N, 3]; RGB [N, 3]; Semantic Label [N]; Instance Label...