Yu-Zhou
Yu-Zhou
I met the excatly same problem, did you fix it? @yuyq96
> Not yet, this also happened when I tried to use GPT-J. A temporary solution is to set fp16=False to use int8+fp32 training. @zhouyu5 Thanks, but after setting fp16=False, the...
> > > Not yet, this also happened when I tried to use GPT-J. A temporary solution is to set fp16=False to use int8+fp32 training. @zhouyu5 > > > >...
> We have also been considering supporting the use of pip to install tllib. Recently, we will check the code and support it after we obtain a more stable version....
If you are busy, I can offer this help. Please tell me if you want me to upload it to pypi. Thanks~
Any further updates? It seems #723 still not solve this bug.
Thanks, sounds great.
@rusty1s @DamianSzwichtenberg Could you please share some of your thoughts on this proposal?
> @zhouyu5 I think you may first start with new example at [examples/multi_gpu](https://github.com/pyg-team/pytorch_geometric/tree/master/examples/multi_gpu), similar to `distributed_sampling_multinode.py` - you have a bit more freedom there. @DamianSzwichtenberg Sounds good. I could provide...
@DamianSzwichtenberg Thanks for your input, it make sense. Will give a example later.