Brady Bonnette comments

Results 24 comments of


                                            Brady Bonnette

Error while fine-tuning model using example from "examples/finetune.ipynb": `RuntimeError: shape '[8, 129, 12, 512]' is invalid for input of size 8306688`

@Ingvarstep I did not create nor set up a new configuration file nor set a configuration path using `--config` when I attempted it via `train.py`, so it defaulted to `configs/config.yaml`...

Error while fine-tuning model using example from "examples/finetune.ipynb": `RuntimeError: shape '[8, 129, 12, 512]' is invalid for input of size 8306688`

@joywang233 good to know, thanks :) That will come in handy if I need to start training on my own data. It's just confusing to me because I'm using the...

Error while fine-tuning model using example from "examples/finetune.ipynb": `RuntimeError: shape '[8, 129, 12, 512]' is invalid for input of size 8306688`

@xingchaozh I have not. I will try again with the latest on Monday. Thanks for the heads up.

Error while fine-tuning model using example from "examples/finetune.ipynb": `RuntimeError: shape '[8, 129, 12, 512]' is invalid for input of size 8306688`

@xingchaozh So I just tried it using the updated codebase, and I am still having the same issue. Can I ask what dataset you are using, and if it differs...

Error while fine-tuning model using example from "examples/finetune.ipynb": `RuntimeError: shape '[8, 129, 12, 512]' is invalid for input of size 8306688`

Well now I am even more confused 🙃 What hardware are you running on?

Error while fine-tuning model using example from "examples/finetune.ipynb": `RuntimeError: shape '[8, 129, 12, 512]' is invalid for input of size 8306688`

@xingchaozh ah geez, sorry, you mentioned that. My fault! Did you run `train.py` in the same manner I did? I saw that you didnt change any of the codebase, but...

Error while fine-tuning model using example from "examples/finetune.ipynb": `RuntimeError: shape '[8, 129, 12, 512]' is invalid for input of size 8306688`

@Ask-sola which dataset are you using?

Error while fine-tuning model using example from "examples/finetune.ipynb": `RuntimeError: shape '[8, 129, 12, 512]' is invalid for input of size 8306688`

@Ask-sola > yield different results when executed on different machines What are the hardware differences between the two machines? Are they also using the same exact versions of torch/transformers/etc?

Error while fine-tuning model using example from "examples/finetune.ipynb": `RuntimeError: shape '[8, 129, 12, 512]' is invalid for input of size 8306688`

@Ask-sola I think you might be onto something. The machine I was trying to run this on is dual GPU (I have two NVIDIA RTX A6000 gpus), and never thought...

Error while fine-tuning model using example from "examples/finetune.ipynb": `RuntimeError: shape '[8, 129, 12, 512]' is invalid for input of size 8306688`

Wow I forgot about this. @uni-dip-gupta From what I could tell and what I could remember, it was an issue when training on a system with multiple GPUs. When I...