david

accelerate autocast on mps device

9

### System Info ```Shell accelerate==0.18.0 system==M2 macos 13.3.1 ``` ### Information - [ ] The official example scripts - [X] My own modified scripts ### Tasks - [ ] One...

loss increase when finetuning on shakespeare datasets with gpt-xl and mps device

2

Hello, I've been attempting to fine-tune a GPT-XL model on a Shakespeare dataset using an MPS device. However, I've noticed that both the training loss and evaluation loss are significantly...

### 是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this? - [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions ### 该问题是否在FAQ中有解答？ | Is there an...