Sherman Siu

Adding `padding=True` when tokenizing both the input and targets does not fix the issue.

Also, when running the script using the CPU only, I get this error: ``` root@pc:~ # CUDA_VISIBLE_DEVICES="" python script_blenderbot_length.py 100%|██████████████████████████| 1/1 [00:00

I've found out why the error seems to appear. I modified `transformers/src/transformers/models/blenderbot/modeling_blenderbot.py:BlenderbotLearnedPositionalEmbedding:forward` (approximately near line 125). ```diff positions = torch.arange( past_key_values_length, past_key_values_length + seq_len, dtype=torch.long, device=self.weight.device ) + print(positions) +...

The length of the positions seems to be equal to `2*CRITICAL_NUMBER + 1`.

And... it goes to a maximum of the tokenizer's max_length-1, which is expected, I guess.

Ah. So the issue is that in the `BlenderbotConfig`, `max_position_embeddings` is set to 128. The publicly available weights only have position embeddings with those dimensions, so either I'd have to...

But seriously, this exception should be caught and re-raised with a more human-readable expression.

(I can contribute a fix after my internship ends, not before)

Catching and re-raising the exception during GPU training doesn't result in a more human-readable expression (It's still `RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasCreate(handle)`, but at least the flood of...

I'm not the author, but it seems like in #1, the author mentions that they could not get permission to release the code because it was deployed into production at...