transformers
transformers copied to clipboard
huggingface
Identifying backend compatibility versions
We are currently working on identifying the backend versions with which we are compatible and with which we want to be compatible. These backends are PyTorch and TensorFlow. We will be considering Flax at a later point in time.
The first step was to identify the number of failures in each PyTorch/TensorFlow version and was done in https://github.com/huggingface/transformers/issues/18181.
Total number of tests: 38,991.
| Framework | No. Failures | Release date | Older than 2 years |
|---|---|---|---|
| PyTorch 1.10 | 50 | Mar 10 2021 | No |
| PyTorch 1.9 | 710 | Jun 15 2021 | No |
| PyTorch 1.8 | 1301 | Mar 4 2021 | No |
| PyTorch 1.7 | 1567 | Oct 27 2020 | No |
| PyTorch 1.6 | 2342 | Jul 28 2020 | Yes |
| PyTorch 1.5 | 3315 | Apr 21 2020 | Yes |
| PyTorch 1.4 | 3949 | Jan 16 2020 | Yes |
| TensorFlow 2.8 | 118 | Feb 2 2022 | No |
| TensorFlow 2.7 | 122 | Nov 4 2021 | No |
| TensorFlow 2.6 | 122 | Aug 11 2021 | No |
| TensorFlow 2.5 | 128 | May 13 2021 | No |
| TensorFlow 2.4 | 167 | Dec 14 2020 | No |
We're proposing to drop versions older than 2 years old and to work towards providing support (support = 0 tests failing) for versions we aim to support. We will drop support for older versions once we reach their two-year-old date.
Here is the proposed plan moving forward:
- [ ] Have a detailed breakdown of failures for the following versions:
- [ ] Torch 1.7
- [ ] Torch 1.8
- [ ] Torch 1.9
- [ ] Torch 1.10
- [ ] Torch 1.11
- [ ] Torch 1.12
- [ ] TensorFlow 2.4
- [ ] TensorFlow 2.5
- [ ] TensorFlow 2.6
- [ ] TensorFlow 2.7
- [ ] TensorFlow 2.8
- [ ] TensorFlow 2.9
- [ ] Start with an initial compatibility document to mention which models are supported in which versions
- [ ] Open good first issues to improve compatibility for models not compatible with all versions, starting from the latest one and moving back in time.
- [ ] As versions become supported, run tests on older versions to ensure no regression.
Work by @ydshieh and @LysandreJik
Past CI - PyTorch 1.11 (Patch release: v4.21.2 | b487096b0)
General
| no. | error | status |
|---|---|---|
| 32 | NameError: name 'kenlm' is not defined | not installed |
| 12 | RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 | fixed in #18303 |
| 6 | OSError: gs555750 is not a valid git identifier (branch name, tag name or commit id) that exists for | fixed in #18531 |
| 6 | NameError: name 'GPT2Tokenizer' is not defined | fixed in #19010 |
| 6 | TypeError: forward() missing 1 required positional argument: 'attention_mask' | fixed in #18303 |
| 3 | ImportError: | detectron2 and accelerate not installed |
| 2 | AssertionError: torch.Size([1, 2]) != torch.Size([1, 32]) | fixed in #18303 |
| 1 | RuntimeError: Caught RuntimeError in replica 0 on device 0. | fixed in #18303 |
Per model
| model | no. of errors | major error | count |
|---|---|---|---|
| wav2vec2_with_lm | 30 | NameError: name 'kenlm' is not defined | 30 |
| owlvit | 21 | RuntimeError: Expected all tensors to be on the same device, | 12 |
| opt | 6 | NameError: name 'GPT2Tokenizer' is not defined | 6 |
| bloom | 6 | OSError: gs555750 is not a valid git identifier (branch name | 6 |
| wav2vec2 | 2 | NameError: name 'kenlm' is not defined | 2 |
| layoutlmv2 | 2 | ImportError: | 2 |
Past CI - PyTorch 1.10 (Patch release: v4.21.2 | b487096b0)
General
| no. | error | status |
|---|---|---|
| 32 | NameError: name 'kenlm' is not defined | see PT 11 |
| 12 | RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 | see PT 11 |
| 6 | OSError: gs555750 is not a valid git identifier (branch name, tag name or commit id) that exists for | see PT 11 |
| 6 | NameError: name 'GPT2Tokenizer' is not defined | see PT 11 |
| 6 | TypeError: forward() missing 1 required positional argument: 'attention_mask' | see PT 11 |
| 4 | RuntimeError: Index is supposed to be an empty tensor or a vector | torch._C issue -> works wth PT >= 11 Fixed in #19122 |
| 3 | ImportError: | see PT 11 |
| 2 | AssertionError: 1.9311904907226562e-05 != 1.9431114196777344e-05 | self.assertEqual is too strict. Fixed in #19200 |
| 2 | AssertionError: torch.Size([1, 2]) != torch.Size([1, 32]) | see PT 11 |
| 1 | RuntimeError: Caught RuntimeError in replica 0 on device 0. | see PT 11 |
Per model
| model | no. of errors | major error | count |
|---|---|---|---|
| wav2vec2_with_lm | 30 | NameError: name 'kenlm' is not defined | 30 |
| owlvit | 21 | RuntimeError: Expected all tensors to be on the same device, | 12 |
| bloom | 8 | OSError: gs555750 is not a valid git identifier (branch name | 6 |
| opt | 6 | NameError: name 'GPT2Tokenizer' is not defined | 6 |
| longt5 | 4 | RuntimeError: Index is supposed to be an empty tensor or a v | 4 |
| wav2vec2 | 2 | NameError: name 'kenlm' is not defined | 2 |
| layoutlmv2 | 2 | ImportError: | 2 |
Past CI - PyTorch 1.9 (Patch release: v4.21.2 | b487096b0)
General
| no. | error | status |
|---|---|---|
| 50 | AttributeError: module 'torch' has no attribute 'pi' | Need PT >= 1.10. But we can use np.pi. See #19201 |
| 44 | TypeError: meshgrid() got an unexpected keyword argument 'indexing' | Vilt needs PT >= 1.10 |
| 32 | NameError: name 'kenlm' is not defined | see PT 11 |
| 18 | AttributeError: module 'torchaudio.functional' has no attribute 'melscale_fbanks' | Need torchaudio >= 0.10. See #19203 |
| 15 | RuntimeError: CUDA error: an illegal memory access was encountered | LeViT re-run OK |
| 12 | RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 | see PT 11 |
| 6 | OSError: gs555750 is not a valid git identifier (branch name, tag name or commit id) that exists for | see PT 11 |
| 6 | NameError: name 'GPT2Tokenizer' is not defined | see PT 11 |
| 6 | TypeError: forward() missing 1 required positional argument: 'attention_mask' | see PT 11 |
| 3 | ImportError: | see PT 11 |
| 2 | RuntimeError: "LayerNormKernelImpl" not implemented for 'BFloat16' | fixed in #19261 |
| 2 | AssertionError: 1.9311904907226562e-05 != 1.9431114196777344e-05 | See PT 10 |
| 2 | AssertionError: -198.98219299316406 != -198.98225 within 4 places (5.7006835930906163e-05 difference | diff acceptable |
| 2 | RuntimeError: Index is supposed to be an empty tensor or a vector | torch._C issue -> works wth PT >= 11 Fixed in https://github.com/huggingface/transformers/pull/19122 |
| 2 | RuntimeError: Expected node type 'onnx::Constant' for argument 'num_classes' of node 'one_hot', got | test already skipped in #19122 (due to another error) |
| 2 | AssertionError: torch.Size([1, 2]) != torch.Size([1, 32]) | see PT 11 |
| 2 | TypeError: Caught TypeError in replica 0 on device 0. | Vilt needs PT >= 1.10 (meshgrid error) |
| 1 | RuntimeError: transform: failed to synchronize: cudaErrorIllegalAddress: an illegal memory access wa | See #20859 (opened) |
| 1 | RuntimeError: Caught RuntimeError in replica 0 on device 0. | see PT 11 |
Per model
| model | no. of errors | major error | count |
|---|---|---|---|
| maskformer | 50 | AttributeError: module 'torch' has no attribute 'pi' | 50 |
| vilt | 46 | TypeError: meshgrid() got an unexpected keyword argument 'in | 44 |
| wav2vec2_with_lm | 30 | NameError: name 'kenlm' is not defined | 30 |
| owlvit | 21 | RuntimeError: Expected all tensors to be on the same device, | 12 |
| mctct | 18 | AttributeError: module 'torchaudio.functional' has no attrib | 18 |
| levit | 16 | RuntimeError: CUDA error: an illegal memory access was encou | 15 |
| bloom | 10 | OSError: gs555750 is not a valid git identifier (branch name | 6 |
| opt | 6 | NameError: name 'GPT2Tokenizer' is not defined | 6 |
| longt5 | 4 | RuntimeError: Index is supposed to be an empty tensor or a v | 2 |
| flava | 2 | AssertionError: -198.98219299316406 != -198.98225 within 4 p | 2 |
| wav2vec2 | 2 | NameError: name 'kenlm' is not defined | 2 |
| layoutlmv2 | 2 | ImportError: | 2 |
Past CI - PyTorch 1.8 (Patch release: v4.21.2 | b487096b0)
General
| no. | error | status |
|---|---|---|
| 570 | AttributeError: module 'torch.jit._state' has no attribute '_clear_class_state' | WIP |
| 50 | AttributeError: module 'torch' has no attribute 'pi' | See PT 1.9 |
| 44 | TypeError: conv1d(): argument 'padding' (position 5) must be tuple of ints, not str | WIP |
| 44 | TypeError: meshgrid() got an unexpected keyword argument 'indexing' | See PT 1.9 |
| 30 | NameError: name 'kenlm' is not defined | see PT 11 |
| 26 | AttributeError: module 'torch' has no attribute 'permute' | WIP |
| 18 | AttributeError: module 'torchaudio.functional' has no attribute 'melscale_fbanks' | See PT 1.9 |
| 12 | RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 | see PT 1.11 |
| 8 | RuntimeError: einsum() operand subscript must be in range [a, z] but found B for operand 0 | WIP |
| 6 | OSError: gs555750 is not a valid git identifier (branch name, tag name or commit id) that exists for | see PT 1.11 |
| 6 | NameError: name 'GPT2Tokenizer' is not defined | see PT 1.11 |
| 6 | TypeError: forward() missing 1 required positional argument: 'attention_mask' | see PT 1.11 |
| 4 | TypeError: Caught TypeError in replica 0 on device 0. | see PT 1.10 |
| 3 | ImportError: | See PT 1.11 |
| 2 | RuntimeError: "LayerNormKernelImpl" not implemented for 'BFloat16' | See PT 1.9 |
| 2 | RuntimeError: "min_cuda" not implemented for 'BFloat16' | WIP |
| 2 | AssertionError: 1.9311904907226562e-05 != 1.9431114196777344e-05 | See PT 10 |
| 2 | AssertionError: -198.98219299316406 != -198.98225 within 4 places (5.7006835930906163e-05 difference | See PT 9 |
| 2 | AssertionError: False is not true | |
| 2 | RuntimeError: Expected node type 'onnx::Constant' for argument 'num_classes' of node 'one_hot', got | See PT 1.9 |
| 2 | AssertionError: torch.Size([1, 2]) != torch.Size([1, 32]) | see PT 11 |
| 2 | TypeError: CheckpointFunctionBackward.forward: expected Tensor or tuple of Tensor (got tuple) for re | WIP |
| 2 | TypeError: save_for_backward can only save variables, but argument 2 is of type bool | WIP |
| 2 | AttributeError: module 'torchaudio.functional' has no attribute 'resample' | WIP |
| 1 | RuntimeError: Caught RuntimeError in replica 0 on device 0. | see PT 11 |
| 1 | AssertionError: 2.9253265857696533 != 2.925307273864746 within 1e-05 delta (1.9311904907226562e-05 d | diff acceptable |
Per model
| model | no. of errors | major error | count |
|---|---|---|---|
| mctct | 64 | TypeError: conv1d(): argument 'padding' (position 5) must be | 44 |
| maskformer | 50 | AttributeError: module 'torch' has no attribute 'pi' | 50 |
| vilt | 46 | TypeError: meshgrid() got an unexpected keyword argument 'in | 44 |
| owlvit | 33 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| wav2vec2_with_lm | 30 | NameError: name 'kenlm' is not defined | 30 |
| longt5 | 26 | AttributeError: module 'torch.jit._state' has no attribute ' | 24 |
| perceiver | 26 | AttributeError: module 'torch' has no attribute 'permute' | 26 |
| bloom | 18 | OSError: gs555750 is not a valid git identifier (branch name | 6 |
| prophetnet | 18 | AttributeError: module 'torch.jit._state' has no attribute ' | 18 |
| data2vec | 18 | AttributeError: module 'torch.jit._state' has no attribute ' | 18 |
| hubert | 14 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| realm | 14 | RuntimeError: einsum() operand subscript must be in range [a | 8 |
| wav2vec2 | 14 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| clip | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| marian | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| opt | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| blenderbot | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| pegasus | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| t5 | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| funnel | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| mvp | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| mbart | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| plbart | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| blenderbot_small | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| bart | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| swin | 8 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| sew | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| resnet | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| xlm_roberta_xl | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| splinter | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| dpt | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| xlm | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| speech_to_text_2 | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| dpr | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| squeezebert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| vit | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| mobilebert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| convnext | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| xlnet | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| glpn | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| segformer | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| cpm | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| nezha | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| bigbird_pegasus | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| megatron_bert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| trocr | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| rembert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| van | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| mobilevit | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| gpt_neox | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| openai | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| albert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| nystromformer | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| distilbert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| gpt2 | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| mpnet | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| roberta | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| deit | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| unispeech | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| flaubert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| codegen | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| wavlm | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| xglm | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| roformer | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| regnet | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| bert_generation | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| convbert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| beit | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| transfo_xl | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| electra | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| ctrl | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| canine | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| groupvit | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| gptj | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| gpt_neo | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| fnet | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| fsmt | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| m2m_100 | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| layoutlm | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| layoutlmv2 | 2 | ImportError: | 2 |
| flava | 2 | AssertionError: -198.98219299316406 != -198.98225 within 4 p | 2 |
| trajectory_transformer | 2 | TypeError: save_for_backward can only save variables, but ar | 2 |
Past CI - PyTorch 1.7 (Patch release: v4.21.2 | b487096b0)
General
| no. | error |
|---|---|
| 550 | AttributeError: module 'torch.jit._state' has no attribute '_clear_class_state' |
| 245 | TypeError: div() got an unexpected keyword argument 'rounding_mode' |
| 150 | RuntimeError: iter.device(arg).is_cuda() INTERNAL ASSERT FAILED at "/pytorch/aten/src/ATen/native/cu |
| 79 | AttributeError: 'builtin_function_or_method' object has no attribute 'fftn' |
| 50 | AttributeError: module 'torch' has no attribute 'pi' |
| 44 | TypeError: meshgrid() got an unexpected keyword argument 'indexing' |
| 28 | NameError: name 'kenlm' is not defined |
| 22 | AttributeError: module 'torch' has no attribute 'permute' |
| 18 | AttributeError: module 'torchaudio.functional' has no attribute 'melscale_fbanks' |
| 12 | AssertionError: Couldn't trace module. |
| 12 | AttributeError: module 'torch' has no attribute 'broadcast_to' |
| 8 | AttributeError: 'Tensor' object has no attribute 'broadcast_to' |
| 6 | OSError: gs555750 is not a valid git identifier (branch name, tag name or commit id) that exists for |
| 6 | RuntimeError: "normal_kernel_cpu" not implemented for 'BFloat16' |
| 6 | NameError: name 'GPT2Tokenizer' is not defined |
| 6 | TypeError: forward() missing 1 required positional argument: 'attention_mask' |
| 6 | AttributeError: 'Tensor' object has no attribute 'tile' |
| 6 | RuntimeError: only lowercase letters a-z allowed as indices |
| 4 | RuntimeError: "log_cuda" not implemented for 'Long' |
| 4 | ValueError: Unsupported ONNX opset version: 13 |
| 4 | TypeError: _conv_forward() takes 3 positional arguments but 4 were given |
| 4 | TypeError: load() got an unexpected keyword argument 'format' |
| 3 | ImportError: |
| 2 | RuntimeError: Caught RuntimeError in replica 0 on device 0. |
| 2 | AssertionError: -198.98219299316406 != -198.98225 within 4 places (5.7006835930906163e-05 difference |
| 2 | AssertionError: False is not true |
| 2 | AssertionError: torch.Size([1, 2]) != torch.Size([1, 32]) |
| 2 | TypeError: CheckpointFunctionBackward.forward: expected Tensor or tuple of Tensor (got tuple) for re |
| 2 | TypeError: save_for_backward can only save variables, but argument 2 is of type bool |
| 2 | TypeError: Caught TypeError in replica 0 on device 0. |
| 2 | AssertionError: 'Toda[15 chars] and we have seen a little of it here on the c[13 chars]ally' != 'Tod |
| 1 | AttributeError: Caught AttributeError in replica 0 on device 0. |
| 1 | AssertionError: 1.2425978183746338 != 1.2425782680511475 within 1e-05 delta (1.9550323486328125e-05 |
Per model
| model | no. of errors | major error | count |
|---|---|---|---|
| mctct | 190 | TypeError: div() got an unexpected keyword argument 'roundin | 172 |
| bloom | 95 | RuntimeError: iter.device(arg).is_cuda() INTERNAL ASSERT FAI | 76 |
| opt | 87 | RuntimeError: iter.device(arg).is_cuda() INTERNAL ASSERT FAI | 74 |
| fnet | 79 | AttributeError: 'builtin_function_or_method' object has no a | 79 |
| layoutlmv3 | 57 | TypeError: div() got an unexpected keyword argument 'roundin | 57 |
| maskformer | 50 | AttributeError: module 'torch' has no attribute 'pi' | 50 |
| vilt | 46 | TypeError: meshgrid() got an unexpected keyword argument 'in | 44 |
| owlvit | 33 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| wav2vec2_with_lm | 30 | NameError: name 'kenlm' is not defined | 28 |
| longt5 | 28 | AttributeError: module 'torch.jit._state' has no attribute ' | 24 |
| perceiver | 26 | AttributeError: module 'torch' has no attribute 'permute' | 22 |
| realm | 18 | AttributeError: 'Tensor' object has no attribute 'tile' | 6 |
| prophetnet | 18 | AttributeError: module 'torch.jit._state' has no attribute ' | 18 |
| data2vec | 18 | AttributeError: module 'torch.jit._state' has no attribute ' | 18 |
| hubert | 14 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| wav2vec2 | 14 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| clip | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| marian | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| blenderbot | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| pegasus | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| t5 | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| funnel | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| mvp | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| mbart | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| wavlm | 12 | AttributeError: 'Tensor' object has no attribute 'broadcast_ | 8 |
| plbart | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| blenderbot_small | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| bart | 12 | AttributeError: module 'torch.jit._state' has no attribute ' | 12 |
| glpn | 10 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| swin | 8 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| xglm | 8 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| sew | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| resnet | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| xlm_roberta_xl | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| splinter | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| dpt | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| xlm | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| speech_to_text_2 | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| dpr | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| squeezebert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| vit | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| mobilebert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| convnext | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| xlnet | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| segformer | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| cpm | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| nezha | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| bigbird_pegasus | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| megatron_bert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| trocr | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| rembert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| van | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| mobilevit | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| gpt_neox | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| openai | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| albert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| nystromformer | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| distilbert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| gpt2 | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| mpnet | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| roberta | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| deit | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| unispeech | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| flaubert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| codegen | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| roformer | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| regnet | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| bert_generation | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| convbert | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| beit | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| transfo_xl | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| electra | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| ctrl | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| canine | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| groupvit | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| gptj | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| gpt_neo | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| fsmt | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| m2m_100 | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| layoutlm | 6 | AttributeError: module 'torch.jit._state' has no attribute ' | 6 |
| layoutlmv2 | 2 | ImportError: | 2 |
| flava | 2 | AssertionError: -198.98219299316406 != -198.98225 within 4 p | 2 |
| trajectory_transformer | 2 | TypeError: save_for_backward can only save variables, but ar | 2 |
Past CI - TensorFlow 2.8 (Patch release: v4.21.2 | b487096b0)
General
| no. | error |
|---|---|
| 66 | RuntimeError: Cannot export model to ONNX using PyTorch because no PyTorch package was found. |
| 30 | NameError: name 'kenlm' is not defined |
| 6 | NameError: name 'GPT2Tokenizer' is not defined |
| 4 | NameError: name 'MaskFormerForInstanceSegmentation' is not defined |
| 4 | ImportError: |
| 2 | NameError: name 'MaskFormerModel' is not defined |
| 2 | tensorflow.python.framework.errors_impl.InvalidArgumentError: required broadcastable shapes [Op:Equa |
| 1 | ValueError: You called set_weights(weights) on layer "tf_segformer_for_image_classification_8" wit |
Per model
| model | no. of errors | major error | count |
|---|---|---|---|
| wav2vec2_with_lm | 28 | NameError: name 'kenlm' is not defined | 28 |
| maskformer | 6 | NameError: name 'MaskFormerForInstanceSegmentation' is not d | 4 |
| opt | 6 | NameError: name 'GPT2Tokenizer' is not defined | 6 |
| speech_to_text | 4 | ImportError: | 4 |
| wav2vec2 | 2 | NameError: name 'kenlm' is not defined | 2 |
| rembert | 2 | tensorflow.python.framework.errors_impl.InvalidArgumentError | 2 |
| segformer | 1 | ValueError: You called set_weights(weights) on layer "tf_s |
1 |
Past CI - TensorFlow 2.7 (Patch release: v4.21.2 | b487096b0)
General
| no. | error |
|---|---|
| 66 | RuntimeError: Cannot export model to ONNX using PyTorch because no PyTorch package was found. |
| 30 | NameError: name 'kenlm' is not defined |
| 6 | TypeError: Invalid keyword argument(s) in compile(): ({'jit_compile'},). Valid keyword arguments i |
| 6 | NameError: name 'GPT2Tokenizer' is not defined |
| 4 | NameError: name 'MaskFormerForInstanceSegmentation' is not defined |
| 4 | ImportError: |
| 2 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 2 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 2 | NameError: name 'MaskFormerModel' is not defined |
| 2 | tensorflow.python.framework.errors_impl.InvalidArgumentError: required broadcastable shapes [Op:Equa |
| 1 | ValueError: You called set_weights(weights) on layer "tf_segformer_for_image_classification_8" wit |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
Per model
| model | no. of errors | major error | count |
|---|---|---|---|
| wav2vec2_with_lm | 28 | NameError: name 'kenlm' is not defined | 28 |
| t5 | 10 | tensorflow.python.framework.errors_impl.InvalidArgumentError | 1 |
| maskformer | 6 | NameError: name 'MaskFormerForInstanceSegmentation' is not d | 4 |
| opt | 6 | NameError: name 'GPT2Tokenizer' is not defined | 6 |
| speech_to_text | 4 | ImportError: | 4 |
| wav2vec2 | 2 | NameError: name 'kenlm' is not defined | 2 |
| gptj | 2 | TypeError: Invalid keyword argument(s) in compile(): ({'ji |
2 |
| bart | 2 | TypeError: Invalid keyword argument(s) in compile(): ({'ji |
2 |
| gpt2 | 2 | TypeError: Invalid keyword argument(s) in compile(): ({'ji |
2 |
| rembert | 2 | tensorflow.python.framework.errors_impl.InvalidArgumentError | 2 |
| segformer | 1 | ValueError: You called set_weights(weights) on layer "tf_s |
1 |
Past CI - TensorFlow 2.6 (Patch release: v4.21.2 | b487096b0)
General
| no. | error |
|---|---|
| 66 | RuntimeError: Cannot export model to ONNX using PyTorch because no PyTorch package was found. |
| 30 | NameError: name 'kenlm' is not defined |
| 10 | ValueError: in user code: |
| 6 | TypeError: Invalid keyword argument(s) in compile: {'jit_compile'} |
| 6 | NameError: name 'GPT2Tokenizer' is not defined |
| 4 | NameError: name 'MaskFormerForInstanceSegmentation' is not defined |
| 4 | ImportError: |
| 2 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 2 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Detected unsupported operations when t |
| 2 | NameError: name 'MaskFormerModel' is not defined |
| 2 | tensorflow.python.framework.errors_impl.InvalidArgumentError: required broadcastable shapes [Op:Equa |
| 1 | ValueError: You called set_weights(weights) on layer "tf_segformer_for_image_classification_8" wit |
| 1 | ValueError: Unable to save function b'__inference_tf_speech2text_model_25_layer_call_and_return_cond |
| 1 | tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected 'tf.Tensor(False, shape=(), d |
| 1 | ValueError: Unable to save function b'__inference_tf_speech2text_model_25_layer_call_and_return_cond |
Per model
| model | no. of errors | major error | count |
|---|---|---|---|
| wav2vec2_with_lm | 28 | NameError: name 'kenlm' is not defined | 28 |
| t5 | 10 | ValueError: in user code: | 10 |
| maskformer | 6 | NameError: name 'MaskFormerForInstanceSegmentation' is not d | 4 |
| opt | 6 | NameError: name 'GPT2Tokenizer' is not defined | 6 |
| speech_to_text | 6 | ImportError: | 4 |
| bart | 3 | TypeError: Invalid keyword argument(s) in compile: {'jit_c |
2 |
| wav2vec2 | 2 | NameError: name 'kenlm' is not defined | 2 |
| gptj | 2 | TypeError: Invalid keyword argument(s) in compile: {'jit_c |
2 |
| gpt2 | 2 | TypeError: Invalid keyword argument(s) in compile: {'jit_c |
2 |
| rembert | 2 | tensorflow.python.framework.errors_impl.InvalidArgumentError | 2 |
| segformer | 1 | ValueError: You called set_weights(weights) on layer "tf_s |
1 |
Past CI - TensorFlow 2.5 (Patch release: v4.21.2 | b487096b0)
General
| no. | error |
|---|---|
| 70 | RuntimeError: Failed to import transformers.models.albert.modeling_tf_albert because of the followin |
| 28 | NameError: name 'kenlm' is not defined |
| 18 | RuntimeError: Failed to import transformers.models.gpt2.modeling_tf_gpt2 because of the following er |
| 4 | NameError: name 'MaskFormerForInstanceSegmentation' is not defined |
| 2 | RuntimeError: Failed to import transformers.models.t5.modeling_tf_t5 because of the following error |
| 2 | RuntimeError: Failed to import transformers.models.distilbert.modeling_tf_distilbert because of the |
| 2 | RuntimeError: Failed to import transformers.models.bert.modeling_tf_bert because of the following er |
| 2 | NameError: name 'MaskFormerModel' is not defined |
Per model
| model | no. of errors | major error | count |
|---|---|---|---|
| wav2vec2_with_lm | 28 | NameError: name 'kenlm' is not defined | 28 |
| maskformer | 6 | NameError: name 'MaskFormerForInstanceSegmentation' is not d | 4 |
| squeezebert | 4 | RuntimeError: Failed to import transformers.models.albert.mo | 4 |
| xglm | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| bert_generation | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| byt5 | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| bloom | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| perceiver | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| layoutlmv2 | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| bort | 2 | RuntimeError: Failed to import transformers.models.bert.mode | 2 |
| tapex | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| plbart | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| barthez | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| layoutxlm | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| nllb | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| canine | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| layoutlmv3 | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| xlm_prophetnet | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| luke | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| mbart50 | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| realm | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| mluke | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| bertweet | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| mvp | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| big_bird | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| phobert | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| fnet | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| speech_to_text_2 | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| prophetnet | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| herbert | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| fsmt | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| codegen | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| retribert | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| m2m_100 | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| bartpho | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
| reformer | 2 | RuntimeError: Failed to import transformers.models.albert.mo | 2 |
I was trying to fix the kenlm issue, but I see it's correctly installed here and has been for a while.
I guess it is an image issue?
I was trying to fix the kenlm issue, but I see it's correctly installed here and has been for a while.
I guess it is an image issue?
Hi @LysandreJik. In fact, Past CI use transformers-past-gpu/Dockerfile:
https://github.com/huggingface/transformers/blame/main/docker/transformers-past-gpu/Dockerfile
It's probably arguable if we should (or should not) include kenlm. I don't remember well if I got issue when installing it. Maybe yes for more elder versions, so I decide not to install it for all versions (to avoid confusion).
We can try with it in the next launch.
I think we can add it, we've had it in the main file for 8 months so it's unlikely to cause an issue. Looking forward to the next launch!