AVT
AVT copied to clipboard
Is there any alternative implementation for 'transformers.GPT2Model'?
Hi, Rohit! May I ask if I can replace the model definition using the package 'transformers' with any other package or code? That would solve my following issue.
The 'transformers' package calls the package 'tokenizers', which calls 'GLIBC_2.18'. My cluster only has an old version of GLIBC, and I have no sudo access to upgrade it. I got the following error when I run it on the cluster:
File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/transformers-4.2.2-py3.8.egg/transformers/models/__init__.py", line 19, in <module>
File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/transformers-4.2.2-py3.8.egg/transformers/models/mt5/__init__.py", line 31, in <module>
File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/transformers-4.2.2-py3.8.egg/transformers/models/t5/tokenization_t5.py", line 26, in <module>
File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/transformers-4.2.2-py3.8.egg/transformers/tokenization_utils.py", line 25, in <module>
File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/transformers-4.2.2-py3.8.egg/transformers/tokenization_utils_base.py", line 87, in <module>
File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/tokenizers/__init__.py", line 79, in <module>
from .tokenizers import (
ImportError: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/tokenizers/tokenizers.cpython-37m-x86_64-linux-gnu.so)
Thanks. Chen