AVT icon indicating copy to clipboard operation
AVT copied to clipboard

Is there any alternative implementation for 'transformers.GPT2Model'?

Open coolbay opened this issue 3 years ago • 0 comments

Hi, Rohit! May I ask if I can replace the model definition using the package 'transformers' with any other package or code? That would solve my following issue.

The 'transformers' package calls the package 'tokenizers', which calls 'GLIBC_2.18'. My cluster only has an old version of GLIBC, and I have no sudo access to upgrade it. I got the following error when I run it on the cluster:

File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/transformers-4.2.2-py3.8.egg/transformers/models/__init__.py", line 19, in <module>
 File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/transformers-4.2.2-py3.8.egg/transformers/models/mt5/__init__.py", line 31, in <module>
 File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/transformers-4.2.2-py3.8.egg/transformers/models/t5/tokenization_t5.py", line 26, in <module>
 File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/transformers-4.2.2-py3.8.egg/transformers/tokenization_utils.py", line 25, in <module>
 File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/transformers-4.2.2-py3.8.egg/transformers/tokenization_utils_base.py", line 87, in <module>
 File "/home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/tokenizers/__init__.py", line 79, in <module>
   from .tokenizers import (
ImportError: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /home/zhaoc/a100/tools/miniconda3/envs/avt2/lib/python3.7/site-packages/tokenizers/tokenizers.cpython-37m-x86_64-linux-gnu.so)

Thanks. Chen

coolbay avatar Jan 27 '23 20:01 coolbay