cannot install faster transformer with nvcr.io/nvidia/pytorch:22.03-py3?

[zhanghaoie]

[byshiue]

Can you provide more details? How do you install and what error do you encounter? Besides, 22.03 docker image is latest one, we still not verify on it. You can try some older docker image.

[Tlntin]

I try to use nvcr.io/nvidia/pytorch:22.05-py3 and transformer==4.20.1. It work very well, but you need to do some change.

install kineto

when i use cmake to generate makefile file, it attention me some warning append, so i need to install it.

git clone --recursive https://github.com/pytorch/kineto.git
cd kineto/libkineto
mkdir build && cd build
cmake ..
make && make install

install cusparse

• when you wan to use sparse argment to run bert_example.py , you need to install it.
• I found a quickly ways to install it (but you need must install cuda-11.2+).

sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
  sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
  sudo apt-get update
sudo apt install libcusparselt0=0.1.0.2-1 libcusparselt-dev=0.1.0.2-1 

install transformers

• default transformers is 2.5.1，if you want install transformer latest, you need to change some code.
• pip install transformers -U
• cd cd examples/pytorch/bert/utils/, open encode.py
• from begin code in line 21, old code is from transformers.modeling_bert import BertEncoder, you need change it to from transformers.models.bert.modeling_bert import BertEncoder. And then, found the last line, old code is output = self.encoder( hidden_states, extended_attention_mask, self.head_mask) you need to add return_dict=False, like this

output = self.encoder(
            hidden_states, extended_attention_mask, self.head_mask,
            return_dict=False)

then you can enjoy it.

[byshiue]

Close this bug because it is inactivated. Feel free to re-open this issue if you still have any problem.