Any demo python I can play with?

[AK51]

Hi,

I have installed LongNet in my Ubuntu with 4090. (Is it enough to run LongNet?) but when I type python example.py, there is error... I have tried "pip install torchscale", no help, same error.. >_< I am still reading the github and the paper, is there any demo python program I can play with first? Thx

Details (No error during LongNet installation)

pip install LongNet
Requirement already satisfied: LongNet in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (0.1.3)
Requirement already satisfied: torch in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (2.0.1)
Requirement already satisfied: einops in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.6.1)
Requirement already satisfied: flash-attn in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (1.0.5)
Requirement already satisfied: accelerate in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.20.3)
Requirement already satisfied: bitsandbytes in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.39.1)
Requirement already satisfied: fairscale in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.4.0)
Requirement already satisfied: timm in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.4.12)
Requirement already satisfied: flamingo-pytorch in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.1.2)
Requirement already satisfied: numpy>=1.17 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from accelerate->LongNet) (1.25.0)
Requirement already satisfied: packaging>=20.0 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from accelerate->LongNet) (23.1)
Requirement already satisfied: psutil in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from accelerate->LongNet) (5.9.5)
Requirement already satisfied: pyyaml in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from accelerate->LongNet) (6.0)
Requirement already satisfied: filelock in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (3.12.2)
Requirement already satisfied: typing-extensions in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (4.7.1)
Requirement already satisfied: sympy in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (1.12)
Requirement already satisfied: networkx in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (3.1)
Requirement already satisfied: jinja2 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (3.1.2)
Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.7.99)
Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.7.99)
Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.7.101)
Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (8.5.0.96)
Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.10.3.66)
Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (10.9.0.58)
Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (10.2.10.91)
Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.4.0.1)
Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.7.4.91)
Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (2.14.3)
Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.7.91)
Requirement already satisfied: triton==2.0.0 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (2.0.0)
Requirement already satisfied: setuptools in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->LongNet) (67.8.0)
Requirement already satisfied: wheel in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->LongNet) (0.38.4)
Requirement already satisfied: cmake in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from triton==2.0.0->torch->LongNet) (3.26.4)
Requirement already satisfied: lit in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from triton==2.0.0->torch->LongNet) (16.0.6)
Requirement already satisfied: einops-exts in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from flamingo-pytorch->LongNet) (0.0.4)
Requirement already satisfied: ninja in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from flash-attn->LongNet) (1.11.1)
Requirement already satisfied: torchvision in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from timm->LongNet) (0.15.2)
Requirement already satisfied: MarkupSafe>=2.0 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from jinja2->torch->LongNet) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from sympy->torch->LongNet) (1.3.0)
Requirement already satisfied: requests in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torchvision->timm->LongNet) (2.31.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torchvision->timm->LongNet) (10.0.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from requests->torchvision->timm->LongNet) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from requests->torchvision->timm->LongNet) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from requests->torchvision->timm->LongNet) (2.0.3)
Requirement already satisfied: certifi>=2017.4.17 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from requests->torchvision->timm->LongNet) (2023.5.7)


python example.py 
Traceback (most recent call last):
  File "/media/ak/HD/LongNet/example.py", line 3, in <module>
    from LongNet import DilatedAttention
  File "/media/ak/HD/LongNet/LongNet/__init__.py", line 6, in <module>
    from LongNet.attention import DilatedAttention
  File "/media/ak/HD/LongNet/LongNet/attention.py", line 6, in <module>
    from LongNet.torchscale import XPOS, RelativePositionBias
ImportError: cannot import name 'XPOS' from 'LongNet.torchscale' (unknown location)