ImageBind import data error in Google Colab

When I try to run the demo in Google Colab, I got the error:

import data

ModuleNotFoundError Traceback (most recent call last) in <cell line: 2>() 1 import torch ----> 2 import data 3 from models import imagebind_model 4 from models.imagebind_model import ModalityType

5 frames /content/ImageBind/data.py in 17 from pytorchvideo import transforms as pv_transforms 18 from pytorchvideo.data.clip_sampling import ConstantClipsPerVideoSampler ---> 19 from pytorchvideo.data.encoded_video import EncodedVideo 20 21 from torchvision import transforms

/usr/local/lib/python3.8/site-packages/pytorchvideo/data/init.py in 1 # Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved. 2 ----> 3 from .ava import Ava # noqa 4 from .charades import Charades # noqa 5 from .clip_sampling import ( # noqa; noqa

/usr/local/lib/python3.8/site-packages/pytorchvideo/data/ava.py in 10 from iopath.common.file_io import g_pathmgr 11 from pytorchvideo.data.clip_sampling import ClipInfo, ClipSampler ---> 12 from pytorchvideo.data.labeled_video_dataset import LabeledVideoDataset 13 14

/usr/local/lib/python3.8/site-packages/pytorchvideo/data/labeled_video_dataset.py in 12 13 from .labeled_video_paths import LabeledVideoPaths ---> 14 from .utils import MultiProcessSampler 15 16

/usr/local/lib/python3.8/site-packages/pytorchvideo/data/utils.py in 14 from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, Union 15 ---> 16 import av 17 import numpy as np 18 import torch

/usr/local/lib/python3.8/site-packages/av/init.py in 18 # MUST import the core before anything else in order to initalize the underlying 19 # library that is being wrapped. ---> 20 from av._core import time_base, library_versions 21 22 # Capture logging (by importing it).

ModuleNotFoundError: No module named 'av._core'

NOTE: If your import is failing due to a missing package, you can manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the "Open Examples" button below.

I try to: !pip install av But the problem has not been resolved.

May 10 '23 14:05 Wallfacer005CN

There are plenty dependencies missing in the list, which are not there in colab. I recommend running it locally. First, run the pip install requirements.txt, and thereafter execute the rest of the code as mentioned in Usage section. It will fail a lot of times, and whatever is missing, keep installing manually. That did the trick for me, as I didn't get this missing module error

May 10 '23 14:05 sachinspanicker

Basically, any dependencies that can be installed locally can also be installed at Colab via pip. Judging from the error, there are not many dependencies missing, just one av._core. I suspect a dependency version discrepancy is the problem, if so, the official version number should be written in requirements.tx file.

May 10 '23 17:05 Wallfacer005CN

av package is part of the pytorchvideo lib. Just install pytorchvideo-0.1.5 in your colab and you will be all set. I tested it on colab and works.

!pip install iopath timm einops pytorchvideo

Collecting av (from pytorchvideo) Downloading av-10.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (31.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.0/31.0 MB 48.3 MB/s eta 0:00:00 Collecting parameterized (from pytorchvideo) Downloading parameterized-0.9.0-py2.py3-none-any.whl (20 kB) Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from pytorchvideo) (3.1) Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.7->timm) (3.12.0) Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.7->timm) (1.11.1) Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7->timm) (3.1.2) Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7->timm) (2.0.0) Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.7->timm) (3.25.2) Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.7->timm) (16.0.3) Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from fvcore->pytorchvideo) (1.22.4) Collecting yacs>=0.1.6 (from fvcore->pytorchvideo) Downloading yacs-0.1.8-py3-none-any.whl (14 kB) Requirement already satisfied: termcolor>=1.1 in /usr/local/lib/python3.10/dist-packages (from fvcore->pytorchvideo) (2.3.0) Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from fvcore->pytorchvideo) (8.4.0) Requirement already satisfied: tabulate in /usr/local/lib/python3.10/dist-packages (from fvcore->pytorchvideo) (0.8.10) Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->timm) (2023.4.0) Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->timm) (2.27.1) Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->timm) (23.1) Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.7->timm) (2.1.2) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->timm) (1.26.15) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->timm) (2022.12.7) Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->timm) (2.0.12) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->timm) (3.4) Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.7->timm) (1.3.0) Building wheels for collected packages: pytorchvideo, fvcore Building wheel for pytorchvideo (setup.py) ... done Created wheel for pytorchvideo: filename=pytorchvideo-0.1.5-py3-none-any.whl size=188694 sha256=5936ff366e5cc6f7553b79e4546ee9c8e0517e7f6f0ef4669ca143416b35e118 Stored in directory: /root/.cache/pip/wheels/ff/4e/81/0f72a543be9ed7eb737c95bfc5da4025e73226b44368074ece Building wheel for fvcore (setup.py) ... done Created wheel for fvcore: filename=fvcore-0.1.5.post20221221-py3-none-any.whl size=61405 sha256=77b75edf002f71841572257a050488d47979e4ace49702aada449464187ef415 Stored in directory: /root/.cache/pip/wheels/01/c0/af/77c1cf53a1be9e42a52b48e5af2169d40ec2e89f7362489dd0 Successfully built pytorchvideo fvcore Installing collected packages: av, yacs, parameterized, fvcore, pytorchvideo Successfully installed av-10.0.0 fvcore-0.1.5.post20221221 parameterized-0.9.0 pytorchvideo-0.1.5 yacs-0.1.8

May 11 '23 02:05 sachinspanicker

libs = [
    "pytorchvideo@git+https://github.com/facebookresearch/pytorchvideo.git@28fe037d212663c6a24f373b94cc5d478c8c1a1d",
    "timm",
    "ftfy",
    "regex",
    "einops",
    "fvcore",
    "decord"
]

for lib in libs:
    command = f"pip install {lib}"
    os.system(command)

os.system("wget -nc -P models https://dl.fbaipublicfiles.com/imagebind/imagebind_huge.pth")
model_path = "./models/imagebind_huge.pth"
os.system("git clone https://github.com/facebookresearch/ImageBind.git")
sys.path.append('/content/ImageBind')
sys.path.append('/content/ImageBind/models')

import ImageBind
import models
import ImageBind.data 
from ImageBind.models import imagebind_model
from ImageBind.models.imagebind_model import ModalityType

May 11 '23 08:05 XinyueZ

But this solution did not work in my project. I don't know why.

May 11 '23 13:05 Wallfacer005CN

need to restart runtime after install, not "disconnect"

May 11 '23 13:05 XinyueZ