wave2vec-recognize-docker icon indicating copy to clipboard operation
wave2vec-recognize-docker copied to clipboard

RuntimeError: [enforce fail at CPUAllocator.cpp:65]

Open raja1196 opened this issue 5 years ago • 5 comments

Followed the installation steps, built the Dockerfile (Which had its own hiccups, the fairseq repository does not have base_architecture definition in their models file, will raise a PR for it separately) and ran the code.

Running command python examples/wav2vec/recognize.py --wav_path /app/data/test.WAV --w2v_path /app/data/wav2vec_small_10m.pt --target_dict_path /app/data/dict.ltr.txt

Error: RuntimeError: [enforce fail at CPUAllocator.cpp:65] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 314663671488 bytes. Error code 12 (Cannot allocate memory)

Screenshot of the error: Screen Shot 2020-11-26 at 2 28 34 PM

I tried with different models of wave2vec with similar error. Do let me know if more information is needed. Running this on Azure DS VM.

raja1196 avatar Nov 26 '20 19:11 raja1196

PR for base_architecture definition merged thanks.

loretoparisi avatar Nov 30 '20 01:11 loretoparisi

I did a couple of things.

  • Changed the memory limit for the docker to 8GB
  • used a different audio file

I got the result: WE THE PEOPLE OF THE UNITED STATES IN ORDER TO FORM A MORE PERFECT UNION ESTABLISH JUSTICE INSURE DOMESTIC TRANQUILLITY PROVIDE FOR THE COMMON DEFENCE

I suspect that the length of the audio file could be a reason it is throwing the error (do let me know if it it is a more prevalent problem). The audio file i was trying to transcribe was around 1hr, compared to the 10 sec clip linked above.

raja1196 avatar Dec 02 '20 19:12 raja1196

https://github.com/pytorch/fairseq/issues/2651#issuecomment-719894563

I followed this comment for reference.

raja1196 avatar Dec 02 '20 19:12 raja1196

@raja1196 thanks we will check this case.

loretoparisi avatar Dec 03 '20 13:12 loretoparisi

I'm trying to run XLRS-53 after fine-tuning with this code but the following error happens. can anyone help me?

/usr/local/lib/python3.8/site-packages/torch/cuda/init.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:100.) return torch._C._cuda_getDeviceCount() > 0 Traceback (most recent call last): File "src/recognize.py", line 170, in model = Wav2VecPredictor(args.w2v_path, args.target_dict_path) File "src/recognize.py", line 122, in init self._model = self._load_model(w2v_path, self._target_dict) File "src/recognize.py", line 145, in _load_model args = base_architecture(w2v["args"]) File "src/recognize.py", line 24, in base_architecture args.no_pretrained_weights = getattr(args, "no_pretrained_weights", False)

iman3sh avatar Feb 01 '21 15:02 iman3sh