opus icon indicating copy to clipboard operation
opus copied to clipboard

Cannot build because media.xiph.org is not responding

Open a-rose opened this issue 1 year ago • 8 comments

At the time of writing, media.xiph.org seems to be inaccessible, which causes a compile block when downloading the model data (https://media.xiph.org/opus/models/opus_data-735117b.tar.gz).

In #337 it was mentioned that this file was too large to push to the repo, but storing it on an independent web server doesn't seem like an ideal solution. Perhaps you could consider git LFS instead?

a-rose avatar Jul 08 '24 14:07 a-rose

Perhaps you could consider git LFS instead?

It was considered

The free limit is 1 GB

xnorpx avatar Jul 08 '24 14:07 xnorpx

Oh wow, I had no idea there was a storage or bandwidth limitation

a-rose avatar Jul 08 '24 15:07 a-rose

What about huggingface.co? I know they host some fairly large models and data sets, but I'm unclear on the pricing.

rillian avatar Jul 08 '24 21:07 rillian

Came across this issue since we also saw lots of build failures in the last weeks / months. I don't have a real solution but we traced this down to this particular file being slow for everyone on a Deutsche Telekom DSL line. (This might as well affect other providers using the same peers as DTAG and OVH).

We could even verify the slowness with their load test files for the Montreal DC https://bhs.proof.ovh.ca/

For the time being we wonder if we should just mirror the file locally and have the build script pick it up from there.

bascht avatar Aug 12 '24 12:08 bascht

Why not just put them in GitHub Releases?

Andarwinux avatar Mar 01 '25 18:03 Andarwinux

Why not just put them in GitHub Releases?

Current docs say there's a 2 GiB per file limit, so seems like this might work.

Each file included in a release must be under 2 GiB. There is no limit on the total size of a release, nor bandwidth usage.

rillian avatar Mar 03 '25 01:03 rillian

~~Even worse, this morning, I'm getting a 404.~~ seems it can't be launched directly, forget about that.

❯ dnn/download_model.sh
Downloading latest model
--2025-05-20 10:30:24--  https://media.xiph.org/opus/models/opus_data-.tar.gz
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving media.xiph.org (media.xiph.org)... 2607:5300:203:2965::, 54.39.18.101
Connecting to media.xiph.org (media.xiph.org)|2607:5300:203:2965::|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2025-05-20 10:30:24 ERROR 404: Not Found.

ticpu avatar May 20 '25 14:05 ticpu

--2025-05-20 10:30:24-- [...]opus_data-.tar.gz

This is a different issue; it's not that the download failed, but that you need to pass the hash of the model you want to download when invoking the script.

$ dnn/download_model.sh b357b95aa2bdc71cf48b098016158b2a989194546bf199caeb83d7f20bf3490e
HTTP response 200 OK [...]

rillian avatar May 20 '25 14:05 rillian