hf_transfer icon indicating copy to clipboard operation
hf_transfer copied to clipboard

Upload choking internet speed

Open silveroxides opened this issue 1 year ago • 0 comments

System Info

hf_transfer==0.1.6, Windows 11, Python version 3.10.9, huggingface_hub==0.23.4

While having the environment variable 'HF_HUB_ENABLE_HF_TRANSFER' set to 1 the upload speeds attempted by hf_transfer completely chokes my connection and pretty much causes Denial Of Service. My internet connection is rated at 250/250 Mbit/s yet it still seems to attempt to push far beyond that and all other connections on Local Network stops working properly due to it. In fact when setting the the variable to 0 I end up uploading faster than with it set to 1 due to it not choking the connection repeatedly

Reproduction

Unsure if it is reproducible that easily however there might be some ways to mitigate it by pre-checking speed or manually setting some form of upper limit

Expected behavior

Being able to upload faster with HF_HUB_ENABLE_HF_TRANSFER set to 1 compared to HF_HUB_ENABLE_HF_TRANSFER set to 0 and not having my entire internet connection getting choked

silveroxides avatar Aug 08 '24 17:08 silveroxides