Aldo Pareja
Aldo Pareja
The package can't be used on travis because the setup fails. Merging this would make the package easier to depend on.
found this link but I'm not sure what kind of license is pytorch using: https://opensource.stackexchange.com/a/10044
when assigning hot-keys, the keyboard might recognize some letters as QUERTY (such as QWER) and others as dvorak. Weirdly enough, when I open the settings with `cmd+,` it both closes...
same here: `Cannot use FlashAttention backend because the flash_attn package is not found`
using the pytorch image (`nvcr.io/nvidia/pytorch:24.03-py3`) with this script: ```shell export HOME=/dev/shm/ export PATH=$HOME/.local/bin/:$PATH pip install vllm==v0.4.1 ray disable-usage-stats ray stop --force export OPENBLAS_NUM_THREADS=32 export OMP_NUM_THREADS=32 pkill python ray start --head...
# Throughput test Performed many experiments in prod cluster trying to get the best throughput using pytorch's[fsdp](https://pytorch.org/tutorials/intermediate/FSDP_adavnced_tutorial.html). In general, when the number of nodes increases above certain threshold the best...
still a problem?
When doing long evals on big models such as llama 70b it fails even with low concurrency (e.g. 4). I'm using 8xA100-80GB and it would time out. I had to...