Andrews Arokiam
Andrews Arokiam
/assign @andyi2it
/assign @andyi2it
@terrytangyuan @yuzisun When configuring speculative decoding, we have to load 2 models. So, in most cases we can have both the models following a pattern of directory structure and download...
@tarukumar If you are pointing to a specific directory instead of prefix, can you try ending it with `/`. Eg: `s3://my-bucket/merlinite-7b-lab/`
/assign @andyi2it
@terrytangyuan @tarilabs @yuzisun Would something like this would work? https://docs.google.com/document/d/1jvm5CWEDhaqoNQbGZlk8jimqs-_XoLL4uZutwQTb4kY/edit?usp=sharing
@pierDipi Are you actively working on this?
/assign @andyi2it
These args are compatible with torchserve 0.9, however they could be redundant in version 0.9 as that is the default in 0.9. However, 0.11.x changes the behavior and hence may...
/approve