Jacob Warren
Jacob Warren
@remiconnesson it looks like @mgoin already approved the changes. It just needs to be reviewed before it can be automatically merged.
Is there any update for this @balazsorban44 ? In v5 beta 4 I'm still gettin the following error despite having implemented the fixes here: ``` [auth][error] OperationProcessingError: "response" is not...
Totally get this. At the same time, LinkedIn has proven they give zero cares about their API or conformity. Realistically, they'll never implement this properly. I'll take a stab at...
Is there any update for 8bit support? That would help Mixtral generate useable outputs on a single (non-overpriced) GPU.
@hmellor it currently works with 4-bit, but not 8-bit. Currently you have to use chu-tianxiang/vllm-gptq to get 8-bit support.