Derek Lewis

Running into this issue myself with 'all-linear' and SFTTrainer using peft main, transformers 4.40.2, and trl 0.8.6. No PEFT and EOS is predicted fine. w/ PEFT and EOS is not...

@BenjaminBossan see below. Tried to simplify as much as possible. Also uploaded fine-tuned models to hub w/ PEFT enabled & disabled. TRL seems to be having some issues w/ chat_templates...

The odd thing is that I can pretty much `po` anything in a model running LLMEval, but something very basic like a simple Swift program that initializes an array with...

Here's a simple, reproducible example that I came up with by modifying Tutorial in mlx-swift-examples by just adding an array initialization with `MLXRandom.normal`. Setting breakpoint after and doing a `po`...

Interesting that doing a `expr eval(y)` also results in a hang. Not sure what to make of that.

It would be helpful if there was something between the current Shared Mode (Single VM to Host) and bridged with the Virtualization framework, which gives the VM access to the...

@bigPYJ1151 This is great - tried running the benchmark w/ `llama-2-7b` and no issues; however, with `mistral-7b`, I am getting an assertion when running the benchmark: ``` python benchmark_throughput.py --backend=vllm...

@bigPYJ1151 again, thanks for the contribution and that was helpful. Still exciting this works with llama-2 models. FYI, I did remove the assert and got this with `mistral-7b`: ``` File...

I encountered this issue when using Z2JH on EKS and an ALB as the ingress. Enabling sticky sessions fixed it for me.