Satoshi Report
Satoshi Report
Didn't work for me
Same issue. Cache fills up and then vLLM stops working.
Same thing for me after I use vLLM for awhile KV cache goes to 100% and the system freezes up. When does cache get cleared? I am using the OpenAI...
Thanks for your reply. I don't understand the underlying concept however. Why not just use the option price in the market and call it a day? That has the best...
OK got it working and the results look good. Thanks for making this code public domain. I will need to purchase your book now so I understand it more.
I tried that and same error. I used: `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`
Both m7i.48xlarge and m7i.xlarge have the same CPU "Intel(R) Xeon(R) Platinum 8488C". What makes this work on the 48x instances but not the smaller one, considering they have the same...
To partially answer my own question: From https://aws.amazon.com/blogs/aws/new-seventh-generation-general-purpose-amazon-ec2-instances-m7i-flex-and-m7i/ "The Intel QAT, Intel IAA, and Intel DSA accelerators will be available on the m7i.metal-24xl and m7i.metal-48xl instances." So I assume it...
To expand on this further: You need to get a whole metal Saphire Rapids server to get the speed ups talked about here. For my use case using the AWS...
same issue