alph4b3th

is it normal 30-minute/token slowness in intel xeon?

20

 I'm running alpaca on my Intel Xeon Silver 4214(4) @ 2.194GHz and I'm having a lot of slowness for artificial intelligence to generate some response. It takes between 20-40...

Support for OpenCL and multiple GPUs, such as Intel Graphics and AMD.

1

### Hello, I need support for GPUs in general, both **AMD** and **Intel Graphics**. > While an Intel Graphics may not provide as much acceleration, it would still be interesting...

What is the project's maturity level?

1

Is the project ready for production use? What is the minimum required hardware to run version 7B? (recommended CPU? how many CPU threads?) Can the project handle 8GB of RAM?...

How to implement INT4/INT8 quantization and optimal way to use AVX instructions?

10

in reply from: @gotzmann Implementing INT4/INT8 quantization and using AVX instructions can be challenging, mainly due to the limitations of INT8 multiplication instructions. However, here are some ideas to help...