nigelzzz comments

Repositories
Issues
Comments

Results 4 comments of


                                            nigelzzz

Introduce New Lookup-Table(LUT)-Based Matrix Multiplication Method (TMAC)

Hi @QingtaoLi1 , can we use `test-backend-ops` to test flops, i would like to show the result from paper image 6, image 7. thanks and i checked the tmac repo,...

Introduce New Lookup-Table(LUT)-Based Matrix Multiplication Method (TMAC)

Can I known if we want to use llama3.2 1b q4 , do we need to recompile kernel?

why arm don't support tl2 kernel

Thanks for your response!! So using `I2_s` and `TL1` can decrease lantency, `TL2` can't improve it? if the simd lane can be 32 or 64, is it helpful?

Prompt eval is 5x slower than in Ollama and maxes out the CPU

Hi I have same issue on rpi5. `prompt eval rate: 27.45 tokens/s vs 2.27 tokens per second)` ollama.cpp total duration: 57.016716541s load duration: 35.149005ms prompt eval count: 948 token(s) prompt...