Aydyn Tairov
Aydyn Tairov
> Thanks @tairov for requesting this (in the the live stream of all places :) ) . We'll make sure this is prioritized. > > Keep up the great work...
Hi @andresnowak thanks for your PR. I'll review it shortly.. Did you get any performance improvement when switched to vectorized rope rotation?
Approach you implemented looks interesting. I did some benchmark, this change doesn't give any performance boost for models up to 110M. However, on bigger models it can show some difference....
Probably for simple loops Mojo already has some levels of optimizations by default. Similarly as gcc optimizes `for` loops.
That's hard to say. Possibly we can manually tune the source code to make it "more optimal" than standard optimizations. But if there is no manually optimized loops, the compiler...
Thanks for taking time to research this topic. I saw in the other PR you tried to leverage autotune for finding optimal nelts. Probably the autotune based implementation might be...
BTW, I don't see any use of `tile` primitive that's imported.
Hi @andresnowak any chance you can validate the ideas from this PR on the latest mojo release? Is there anything we can improve?
Hi @shroominic , thanks for reaching out! I really appreciate you taking the time to share your thoughts on how this project could grow. You've raised some great ideas, part...
Hi . Thanks for you question. GGUF is not yet supported. AFAIK, gguf models are just originally bin models converted to gguf, so. It also depends on the exact "bin"...