sdmorrey

Is Infini-attention support possible?

1

# Prerequisites Please answer the following questions for yourself before submitting an issue. - [ x] I am running the latest code. Development is very rapid so there are no...

Please publish train_state_unpadded_shape_dtype_struct

4

We really need this file very soon the model will not run at all `Restored checkpoint in 6.29 seconds. Jitting decoding. ERROR:absl:For checkpoint version > 1.0, we require users to...

Help isn't offered?

1

I downloaded this from the App Store, then when I ran into an issue where I wanted a user guide or something. I eventually found my answer here in the...

Is it possible to add support for Infini-attention?

2

There's some work being done to implement Infini-attention from https://arxiv.org/pdf/2404.07143 In a nutshell it allows for essentially an unlimited context length without incurring the quadratic penalty. There's a proof of...

There is a discussion about implementing this into llama.cpp so models trained this way can run. Can you please chime in? On this issue https://github.com/ggerganov/llama.cpp/issues/7168

Support for Gemma 2?

2

What would be required to support Gemma 2? I'd be happy to chip in and help with the code, I just need to have a bit of insight into what...

[Feature Request]: Background sync of models

4

### Problem Description When I start a new chat often I want to chat a different model than the last one I talked to. At present this is handled via...

Why Transformer++

1

I found this project being discussed in local llama subreddit. I read the paper but had questions. One of the questions that came up that is gnawing at me... Why...