prashant comments

Repositories
Issues
Comments

Results 3 comments of


                                            prashant

kv cache manipulation?

> @l3utterfly thanks for offering help! We have been talking about implementing different kv cache manipulation techniques but haven't got a chance to that part. You mentioned implementing _different_ kv...

kv cache manipulation?

Thanks for the response, you must mean [llama_transformer.py](https://github.com/pytorch/executorch/blob/d59419c4d56f7f14b3e6ac65848ce23c1b3ee108/examples/models/llama2/llama_transformer.py#L146). I had a basic question, is kv cache part of the pytorch llama model(*.pt/pth) or is it implemented separately outside of the...

kv cache manipulation?

Thanks for the response. Where can I find the earlier version?