based
based copied to clipboard
HazyResearch
Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"
Hey There, appreciate what you guys are doing, its great work. I'm trying to access the model weights from HF using transformer Library but stuck due to a swiglu error,...
I am having a go at running inference and evaluation for this model, and running into a TypeError in `GPTLMHeadModel`: ``` In [1]: import torch ...: from transformers import AutoTokenizer...
Hello, I would like to extend my sincere appreciation for the outstanding work you have done. While going through the paper, I came across a parameter labeled 'params' in the...
Hi all, just a heads up: I filed an [issue](https://github.com/huggingface/transformers/issues/29466) with `huggingface/transformers` requesting model support for BASED via their library. My engagement over the past few days has been part...
Hi, When running the training script with Hydra configuration, an import error occurs indicating that the module train cannot be found. ``` (venv) (base) denghaoran@scai3:~/workspace/tmp/based/train$ python run.py experiment=example/based-360m trainer.devices=1 /home/denghaoran/workspace/tmp/based/train/run.py:33:...
Metadata
Owner
Metadata
Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"