Language-Model-SAEs
Language-Model-SAEs copied to clipboard
OpenMOSS
For OpenMOSS Mechanistic Interpretability Team's Sparse Autoencoder (SAE) research.
Automatic testing is fundamental to keep a collaborative developed project from endless bugs corrupting modules that originally work. As for a deep learning library, always running the whole training or...
It is much easier for people (who may be new to mechanistic interpretability) to get started with detailed tutorial and documentation. Currently this project lacks documentation and comments in many...
We can publish this library on [PyPI](https://pypi.org/) so that people can use this package simply using `pip install lm-saes`! However, before this we should first get this library well-tested and...
Thanks for open-sourcing the code! It seems the update of the repo makes the instructions in the README unusable. Is there any possible fix so the community can reproduce your...
Hi, Is there any released SAE is trained on the LLama instruct version? Thank you
Hi, Thank you for your great efforts on open-sourcing everything! However, for your dictionary learning on Othello paper (https://www.open-moss.com/en/DictCircuits_Othello/), it seems that the Github repo is now 404 (https://github.com/OpenMOSS/DictionaryCircuits-Othello). Will...
Hi, i have been training with your work recently and i'am having some issues around how to change the path of dataset and model to my own. I found that...
Following ./examples/loading_llamascope_saes.ipynb, and received the following error on the line below... Please instruct on how to fix. Thanks! `sae = SparseAutoEncoder.from_pretrained("fnlp/Llama3_1-8B-Base-L15R-8x")` ``` Local path `fnlp/Llama3_1-8B-Base-L15R-8x` not found. Downloading from huggingface...
Hi, I am new to this repo, and I got this error when I followed the readme to train the SAE: ``` bash (llama_scope) [email protected]:/Language-Model-SAEs$ lm-saes train examples/configuration/train.toml bash: lm-saes:...
Hello LLAMASCOPE team, thank you for your work! I'm having trouble with loading the SAE using the SAElens library, I keep getting the error below: ``` File "/anaconda3/envs/mechinterp/lib/python3.12/site-packages/sae_lens/sae.py", line 616,...
