tuned-lens

tuned-lens copied to clipboard

Reproducing table 1 in the paper

1

Hello, Thank you so much for your hard work! Is there any code to reproduce the experiments for table 1? Thanks!

Nicole-Nobili

Support for gemma architecture models

2

We also need to bump some dependencies for this since gemma does not exist in the version of transformers we were requiring.

levmckinney

Drop Python 3.9 support too while we're at it (enabling `|` for union types) _Originally posted by @norabelrose in https://github.com/AlignmentResearch/tuned-lens/issues/125#issuecomment-1968239228_

levmckinney

Has anyone trained a tuned lens on Gemma-2b or other Gemma models?

2

jbloomAus

Currently, if you try to create a prediction trajectory from a model and lens loaded in `bfloat16` error. ``` 294 traj_log_probs.append( --> 295 logits.log_softmax(dim=-1).squeeze().detach().cpu().numpy() 296 ) 298 # Add model...

levmckinney

All layers translators with nan loss while training the lens

2

Hello! Thanks for sharing this amazing work! I am trying to train the lens over a new dataset [HF Dataset](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) (note that the original "the pile" dataset was removed from...

AmeenAli

Add causal basis extraction stuff back to the CLI somehow

2

Before white-box -> tuned-lens rename, the CLI was like this: ```white-box lens train ``` and ```white-box cbe extract ``` But after the rename we have ```tuned-lens train ``` without the...

norabelrose

**Describe the bug** Checkpointing crashes when `--zero` is set, with the error `RuntimeError: Tensors must be CUDA and dense` being thrown inside the method `consolidate_state_dict()` **Expected behavior** Shouldn't crash **Screenshots**

norabelrose

Prediction depth

2

In the paper there is a nice visualization of prediction depth. Prediction depth is defined in the paper is the first layer where the most likely token is equal to...

levmckinney

This feature will be removed in #63. It would be nice to reimplement this so that we can not only see which tokens have a high probability at each layer...

levmckinney