Nora Belrose

Will not compile on GCC 11.1.0, CUDA 11.3

6

I've been trying to build this package for the last two nights to no avail. Every time I run `python setup.py install`, I get a big wall of compiler warnings...

JAX SpectralNormalization shape error with vanilla nn.Dense layer

1

Applying `edward2.jax.nn.SpectralNormalization` to a vanilla Flax `nn.Dense` layer fails with a shape error because the kernel is left-multiplied with the input instead of right-multiplied. On line 143 of `spectral_norm.py` the...

Pylance/VSCode says "Expected 1 more positional argument" when using pipe operator

2

Hi, thanks for the great library. I've noticed that when I'm using an iPython notebook in VSCode with type checking turned on, Pylance doesn't recognize the verb functions as being...

Add causal basis extraction stuff back to the CLI somehow

2

Before white-box -> tuned-lens rename, the CLI was like this: ```white-box lens train ``` and ```white-box cbe extract ``` But after the rename we have ```tuned-lens train ``` without the...

**Describe the bug** Checkpointing crashes when `--zero` is set, with the error `RuntimeError: Tensors must be CUDA and dense` being thrown inside the method `consolidate_state_dict()` **Expected behavior** Shouldn't crash **Screenshots**

Adds `LdaFitter` for supervised LDA reporters

Use shrinkage for (cross-)covariance estimation

3

We're now using the shrinkage technique from [this paper](https://arxiv.org/abs/0907.4698) in the concept-erasure repo; it makes covariance estimation robust to small sample sizes. Might make CRC-TPC, VINC, etc. work better ```py...

Support Neel Nanda's counterfact dataset

3

In general, we should support datasets where the statements don't take the form of question-answer pairs. The first obvious dataset to support is [counterfact](https://huggingface.co/datasets/NeelNanda/counterfact-tracing).

Use dill and Apache Arrow directly for caching & storing hidden states

1

Right now we save hidden states as a HF dataset so that we can use its magical caching abilities. We're clearly wrangling HF into supporting something it wasn't designed for...

Right now it appears that we're getting numerical overflows when casting to float16 in `float32_to_int16` for some models (RWKV-LM, T0pp, Unified QA). These models were trained in bfloat16, not float16,...