Henri Vasserman

ROCm Port

116

Currently I can say that for regular users the CLBlast version is much easier to run. If you want the most performance, though, HIP is for you. --- **Remember to...

cuBLAS: non-contiguous tensor support

4

This will allow cuBLAS to multiply tensors that are not contiguous in the row or column (I don't think llama has that situation) level by using [cudaMemcpy2dAsync](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__MEMORY.html#group__CUDART__MEMORY_1g373dacf191566b0bf5e5b807517b6bf9). Testing perplexity right...

Change CMake files to library target

3

The main change here is to get rid of the file and library lists and use targets instead. Here is an example of the dependencies of main.cpp when building with...

[Research] Steering vectors

27

For #1460 Original paper: [Steering GPT-2-XL by adding an activation vector](https://www.lesswrong.com/posts/5spBue2z2tw4JuDCx/steering-gpt-2-xl-by-adding-an-activation-vector) ```sh ./main -m ... --seed 123 -n 64 \ --steering-add "Love" \ --steering-sub "Hate" \ --steering-source 4 \ --steering-layer...

**Background:** Currently, the development server opens a socket on `localhost:1990` by default. While the port 1990 can be changed on the command line with `-Dport=`, there is no way to...