Simran Arora
Simran Arora
Awesome, thanks for putting in the request!
Hi, I think it's because this RMSNorm is being set to None https://github.com/HazyResearch/based/blob/e8de5648f7e84248be8ebc1499e817641b0f577b/based/models/gpt.py#L371 Due to the import structure here: https://github.com/HazyResearch/based/blob/e8de5648f7e84248be8ebc1499e817641b0f577b/based/models/gpt.py#L52 The options are to - install the norm from flash...
That line you pointed out requires this to be installed: https://github.com/Dao-AILab/flash-attention/tree/main/csrc/fused_dense_lib Would recommend cloning flash-attention and python setup.py install within this directory An alternative workaround, without the install, is to,...
yes that was due to the changes, please try again and let me know if you run into issues
The implementation already exists You can learn more about our implementation here: https://hazyresearch.stanford.edu/blog/2024-11-27-tk-fp8
That would be great!!