Siddharth Narayanan

icon location San Francisco, CA icon location Interested in the application of statistical methods to the natural sciences. Technical staff at @Future-House.

Results 2 comments of Siddharth Narayanan

Performance gap between triton and flash attn

Hi, I see a similar scaling gap when running the tutorial example. I ran the tests with CUDA 11.3, Pytorch 1.11, and an A10 GPU. I installed Triton from source....

Error with PPO training about hidden state in-place modification

@haosdent here's how I fixed the same error in my PPO code. I think this solution will work for trl's PPOTrainer too: Wrap [these lines](https://github.com/huggingface/trl/blob/main/trl/trainer/ppo_trainer.py#L798-L804) with this context manager: ```python...