Farhang Dehzad

icon indicating an email [email protected]

icon company Wise Technologies icon location Eindhoven, Netherlands icon location Health tech entrepreneur with an interest in improving healthcare through AI

Results 1 comments of Farhang Dehzad

fine tuning the updated Phi-2 with flash-attn-2 produces very high loss > 2

Just wanted to acknowledge I have the same issue with using Fast Attention 2 with phi-2, the training loss hardly decreases with FA2 turned on, and works pretty well with...