BMTrain icon indicating copy to clipboard operation
BMTrain copied to clipboard

Output hidden states and attention scores for each transformer layer

Open xcjthu opened this issue 4 years ago • 1 comments

Existing TransformerBlockList cannot output the hidden states and attention scores for each transformer layer. Sometimes we want to get the hiddens and attention scores to conduct analysis and feed them into the next modules.

xcjthu avatar Apr 23 '22 16:04 xcjthu

Support backward of middle hidden states by PR #40 .

Achazwl avatar Aug 30 '22 12:08 Achazwl