deepmind-research icon indicating copy to clipboard operation
deepmind-research copied to clipboard
verified publisher icon google-deepmind

implementation of stoch depth in nfnets

Open mabingqi opened this issue 5 years ago • 0 comments

The implementation of stoch depth in the code of nfnet seems to be batch-wise dropout, but not block-level dropout as described in paper.

mabingqi avatar Feb 17 '21 12:02 mabingqi