Jinchen Ge

Datasets' cache not re-used

28

## Describe the bug For most tokenizers I have tested (e.g. the RoBERTa tokenizer), the data preprocessing cache are not fully reused in the first few runs, although their `.arrow`...

Set LayerNorm's eps to a number that is larger than 6e-5

2

Currently, many LayerNorm's eps are smaller than 6.1e-5 (smallest fp16 value), which might cause underflow.