DeepLearningExamples
DeepLearningExamples copied to clipboard
BERT pre-training - Data preparation - Sharding speed-up
Changed the way we keep track of sentences counts in each shard training and test file to avoid re-calculating from scratch. This results in substantial speed-up of sharding on huge text files especially.