parmeet

Regarding adding shuffling and sharding datapipes to in-built datasets

1

## 🚀 Feature **Motivation** * To avoid pitfall with shuffling and sharding of datapipes in distributed training environments * To ensure consistent experience of TorchData based datasets across domains. **Pitch**...

add projection layer to roberta encoder

5

User may want to additionally project the features from encoder. This PR add support for projecting features to different dimensional space.

https://github.com/pytorch/text/issues/1359

[WIP] using std::array in vocab for additional speed-ups

1

We replace container type of stoi_ from std::vector to std::array. This bring in slight additional improvements in look-up speed

Convert iterator-style raw datasets to map-style raw datasets

2

## 🚀 Feature **Motivation** torchtext provide several open source nlp datasets in raw form. These datasets are provide as [Iterables](https://pytorch.org/docs/stable/data.html#iterable-style-datasets). Although there are times when user may prefer [map-style](https://pytorch.org/docs/stable/data.html#map-style-datasets) datasets....