New models via AutoModel

[bhavsarpratik]

Hi Kaushal,

Current some models are not available in fast-bert and this will always be the case as new models keep coming up. I wanted to use xlm-roberta and one way I could make it work was to use AutoTokenizer and AutoModel. I passed the tokenizer object to databunch and saved the Automodel to a path and passed the path to learner.

Can you make changes in a way that the logic tries to load model via Auto mechanism. This might save headaches for you too.

from transformers import AutoModel, AutoTokenizer, AutoModelWithLMHead, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
model = AutoModelWithLMHead.from_pretrained(model_name_or_path)