UnpicklingError with different BERT models

[alpoktem]

Hi, I'm trying to perform document classification with Hindi language. I want to use BERT models that are adapted to Hindi and Indian languages like muril-base-cased and muril-large-cased.

In order to load them, I downloaded the models into hedwig-data/models/bert_pretrained directory and I added these lines to constants.py:

PRETRAINED_MODEL_ARCHIVE_MAP = {
    ...
    'muril-large-cased': os.path.join(MODEL_DATA_DIR, 'bert_pretrained', 'muril-large-cased'),
    'muril-base-cased': os.path.join(MODEL_DATA_DIR, 'bert_pretrained', 'muril-base-cased')

}
PRETRAINED_VOCAB_ARCHIVE_MAP = {
    ...
    'muril-large-cased': os.path.join(MODEL_DATA_DIR, 'bert_pretrained', 'muril-large-cased', 'vocab.txt'),
    'muril-base-cased': os.path.join(MODEL_DATA_DIR, 'bert_pretrained', 'muril-base-cased', 'vocab.txt')
}

I'm getting this UnpicklingError which I think is because of the transformers package version.

.../hedwig$ python -m models.bert --dataset MFIN --model muril-base-cased --max-seq-length 256 --batch-size 8 --lr 2e-5 --epochs 1
Device: CUDA
Number of GPUs: 2
FP16: False
Traceback (most recent call last):
  File "/home/twbgmy/anaconda3/envs/hindiclass/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/twbgmy/anaconda3/envs/hindiclass/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/twbgmy/play/MFIN/hedwig/models/bert/__main__.py", line 87, in <module>
    model = BertForSequenceClassification.from_pretrained(pretrained_model_path, num_labels=args.num_labels)
  File "/home/twbgmy/anaconda3/envs/hindiclass/lib/python3.6/site-packages/transformers/modeling_utils.py", line 345, in from_pretrained
    state_dict = torch.load(resolved_archive_file, map_location='cpu')
  File "/home/twbgmy/anaconda3/envs/hindiclass/lib/python3.6/site-packages/torch/serialization.py", line 358, in load
    return _load(f, map_location, pickle_module)
  File "/home/twbgmy/anaconda3/envs/hindiclass/lib/python3.6/site-packages/torch/serialization.py", line 532, in _load
    magic_number = pickle_module.load(f)
_pickle.UnpicklingError: A load persistent id instruction was encountered,
but no persistent_load function was specified.

Am I doing something wrong? I'd appreciate any guidance.