Hanlin Zhang

Add MPT (mpt-1b-redpajama-200b) support

2

https://huggingface.co/mosaicml/mpt-1b-redpajama-200b

https://huggingface.co/mosaicml/mpt-1b-redpajama-200b

CMakeFiles/profile_dipcc.dir/dipcc/cc/cfrstats.cc.o: In function `pybind11::capsule::name() const [clone .isra.564]': cfrstats.cc:(.text+0x2297): undefined reference to `PyCapsule_GetName' cfrstats.cc:(.text+0x22b1): undefined reference to `PyErr_Occurred' CMakeFiles/profile_dipcc.dir/dipcc/cc/cfrstats.cc.o: In function `pybind11::setattr(pybind11::handle, char const*, pybind11::handle)': cfrstats.cc:(.text._ZN8pybind117setattrENS_6handleEPKcS0_[_ZN8pybind117setattrENS_6handleEPKcS0_]+0x7): undefined reference to `PyObject_SetAttrString' CMakeFiles/profile_dipcc.dir/dipcc/cc/cfrstats.cc.o:...

Cannot load the fine-tuned ckpt

4

Unexpected key(s) in state_dict: "head.weight", "head.bias", "fc_norm.weight", "fc_norm.bias". size mismatch for pos_embed: copying a param with shape torch.Size([1, 197, 1024]) from checkpoint, the shape in current model is torch.Size([1, 257,...