Henry Xiao

Could this tool apply for encoder-decoder model, like Flan-T5?

3

It's great tool to show how to build a chatgpt-like model based on some foundation models. I wonder could it support the encoder-decoder model, like Flan-T5? Could we just directly...

could we use load_checkpoint_and_dispatch in a deepspeed framework?

1

### System Info ```Shell - `Accelerate` version: 0.19.0 - Platform: Linux-5.10.173-154.642.amzn2.x86_64 - Python version: 3.9.16 - Numpy version: 1.24.3 - PyTorch version: 2.0.0+cu117 ``` ### Information - [ ] The...

Does it provide the training interface?

2

Excellent works! I wonder whether this package provide the api to train a tokenizer (i.e. get the vocab) from huge corpus? Thanks!