Not finetuned generalised checkpoint
Hi Authors,
There are checkpoints available for models finetuned for the generation of Chairs, tables, lamps, and benches.
Can you please share the checkpoint file of the generalized "not finetuned" model for the 3 variations of the number of parameters?
What does the "testing the model from scratch" prompt mean in the code. Does it mean it is taking the non fine-tuned model for inference?
We provide pre-trained weights for different sizes of models (i.e. 125m, 350m, and 1.3b) on huggingface🤗. Download the pre-trained weights from the links below to replace the pytorch_model.bin files in the corresponding folders under the ./mesh-xl/ folder. The model details are shown below:
| Model Size | #Layers | #Heads | $d_\text{model}$ | $d_\text{FFN}$ | GPU Hours | Download Link |
|---|---|---|---|---|---|---|
| 125M | 12 | 12 | 768 | 3072 | 1944 | download link |
| 350M | 24 | 16 | 1024 | 4096 | 6000 | download link |
| 1.3B | 24 | 32 | 2048 | 8192 | 23232 | download link |
The "testing the model from scratch" is only used for debug propose.
For more information, please refer to the readme.md file.