continual-pre-training topic

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)

pjlab-sys4nlp

Llama-3-SynE: A Significantly Enhanced Version of Llama-3 with Advanced Scientific Reasoning and Chinese Language Capabilities | 继续预训练提升 Llama-3 的科学推理和中文能力

RUC-GSAI

Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"

GAIR-NLP