Varun Gumma

# Before submitting - [*] Was this discussed/approved via a Github issue? (no need for typos, doc improvements) - [*] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)? - [*] Did you...

Knowledge Distillation

10

Is there any implementation of Knowledge Distillation in Fairseq? I need to distilled a large multilingual Transformer model and I am not finding any suitable implementation for it here.

Integrated UrduHack and IndicNLP Resources directly into the module

2

Integrated `UrduHack` and `indic_nlp_resources` directly into the module. This negates the need to install the TensorFlow-based Urdu library which was causing some conflicts. Also, the resources are directly added to...

Difference between cli and corpus_score

1

I am attempting to compute chrF++ for a set of predictions and references. If I use `sacrebleu` cli (`sacrebleu ref.eng_Latn.tok < pred.eng_Latn.tok -m bleu chrf --chrf-word-order 2`), I find a...

Missing documentation on how to train a model

22

Is there any documentation or `examples` that I can refer to train a transformer model from scratch using `fairseq2`? The `examples` folder in the repository seems empty.

Since LORA is implemented in fairseq2, it would be good to have general adapter modules available as part of PEFT. Can someone please add them to the repository?

@lucidrains, it would be really helpful to have an implementation of YaRN [(Peng _et al._)](https://openreview.net/forum?id=wHBfxhZu1u) in this repository as well.