Jiarui Yao

Armo-rm env set-up and data processing

1

Hi, I plan to reproduce the armo-rm results but haven't found the env requirements. Is it the same as the bt model? Currently I used the bt model env but...

There's an error while I ran the generation code. For example, xlora_model.generate(torch.randint(100, 1000, (1, 8)).to('cuda'), max_new_tokens=1) throws: RuntimeError: The expanded size of the tensor (16) must match the existing size...

add fsdp sft checkpoint manager

2

### Checklist Before Starting - [x] Search for similar PR(s). ### What does this PR do? Add checkpoint manager to support save & load checkpoints for fsdp sft trainer. ###...