Cathy0908
Cathy0908
some codes in `train.py` is like: ```python if world_size > 1: with torch.no_grad(): all_z = [torch.zeros_like(z) for _ in range(world_size)] torch.distributed.all_gather(all_z, z) all_z[cfg.local_rank] = z z = torch.cat(all_z) ``` I'm...
### What happened + What you expected to happen I want to use ray to do large-scale text data cleaning tasks, and extracted 5 million data from the open source...
Add new operators for llm data augmentation: - GenerateInstructionMapper - OptimizeInstructionMapper - ExtractQAMapper And support vllm infer for these ops.