YLGH
YLGH
Summary: Context: We want to be able to use DistributedDataParallel in a composable way, e.g. after applying this paralleization scheme - the model properties (including methods on the model) should...
my df consists of int_0 through int_12 for, I'm trying to turn these into an array of features, however df["dense_features"] = functional.array_constructor( *[df[int_name] for int_name in DEFAULT_INT_NAMES] ) fails with...
Summary: The current EmbeddingBagCollection/FusedEmbeddingBagCollection are only usable through the DistributedModelParallel wrapper which override common torch.nn.module APIs (named_parameters/state_dict) etc. However, this makes these modules inflexible, and sometimes unusable without using DMP....
Summary: Since we no longer rely on DistributedModelParallel (for composability piece), we need an alternative way of getting the fused optimizer. get_fused_optimizer implements this, logically it's the same as the...
## Desired behavior We want to make TorchRec sharding composable w/ other sharding/parallelism techniques. This practically means that after applying TorchRec sharding model characteristics remain the same (e.g. state_dict() doesn’t...
Summary: Motivation is that we want to OSS quantized comm library, and refactor torchrec quant comms support This diff is rather large (used to be a bunch of small diffs)....
Base training loop examples run cmd `torchx run -s local_cwd dist.ddp -j 1x8 --script train_dlrm.py ` Some TODO items: 1. Add NE/QPS metrics checkpointing 2. Show saving this model and...
Summary: Rename quantized comms config Differential Revision: D37221312