Rithwik Ediga Lakhamsani

Add Spark DataFrame as a Document Loader

1

# Add Spark DataFrame as a Document Loader This is currently a work in progress PR on adding Spark DataFrames as a Document Loader **(tests haven't been added yet**). Langchain...

Horovod 0.28.1 incompatibility with PyTorch 2.1.0

2

**Environment:** 1. Framework: PyTorch 3. Framework version: 2.1.0 4. Horovod version: 0.28.1 **Checklist:** 1. Did you search issues to find if somebody asked this question before? Yes 2. If your...

WIP: 1. Added troubleshooting guide 2. Added comments on all DBFS locations to introduce UC Volumes Note: We might need to wait until after MLR 14.3 to submit these PRs.

Draft PR for supporting automicrobatching on FSDP2 This isn't added yet because we ran into some hiccups with how FSDP2 handles state transitions. As FSDP2 is stateful, it expects the...

# What does this PR do? This is tied to the changes in https://github.com/databricks/compose-rl/pull/109 to make sure that FSDP2 is correctly supported in compose-rl