anj-s

Results 10 comments of anj-s

Support Tensor parallel or pipeline parallel out of the box

This is something we have on our roadmap. Is it a different effort from what we have planned? 1. We need to land the tensor parallel code that is already...

Add tests for MultiplePadDataset class

I think this says "Documentation" but you meant tests? Can you update the description?

Model_parallel=2 and 2 gpus on FAIR cluster: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:1! (when checking arugment for argument index in method wrapper_index_select)

can you specify --max-tokens?

Model_parallel=2 and 2 gpus on FAIR cluster: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:1! (when checking arugment for argument index in method wrapper_index_select)

Can you try running this on 8 GPUs? I have a hunch of what might be wrong...

Model_parallel=2 and 2 gpus on FAIR cluster: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:1! (when checking arugment for argument index in method wrapper_index_select)

Lets try world size = 8 and MP size =2. Can you also confirm model params and inputs are on the same device before the FW pass? BTW the error...

Model_parallel=2 and 2 gpus on FAIR cluster: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:1! (when checking arugment for argument index in method wrapper_index_select)

Glad you are unblocked :) but we should still not fail if TP=2 and world size =2. Can you update the title of the issue to reflect this? I (or...

use torchDynamo instead of tracing to get graph?

Yes, definitely. We do plan to support torchdynamo as the frontend in the near future.

Vertex-AI Authentication does not prompt for an oauth screen

Can you use /auth and use the Login with Google option which will map to using your Code Assist license. See instructions [here](https://github.com/google-gemini/gemini-cli/blob/main/docs/cli/authentication.md).

Model sometimes loops or continues in thinking mode for a very long time

We should also fix the exist status to represent this more effectively.

Evaluate quality of edit tool and improve it

Other related issues: b/427299140