exo icon indicating copy to clipboard operation
exo copied to clipboard
Published 3 weeks ago • verified publisher icon exo-explore

Add `--gpu` CLI option so Linux users can choose a single GPU ID for tinygrad

Open freerainboxbox opened this issue 1 year ago • 3 comments

Original Behavior

  1. User has to set VISIBLE_DEVICES in shell for tinygrad to accept choice of GPUs.
  2. Even if set to something other than 0, linux_device_capabilities() will fetch only index 0 (this was hardcoded in).

New Behavior

  1. The user may specify --gpu followed by an integer. If parsed incorrectly or omitted, it will default to 0. VISIBLE_DEVICES in the parent environment is effectively abstracted away from the user, so if they want to pick a device, they can pick only one for the instance.*
  2. Device capabilities will fetched according to this index, and the user will see the correct GPU as confirmation in the TUI (see behavior of topology.py).

*Users can run multiple GPUs at once by running multiple instances of exo. This workaround I wrote may be obsolete if we implement multi-GPU in a single instance of exo, but for now, this will allow users to take advantage of multiple GPUs in one device.

freerainboxbox avatar Jan 30 '25 04:01 freerainboxbox

I am not able to get this to work.

Setup:

1 node with 2x3090 running two instances 1 Mac Studio

Thunderbolt connection.

See attached image.

Screenshot 2025-03-25 at 8 55 41 AM

frenzybiscuit avatar Mar 25 '25 15:03 frenzybiscuit

This would be useful. In the meanwhile to get exo to support multiple GPUs, you could run multiple exo instances on a single machine, each for different GPUs.

pharazon avatar May 23 '25 17:05 pharazon

Also I can't get this to work. I have 2xGTX3060 running on screen. The first exo -instance connects to the cluster ok.

The second exo-instance that has been started by exo --gpu 1 creates it's own cluster with 1 node, but does not connect to the first instance or the cluster.

pharazon avatar May 23 '25 18:05 pharazon

Thanks so much for your contribution and for taking the time to open this PR.

Since this repository has been fully rewritten and the license has changed, I’m closing all existing open PRs to avoid confusion and to align with the new codebase.

I really appreciate your interest in the project, and you’re very welcome to open a new PR against the updated version if you’d like and we look forward to reviewing it!

Evanev7 avatar Dec 18 '25 14:12 Evanev7