Generation working on Macbook with Python 3.11, Torch 2.5dev, Numpy 1.26.4 using MPS

[Grant-CP]

Intro

I pushed the code that I'm using to run the model on my macbook GPU here: https://github.com/Grant-CP/motion-diffusion-model-mps-hack

I wanted to try to get it working without using the requirement.txt file to make minimal dependencies, and I also wanted to have it run using MPS instead of an Nvidia GPU.

The only library I made a concession for was matplotlib. I had it working temporarily on the newest version but then I broke it, so it definitely can work. Fix from https://github.com/GuyTevet/motion-diffusion-model/issues/6 which can work on matplotlib 3.9 is included but commented out for now.

Note that my repo pulls chumpy from my github instead of pypi.

Summary of Changes

Review this commit if you are interested: https://github.com/Grant-CP/motion-diffusion-model-mps-hack/commit/25930b79ff2562e52b8bf2d85d109bf67a0ab0ba

Some changes are just replacing .cuda calls with platform agnostic alternatives and things like that. I did have to reduce the precision of the timestep embeddings from float64 to float32. I also set it temporarily to only accept text conditioning.

I'm happy to discuss any changes that I made.

Performance

Computer: M1 MacBook Pro 32 GB.

For the default of 120 frames with text conditioning, it takes me about 20 seconds for the 1k step model and .9 seconds for the 50 step model. Note that the overall time is a bit more than 1 second per gen because of matplotlib rendering and loading times.

VRAM usage is negligible except if I try to load the training data, in which case my computer gets overloaded and tries to use all of the swap possible. Probably just the hacky way I was trying to load it.

from data_loaders.get_data import get_dataset_loader
data = get_dataset_loader(name='uestc', batch_size=1, num_frames=10)
for motion, cond in data:
    sample_motion = motion
    sample_cond = cond
    break

Visuals

Matplotlib renders work, but SMPL rendering is VERY SLOW. I recommend trying to follow the steps from https://github.com/HanchaoLiu/ProgMoGen where they use TEMOS blender-based rendering.

I tried building mesh and some other tools related to SMPLx and it was a disaster on python 3.11. The other option is to try and get a .npz file to be created by this tool that is compatible with SMPLx. The SMPLx blender extension lets you import a .npz file which would be the easiest alternative. Please ping me if you figure out a nice low-dependency or native way to make that happen.