Alex Havrilla

Results 17 comments of Alex Havrilla

caffe2 error in forward method when using fsdp

Thanks! I can run generate in accelerate with this setup after changing the wrapping policy: ``` compute_environment: LOCAL_MACHINE deepspeed_config: {} distributed_type: FSDP fsdp_config: fsdp_auto_wrap_policy: SIZE_BASED_WRAP fsdp_backward_prefetch_policy: BACKWARD_PRE min_num_params: 2000 offload_params:...

Using calculate_frechet_distance.py, The FID between cifar-10 and the standard npz provided by TTUR, the FID is 15. Any idea why?

Any update on this?

initial commit for trlx LORA support

@ethankim00 just a gentle push on when you expect to finish this?

Add LORA support to TRLX

Closing this.

How to attribute different rewards to parts of the same rollout with PPO?

From the PPO perspective each token receives a reward given by the kl-distance of the fine-tuned model to the reference model plus the score provided by the user provided `reward_fn`...

Amos optimizer support

Let us know if you run into any issues here or on the trlx channel.

Benchmark suite

# Ideas for tasks - web searching: wikipedia race - chess - A chess DT is not trained on natural language, it’s trained on a formal language encoding chess moves....

Inference pipeline

Depends on #529

8-bit inference (#512)

@glerzing Do you have an example run using 8bit?

8-bit inference (#512)

@PhungVanDuy If you have time can you help to debug this? I think having lower precision inference and training options will be very useful.