KKelly

> You can do something like this > > ``` > from mistral_inference.model import Transformer > model = Transformer.from_folder(args.model_path, device=f"cuda:0") > model.load_lora("/path/to/lora.safetensors", device=f"cuda:0") > safetensors.torch.save_model(model, "/path/to/merged.safetensors") > ``` When I...

A single GPU doesn't seem to be able to load the entire Mixtral-8x7B-v0.1-Instruct model, how should I merge the model using multiple cards? Thanks!

@matheus-prandini Hi! I met the same problem. I merged the base model and lora by the script below ``` python utils/merge_lora.py \ --initial_model_ckpt Mixtral-8x7B-v0.1-Instruct_extended/consolidated.00.pth \ --lora_ckpt mistral_8x7B/snapshots/f7ef571615ca95187c38d1e5451758532cf795c3/checkpoint_000948/consolidated/lora.safetensors \ --dump_ckpt Model/merged_model.safetensors...

Hi, thanks for your remarkable work! I'm attempting to visualize the results using the script mentioned above. However, I can not obtain the true visualized outcomes. The relevant revised code...

@ruili3 Hi, I utilized the `save_concat_res(output_dir, sample_pred_disp, sample_img, sample_gt_depth)` function from `plot_depth_utils.py`, and made revisions to the code in `evaluate.py`. Here is the result I obtained. (I have not modified...