zzhang2816

Results 7 comments of zzhang2816

The model can not handle short time series data

Besides this problem, with the apple m1 chip, there will be an "illegal hardware instruction" error.

The model can not handle short time series data

Thank you! Sorry for the later reply. I am not working in the field of time-series data now. The version I used is fedot == 0.3.0, I haven't tried the...

Inference with LoRA weights

Add these lines to inference code ``` if lora_folder != "": unet.load_attn_procs(lora_folder) unet.to(torch.float16) unet.requires_grad_(False) ```

NaN Loss after Forward Pass on Step 4 when Full Fine-Tuning Wan 2.2 5B

@wyyfffff I encountered a similar issue. I reduced the learning rate to 1e-8 and applied gradient clipping with accelerator.clip_grad_norm_(model.trainable_modules(), 1.0). My training uses DeepSpeed ZeRO-2 on four GPUs, and I’ve...

NaN Loss after Forward Pass on Step 4 when Full Fine-Tuning Wan 2.2 5B

I will try it soon, thanks a lot! @wyyfffff

NaN Loss after Forward Pass on Step 4 when Full Fine-Tuning Wan 2.2 5B

I have rewritten the model loading logic using traditional initialization method, but the problem persist. Not sure what is wrong. ``` def load_dit(file_path, model_class, torch_dtype, device): state_dict = {} for...

NaN Loss after Forward Pass on Step 4 when Full Fine-Tuning Wan 2.2 5B

Thanks for sharing! I’ll give it a try. @wyyfffff