Open-Sora
Open-Sora copied to clipboard
hpcaitech
Open-Sora: Democratizing Efficient Video Production for All
为什么魔改DiT？
 DiT原文结构采用了AdaLN，condition内的class label不包含sequence维度，因此需要加入cross attention才能处理复杂文本序列和patch序列的关系。但是本项目的实现魔改了这个结构，把patch的self attention直接修改为patch序列和text condition的cross attention，忽略了patch序列的self attention，这样做的目的是？ 忽略patch的self attention，不会有帧生成的质量问题吗？
I am running the inference and this is what I am getting. The command that I ran:` python sample.py -m "DiT-XL/2" --text "a person is walking on the street" --ckpt...
`python sample.py -m "DiT/XL-2" --text "a person is walking on the street" --ckpt /path/to/checkpoint --height 256 --width 256 --fps 10 --sec 5 --disable-cfg` What is the path to the checkpoint?...
FileNotFoundError: [Errno 2] No such file or directory: '/root/miniconda3/lib/python3.10/site-packages/colossalai/kernel/extensions/csrc/cuda/cpu_adam.cpp' 这是服务器的信息：Linux #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux 找不到这个模块:ModuleNotFoundError: No module named 'colossalai._C.cpu_adam_x86'
----------------- nvcc -V nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Mon_Apr__3_17:16:06_PDT_2023 Cuda compilation tools, release 12.1, V12.1.105 Build cuda_12.1.r12.1/compiler.32688072_0 ---------------- nvidia-smi NVIDIA GeForce GTX...
[03/07/24 14:50:30] INFO colossalai - colossalai - INFO: train.py:155 main INFO colossalai - colossalai - INFO: Dataset contains 105060 samples [03/07/24 14:52:00] INFO colossalai - colossalai - INFO: train.py:165 main...
我现在手头只有一台A100 40G、128G内存、1T的NVME硬盘，官方说可以在8块A100 80G上训练，如果采用ZeRO-Infinity技术，我的这个机器应该也可以训练，请问我的这个硬件可以支持全参数训练吗？ 另外，想问一下，支持LoRA等PEFT微调方法吗？
Runing the inerence sample `python sample.py -m "DiT/XL-2" --text "a person is walking on the street" --ckpt /path/to/checkpoint --height 256 --width 256 --fps 10 --sec 5 --disable-cfg` I got the...
