TATS
TATS copied to clipboard
songweige
Official PyTorch implementation of TATS: A Long Video Generation Framework with Time-Agnostic VQGAN and Time-Sensitive Transformer (ECCV 2022)
Dear author: In the training of Interpolation Transformer, given the latent space is 5 * 16 * 16, I found the first 16 * 16 and the last 16 *...
Thanks for the awesome project. How can I train final model condition on text?
Few Queries..... (a)Can you Please provide the evaluation code for reproducing Table 1(a), 1(b), 1(c) and 1(d). (b)Can you Please let me know the total computation hours needed to train...
Great work and thanks for kindly releasing both codes and pretrained models! I could access all the checkpoints except the one on Mugen dataset. Could you please upload it to...
I have a few queries regarding the CausalSelfAttention module in the codebase. 1. In lines https://github.com/songweige/TATS/blob/main/tats/modules/gpt.py#L100C13-L102C86, ``` mask[:, :config.n_unmasked+1] = 1 mask[:, -config.n_unmasked+1:] = 1 mask[-config.n_unmasked+1:, config.n_unmasked+1:-config.n_unmasked+1] = 0 ```...
I try to train VQGAN on UCF-101 dataset with 4 A100s, and 24 samples for each device, recontruction and perceptual loss could converge normally, until the discriminator is opened in...
Hi, Thanks for your great work. When I train the mmnist dataset with a downsampling parameter of 4, 8，8， mmnist dataset's input and output in T-dimension are both 10, so...
Hi, Congratulations for your excellent work! I am interested in using audio to generate video. I found that the scripts you provided for audio conditioning generation is just using audio...
https://github.com/songweige/TATS/blob/03bbce1daa3da7e9bff8221e9ec433df9484817f/tats/data.py#L281 In the above line there's a `BalancedRandomSampler` class that does not seem to be defined or imported anywhere, which causes an error when I try to create the dataloader...
Hello, I’m currently exploring the functionality of the audio-to-video script in this repository and would like to understand how the audio features are extracted as part of the process, specifically...
