interactdiffusion icon indicating copy to clipboard operation
interactdiffusion copied to clipboard
Published 10 months ago • verified publisher icon jiuntian

Custom Dataset

Open Hayeon-kimm opened this issue 1 year ago • 3 comments

Thank you for sharing your research in this code. Thanks to you, I am studying a lot. I want to conduct an experiment on custom dataset. If you look at the HICO_DET_CLIP you share, I think I need 'action' image_embedding / text_embedding as well. To use it on custom dataset, can I get the action bbox, cut it, and pass it through clip embedding correspond to the action image embedding you provided?

Hayeon-kimm avatar Jul 02 '24 12:07 Hayeon-kimm

Yes, I think you can obtain the clip embeddings in such way. In InteractDiffusion, only text_embedding is used.

jiuntian avatar Jul 02 '24 14:07 jiuntian

If you okay, can you share your process_grounding.py code for HICO-DET? To get embedding for custom data, I make the loader for this. But, your hico-det-clip and gligen.tsv is little different. So, I want to show your preprocess step for HICO-DET.

Hayeon-kimm avatar Jul 03 '24 10:07 Hayeon-kimm

You may refer to extract_embedding.py.zip.

jiuntian avatar Jul 04 '24 04:07 jiuntian