open_clip icon indicating copy to clipboard operation
open_clip copied to clipboard
verified publisher icon mlfoundations

Matryoshka Representation Learning for Open CLIP

Open adityakusupati opened this issue 3 years ago • 4 comments

The PR hardcodes 4 granularities of dimensions to train CLIP on.

While I hope the default hparams will work, it might be good to train it for slightly longer epochs.

adityakusupati avatar Dec 15 '22 22:12 adityakusupati

Hey Aditya thanks for the PR with MRL -- however if you want to make MRL an option it would be good to have a flag so that this PR does not affect default training and its not hard coded. Also, we'd want to do experiments to verify MRL works in this setting, which I can help you get set up on the UW compute -- we have a subset of LAION there.

mitchellnw avatar Dec 15 '22 23:12 mitchellnw

Sure thing Mitchell. I was hard coding it so that Rom could use it to train a model. I can probably cleanup the code and make it a flag so that it does not break training.

I don't think we will be merging this PR anyway. I can run things on LAION subset at UW with the generalized or hard coded variant.

adityakusupati avatar Dec 15 '22 23:12 adityakusupati

sure can you convert to draft in the meantime?

mitchellnw avatar Dec 22 '22 20:12 mitchellnw

Done.

adityakusupati avatar Dec 22 '22 22:12 adityakusupati