table-transformer Fine-tuning dataset size

Hi Everyone, I'm trying to use the pretrained weights of PubTabNet 1M, to fine-tune on my own data.

As you know transformers are data hungry but I'd expect the need for huge amounts of data will be less in fine-tuning.

Any idea about the size of data needed for fine-tuning? will 1k images be enough?

Apr 18 '23 20:04 bely66

@bsmock Would it be hard to get your opinion on that?

Apr 18 '23 20:04 bely66

Hi @bsmock, i am fine tuning on 500k sample dataset for table detection. I'm not getting very accurate results. I would like top know what size of the dataset is sufficient for fine tuning the model for both table detection and table structure recognition. can share your thoughts on this?

May 19 '23 06:05 IrfanSk-AI

@IrfanSk-AI PubTables-1M contains 500k samples (page images) for detection. 500k is enough to train from scratch if your data is high-quality. Could there be a data quality issue?

@bely66 1k could be enough for either detection or structure recognition. The number you need depends on a lot of factors, such as how similar the tables are to tables in PubTables-1M, whether the bounding boxes are annotated consistently, how similar the tables in your data are to each other, etc.

May 19 '23 22:05 bsmock

@bsmock Thanks for your reply Mostly the data are very close to each other, at this stage, I'm training around 400 image The table format and color are very close, the only variation is in the table size and content.

I'm getting a relatively low scores on the dataset AP50: 0.794, AP75: 0.458, AP: 0.472, AR: 0.627

And It's not obvious why this is happening

Also I wanted to ask what's the recommended amount of epochs and hyper-parameter optimization for fine-tuning?

Thanks again

Jun 01 '23 11:06 bely66

Hey @bely66 , could you share what were your parameters for the fine-tuning? Number of epochs and learning rate are parameters that usually you don't want to set too high when fine-tuning so that the model doesn't 'forget' what it saw while pre-training.

Jun 05 '23 20:06 thiagodma

I used the default parameter in structure_config

What would be the recommended weights?

Jun 05 '23 23:06 bely66

As you're training with few data (400 samples) probably it is ok for you to make some experiments but I would start changing just the learning rate and the number of epochs. Try something like a learning rate 40x smaller than the one used on pre-training and 5 epochs. If you think the model is taking to long to converge, increase a bit the learning rate and so on

Jun 06 '23 11:06 thiagodma

Can any of you share any alternate links to the pretrained pubtables weights (pubtables1m_detection_detr_r18.pth) for both detection and TSR? The links in the readme are not working and have been waiting for it since 2 weeks now. Thanks in advance!

Jun 15 '23 06:06 veyyil

Can any of you share any alternate links to the pretrained pubtables weights (pubtables1m_detection_detr_r18.pth) for both detection and TSR? The links in the readme are not working and have been waiting for it since 2 weeks now. Thanks in advance!

Just shared them here https://github.com/microsoft/table-transformer/issues/116

Jun 15 '23 09:06 WalidHadri-Iron

@bsmock Thanks for your reply Mostly the data are very close to each other, at this stage, I'm training around 400 image The table format and color are very close, the only variation is in the table size and content.

I'm getting a relatively low scores on the dataset AP50: 0.794, AP75: 0.458, AP: 0.472, AR: 0.627

And It's not obvious why this is happening

Also I wanted to ask what's the recommended amount of epochs and hyper-parameter optimization for fine-tuning?

Thanks again

can u pls help me on this #181 ... @bely66 @thiagodma

May 16 '24 05:05 tzktok