Transformers-Tutorials icon indicating copy to clipboard operation
Transformers-Tutorials copied to clipboard

Why do you need custom Features?

Open gordon-lim opened this issue 1 year ago • 1 comments

Why do we need the following in the Fine-tuning LayoutLMv2ForSequenceClassification on RVL-CDIP notebook?

# we need to define custom features
features = Features({
    'image': Array3D(dtype="int64", shape=(3, 224, 224)),
    'input_ids': Sequence(feature=Value(dtype='int64')),
    'attention_mask': Sequence(Value(dtype='int64')),
    'token_type_ids': Sequence(Value(dtype='int64')),
    'bbox': Array2D(dtype="int64", shape=(512, 4)),
    'labels': ClassLabel(num_classes=len(labels), names=labels),
})

gordon-lim avatar Jun 19 '24 16:06 gordon-lim

Hi,

I should actually update this notebook as now we can use the native Image feature rather than Array3D for the image column. Did you try to run the code without defining custom features, and if so, did it work?

NielsRogge avatar Jun 28 '24 10:06 NielsRogge

My apologies for the late response. I was asking because I faced some bug when I did not define custom features as you have. However, I don't recall how I eventually solved the bug (nor what the bug was) but regardless I was curious why we needed that block of code.

gordon-lim avatar Jul 04 '24 22:07 gordon-lim