datasets icon indicating copy to clipboard operation
datasets copied to clipboard

Fix rlds `episode_metadata`

Open Tavish9 opened this issue 6 months ago • 10 comments

This PR fixes rlds's build_info when episode_metadata is not None

Reproduce

from tensorflow_datasets.rlds import rlds_base
features = rlds_base.build_info(
    rlds_base.DatasetConfig(
        name="xxx",
        steps={xxx},
        episode_metadata_info={
            "test": tfds.features.Text()
        }
    ),
    self,
)
print(features)
FeaturesDict({
    'steps': Dataset({
        ...
    }),
    'test': Text(shape=(), dtype=string),
}),

Currently, the returned features will get rid of episode_metadata key, which is not consistent with widely used rlds dataset:

Tavish9 avatar Aug 05 '25 11:08 Tavish9

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

google-cla[bot] avatar Aug 05 '25 11:08 google-cla[bot]

@Tavish9 thank for the PR!

IIUC, this changes the dataset info, so you need to change the example generation as well, see the failed tests.

fineguy avatar Aug 17 '25 15:08 fineguy

@fineguy already fix

Tavish9 avatar Aug 18 '25 01:08 Tavish9

@Tavish9 please fix the failed tests.

fineguy avatar Aug 18 '25 08:08 fineguy

I also updated the logic of generate_examples, please check

Tavish9 avatar Aug 18 '25 09:08 Tavish9

@Tavish9 please confirm locally that tests run successfully, this will be easier for the review.

fineguy avatar Aug 19 '25 09:08 fineguy

@fineguy sorry for bothering you. I don't find any steps for local testing in README.

Could you please show me the way?

Tavish9 avatar Aug 19 '25 10:08 Tavish9

@Tavish9 check this documentation: https://www.tensorflow.org/datasets/contribute#add_and_run_unittests

fineguy avatar Aug 19 '25 17:08 fineguy

image

Already passed all tests in rlds.

Tavish9 avatar Aug 20 '25 01:08 Tavish9

@fineguy, it has been a very long time, take a look if you are free, thanks.

Tavish9 avatar Nov 22 '25 03:11 Tavish9