sprint_statistics() method gives KeyError: 'test_loss' in 0.2 version

[shabir1]

I got an error while calling sprint_statistics() in autopytorch version 0.2

model.sprint_statistics()

error:
...
cost_info = run_value.additional_info[f'{inference_name}_loss']
KeyError: 'test_loss'

[ravinkohli]

Could you provide the code to reproduce this error?

[shabir1]

@ravinkohli Thank you for your quick response, the below are the code snipt, the same code i was using in the previous version (0.1.1)

model = TabularClassificationTask(
                    ensemble_size=5,
                    ensemble_nbest=5,
                    max_models_on_disc=5
                )
model.search(
                X_train=X_train,
                y_train=y_train,
                total_walltime_limit=100,
                memory_limit=4072,
                enable_traditional_pipeline=False
            )
ss = model.sprint_statistics()  # Error is on this statement, In 0.1.1 version it  is working but 0.2 version it throws error.
print(ss)

[ravinkohli]

Okay, I see. Thanks for pointing out this issue. We'll release a version with a bug fix for this ASAP. If you can, you can use version 0.1.1 till then. Or, you can simply pass X_train and y_train or some of it to the test data as well, because I think the issue is that model.sprint_statistics() now assumes that there will always be test data.

[ravinkohli]

Okay, I see. Thanks for pointing out this issue. We'll release a version with a bug fix for this ASAP. If you can, you can use version 0.1.1 till then. Or, you can simply pass X_train and y_train or some of it to the test data as well, because I think the issue is that model.sprint_statistics() now assumes that there will always be test data.

Test data will not interfere with the optimisation process, it will only add to the time it takes for a configuration as we will also predict using the test data. Sorry for the inconvenience.

[ravinkohli]

Hi, we have fixed this issue in the latest release. You can install it using pip install autoPyTorch==0.2.1. I am closing this issue for now.