evaluate
evaluate copied to clipboard
HuggingFace evaluate-cli create command results Error: "datasets[0]" must be a string
I follow the official documentation of evaluate 0.4.0 to freshly create a custom metric in an empty folder with the following command:
evaluate-cli create "My Metric" --module_type "metric"
The command returns the following error:
Error: "datasets[0]" must be a string
The command creates these files but I am unsure whether the files are intact:
app.py my_metric.py README.md requirements.txt tests.py
How to resolve this error?
using evaluate version 0.4.0
Check your README.md, then there is datasets in YAML style. Remove that.
from
---
title: My metric
datasets:
-
tags:
- evaluate
- metric
description: "TODO: add a description here"
sdk: gradio
sdk_version: 3.19.1
app_file: app.py
pinned: false
---
to
---
title: My metric
tags:
- evaluate
- metric
description: "TODO: add a description here"
sdk: gradio
sdk_version: 3.19.1
app_file: app.py
pinned: false
---
Where exactly is this yaml file located?