evaluate icon indicating copy to clipboard operation
evaluate copied to clipboard

HuggingFace evaluate-cli create command results Error: "datasets[0]" must be a string

Open shivanraptor opened this issue 2 years ago • 2 comments

I follow the official documentation of evaluate 0.4.0 to freshly create a custom metric in an empty folder with the following command:

evaluate-cli create "My Metric" --module_type "metric"

The command returns the following error:

Error: "datasets[0]" must be a string

The command creates these files but I am unsure whether the files are intact:

app.py  my_metric.py  README.md  requirements.txt  tests.py

How to resolve this error?

using evaluate version 0.4.0

shivanraptor avatar Nov 07 '23 02:11 shivanraptor

Check your README.md, then there is datasets in YAML style. Remove that.

from

---
title: My metric
datasets:
-  
tags:
- evaluate
- metric
description: "TODO: add a description here"
sdk: gradio
sdk_version: 3.19.1
app_file: app.py
pinned: false
---

to

---
title: My metric
tags:
- evaluate
- metric
description: "TODO: add a description here"
sdk: gradio
sdk_version: 3.19.1
app_file: app.py
pinned: false
---

yoon-gu avatar Nov 17 '23 15:11 yoon-gu

Where exactly is this yaml file located?

xxshubhamxx avatar Apr 11 '24 14:04 xxshubhamxx