promptbase icon indicating copy to clipboard operation
promptbase copied to clipboard

Missing mmlu as valid_dataset even though readme encourages download

Open jackfreeman88 opened this issue 2 years ago • 1 comments

(gpt5) [jfreeman@mir-81 promptbase]$ python -m promptbase mmlu usage: main.py [-h] [--subject SUBJECT] [--list_subjects] {gsm8k,humaneval,math,drop,bigbench} main.py: error: argument dataset: invalid choice: 'mmlu' (choose from 'gsm8k', 'humaneval', 'math', 'drop', 'bigbench')

Head of main.py :

import argparse from promptbase.gsm8k import gsm8k from promptbase.humaneval import humaneval from promptbase.math import math from promptbase.drop import drop from promptbase.bigbench import bigbench from promptbase.bigbench.consts import BIGBENCH_SUBJECTS

VALID_DATASETS = ["gsm8k", "humaneval", "math", "drop", "bigbench"]

:/

jackfreeman88 avatar Dec 29 '23 20:12 jackfreeman88

I'm looking in to this as part of #23

riedgar-ms avatar Jan 03 '24 20:01 riedgar-ms