promptbase
promptbase copied to clipboard
Missing mmlu as valid_dataset even though readme encourages download
(gpt5) [jfreeman@mir-81 promptbase]$ python -m promptbase mmlu usage: main.py [-h] [--subject SUBJECT] [--list_subjects] {gsm8k,humaneval,math,drop,bigbench} main.py: error: argument dataset: invalid choice: 'mmlu' (choose from 'gsm8k', 'humaneval', 'math', 'drop', 'bigbench')
Head of main.py :
import argparse from promptbase.gsm8k import gsm8k from promptbase.humaneval import humaneval from promptbase.math import math from promptbase.drop import drop from promptbase.bigbench import bigbench from promptbase.bigbench.consts import BIGBENCH_SUBJECTS
VALID_DATASETS = ["gsm8k", "humaneval", "math", "drop", "bigbench"]
:/
I'm looking in to this as part of #23