benbench icon indicating copy to clipboard operation
benbench copied to clipboard

verified publisher icon GAIR-NLP
→

Metadata

Benchmarking Benchmark Leakage in Large Language Models

Results 4 benbench issues
Sort by recently updated
recently updated
newest added

When I use the ppl indicator, it shows nan

3 comment

orgn-GSM8K-train Average_ppl_accuracy: nan PPL of xxxx GSM8K_rewritten-test-1: nan GSM8K_rewritten-test-2: nan GSM8K_rewritten-test-3: nan GSM8K_rewritten-train-1: nan GSM8K_rewritten-train-2: nan GSM8K_rewritten-train-3: nan orgn-GSM8K-test: nan orgn-GSM8K-train: nan why this happens?

MrZhang1996 avatar MrZhang1996

Evaluate new proprietary LLMs through APIs: GPT4

pfliu-nlp avatar pfliu-nlp

Multi-GPU Support

Are the ngram and ppl scripts currently capable of leveraging multi-GPU setups for inference? If not, are there any planned updates or workarounds that might support this feature? Additionally, is...

Wyyyb avatar Wyyyb

error: RPC failed; curl 92 HTTP/2 stream 0 was not closed cleanly: CANCEL (err 8) error: 5929 bytes of body are still expected fetch-pack: unexpected disconnect while reading sideband packet fatal: early EOF fatal: fetch-pack: invalid index-pack output

error: RPC failed; curl 92 HTTP/2 stream 0 was not closed cleanly: CANCEL (err 8) error: 5929 bytes of body are still expected fetch-pack: unexpected disconnect while reading sideband packet...

hellocrystal avatar hellocrystal

← Metadata

36
Stars
1
Forks
Watchers

Owner

verified publisher icon GAIR-NLP

Metadata

Benchmarking Benchmark Leakage in Large Language Models

Back