evaluate icon indicating copy to clipboard operation
evaluate copied to clipboard

Allow for specify coda device in perplexity evaluation

Open manuelbrack opened this issue 2 years ago • 0 comments

The current implementation of perplexity.compute does not allow to provide a specific coda device (e.g. cuda:1 instead of just cuda.

https://github.com/huggingface/evaluate/blob/8dfe05784099fb9af55b8e77793205a3b7c86465/metrics/perplexity/perplexity.py#L108

Should probably be changed to something along the lines of:

assert device in ["gpu", "cpu", "cuda"] or re.search(r'^cuda:[0-9]+$', device), "device should be either gpu or cpu." 

manuelbrack avatar Apr 08 '24 14:04 manuelbrack