Does this model benefit from a GPU

Open geraldthewes opened this issue 11 months ago • 2 comments

I see no documentation on how to leverage one. Is this because it would not help?

Mar 04 '25 14:03 geraldthewes

It does! You can see here how to set it onto the GPU for inference: https://github.com/urchade/GLiNER/blob/9068a2f5ae854eb6288c24ae3d7f36e9d84c06ba/gliner/model.py#L107

The device can be set as a model property.

Mar 16 '25 21:03 wjbmattingly

Yes, Gliner is much more performant on GPU. Base on the test done on my side, it will run about 250 times faster on GPU then on CPU. For CPU base, spacy would be a better choice.

Oct 17 '25 13:10 jpmallette