Medusa icon indicating copy to clipboard operation
Medusa copied to clipboard

How to run inference of a baseline model without medusa support?

Open kailashg26 opened this issue 1 year ago • 0 comments

Hello! I'd like to learn how to run an inference with a baseline model of Vicuna without Medusa support. Additionally, I’m curious if there has been any analysis done on the memory footprint, and whether Medusa provides any notable improvements compared to the baseline.

kailashg26 avatar Sep 16 '24 21:09 kailashg26