is cpu peak_mem monitoring work?

[dlfrnaos19]

Thank you for the amazing project!

I was on checking opt-30B model with the provided code in readme.

python3 -m flexgen.flex_opt --model facebook/opt-30b --percent 0 100 100 0 100 0

and the result is

as I watched the progress, peak memory was 95/126GB, So I wonder is this right or, bug?

any keyword would help me, thanks!

[henrywoo]

this is not working as you expected. it is calculated by tensor size :-)

[dlfrnaos19]

thanks for your help!