ReWOO Token price human error issue

Hi, I read the paper ReWOO very impressive.

I think I found some human error when counting the token cost on the PWS.py https://github.com/billxbf/ReWOO/blob/9cd0283043ff4be0c9d614fda2789d143ca6ffd1/algos/PWS.py#L66-L68

OpenAI count the input token and output token because they have different pricing policy, but these codes just count the input and output token as same. Maybe I can make a mistake, but I want you to know the issue.

Thanks a lot for the nice research!!

May 19 '24 17:05 gigio1023

Thanks for your kind words! You are right, this function doesn't treat in/out token prices separately. If I remember correctly, openai's API didn't split the in/out token prices at the time of this work. So I'd rather treat it as an estimation for experiments at the time. Now they include token counts in the API's completion object, accessed by completion["usage"]

May 21 '24 15:05 billxbf

Because of your response, now I totally understand about the history of the works. Thanks a lot!!

May 21 '24 15:05 gigio1023

Closing this thread as solved.

Oct 04 '24 00:10 billxbf