Need help for instruction translation

Open XvHaidong opened this issue 10 months ago • 1 comments

Dear BrowserGym team, I really like your work. Is it possible to translate the user's clicking behavior into instructions defined by browsergym? For example, if I click on a button in the browser, the backend could output: The user's action: click(ID)

Mar 13 '25 03:03 XvHaidong

Hi @XvHaidong, thank you for your interest!

This feature is not supported by bgym as of now, and isn't in our plans atm. Can you confirm this @recursix ? To implement this feature, it would most likely involve using EventListeners to record the actions used, and then remap those actions to BrowserGym actions.

Mar 25 '25 20:03 TLSDC

Exact. To fully suport this we would need something that maps playwrigth recording of user action to browsergym actions. This would be tedious to achieve.

Jul 16 '25 20:07 recursix