Researcher problem: Browser needs to login or apply a user agent

[steinhaug]

Describe your issue

See the screenshot below. My issue is that I would like to login to this service, and some other services having same issues. How would this be possible with the current code ? Is there some way of having a "cookie dir" that I could make sure has the appropriate logged in cookies required? Another way would be to have a list of sites and their username and password, together with the login url. This way upon crawling a url on a domain in this list - there could be a login before crawling ? How does the other crawlers solve this ? Im guessing this is a very common problem.

Expected behavior

When Devika is doing research or browsing webpages, it should make sure that certain websites are logged in. Or as from the screen capture from reddit, atleast make sure Devika has a user agent as a browser and not empty.

Screenshots and logs

Configuration

- OS: windwos
- Python version: 3.11
- Node version: v20.14.0
- bun version: 1.1.10
- search engine: google
- Model: cluade

[suryadev777]

I am planing to fix this issue if anyone is already working this issue please indicate to me or i provide my planing below feel free to share your suggestions

my Plan is here

• built a credentials / cookie store
• built a functions that checks current page has need login or not
• if login is needed check if credential is exist use it or else
• built a function that automatically enter the credentials or wait for user to fill it up and store it in credentials / cookie store

[steinhaug]

built a function that automatically enter the credentials or wait for user to fill it

Devika could ask for the credentials if hitting a known site that requires a login, this way the usernames and passwords could be added from the main operation of Devika instead of the configuration - sounds sweet.

built a functions that checks current page...

I was thinking adding the domains as needed manually, maby even having to add the url where login form is. Regarding such a check, how would you distinguish a social media login like reddit from a personal admin login ?

Looking forward to see your work on this.

[suryadev777]

sounds perfect for me !!! could you provide some user stories like login to mid journey generate a image ( we can do it through API but still)