syed-al

[Bug] Error on `agenta init`

2

**Describe the bug** A clear and concise description of what the bug is. Unable to set-up the initial `agenta init` **To Reproduce** Steps to reproduce the behavior: 0. [Get an...

Remove Headers, Footers, External Links and their related data

18

Hi, Thanks for this great work. I have been playing around with this, to crawl webpages and get content in markdown format, which can be used to provide to LLMs...

Issue with website with anti-bot detection.

7

Hi, I am facing this error, quite a few times for multiple articles. ``` [ERROR] 🚫 arun(): Failed to crawl https://www.washingtonpost.com/technology/2024/10/31/openai-chatgpt-search-ai-upgrade-google, error: [ERROR] 🚫 crawl(): Failed to crawl https://www.washingtonpost.com/technology/2024/10/31/openai-chatgpt-search-ai-upgrade-google: Page.goto:...