crawl4ai
crawl4ai copied to clipboard
Issue running with Colab
when I run the colab I get the following error
[INIT].... → Crawl4AI 0.4.247
[ERROR]... × https://crawl4ai.com/... | Error:
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ × Unexpected error in _crawl_web at line 833 in execute_hook (../usr/local/lib/python3.10/dist- │
│ packages/crawl4ai/async_crawler_strategy.py): │
│ Error: custom_hook_workflow.<locals>.<lambda>() got an unexpected keyword argument 'url' │
│ │
│ Code context: │
│ 828 hook = self.hooks.get(hook_type) │
│ 829 if hook: │
│ 830 if asyncio.iscoroutinefunction(hook): │
│ 831 return await hook(*args, **kwargs) │
│ 832 else: │
│ 833 → return hook(*args, **kwargs) │
│ 834 return args[0] if args else None │
│ 835 │
│ 836 def update_user_agent(self, user_agent: str): │
│ 837 """ │
│ 838 Update the user agent for the browser. │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
when this piece of code is executed. let me know if I missed some instruction or setting
async def custom_hook_workflow(verbose=True):
async with AsyncWebCrawler(config=BrowserConfig( verbose=verbose)) as crawler:
# Set a 'before_goto' hook to run custom code just before navigation
crawler.crawler_strategy.set_hook("before_goto", lambda page, context: print("[Hook] Preparing to navigate..."))
# Perform the crawl operation
result = await crawler.arun(
url="https://crawl4ai.com",
config=CrawlerRunConfig(cache_mode=CacheMode.BYPASS)
)
print(result.markdown_v2.raw_markdown[:500].replace("\n", " -- "))