[Bug]: Cannot bypass automated crawler detection by creating User Data Directory

[Hnam29]

crawl4ai version

v0.5.x

Expected Behavior

I follow the instruction in "Creating a User Data Directory (Command-Line Approach via Playwright)" to save the profile data in user_data_dir. browser_config = BrowserConfig( headless=False, verbose=True, use_managed_browser=True, user_data_dir="/Users/{my_name}/my_chrome_profile", browser_type="chromium" )

Current Behavior

The cloudflare still displays the checkbox captcha

Is this reproducible?

Yes

Inputs Causing the Bug

Steps to Reproduce

Code snippets

OS

macOS

Python version

3.11

Browser

Chrome

Browser version

No response

Error logs & Screenshots (if applicable)

URL: https://ahrefs.com/website-authority-checker/