Using Hero bypassing Cloudflare in Docker

[s41r4j]

Issue Title: Unable to Bypass Cloudflare in Docker Container with Ulixee

Description:

I am currently facing an issue while attempting to bypass Cloudflare when running the Ulixee's Hero within a Docker container. Despite my efforts, I have been unable to successfully bypass Cloudflare, which is preventing me from using the application effectively.

Screenshot from Docker Container	Script output

Details:

• Project: Ulixee
• Environment: Docker container
• Issue: Unable to bypass Cloudflare
• Expected Behavior: Successful bypass of Cloudflare to run next automation.
• Actual Behavior: Cloudflare remains active, unsolved.

Additional Information:

• I have tried using both the ulixee/ulixee-cloud and ubuntu Docker images, but the issue persists.
• The Dockerfiles I am using are as follows:

Dockerfile (using ulixee-cloud):

FROM ulixee/ulixee-cloud:latest

# Install Chrome dependencies with correct package names for node:20-slim
RUN apt-get update && apt-get install -y \
    wget \
    gnupg \
    ca-certificates \
    procps \
    libxss1 \
    libxtst6 \
    libxrandr2 \
    libasound2 \
    libpangocairo-1.0-0 \
    libatk1.0-0 \
    libcairo-gobject2 \
    libgtk-3-0 \
    libgdk-pixbuf-2.0-0 \
    libxcomposite1 \
    libxcursor1 \
    libxdamage1 \
    libxext6 \
    libxfixes3 \
    libxi6 \
    libxrender1 \
    libxtst6 \
    fonts-liberation \
    libnss3 \
    lsb-release \
    xdg-utils \
    && rm -rf /var/lib/apt/lists/*

# Create app directory
WORKDIR /usr/src/app

# Copy package files first to cache layer
COPY package*.json ./

# Install dependencies
RUN timeout 300 npm ci —verbose && npm cache clean —force
# RUN apt-get update && apt-get install -y xvfb

# Copy your scraping script
COPY script.js .

# (Optional) Fix chrome-sandbox permission just in case
RUN chmod 4755 /root/.ulixee/chrome/*/chrome-sandbox || true

# Default command
CMD [“node”, “script.js”]
# CMD [“sh”, “-c”, “xvfb-run -a node script.js 2>&1”]

Dockerfile (using ubuntu):

# stable ubuntu image
FROM ubuntu:24.04

# Install Node.js and npm plus required browser dependencies
RUN apt-get update && apt-get install -y \
    curl \
    gnupg \
    ca-certificates \
    libglib2.0-0 \
    libgobject-2.0-0 \
    libnss3 \
    libnspr4 \
    libdbus-1-3 \
    libatk1.0-0 \
    libatk-bridge2.0-0 \
    libcups2 \
    libgio-2.0-0 \
    libxcb1 \
    libxkbcommon0 \
    libatspi2.0-0 \
    libx11-6 \
    libxcomposite1 \
    libxdamage1 \
    libxext6 \
    libxfixes3 \
    libxrandr2 \
    libgbm1 \
    libpango-1.0-0 \
    libcairo2 \
    libasound2t64 \
    fonts-liberation \
    libappindicator3-1 \
    libdrm2 \
    libgtk-3-0 \
    libxss1 \
    libxtst6 \
    xdg-utils \
    wget \
    procps \
    htop \
    strace \
    lsof \
    && curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
    && apt-get install -y nodejs \
    && rm -rf /var/lib/apt/lists/*

# Create app directory
WORKDIR /usr/src/app

# Copy package files first to cache layer
COPY package*.json ./

# Install dependencies
RUN timeout 300 npm ci —verbose && npm cache clean —force
# RUN apt-get update && apt-get install -y xvfb

# Copy your scraping script
COPY script.js .

# (Optional) Fix chrome-sandbox permission just in case
RUN chmod 4755 /root/.ulixee/chrome/*/chrome-sandbox || true

# Default command
CMD [“node”, “script.js”]
# CMD [“sh”, “-c”, “xvfb-run -a node script.js 2>&1”]

Request:

Could you please provide any guidance or recommendations on how to bypass Cloudflare in this Docker environment? Any tips or solutions you might have would be greatly appreciated, as I am eager to resolve this issue and continue using your project effectively.

Best regards,
Sairaj J

[s41r4j]

Hey @blakebyrnes , does hero support docker!?

[blakebyrnes]

Hey @blakebyrnes , does hero support docker!?

We provide a docker, but Linux in general has a lot of flaws that make it hard to operate (as an example) like Windows (fonts, gpus, etc, etc). If you're hitting issues with a target, it's usually better to just lock down to a single OS in your userAgent string and run as the host OS.

[s41r4j]

Yes, indeed! (would like to keep this open, incase someone finds a solution!)

[adsmediatg]

Hi, I can help you solve the problem with captcha. Write to me in Telegram @sakulchai_place or in my mail [email protected]

[coud]

I'm also facing a similar issue with google search where ulixee is not detected on a linux desktop in host os installed from a bootable iso and amazingly works in a VM environment as well as long as it is desktop from the bootable iso.

However, ulixee is detected on any linux server regardless of installation, whether it is host, VM or docker, and even not working on a linux server first then a desktop environment from a package manager.

[vikito755]

@s41r4j, this example should help you. Use at your own discretion.

import Hero, { ConnectionToHeroCore } from "@ulixee/hero";

const hero = new Hero({ mode: "development", showChrome: true, showChromeInteractions: true, noChromeSandbox: true, });

await hero.goto(url); await hero.waitForMillis(10000);

const iframeElement = await hero
  .querySelector("#uMtSJ0 > div:nth-child(1) > div:nth-child(1)")
  .shadowRoot.querySelector("iframe");
const iframe = await hero.getFrameEnvironment(iframeElement);

const input = await iframe?.document.body.shadowRoot.querySelector(
  "label > input"
);
const iframeBounds = await iframeElement.getBoundingClientRect();
const inputBounds = await input?.getBoundingClientRect();
// const textShadow = await iframe?.document.body.shadowRoot.querySelector('span.cb-lb-t').textContent

const finalX = (await iframeBounds.left) + (await inputBounds!.left);
const finalY = (await iframeBounds.top) + (await inputBounds!.top);
hero.interact({ click: [finalX, finalY] });

How it works:

1. Go to URL.
2. Wait 10 seconds (or however much for Cloudflare to load)/
3. Select the iframe inside the shadow root.
4. Switch context to the iframe.
5. Get the input inside the shadow label.
6. Calculate the X,Y position, then click.