midscene
midscene copied to clipboard
Driving all platforms UI automation with vision-based model
/Users/hankunfang/workspace/midscene-example/puppeteer-demo/node_modules/@midscene/web/dist/lib/puppeteer.js:1291 throw new Error(`${errorTask == null ? void 0 : errorTask.error} ^ Error: 400 Missing required parameter: 'response_format.json_schema'. (request id: 202412201428294827994785579509) Error: 400 Missing required parameter: 'response_format.json_schema'. (request id: 202412201428294827994785579509)...
[The FAQ](https://midscenejs.com/faq.html#can-midscene-smartly-plan-the-actions-according-to-my-one-line-goal-like-executing-tweet-hello-world) mentioned > If you require a 'goal-to-task' AI planning tool, you can develop one based on Midscene. But I'm not sure how to leverage Midscene to make this,...
Midscene is currently limited to interacting with the current viewport, but in real-world scenarios, we need to deal with elements not visible in the current viewport (e.g., elements that need...
执行操作异常
Failed to plan actions with context: undefined AssertionError [ERR_ASSERTION]: Failed to plan actions with context: undefined at Sr (chrome-extension://gbldofcpkknbggpkmbdaefngejllnief/lib/popup.js:1:2816013) at async Object.executor (chrome-extension://gbldofcpkknbggpkmbdaefngejllnief/lib/popup.js:1:2873406) at async Vt.flush (chrome-extension://gbldofcpkknbggpkmbdaefngejllnief/lib/popup.js:1:2790577) at async N.action...
使用 qwen 配置为: OPENAI_API_KEY sk-xxxxxxxxxxx OPENAI_BASE_URL https://dashscope.aliyuncs.com/compatible-mode/v1 MIDSCENE_MODEL_NAME qwen-vl-max-lates 运行 plawright-demo e2e 文件夹中的 “todo-mvc-zh.spec.ts” 文件,即: import { expect } from "@playwright/test"; import { test } from "./fixture"; test.beforeEach(async ({ page...
I tried the page with the iframe and found that its image analysis could not cover the contents of the iframe, as shown in the following figure:   I'm...
# Background I wish to experiment with other LLM models like Tongyi or Claude, but it's not feasible. https://github.com/web-infra-dev/midscene/blob/863e5a54d0380004038b6012d4810d79a437a401/packages/midscene/src/ai-model/openai/index.ts#L94 # Aspiration Provide a solution akin to [browser-use](https://github.com/gregpr07/browser-use), allowing the integration...