langflow-ai
feat: Add AgentQL integration
AgentQL is a query language and a set of supporting developer tools designed to identify web elements and their data using natural language and return them in the shape you define. Added AgentQL data extraction component. See: https://docs.agentql.com/rest-api/api-reference
This pull request introduces a new component, AgentQL, to both the backend and frontend of the project. The changes include the implementation of the AgentQL component, its integration into the frontend, and the addition of a new icon for AgentQL.
Backend changes:
-
src/backend/base/langflow/components/agentql/__init__.py: Added theAgentQLimport and included it in__all__. -
src/backend/base/langflow/components/agentql/agentql_api.py: Implemented theAgentQLclass, which uses the AgentQL API to extract structured data from a given URL. This includes defining inputs, outputs, and thebuild_outputmethod for making API requests and handling responses.
Frontend changes:
-
src/frontend/src/icons/AgentQL/AgentQL.jsx: Added the SVG component for the AgentQL icon. -
src/frontend/src/icons/AgentQL/index.tsx: Created a forward reference component for the AgentQL icon. -
src/frontend/src/utils/styleUtils.ts: Integrated the AgentQL icon and added AgentQL to the sidebar bundles and node icons. [1] [2] [3]
cc @ogabrielluiz for feedback and thoughts. Plus, can you point me to the right way to add examples for Langflow + AgentQL, please?
@colriot Thank you for the pull request. We will test it and let you know if any changes are required.
I was trying out the example at https://github.com/tinyfish-io/agentql/blob/main/examples/python/run_script_online_in_google_colab/main.ipynb.
I received a timeout error. If possible, could you also share a sample flow that works well and look into this timeout issue?
Error:
"""" ReadTimeout: The read operation timed out
The above exception was the direct cause of the following exception:
╭───────────────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────────────╮
│ /Users/edwin.jose/Documents/GitHub/langflow/src/backend/base/langflow/graph/vertex/base.py:709 in _build_results │
│ │
│ 706 │ │ self, custom_component, custom_params, base_type: str, *, │
│ fallback_to_env_vars=False │
│ 707 │ ) -> None: │
│ 708 │ │ try: │
│ ❱ 709 │ │ │ result = await initialize.loading.get_instance_results( │
│ 710 │ │ │ │ custom_component=custom_component, │
│ 711 │ │ │ │ custom_params=custom_params, │
│ 712 │ │ │ │ vertex=self, │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/src/backend/base/langflow/interface/initialize/loading.py:68 in get_instance_results │
│ │
│ 65 │ │ if base_type == "custom_components": │
│ 66 │ │ │ return await build_custom_component(params=custom_params, │
│ custom_component=custom_component) │
│ 67 │ │ if base_type == "component": │
│ ❱ 68 │ │ │ return await build_component(params=custom_params, │
│ custom_component=custom_component) │
│ 69 │ │ msg = f"Base type {base_type} not found." │
│ 70 │ │ raise ValueError(msg) │
│ 71 │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/src/backend/base/langflow/interface/initialize/loading.py:145 in build_component │
│ │
│ 142 ): │
│ 143 │ # Now set the params as attributes of the custom_component │
│ 144 │ custom_component.set_attributes(params) │
│ ❱ 145 │ build_results, artifacts = await custom_component.build_results() │
│ 146 │ │
│ 147 │ return custom_component, build_results, artifacts │
│ 148 │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/src/backend/base/langflow/custom/custom_component/component.py:847 in build_results │
│ │
│ 844 │ │ │ session_id = None │
│ 845 │ │ try: │
│ 846 │ │ │ if self._tracing_service: │
│ ❱ 847 │ │ │ │ return await self._build_with_tracing() │
│ 848 │ │ │ return await self._build_without_tracing() │
│ 849 │ │ except StreamingError as e: │
│ 850 │ │ │ await self.send_error( │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/src/backend/base/langflow/custom/custom_component/component.py:829 in _build_with_tracing │
│ │
│ 826 │ │ inputs = self.get_trace_as_inputs() │
│ 827 │ │ metadata = self.get_trace_as_metadata() │
│ 828 │ │ async with self._tracing_service.trace_context(self, self.trace_name, inputs, │
│ metadata): │
│ ❱ 829 │ │ │ results, artifacts = await self._build_results() │
│ 830 │ │ │ self._tracing_service.set_outputs(self.trace_name, results) │
│ 831 │ │ │
│ 832 │ │ return results, artifacts │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/src/backend/base/langflow/custom/custom_component/component.py:895 in _build_results │
│ │
│ 892 │ │ │ │ │ │ if inspect.iscoroutinefunction(method): │
│ 893 │ │ │ │ │ │ │ result = await method() │
│ 894 │ │ │ │ │ │ else: │
│ ❱ 895 │ │ │ │ │ │ │ result = await asyncio.to_thread(method) │
│ 896 │ │ │ │ │ │ if ( │
│ 897 │ │ │ │ │ │ │ self._vertex is not None │
│ 898 │ │ │ │ │ │ │ and isinstance(result, Message) │
│ │
│ /Users/edwin.jose/.local/share/uv/python/cpython-3.12.6-macos-aarch64-none/lib/python3.12/asyncio/threads.py:25 in to_thread │
│ │
│ 22 │ loop = events.get_running_loop() │
│ 23 │ ctx = contextvars.copy_context() │
│ 24 │ func_call = functools.partial(ctx.run, func, *args, **kwargs) │
│ ❱ 25 │ return await loop.run_in_executor(None, func_call) │
│ 26 │
│ │
│ /Users/edwin.jose/.local/share/uv/python/cpython-3.12.6-macos-aarch64-none/lib/python3.12/concurrent/futures/thread.py:58 in run │
│ │
│ 55 │ │ │ return │
│ 56 │ │ │
│ 57 │ │ try: │
│ ❱ 58 │ │ │ result = self.fn(*self.args, **self.kwargs) │
│ 59 │ │ except BaseException as exc: │
│ 60 │ │ │ self.future.set_exception(exc) │
│ 61 │ │ │ # Break a reference cycle with the exception 'exc' │
│ in build_output:60 │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/.venv/lib/python3.12/site-packages/httpx/_api.py:319 in post │
│ │
│ 316 │ │
│ 317 │ **Parameters**: See `httpx.request`. │
│ 318 │ """ │
│ ❱ 319 │ return request( │
│ 320 │ │ "POST", │
│ 321 │ │ url, │
│ 322 │ │ content=content, │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/.venv/lib/python3.12/site-packages/httpx/_api.py:106 in request │
│ │
│ 103 │ │ timeout=timeout, │
│ 104 │ │ trust_env=trust_env, │
│ 105 │ ) as client: │
│ ❱ 106 │ │ return client.request( │
│ 107 │ │ │ method=method, │
│ 108 │ │ │ url=url, │
│ 109 │ │ │ content=content, │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/.venv/lib/python3.12/site-packages/httpx/_client.py:827 in request │
│ │
│ 824 │ │ │ timeout=timeout, │
│ 825 │ │ │ extensions=extensions, │
│ 826 │ │ ) │
│ ❱ 827 │ │ return self.send(request, auth=auth, follow_redirects=follow_redirects) │
│ 828 │ │
│ 829 │ @contextmanager │
│ 830 │ def stream( │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/.venv/lib/python3.12/site-packages/httpx/_client.py:914 in send │
│ │
│ 911 │ │ │
│ 912 │ │ auth = self._build_request_auth(request, auth) │
│ 913 │ │ │
│ ❱ 914 │ │ response = self._send_handling_auth( │
│ 915 │ │ │ request, │
│ 916 │ │ │ auth=auth, │
│ 917 │ │ │ follow_redirects=follow_redirects, │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/.venv/lib/python3.12/site-packages/httpx/_client.py:942 in _send_handling_auth │
│ │
│ 939 │ │ │ request = next(auth_flow) │
│ 940 │ │ │ │
│ 941 │ │ │ while True: │
│ ❱ 942 │ │ │ │ response = self._send_handling_redirects( │
│ 943 │ │ │ │ │ request, │
│ 944 │ │ │ │ │ follow_redirects=follow_redirects, │
│ 945 │ │ │ │ │ history=history, │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/.venv/lib/python3.12/site-packages/httpx/_client.py:979 in _send_handling_redirects │
│ │
│ 976 │ │ │ for hook in self._event_hooks["request"]: │
│ 977 │ │ │ │ hook(request) │
│ 978 │ │ │ │
│ ❱ 979 │ │ │ response = self._send_single_request(request) │
│ 980 │ │ │ try: │
│ 981 │ │ │ │ for hook in self._event_hooks["response"]: │
│ 982 │ │ │ │ │ hook(response) │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/.venv/lib/python3.12/site-packages/httpx/_client.py:1015 in _send_single_request │
│ │
│ 1012 │ │ │ ) │
│ 1013 │ │ │
│ 1014 │ │ with request_context(request=request): │
│ ❱ 1015 │ │ │ response = transport.handle_request(request) │
│ 1016 │ │ │
│ 1017 │ │ assert isinstance(response.stream, SyncByteStream) │
│ 1018 │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/.venv/lib/python3.12/site-packages/httpx/_transports/default.py:232 in handle_request │
│ │
│ 229 │ │ │ content=request.stream, │
│ 230 │ │ │ extensions=request.extensions, │
│ 231 │ │ ) │
│ ❱ 232 │ │ with map_httpcore_exceptions(): │
│ 233 │ │ │ resp = self._pool.handle_request(req) │
│ 234 │ │ │
│ 235 │ │ assert isinstance(resp.stream, typing.Iterable) │
│ │
│ /Users/edwin.jose/.local/share/uv/python/cpython-3.12.6-macos-aarch64-none/lib/python3.12/contextlib.py:158 in __exit__ │
│ │
│ 155 │ │ │ │ # tell if we get the same exception back │
│ 156 │ │ │ │ value = typ() │
│ 157 │ │ │ try: │
│ ❱ 158 │ │ │ │ self.gen.throw(value) │
│ 159 │ │ │ except StopIteration as exc: │
│ 160 │ │ │ │ # Suppress StopIteration *unless* it's the same exception that │
│ 161 │ │ │ │ # was passed to throw(). This prevents a StopIteration │
│ │
│ /Users/edwin.jose/Documents/GitHub/langflow/.venv/lib/python3.12/site-packages/httpx/_transports/default.py:86 in map_httpcore_exceptions │
│ │
│ 83 │ │ │ raise │
│ 84 │ │ │
│ 85 │ │ message = str(exc) │
│ ❱ 86 │ │ raise mapped_exc(message) from exc │
│ 87 │
│ 88 │
│ 89 HTTPCORE_EXC_MAP = { │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ReadTimeout: The read operation timed out
""""
To integrate this component as a tool for agents, we also need to ensure tool_mode=True is set for the relevant inputs. This is important for enabling the tool functionality as described in the documentation.
You can refer to the implementation in the Langflow URL component code for guidance on how to properly configure this.
Let me know if you need assistance updating the inputs to align with this requirement!
cc @ogabrielluiz for feedback and thoughts. Plus, can you point me to the right way to add examples for Langflow + AgentQL, please?
Hey @colriot
Thank you for the PR! About the examples, we are still managing those internally just so we have good ways of making sure they are all working.
Hi @edwinjosechittilappilly @ogabrielluiz , thanks for you feedback! Let me address it.
Regarding the examples, I found this repo: https://github.com/langflow-ai/langflow_examples. Would you recommend it for adding an example or is there a better/other way?
Updated PR with tool support, different Input types and better error messages
Tested! Changes looks good to me! @ogabrielluiz and @NadirJ Awaiting your approval!
LGTM!