Don't prepare statements unless explicitly specified
This is linked to #148 but since it explores a different/wider aspect of this issue, I opted for creating another issue.
The root cause in #148 is that statements are prepared, which combined with a large number of values results in too large header sizes.
As I explored it, it got me thinking about whether we should be preparing each and every query before executing it. Especially since currently it seems everything is happening implicitly, without the knowledge of the user. Any thoughts?
If I am reading the code correctly the prepared statements path is only used when params are provided to execute. If not plain SQL is sent as it is.
Are you observing different behaviour?
I had to dig into the stack traces since we're not calling execute directly, but yes, the params are provided.
For context, we're using:
- pandas - which in turn calls sqlalchemy
- sqlalchemy - which relies on sqlalchemy-trino dialect to communicate with Trino
- finally, sqlalchemy-trino then calls the Trino client
I think the issue here is that the client is doing more than it's asked for. In my opinion, deciding which statement should be prepared (or not) is the end-user's responsibility.
There is no way today (or before the PR which added prepared statements support) to be able to pass parameters to a query without using prepared statements.
If you want to avoid prepared statements you need to stop passing params by constructing the SQL as a string and then calling cursor.execute(sql) instead of cursor.execute(sql, params).
I'm not sure if there's a way to have both:
- Ability for client to handle params.
- Not use prepared statements when params are passed.
What solution are you suggesting?