node-postgres icon indicating copy to clipboard operation
node-postgres copied to clipboard

Pipelining

Open mcollina opened this issue 1 year ago • 9 comments

This is a reopen of https://github.com/brianc/node-postgres/pull/2706, hopefully completing it soon.

mcollina avatar Jan 14 '25 12:01 mcollina

Just for remind, the pipeline mode should also be implemented in the pg-native client: https://github.com/brianc/node-postgres/blob/master/packages/pg-native/index.js#L9

Thanks so much!

cesco69 avatar Jan 15 '25 08:01 cesco69

Just for remind, the pipeline mode should also be implemented in the pg-native client:

Also, I'm fine accepting additions in just JavaScript which can be ported to the native module at a later date if you're not comfortable working on it. Particularly if it requires c/c++ binding changes. There are several "pure javascript only" features already such as pg-cursor and pg-query-stream only supporting the pure JS client.

pls lmk how I can help out here!

brianc avatar Jan 17 '25 21:01 brianc

@brianc can you approve the workflow run so we know where things are in CI?

mcollina avatar Jan 23 '25 16:01 mcollina

It doesnt give me the option while its a draft. Lemme mark it as ready and try.

Edit: nevermind. It did. Was behind a dropdown. Anyways, running now!

brianc avatar Jan 23 '25 16:01 brianc

I changed the rules on the CI runner to be more lax & auto-run in more situations. I might still have to approve some, but shouldn't require me to approve it quite as much which is good. :)

brianc avatar Jan 23 '25 17:01 brianc

Hi @mcollina,

I hope you're doing well! I wanted to kindly check if you're still interested in working on this feature. I'd really love to see it implemented, as it would be a valuable addition to the project.

I've contributed to the project myself, but unfortunately, I don't have the skills to tackle this feature. If you decide not to move forward with it, that's completely understandable, your time and efforts on so many projects like Fastify are truly appreciated!

Thanks again for your contributions.

cesco69 avatar Feb 11 '25 08:02 cesco69

I'll get to it at some point ;). If anyone has bandwidth, happy to collaborate

mcollina avatar Feb 11 '25 08:02 mcollina

Hi @mcollina and @brianc,

Thank you for your work on node-postgres.

I'm very interested in the query pipelining feature in this PR. I've reviewed the code and would like to help get it merged.

What are the most important next steps? I'm happy to help with tests, exploring the native client implementation, or any other remaining tasks.

Looking forward to collaborating with you.

breakpointninja avatar Oct 12 '25 13:10 breakpointninja

Side note for those who want to try finishing this PR:

The current implementation relies only on prepared statements (named queries) to connect the result to the correct query, but this is not mandatory.

Every query can be "pipelined", e.g., other clients use the hash (SHA-1) of the query to return the correct result.

I have tried (and failed) to extend pipeline mode to every query in this PR: https://github.com/brianc/node-postgres/pull/3534

cesco69 avatar Oct 13 '25 14:10 cesco69