node-pg-query-stream icon indicating copy to clipboard operation
node-pg-query-stream copied to clipboard

Is there a way to get the total number of rows as soon as the query executor knows the total result size?

Open gajus opened this issue 6 years ago • 1 comments

I am streaming results and I would to cut off the stream after X rows, but I need to know how many rows there are in total.

I have tried to simply count chunks after I stop reading them, but the overhead is too big. It takes +30 seconds to count 30k+ rows.

gajus avatar Mar 15 '19 22:03 gajus

Add a count(*) as total in you select.

You can then make use of async iterables:

let current = 0
let max = 10
let total = 0
for await (const row of stream) {
  total = row.total
  current ++
  if (curent >= max) break // no need to go further
}

console.log("consumed " + current + " out of " + total + " rows")

Be aware of issues #52, #56, #66 if you choose to do that.

matthieusieben avatar Dec 04 '19 13:12 matthieusieben