cornucopia icon indicating copy to clipboard operation
cornucopia copied to clipboard

Add doc section on pipelining

Open LouisGariepy opened this issue 3 years ago • 0 comments

Concept described here https://docs.rs/tokio-postgres/latest/tokio_postgres/#pipelining.

Transcript of the Discord thread that sparked this (thanks to user korin):

So maybe more context. I have 2 tables. And I want to copy one to another. But I do not know how to do it in SQL I am doing it through Rust code.

So first what I am create is a stream of that table (it's 800GB so I can't use all()).

   let mut stmts = select_all_transfer_cache_old();
   let mut stream_of_data = stmts.bind(&client).iter().await?.boxed();

and now I tried with stream_of_data.try_for_each_conncurent(1, |input_data| async move { insert_function().bind(&client, ...) } ).await?; but I want to move input_data only and not client.

Ok I have it:

stream_of_data
    .try_for_each_concurrent(64, |v| {
         let client = &client;
         async move { ... }
}).await?;

just make sure that v is moved and client is moved by reference and this works

LouisGariepy avatar Sep 15 '22 22:09 LouisGariepy