Ksawery Smoczyński comments

Results 5 comments of


                                            Ksawery Smoczyński

Problem with multigpu parallel External Source operator

Thanks for your response @JanuszL I've implemented it according to your suggestion and example but the error unfortunately persists.

Problem with multigpu parallel External Source operator

Bingo, that was probably it, thanks a lot @stiepan ! Now the problem is that after first epoch execution hangs, then it proceeds with one batch per epoch skipping validation...

Problem with multigpu parallel External Source operator

Thanks for suggestion @JanuszL Before adding external source operator (using DALI readers) there was no problem with buffering. Using flag `python -u (...)` doesn't resolve this issue. In my setup...

Problem with multigpu parallel External Source operator

Thanks for your response @JanuszL With simple code, when I multiplied the list with images paths by 100k the lag also occurred but was significantly shorter than in the case...

Ksawery Smoczyński

Problem with multigpu parallel External Source operator

Problem with multigpu parallel External Source operator

Problem with multigpu parallel External Source operator

Problem with multigpu parallel External Source operator

Compatibility with langchain-core >= 1.0.0