Philippe Moussalli
Philippe Moussalli
@rezarokni @TheNeuralBit
> > Have only one installation script for Beam with the latest implemented functions in the Dataframe API instead of installing from source > > This is just blocked on...
> > > I think the "No producer" error is a bug with PCollection pruning in interactive beam: #21430 > > > CC: @KevinGG > > > > > >...
> > I think the "No producer" error is a bug with PCollection pruning in interactive beam: #21430 > > CC: @KevinGG > > Commented in #21430, we can disable...
> @PhilippeMoussalli Could you please take a look at #23069? @KevinGG I just tested it out and it checks out! Thanks again for taking this up.
Implemented latest feedback @TheNeuralBit @davidcavazos :)
I am not sure if we should provide a generic way of implementing this, you could still copy over the bash script and execute it either at at the beginning...
> > However, because we're running containers in sequence, this means that the whole pipeline will abort after the first component has finished executing. > > I would say this...
> But why would we want to execute the rest of the pipeline if a component has failed? Or does it also not execute if the component succeeds? It does...
I feel like this feature is needed but would not really prioritize it at the moment unless it's currently blocking us/other users from implementing a given pipeline