fairseq2
fairseq2 copied to clipboard
`from_generator` function for BuilderDataPipeline
Is your feature request related to a problem? Please describe:
Currently there's only read_sequence method that requires to know all list of sample in advance,
from_generator would offer more flexible interface useful in various contexts (such as async processing).