square
Is it possible to load data from Source to Dest between servers?
Is it possible to load data from Source to Dest between servers? Like what ETL usually does. Thanks! And I don't know where to add questions, just put it here...
hmm, you'd probably need to expand the ETLs class's ability to hold more than one connection (source_connection / destination_connection) so you could pull from one server and send to another. Let me know if that makes sense and answers your question.
thank you! I tried another one (activewarehouse-etl), it's more complex and support multiple servers. I'm new in Ruby and trying to break it down...
interesyed in this functionality as well. Wondering what the API might look like.
@Emerson @shut000 was just giving this a look over and have an idea on what could work here: right now we have ETL.connection that's used to speak with a single db via #query. This works fine when doing data processing within the same server / db, but totally see your use case. My thought: add a ETL.read_connection and ETL.write_connectionthat you can set and then interact with through two new methods#readand#writewhich would use either{read,write}_connection#query`. One requirement is we'd need to hold the read data in memory and then prepare it to be written, but that shouldn't been too bad!
Happy to take a pass at doing the above, let me know if it's still something you're interested in!