Bryan Keller
Bryan Keller
Thanks @kumarpritam863 , having an SMT to (optionally) convert time to int makes sense to me, I had a couple of comments on improving code reuse.
I feel like we can take a much simpler approach for this. For more complex routing needs, an SMT makes more sense to me.
I don't feel we need a new `RecordRouter` abstraction to implement this feature, which introduces complexity. `SinkWriter.extractRouteValue()` could be enhanced to extract the source topic name, if configured, in addition...
I was thinking something [like this](https://github.com/bryanck/iceberg/commit/1967c5e823e86b6f4ca8595db967e52a44688fd9)
The topic name can be mapped to a table via static routing, isn't that what your `TopicRecordRouter` is doing?
I feel your solution is reasonable, though I'm trying to reconcile this with the need for a more flexible, pluggable way to route records. For example, one case we had...
> @bryanck copied over the code as is. > > Im planning to refactor upsert mode (delta writer) code, planning to add few improvements to it, potentially changing existing behavior....
@danielcweeks Do you feel our stance on this evolved or should we hold off on adding this until there is more clarity on the future of equality deletes?
I feel we should close this PR until we discuss this with the community. If the community feels we can move forward, I can handling porting over my code.
We should resolve concerns around relying on equality deletes before going down that road, or open a new PR for a solution that does not rely on equality deletes. Here...