apache
[Feature Request]: Enhancement Request for Table Input: Explicit Parameter Mapping and Option to Preserve Incoming Fields
What would you like to happen?
In its current form, the Table Input transform has two limitations when used with the options “Insert data from transform” and “Execute for each row”:
No explicit parameter-mapping interface. When the transform receives upstream fields and uses ? placeholders in the SQL, Hop automatically assigns parameters based on field order. This behavior is implicit and error-prone, especially when several fields arrive in the input stream. Users have no way to select which fields should map to each parameter.
Upstream fields are lost after execution. When the SQL is executed, the transform outputs only the fields returned by the query. The fields coming from previous transforms do not flow forward unless additional steps are added. This breaks the data stream and forces unnecessary workarounds.
Requested Improvements:
Parameter Mapping UI Add a configuration panel that allows users to explicitly map each ? placeholder in the SQL to a specific upstream field. This would eliminate ambiguity, reduce errors, and prevent users from resorting to Database Join solely to achieve parameterized execution.
Option to Preserve Input Stream Fields Add a checkbox such as “Keep incoming fields”. When enabled, the output rows should include both:
the original upstream fields, and
the fields returned by the SQL query.
Why this enhancement is important:
It enables parameterized SQL execution directly in Table Input, without falling back to slower transforms like Database Join.
It keeps the data stream intact, simplifying pipeline design.
It gives users explicit control over parameter mapping, improving reliability and usability.
This change would significantly enhance the flexibility and efficiency of workflows that rely on dynamic SQL parameters.
Issue Priority
Priority: 3
Issue Component
Component: Transforms
Hi @CarlosJuncher03, let me disagree on your first point explicit parameter-mapping interface. You can reach the same goal by reconfiguring the incoming stream using the available transforms to satisfy your requirements. I personally don't see any real added value in this. About keeping the incoming field, well, this could be useful that could be interesting. Let us wait for opinions from other community members before thinking about implementing the feature.
I agree that this transform is showing its age and could use improvements. After all, it was one of the very first first steps ever to be created almost 25 years ago.
But, to remain absolutely 100% compatible I wouldn't change the transform at all and opt to create a new "SQL Input" transform with the aforementioned features. Indeed, I would allow for a better GUI and better parameter mapping options. I also would think about allowing explicit definition of expected output fields to prevent surprises with SELECT * FROM queries. Perhaps we could also allow the data types and lengths to be specified to do possible conversions right on the spot.
