[BUG] Make return type inferrable for TRANSFORM and other lambda functions

[yuancu]

Query Information

PPL Command/Query:

source=people | eval array = array(1, -2, 3), result = transform(array, x -> x + 2) | fields result

Expected Result: [3,0,5] of type INTEGER ARRAY

Actual Result: [3,0,5] of type ANY ARRAY

Dataset Information

Dataset/Schema Type

• [ ] OpenTelemetry (OTEL)
• [ ] Simple Schema for Observability (SS4O)
• [ ] Open Cybersecurity Schema Framework (OCSF)
• [x] Custom (details below)

Index Mapping Mapping not relevant

Bug Description

Before #4892, the types are injected at runtime:

https://github.com/opensearch-project/sql/blob/d19ba53249f32ca8429f8404ca8ef74ab7a1d935/core/src/main/java/org/opensearch/sql/expression/function/CollectionUDF/TransformFunctionImpl.java#L183

https://github.com/opensearch-project/sql/blob/d19ba53249f32ca8429f8404ca8ef74ab7a1d935/core/src/main/java/org/opensearch/sql/expression/function/CollectionUDF/TransformFunctionImpl.java#L190-L191

However, the standard SQL requires the return type to be inferrable before executing the query. Currently, the argument is of type ANY, making the result also typed ANY. Therefore, the result array is an ANY array. For this case, it is indeed inferrable -- the array is an INTEGER ARRAY, meaning that x should be of type INTEGER instead of ANY.

Impact: doctest in collections.md

Environment Information

OpenSearch Version: 3.4 (with PR #4892 )

Screenshots

File "/Users/yuanchu/src/sql/doctest/../docs/user/ppl/functions/collection.md", line 167, in /Users/yuanchu/src/sql/doctest/../docs/user/ppl/functions/collection.md
Failed example:
    ppl_cmd.process('source=people | eval array = array(1, -2, 3), result = transform(array, x -> x + 2) | fields result | head 1')
Expected:
    fetched rows / total rows = 1/1
    +---------+
    | result  |
    |---------|
    | [3,0,5] |
    +---------+
Got:
    fetched rows / total rows = 1/1
    +---------------+
    | result        |
    |---------------|
    | [3.0,0.0,5.0] |
    +---------------+

[coderabbitai[bot]]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• [ ] Create Plan

Examples

• Example 1
• Example 2

---

🔗 Related PRs

opensearch-project/sql#4839 - Support mvfind eval function [merged] opensearch-project/sql#4856 - Support mvmap eval function [merged]

👤 Suggested Assignees

• ahkcs

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord or schedule a call!