[Feature Request] Support optional data types

[bastbu]

Is your feature request related to a problem? Please describe.

Based on how we use prompt flow, we sometimes would need to pass optional integers into flows. The reason is that we distinguish between the absence of a number (None) and its default (0). As of today, prompt flow does not support optional integer input types. This can be verified by running:

• pf flow init --flow foo
• Change type of the text input to int
• Change the value of text in data.jsonl to null
• Run pf run create --flow foo --data foo/data.jsonl --column-mapping text='${data.text}' --stream

Describe the solution you'd like

Prompt flow should support optional data types and with that the ability to express absence of primitive input values.

Describe alternatives you've considered

As a workaround, we're currently expressing absent integers as integers that have no meaning in our context (e.g. -1 for integers that are always expected to be positive).

[brynn-code]

Thank you for the feedback, we will discuss this. It would be helpful if you could provide more detail about your scenario, could you please give a short description on your use scenario?

[bastbu]

Sure. The feature would apply to how we currently use evaluation flows. We pass numbers from the main flow to our evaluation flows, on which we then base our evaluation metrics. One example would be the number of tokens (prompt, completion) that was used for the main flow. Other numbers might be related to the records we retrieve with RAG.

In most cases, we want to be able to differentiate between a None (e.g. if the main flow uses an execution path without RAG), and a 0 (e.g. if no records were retrieved, but we expected to use RAG, or if we did not end up accessing the LLM and hence no tokens were consumed due to taking an execution path that does not involve an LLM).

The same goes for our output evaluation metrics: we sometimes skip calculating metrics for certain lines, in which case we would not want to get a score of 0, but rather of None (because the evaluation did not "fail", but it was not executed at all).

[brynn-code]

Thank you for the feedback, really appreciate it. We have a short discussion about your case this morning, for the optional (or we just say the 'nullable') input for flow, we had discussed about it before but not planned to support it for now, because nullable data is a very basic feature and links to many related features' design, it's a long-term work that deserves design cautiously, sorry for the inconvenience, we'll track this as a long-term feature ask, and will update here if we're going to support this in the future.