Additional sequential process to return full task output

[bryetz]

Add Full Sequential Task Output Feature

Overview

This pull request introduces a new feature to the existing codebase that enhances the functionality of sequential task execution. The new feature, identified as sequential_full_output, allows for the collection and return of the outputs from each individual task in a sequential process, rather than just the final task's output.

Changes

• New Method _sequential_loop_full_output: Implemented in the main processing class, this method extends the functionality of the existing _sequential_loop. It accumulates the outputs of each task in a sequential process and returns them as a list.
• Process Enumeration Update: Added Process.sequential_full_output to the Process enumeration to support the new method.
• Main Process Handling: Updated the main processing method to handle the new Process.sequential_full_output case, directing it to use the new _sequential_loop_full_output method.

Benefits

• Flexibility: It offers additional flexibility for use cases where individual task outputs are required for further processing or analysis.

[joaomdmoura]

@gvieira i know you're also working on this, so you might want to review

[gvieira]

Hey @bryetz! Thank you so much for taking the time to work on this. It's looking great!

It offers additional flexibility for use cases where individual task outputs are required for further processing or analysis.

Would you mind sharing a real-life(-ish?) use case of accessing intermediate outputs?

[joaomdmoura]

Oh also the output from the individual task is already available at the end of the execution in the task itself task.output

[bryetz]

Hey @bryetz! Thank you so much for taking the time to work on this. It's looking great!

It offers additional flexibility for use cases where individual task outputs are required for further processing or analysis.

Would you mind sharing a real-life(-ish?) use case of accessing intermediate outputs?

Absolutely, I'd be happy to explain how this applies to a project I'm currently involved in at my university. We're working on developing AI agents and integrating them with a graphical user interface (GUI). In this setup, the output of each task executed by an AI agent is crucial for multiple reasons. Displaying each task's output in the GUI makes the process more transparent and engaging for the users. They can see step by step how the AI is processing their request, which enhances trust and understanding of the system. Also sometimes the final output of an AI agent might be a summarized or condensed version of all the tasks it has performed. While this is useful for quick insights, it can leave out important details that were present in the intermediate outputs. By having access to these outputs, users can optionally dive deeper into the data for a more comprehensive understanding.

[bryetz]

Hey @bryetz! Thank you so much for taking the time to work on this. It's looking great!

It offers additional flexibility for use cases where individual task outputs are required for further processing or analysis.

Would you mind sharing a real-life(-ish?) use case of accessing intermediate outputs?

Absolutely, I'd be happy to explain how this applies to a project I'm currently involved in at my university. We're working on developing AI agents and integrating them with a graphical user interface (GUI). In this setup, the output of each task executed by an AI agent is crucial for multiple reasons. Displaying each task's output in the GUI makes the process more transparent and engaging for the users. They can see step by step how the AI is processing their request, which enhances trust and understanding of the system. Also sometimes the final output of an AI agent might be a summarized or condensed version of all the tasks it has performed. While this is useful for quick insights, it can leave out important details that were present in the intermediate outputs. By having access to these outputs, users can optionally dive deeper into the data for a more comprehensive understanding.

In my university project, AI agents simulate a business environment. For example, in a software engineering business each agent's intermediate output from a given task is critical as it consists of actual code that needs to be written to specific real life files. This is essential because the intermediate output from each agent forms a different part of a larger software project that cannot be summarized by one task/agent.

[gvieira]

Got it. It makes sense. Do you think @joaomdmoura's suggestion of using task.output would be helpful?

[bryetz]

Got it. It makes sense. Do you think @joaomdmoura's suggestion of using task.output would be helpful?

Oh also the output from the individual task is already available at the end of the execution in the task itself task.output

Do you mean from using task.execute() directly instead of using crew.kickoff()?

[gvieira]

Do you mean from using task.execute() directly instead of using crew.kickoff()?

You can simply use the plain sequential process and call the kickoff() like you would normally do – it will return only the final output. Then you go through your tasks and you'll be able to retrieve their outputs directly by calling task.output.

[gvieira]

Hey @bryetz! Do you think my suggestion above makes sense for your use case?

[joaomdmoura]

Since then we added a flag full_output that bundles the crew and tasks outputs :)

return {
                "final_output": output,
                "tasks_outputs": [task.output for task in self.tasks if task],
            }