[RFC]: Dynamic pipeline composition

[bharagha]

This RFC is submitted to discuss the dynamic pipeline composition requirement. The attachment contains the RFC in the prescribed template along with the images it references. 24-06-08-OPEA-00x-Dynamic-Pipelines.md

[rdiazintel]

I cannot download the images. Is there a process to recreate the RFC either online or locally on my computer?

[rdiazintel]

How do I edit and contribute to the Dynamic pipeline composition RFC?

[devpramod]

I'll escalate this issue to the engineering team

[irisdingbj]

OPEA already has GenAI Microservices Connector for composing pipeline dynamically, please check via https://github.com/opea-project/GenAIInfra/blob/673e7c820667c92f56460161a20c054803cb431f/microservices-connector/README.md for some introduction.

[amberjain1]

I cannot download the images. Is there a process to recreate the RFC either online or locally on my computer?

[bharagha]

How do I edit and contribute to the Dynamic pipeline composition RFC?

The md file is downloadable and editable. Let me know if you are able to do that. Alternatively, I can send it as attachment (including the figures) over email.

[bharagha]

OPEA already has GenAI Microservices Connector for composing pipeline dynamically, please check via https://github.com/opea-project/GenAIInfra/blob/673e7c820667c92f56460161a20c054803cb431f/microservices-connector/README.md for some introduction.

This is not the dynamic pipeline this RFC targets. The GMC ability to dynamically manage the deployment is well understood. The routing ability of the GMC to direct the end user request to the correct user chosen endpoint is not clear from the provided documentation. Let's target to resolve that in the POC underway. Until then I wouldn't confirm that GMC is addressing the requirement listed in this RFC.

[rdiazintel]

Hi Raghu,

Yes, I downloaded it. I read the text and it references the images, but when I download the images, I cannot get them to show within the text even with an MD editor.

What MD editor are you using?

Thanks, Raul

From: Raghu Bhat @.> Sent: Friday, June 14, 2024 07:27 To: opea-project/GenAIExamples @.> Cc: Diaz, Raul @.>; Comment @.> Subject: Re: [opea-project/GenAIExamples] [RFC]: Dynamic pipeline composition (Issue #270)

How do I edit and contribute to the Dynamic pipeline composition RFC?

The md file is downloadable and editable. Let me know if you are able to do that. Alternatively, I can send it as attachment (including the figures) over email.

— Reply to this email directly, view it on GitHubhttps://github.com/opea-project/GenAIExamples/issues/270#issuecomment-2168164789, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANP3BUSM4S6ILR4W33G7M3TZHL4TRAVCNFSM6AAAAABI77U2VCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRYGE3DINZYHE. You are receiving this because you commented.Message ID: @.@.>>

[hshen14]

OPEA already has GenAI Microservices Connector for composing pipeline dynamically, please check via https://github.com/opea-project/GenAIInfra/blob/673e7c820667c92f56460161a20c054803cb431f/microservices-connector/README.md for some introduction.

This is not the dynamic pipeline this RFC targets. The GMC ability to dynamically manage the deployment is well understood. The routing ability of the GMC to direct the end user request to the correct user chosen endpoint is not clear from the provided documentation. Let's target to resolve that in the POC underway. Until then I wouldn't confirm that GMC is addressing the requirement listed in this RFC.

As we agreed, let's use GMC as the example and see whether the dynamic pipeline requirement could be addressed properly. @rdiazintel also wanted to share the inputs in this thread. I'd like to suggest to make this RFC as a starting point and improve this RFC/repurpose this RFC.

PS: @bharagha I guess you may need to share the permission of your RFC branch with @rdiazintel to edit the RFC online together.

[bharagha]

Hi Raul

I use VSCode for the same. Ctrl+Shift+V shows the contents including image in preview mode.

-- Raghu

From: Raul Diaz @.> Sent: Friday, June 14, 2024 11:43 PM To: opea-project/GenAIExamples @.> Cc: Bhat, Raghavendra @.>; Author @.> Subject: Re: [opea-project/GenAIExamples] [RFC]: Dynamic pipeline composition (Issue #270)

Hi Raghu,

Yes, I downloaded it. I read the text and it references the images, but when I download the images, I cannot get them to show within the text even with an MD editor.

What MD editor are you using?

Thanks, Raul

From: Raghu Bhat @.mailto:***@***.***> Sent: Friday, June 14, 2024 07:27 To: opea-project/GenAIExamples @.mailto:***@***.***> Cc: Diaz, Raul @.mailto:***@***.***>; Comment @.mailto:***@***.***> Subject: Re: [opea-project/GenAIExamples] [RFC]: Dynamic pipeline composition (Issue #270)

How do I edit and contribute to the Dynamic pipeline composition RFC?

The md file is downloadable and editable. Let me know if you are able to do that. Alternatively, I can send it as attachment (including the figures) over email.

— Reply to this email directly, view it on GitHubhttps://github.com/opea-project/GenAIExamples/issues/270#issuecomment-2168164789, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANP3BUSM4S6ILR4W33G7M3TZHL4TRAVCNFSM6AAAAABI77U2VCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRYGE3DINZYHE. You are receiving this because you commented.Message ID: @.@.mailto:***@***.******@***.***>>

— Reply to this email directly, view it on GitHubhttps://github.com/opea-project/GenAIExamples/issues/270#issuecomment-2168529501, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJOGEDZTJKN5ZQCCU5CNWKTZHMXCZAVCNFSM6AAAAABI77U2VCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRYGUZDSNJQGE. You are receiving this because you authored the thread.Message ID: @.@.>>

[bharagha]

I don’t see to have the permission to provide edit rights t @rdiazintel.

-- Raghu

From: Haihao Shen @.> Sent: Monday, June 17, 2024 12:12 PM To: opea-project/GenAIExamples @.> Cc: Bhat, Raghavendra @.>; Mention @.> Subject: Re: [opea-project/GenAIExamples] [RFC]: Dynamic pipeline composition (Issue #270)

OPEA already has GenAI Microservices Connectorhttps://github.com/opea-project/GenAIInfra/tree/main/microservices-connector for composing pipeline dynamically, please check via https://github.com/opea-project/GenAIInfra/blob/673e7c820667c92f56460161a20c054803cb431f/microservices-connector/README.md for some introduction.

This is not the dynamic pipeline this RFC targets. The GMC ability to dynamically manage the deployment is well understood. The routing ability of the GMC to direct the end user request to the correct user chosen endpoint is not clear from the provided documentation. Let's target to resolve that in the POC underway. Until then I wouldn't confirm that GMC is addressing the requirement listed in this RFC.

As we agreed, let's use GMC as the example and see whether the dynamic pipeline requirement could be addressed properly. @rdiazintelhttps://github.com/rdiazintel also wanted to share the inputs in this thread. I'd like to suggest to make this RFC as a starting point and improve this RFC/repurpose this RFC.

PS: @bharaghahttps://github.com/bharagha I guess you may need to share the permission of your RFC branch with @rdiazintelhttps://github.com/rdiazintel to edit the RFC online together.

— Reply to this email directly, view it on GitHubhttps://github.com/opea-project/GenAIExamples/issues/270#issuecomment-2172417755, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJOGED2Q7AUUDXLCHVFHQY3ZH2AK5AVCNFSM6AAAAABI77U2VCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZSGQYTONZVGU. You are receiving this because you were mentioned.Message ID: @.@.>>

[rdiazintel]

Thanks Raghu! I’ll use it. I do already use it for my development work, so easy adoption 😊

From: Raghu Bhat @.> Sent: Thursday, June 20, 2024 01:16 To: opea-project/GenAIExamples @.> Cc: Diaz, Raul @.>; Mention @.> Subject: Re: [opea-project/GenAIExamples] [RFC]: Dynamic pipeline composition (Issue #270)

Hi Raul

I use VSCode for the same. Ctrl+Shift+V shows the contents including image in preview mode.

-- Raghu

From: Raul Diaz @.mailto:***@***.***> Sent: Friday, June 14, 2024 11:43 PM To: opea-project/GenAIExamples @.mailto:***@***.***> Cc: Bhat, Raghavendra @.mailto:***@***.***>; Author @.mailto:***@***.***> Subject: Re: [opea-project/GenAIExamples] [RFC]: Dynamic pipeline composition (Issue #270)

Hi Raghu,

Yes, I downloaded it. I read the text and it references the images, but when I download the images, I cannot get them to show within the text even with an MD editor.

What MD editor are you using?

Thanks, Raul

From: Raghu Bhat @.<mailto:@.mailto:***@***.***%3cmailto:***@***.***>> Sent: Friday, June 14, 2024 07:27 To: opea-project/GenAIExamples @.<mailto:@.mailto:***@***.***%3cmailto:***@***.***>> Cc: Diaz, Raul @.<mailto:@.mailto:***@***.***%3cmailto:***@***.***>>; Comment @.<mailto:@.mailto:***@***.***%3cmailto:***@***.***>> Subject: Re: [opea-project/GenAIExamples] [RFC]: Dynamic pipeline composition (Issue #270)

How do I edit and contribute to the Dynamic pipeline composition RFC?

The md file is downloadable and editable. Let me know if you are able to do that. Alternatively, I can send it as attachment (including the figures) over email.

— Reply to this email directly, view it on GitHubhttps://github.com/opea-project/GenAIExamples/issues/270#issuecomment-2168164789, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANP3BUSM4S6ILR4W33G7M3TZHL4TRAVCNFSM6AAAAABI77U2VCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRYGE3DINZYHE. You are receiving this because you commented.Message ID: @.@.<mailto:@.@.mailto:***@***.******@***.***%3cmailto:***@***.******@***.***>>>

— Reply to this email directly, view it on GitHubhttps://github.com/opea-project/GenAIExamples/issues/270#issuecomment-2168529501, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJOGEDZTJKN5ZQCCU5CNWKTZHMXCZAVCNFSM6AAAAABI77U2VCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRYGUZDSNJQGE. You are receiving this because you authored the thread.Message ID: @.@.mailto:***@***.******@***.***>>

— Reply to this email directly, view it on GitHubhttps://github.com/opea-project/GenAIExamples/issues/270#issuecomment-2180088531, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANP3BUQIMB6PULQSKTMQYS3ZIKFU3AVCNFSM6AAAAABI77U2VCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBQGA4DQNJTGE. You are receiving this because you were mentioned.Message ID: @.@.>>

[bharagha]

Progress on this RFC:

• Exploring using GMC as the router to support dynamic pipeline. ChatQnA application is being done to provide an option on the LLM to switch thereby showcasing the routing capabilities.
• Issues faced as part of the exploration is being addressed with separate PRs. Two related PRs are: https://github.com/opea-project/GenAIInfra/issues/143 https://github.com/opea-project/GenAIInfra/issues/172

Working group is in discussion on how to address the opens.

Just for reference, the original RFC proposed approach (non-GMC) is also available as a PR: https://github.com/opea-project/GenAIExamples/pull/319.

[bharagha]

Further updates from the discussion: https://github.com/opea-project/GenAIInfra/issues/166 provides a way forward to address the issue raised in 172.

There was also a discussion on a possible approach using service registry to provide more flexibility, configurability, and dynamic manageability for the entire decision making of a pipeline handling. Attached the files. 24-07-11-OPEA-00x-Dynamic-Pipelines.md

[bharagha]

Closing this issue with all the documentation available in this RFC. Final alignment is to use GMC dynamic routing and provide the same for ProductivitySuite. PR #319 is the alternative approach for reference