flyte icon indicating copy to clipboard operation
flyte copied to clipboard

Support uploading files to Flyte blob store in the Flyte Console launch form

Open cosmicBboy opened this issue 3 years ago • 7 comments

There isnt a straightforward way of uploading data/artifacts (e.g. models) to Flyte blob store without writing a task, registering it, executing it.

Assumption: As a user, I've registered a workflow that has a FlyteFile as an input.

When I go to the launch form on the Flyte UI to execute a workflow, currently I need to reference a specific URI string (e.g. S3 path) pointing to a file that I want to use as input.

As a user, I want the option to upload a file from my local computer as part of the launchform UI so that I don't have to manually upload an s3 file and point the workflow to it.

Comps: comps: Launch Plan File upload

cosmicBboy avatar Feb 06 '23 18:02 cosmicBboy

We can leverage DataProxy.CreateUploadLocation api (take a look at how flyte decks get downloaded for a close example)

EngHabu avatar Feb 20 '23 18:02 EngHabu

@EngHabu can you point me to an example for DataProxy.CreateUploadLocation.

How does this work? Does it upload file to a public s3 and return the URI that I should then append to the form which uses/requires file upload ?

cloud-shannon avatar Mar 03 '23 16:03 cloud-shannon

This is where flyteconsole calls create download link: https://github.com/flyteorg/flyteconsole/blob/957f9fa4e2406d34052f1ef8cace8287d121a691/packages/console/src/models/Execution/api.ts#L73

It uploads to the private s3 bucket used for executions... and gives you back an s3 link that, as you said, you can pass along in the inputs...

https://github.com/flyteorg/flyteidl/blob/master/protos/flyteidl/service/dataproxy.proto#L108

EngHabu avatar Apr 27 '23 18:04 EngHabu

This PR includes the "upload" component that should be implemented into the launch form

https://github.com/flyteorg/flyteconsole/pull/690

jsonporter avatar May 01 '23 16:05 jsonporter

@cosmicBboy Is this for Blob type? or do we have a new type?

james-union avatar May 03 '23 15:05 james-union

Hello 👋, this issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will engage on it to decide if it is still applicable. Thank you for your contribution and understanding! 🙏

github-actions[bot] avatar Feb 19 '24 00:02 github-actions[bot]

@FrankFlitton @jsonporter

EngHabu avatar Feb 19 '24 00:02 EngHabu