appsmithorg
[Epic]: Problems Uploading Large Files
Is there an existing issue for this?
- [X] I have searched the existing issues
Description
When we try to upload a file to a database that is larger than 20mb the file upload fails, this problem happens with different sizes for different users for some user the file upload fails when it is larger than 5mb but another when it is larger than 20mb this doesn't make much sense.
To get a better idea of this issue, create a repository with the tests that have been done.
appsmith-problems-uploading-large-files
Apart from this, here I attach issues that are related to the issue.
- Base64 files larger than 5 Mb fail to display in Appsmith
- Upload of files larger than 5mb fails when changing port
- Not able to upload files greater than 75 mb
- How to increase Filepicker upload size on self-hosted
- Issue with video uploads to S3 with files greater than 5MB
- Increase the filepicker supported size on cloud
Steps To Reproduce
- Enter this application: app
- Try to upload a video larger than 25mb
Public Sample App
https://app.appsmith.com/app/filesuploaderror/page1-636914a6b7d12f577aa56521
Issue video log
No response
Version
Cloud : Appsmith v1.8.7-SNAPSHOT
Another user reported facing this error while trying to upload a file larger than 50mb.
I spoke to one more user who wants to upload large files( > 100mb ). Is there any workaround for this.
Limitations
- Spring in memory limit,
- plugin specific limit
- and a few more things
If we are going to allow >100MB, we'll need to test and verify that it actually works
More context -https://theappsmith.slack.com/archives/CGBPVEJ5C/p1669722874855689?thread_ts=1669700677.989409&cid=CGBPVEJ5C
Just adding as another example, we have files that are upwards of 300mb that we need uploaded.
@ereio can you also add your use-case on why these large files need to be uploaded in your app? How is appsmith going to help here?
@dilippitchika For sure!
We have tool where management is going to upload large statement files instead of pulling them from where we normally do automatically. We don’t want to build a webpage outside appsmith that’s only for management to drag and drop large files to upload.
We have an API endpoint that can handling uploading very large files through multipart requests, and were hoping AppSmith could help us build out the UI part.
If we are going to allow >100MB, we'll need to test and verify that it actually works
@dilippitchika Are you going to allow >100MB finally? We also need this for uploading video files in our admin panel
Hey @pfilatov92 if you can help me understand your use case better, it will help us prioritise. Can you let me know what your users are doing next after uploading video files? What is their flow like?
@dilippitchika We choose video files via admin panel, attach some meta-information, then video is uploading to s3 through our backend and then becomes available on our website
@pfilatov92 we're working on this soon. Currently, it's a backend limitation with how we process these files. We're working on making it so that the system can upload larger files.
@nidhi-nair wanted to see what PR will fix / has fixed this issue! This is still a big blocker for us!
This is the main blocker stopping me from moving an app to appsmith.
My users need to upload PDF files that can be hundreds of megabytes. I'm uploading to Dropbox, which has a separate API flow for large uploads, so all the file input needs to do is allow the user to select files and I do the rest with javascript. I believe the Filepicker has all the functionality I need (based on https://docs.appsmith.com/connect-data/how-to-guides/how-to-integrate-dropbox), except I can't override the file size limit. It seems like the use case of "use appsmith to handle uploads" vs "use dropbox or some other host" are stuck in the same widget which is maybe causing issues -- maybe there should be 2 different ones?
What is the recommended escape hatch for this kind of situation? In the meantime I've embedded an iframe with some html and javascript which can handle the upload process, but I have to host that code outside of appsmith so it's not ideal.
@devdoshi are you using Appsmith Cloud, or your own self-hosted instance? If self-hosted, what version are you running?
@devdoshi are you using Appsmith Cloud, or your own self-hosted instance? If self-hosted, what version are you running?
Appsmith cloud
@devdoshi there's no way to increase the 100 Mb limit on Appsmith cloud. I think you might be able to increase it on a self-hosted instance though. I think your approach using an iframe is the only way around the limit on Appsmith cloud.
I too thave the same problem. I am developing an application for medical purposed and I need a functionality to upload large files like videos, ctct scans, 3d files ... I am using Firebase Storage as the backend.
