label-studio icon indicating copy to clipboard operation
label-studio copied to clipboard

Exporting large number of data records is slow

Open zhoumingcheng opened this issue 1 year ago • 1 comments

Describe the bug When the amount of data is large, exporting data will be very slow. Here, it's not a failure caused by exporting data timeout. When I uploaded a text annotation data of 50,000 records, with a file size of approximately 30 megabytes, and tried to export it, it took nearly 4 minutes.

To Reproduce Steps to reproduce the behavior:

Expected behavior

Screenshots

Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Label Studio Version [e.g. 1.12.0]

Additional context Add any other context about the problem here.

zhoumingcheng avatar May 13 '24 06:05 zhoumingcheng

hey @zhoumingcheng

Thanks for the bug report, are you able to reproduce on the latest version? Furthermore, are you deploying your LS instance locally or is it hosted on a service? Wondering what the bottleneck is on this export process. What are your impressions?

sajarin avatar Jun 21 '24 11:06 sajarin

I have a similar issue, but the amount of data is much smaller (1200 Taks). I cannot even export because I run into an HTTP Request Timeout of 30s. It would be better to export asynchronously and then to provide a download link once the export is done, like other applications do.

mjunker avatar Nov 05 '24 09:11 mjunker