parseable icon indicating copy to clipboard operation
parseable copied to clipboard

Remove redundant ListObjectsV2 call when fetching from S3

Open trueleo opened this issue 3 years ago • 0 comments

In S3 query flow, datafusion ListingOptions is configured with extension to look for, this internally makes datafusion list files using ListObjectsV2 before actually fetching the file. This call maybe removed if filename is known beforehand.

Todo:

  • [ ] What to do in case of multiple parseable instances with different hostnames.
  • [ ] Figure out how to provide table without doing list object calls.

trueleo avatar Sep 26 '22 13:09 trueleo