VectorDBBench icon indicating copy to clipboard operation
VectorDBBench copied to clipboard

s3 source files are not downloading in aws east region

Open kish5432 opened this issue 1 year ago • 0 comments

HI Team,

We have setup the Vector DB Bench in aws east region. While running the test, it's unable to download the files from zilliz s3 location.

But same code is working in aws west2 region. Can you plz fix the issue.

2024-04-26 20:34:43,346 | INFO: Milvus create collection: VectorDBBenchCollection (milvus.py:55) (404) 2024-04-26 20:35:12,236 | WARNING: pre run case error: Connect timeout on endpoint URL: "https://s3.us-west-2.amazonaws.com/assets.zilliz.com/benchmark/cohere_medium_1m/test.parquet" (task_runner.py:92) (404) 2024-04-26 20:35:12,236 | WARNING: [1/1] case {'label': <CaseLabel.Performance: 2>, 'dataset': {'data': {'name': 'Cohere', 'size': 1000000, 'dim': 768, 'metric_type': <MetricType.COSINE: 'COSINE'>}}, 'db': 'Milvus-1M_HNSW_DEFAULT'} failed to run, reason=Connect timeout on endpoint URL: "https://s3.us-west-2.amazonaws.com/assets.zilliz.com/benchmark/cohere_medium_1m/test.parquet" (interface.py:187) (404) Traceback (most recent call last): File "/app/VectorDBBench/lib64/python3.11/site-packages/vectordb_bench/interface.py", line 168, in _async_task_v2 case_res.metrics = runner.run(drop_old) ^^^^^^^^^^^^^^^^^^^^ File "/app/VectorDBBench/lib64/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 96, in run self._pre_run(drop_old) File "/app/VectorDBBench/lib64/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 93, in _pre_run raise e from None File "/app/VectorDBBench/lib64/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 87, in _pre_run self.ca.dataset.prepare(self.dataset_source) File "/app/VectorDBBench/lib64/python3.11/site-packages/vectordb_bench/backend/dataset.py", line 202, in prepare source.reader().read( File "/app/VectorDBBench/lib64/python3.11/site-packages/vectordb_bench/backend/data_source.py", line 135, in read if (not local_file.exists()) or (not self.validate_file(remote_file, local_file, check_etag)): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/VectorDBBench/lib64/python3.11/site-packages/vectordb_bench/backend/data_source.py", line 152, in validate_file info = self.fs.info(remote) ^^^^^^^^^^^^^^^^^^^^ File "/app/VectorDBBench/lib64/python3.11/site-packages/fsspec/asyn.py", line 118, in wrapper return sync(self.loop, func, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Thanks

kish5432 avatar Apr 26 '24 21:04 kish5432