PySUS
PySUS copied to clipboard
Pandas does not concatenate the parquet files
Unable to concatenate parquet files to dataframe.
disease dengue year 2020
https://github.com/AlertaDengue/PySUS/blob/master/pysus/tests/test_data/test_sinan.py#L43
48 DENGBR20.parquet/6d35585c41984b459cc3f72986f9aa6c-0.parquet
49 DENGBR20.parquet/f1d9168f79e1431a933faa433859c305-0.parquet
50 DENGBR20.parquet/5d8a203ccca04e87b12c5ef465a4d32d-0.parquet
Killed
pysus/tests/test_data/test_sinan.py::TestSINANDownload::test_chunked_df_size Killed
Reproduce the error:
fn = download(2020, "dengue", return_fname=True)
for i, f in enumerate(glob(f"{fn}/*.parquet")):
if i == 0:
df2 = pd.read_parquet(f)
else:
df2 = pd.concat([df2, pd.read_parquet(f)], ignore_index=True)