PySUS icon indicating copy to clipboard operation
PySUS copied to clipboard

Update download method of SIA data for PA

Open alfonso777 opened this issue 5 years ago • 2 comments

Currently, DBC files of SIA data for PA have prefixes a,b and c. The following image show that: image

So, the download method pysus.online_data.SIA.download( estado, ano, mes ) is outdated.

alfonso777 avatar Oct 28 '20 23:10 alfonso777

@alfonso777 how do you suggest he handle these variants? Currently I don't have the time to inspect these datafiles.

fccoelho avatar Oct 29 '20 15:10 fccoelho

Hello guys, I would like to try to contribute here. If possible, I can create the PR with the code suggestion.

My suggestion would be changing download() function inside pysus/online_data/SIA.py.

A small draft from solution:

  1. transform lines 107-118 into a function wich receives file name as parameter;
  2. try to read() with the current filename format 2.1 in case of success handle with the same code; 2.2 in case of error, handle exception of file not found (or not available) into FTP and start a loop appending letters (a, b, c ....) to the filename until a file not found exception occurs; 2.3. for each file returned in 2.2, append the content to the same parquet file.

We can also check if exceptions raised by ftp.retrbinary(), inside _fetch_file() function, helps to differentiate between different scenarios.

heber-augusto avatar Jan 05 '22 16:01 heber-augusto

:tada: This issue has been resolved in version 0.10.1 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

github-actions[bot] avatar Sep 21 '23 18:09 github-actions[bot]