PySUS
PySUS copied to clipboard
UnicodeDecodeError while trying to download 'Dengue' data from SINAN
Bug report
I've encountered a bug while trying to download data for 'Dengue' in 2001 to 2008.
How to reproduce
from pysus.online_data import SINAN
SINAN.download(2001, 'dengue', return_fname=True)
LOG error
>>> SINAN.download(2001, 'dengue', return_fname=True)
/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pysus/online_data/SINAN.py:62: UserWarning: Now SINAN tables are no longer split by state. Returning countrywide years
warnings.warn("Now SINAN tables are no longer split by state. Returning countrywide years")
/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pysus/online_data/SINAN.py:87: UserWarning: Now SINAN tables are no longer split by state. Returning country table
warnings.warn("Now SINAN tables are no longer split by state. Returning country table")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pysus/online_data/SINAN.py", line 105, in download
filename = get_chunked_dataframe(fname, 'DBC')
File "/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pysus/online_data/__init__.py", line 86, in get_chunked_dataframe
df.applymap(lambda x: x.decode() if isinstance(x, bytes) else x)
File "/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pandas/core/frame.py", line 8930, in applymap
return self.apply(infer).__finalize__(self, "applymap")
File "/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pandas/core/frame.py", line 8845, in apply
return op.apply().__finalize__(self, method="apply")
File "/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pandas/core/apply.py", line 733, in apply
return self.apply_standard()
File "/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pandas/core/apply.py", line 857, in apply_standard
results, res_index = self.apply_series_generator()
File "/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pandas/core/apply.py", line 873, in apply_series_generator
results[i] = self.f(v)
File "/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pandas/core/frame.py", line 8928, in infer
return lib.map_infer(x.astype(object)._values, func, ignore_na=ignore_na)
File "pandas/_libs/lib.pyx", line 2870, in pandas._libs.lib.map_infer
File "/home/luabida/miniconda3/envs/epigraphhub/lib/python3.9/site-packages/pysus/online_data/__init__.py", line 86, in <lambda>
df.applymap(lambda x: x.decode() if isinstance(x, bytes) else x)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 18: invalid continuation byte