PyCQA
[BUG] prospector fails with UnicodeDecodeError on requirements.txt having DOS (CRLF) line endings
Describe the bug
prospector fails with UnicodeDecodeError (see below) if the requirements.txt has DOS line endings.
dos2unix requirements.txt
cures the problem.
To Reproduce
-
prospector .
Expected behavior Either accept the file - since pip can handle it - or show a helpful error message
Screenshots
Traceback (most recent call last):
File "/home/me/project/mysite/venv/bin/prospector", line 8, in <module>
sys.exit(main())
File "/home/me/project/mysite/venv/lib/python3.10/site-packages/prospector/run.py", line 188, in main
config = ProspectorConfig()
File "/home/me/project/mysite/venv/lib/python3.10/site-packages/prospector/config/__init__.py", line 36, in __init__
self.libraries = self._find_used_libraries(self.config, self.profile)
File "/home/me/project/mysite/venv/lib/python3.10/site-packages/prospector/config/__init__.py", line 215, in _find_used_libraries
for found_dep in autodetect_libraries(self.workdir):
File "/home/me/project/mysite/venv/lib/python3.10/site-packages/prospector/autodetect.py", line 89, in autodetect_libraries
libraries = find_from_requirements(path)
File "/home/me/project/mysite/venv/lib/python3.10/site-packages/prospector/autodetect.py", line 72, in find_from_requirements
reqs = find_requirements(path)
File "/home/me/project/mysite/venv/lib/python3.10/site-packages/requirements_detector/detect.py", line 83, in find_requirements
requirements += from_requirements_txt(reqfile)
File "/home/me/project/mysite/venv/lib/python3.10/site-packages/requirements_detector/detect.py", line 146, in from_requirements_txt
for req in f.readlines():
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
> file requirements.txt
requirements.txt: Unicode text, UTF-16, little-endian text, with CRLF line terminators
Environment (please complete the following information):
- OS: Linux (Ubuntu 22.04)
- Tool: any?
- Prospector version: 1.10.3
- Python version: 3.10
Additional context
dependencies installed, e.g. the output of pip freeze also helps.
doesn't matter imho
Additional context dependencies installed, e.g. the output of pip freeze also helps. doesn't matter imho
Actually it does, because this isn't a bug in prospector, it's a bug in one of its dependencies.
Specifically, requirements-detector. If you're not happy sharing the entire pip freeze fair enough but please share pip freeze | grep requirements-detector as that will add useful context
@carlio Thanks for the fast response. Here's more info:
me@computer:~/project/mysite$ pip freeze | grep requirements
pip-requirements-parser==32.0.1
requirements-detector==1.2.2
