python
gh-69753: added decode bytes to utf-8
The following commit authors need to sign the Contributor License Agreement:
Click the button to sign:
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.
If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.
![bedevere-app[bot] avatar](https://avatars.githubusercontent.com/in/388350?v=4 "Published by a pub.dev verified publisher"){kind=small}
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.
If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.
If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.
Thank you for taking this on!
As stated in the issue:
some operations (e.g. file access to some known paths) decoding and encoding from/to any sort of unicode interpretation can be lossy, specifically when the file path on the filesystem has broken/mixed encoding characters.
In cases where encoding to UTF-8 helps, the user can do it themselves.
Encoding and decoding with surrogateescape, as suggested by @bitdancer, would work better.
The feature will also need tests and a note in the documentation.
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.
If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.
I afraid that this does not fix the original issue. Please read the discussion. It was also suggested to add support of bytes to all shlex module.
