Absolute-Series-Scanner icon indicating copy to clipboard operation
Absolute-Series-Scanner copied to clipboard
verified publisher icon ZeroQI

Unicode crashing ASS with "Exception: ''ascii' codec can't encode character u'\xe9' in position 155: ordinal not in range(128)'

Open 2600box opened this issue 3 years ago • 1 comments

Platform

Operating system and version: Ubuntu Plex version: Version 1.28.1.6041

Expected Behavior

Scan youtube videos from tubesync together with YouTuve-Agent.bundle.

Current Behavior

Crashes with:

# cat _root_.scanner.log
=============================================================================================================================================================
Call: "Plex", path: "", folder_show: "", dirs (3), files (0)
=============================================================================================================================================================
-------------------------------------------------------------------------------------------------------------------------------------------------------------
=============================================================================================================================================================
Library root ([R] Series in Grouping folder Root call (uncached), [_] Normal (cached) Plex call, include grouping folder itself, [S][s] Season folders (uppercase for Root call, lowercase for Plex standard Call)
[_] Little Baby Bum 2 Hours playlist                                                                                        ( 32 files)
Exception: ''ascii' codec can't encode character u'\xe9' in position 155: ordinal not in range(128)'
[_] Mimi Cracra 58 Episodes Playlist
Exception: ''ascii' codec can't encode characters in position 113-114: ordinal not in range(128)'
[_] Perspective Arts                                                                                                        (  7 files)
=============================================================================================================================================================
Dirs left for normal Plex calls:
[_] Little Baby Bum 2 Hours playlist [PL0VE_cI7-AYR1vkdKi3y8kO50eWwN2FEc]
[_] Mimi Cracra 58 Episodes Playlist [PLef2q6KaqVUear4ST0zg82kNo_TdHkz5v]
[_] Perspective Arts [UCN8V_pO0xOFKLL4XG1tshnw]
=============================================================================================================================================================

# cat Plex\ Media\ Scanner.log
Aug 14, 2022 10:23:49.799 [0x7ff1e1c8bb00] INFO - Plex Media Scanner v1.28.1.6041-738907df3 - Ubuntu PC x86_64 - build: linux-x86_64 - GMT 00:00
Aug 14, 2022 10:23:49.799 [0x7ff1e1c8bb00] INFO - Linux version: 20.04.4 LTS (Focal Fossa), language: en-US
Aug 14, 2022 10:23:49.799 [0x7ff1e1c8bb00] INFO - Processor: 6-core Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
Aug 14, 2022 10:23:49.799 [0x7ff1e1c8bb00] INFO - /usr/lib/plexmediaserver/Plex Media Scanner --scan --refresh --section 41 --activity 8fd8e839-ca93-4627-84cb-b7c80a2d916a
Aug 14, 2022 10:23:49.802 [0x7ff1e5658140] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.35.5, threadsafe=1
Aug 14, 2022 10:23:49.892 [0x7ff1df934b00] DEBUG - [HCl#2] HTTP requesting GET http://127.0.0.1:32400/:/metadata/notify/cullTimeline?librarySectionID=41&sinceTime=1660471429
Aug 14, 2022 10:23:49.892 [0x7ff1e5658140] DEBUG - [HCl#1] HTTP requesting PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=0
Aug 14, 2022 10:23:49.893 [0x7ff1df907b00] DEBUG - [HttpClient/HCl#2] HTTP/1.1 (0.0s) 200 response from GET http://127.0.0.1:32400/:/metadata/notify/cullTimeline?librarySectionID=41&sinceTime=1660471429
Aug 14, 2022 10:23:49.893 [0x7ff1df907b00] DEBUG - [HttpClient/HCl#1] HTTP/1.1 (0.0s) 200 response from PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=0
Aug 14, 2022 10:23:49.893 [0x7ff1e5658140] DEBUG - Scanning Youtube Offline using xn(Absolute Series Scanner) with 32 current media items and 1 section locations in the database.
Aug 14, 2022 10:23:50.105 [0x7ff1e5658140] DEBUG - Performing a scan with 'Absolute Series Scanner' (language: xn virtual: 0).
Aug 14, 2022 10:23:50.105 [0x7ff1e5658140] DEBUG -   * Scanning /YouTube/tubesync-downloads/video
Aug 14, 2022 10:23:50.105 [0x7ff1e5658140] DEBUG - Scanner: Processing directory /YouTube/tubesync-downloads/video (parent: no)
Aug 14, 2022 10:23:50.107 [0x7ff1e5658140] DEBUG - Skipping over directory '', as nothing has changed; removing 0 media items from map.
Aug 14, 2022 10:23:50.467 [0x7ff1e5658140] DEBUG - Scanner: Processing directory /YouTube/tubesync-downloads/video/Little Baby Bum 2 Hours playlist [PL0VE_cI7-AYR1vkdKi3y8kO50eWwN2FEc] (parent: yes)
Aug 14, 2022 10:23:50.473 [0x7ff1e5658140] DEBUG - Skipping over directory 'Little Baby Bum 2 Hours playlist [PL0VE_cI7-AYR1vkdKi3y8kO50eWwN2FEc]', as nothing has changed; removing 32 media items from map.
Aug 14, 2022 10:23:50.474 [0x7ff1e5658140] DEBUG - [HCl#3] HTTP requesting PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=%2B33&subtitle=Little%20Baby%20Bum%202%20Hours%20playlist%20%5BPL0VE_cI7-AYR1vkdKi3y8kO50eWwN2FEc%5D
Aug 14, 2022 10:23:50.474 [0x7ff1df934b00] DEBUG - [HCl#4] HTTP requesting GET http://127.0.0.1:32400/:/metadata/updateProgressMessage?message=Scanning%20Little%20Baby%20Bum%202%20Hours%20playlist%20%5BPL0VE_cI7-AYR1vkdKi3y8kO50eWwN2FEc%5D
Aug 14, 2022 10:23:50.474 [0x7ff1df907b00] DEBUG - [HttpClient/HCl#4] HTTP/1.1 (0.0s) 200 response from GET http://127.0.0.1:32400/:/metadata/updateProgressMessage?message=Scanning%20Little%20Baby%20Bum%202%20Hours%20playlist%20%5BPL0VE_cI7-AYR1vkdKi3y8kO50eWwN2FEc%5D (reused)
Aug 14, 2022 10:23:50.475 [0x7ff1df907b00] DEBUG - [HttpClient/HCl#3] HTTP/1.1 (0.0s) 200 response from PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=%2B33&subtitle=Little%20Baby%20Bum%202%20Hours%20playlist%20%5BPL0VE_cI7-AYR1vkdKi3y8kO50eWwN2FEc%5D (reused)
Aug 14, 2022 10:23:50.475 [0x7ff1e5658140] DEBUG - Scanner: Processing directory /YouTube/tubesync-downloads/video/Perspective Arts [UCN8V_pO0xOFKLL4XG1tshnw] (parent: yes)
Aug 14, 2022 10:23:50.512 [0x7ff1e5658140] DEBUG - Directory had 250 files, database had 0 files, can't skip.
Aug 14, 2022 10:23:50.521 [0x7ff1e5658140] ERROR - Error in Python: Running scanner:
Traceback (most recent call last):
  File "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py", line 512, in Scan
    files        = [sanitize_path(p) for p in files] #unicode
  File "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py", line 495, in sanitize_path
    return p.decode(sys.getfilesystemencoding() or 'utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 100: ordinal not in range(128)
Aug 14, 2022 10:23:50.521 [0x7ff1e5658140] DEBUG - [HCl#5] HTTP requesting PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=%2B33&subtitle=Perspective%20Arts%20%5BUCN8V_pO0xOFKLL4XG1tshnw%5D
Aug 14, 2022 10:23:50.522 [0x7ff1df907b00] DEBUG - [HttpClient/HCl#5] HTTP/1.1 (0.0s) 200 response from PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=%2B33&subtitle=Perspective%20Arts%20%5BUCN8V_pO0xOFKLL4XG1tshnw%5D (reused)
Aug 14, 2022 10:23:50.522 [0x7ff1e5658140] ERROR - We got an error scanning in /YouTube/tubesync-downloads/video
Aug 14, 2022 10:23:50.522 [0x7ff1e5658140] DEBUG - Scanner: Processing directory /YouTube/tubesync-downloads/video/Mimi Cracra 58 Episodes Playlist [PLef2q6KaqVUear4ST0zg82kNo_TdHkz5v] (parent: yes)
Aug 14, 2022 10:23:50.530 [0x7ff1e5658140] DEBUG - Directory had 58 files, database had 0 files, can't skip.
Aug 14, 2022 10:23:50.531 [0x7ff1e5658140] ERROR - Error in Python: Running scanner:
Traceback (most recent call last):
  File "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py", line 512, in Scan
    files        = [sanitize_path(p) for p in files] #unicode
  File "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py", line 495, in sanitize_path
    return p.decode(sys.getfilesystemencoding() or 'utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 156: ordinal not in range(128)
Aug 14, 2022 10:23:50.531 [0x7ff1e5658140] DEBUG - [HCl#6] HTTP requesting PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=%2B33&subtitle=Mimi%20Cracra%2058%20Episodes%20Playlist%20%5BPLef2q6KaqVUear4ST0zg82kNo_TdHkz5v%5D
Aug 14, 2022 10:23:50.532 [0x7ff1df907b00] DEBUG - [HttpClient/HCl#6] HTTP/1.1 (0.0s) 200 response from PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=%2B33&subtitle=Mimi%20Cracra%2058%20Episodes%20Playlist%20%5BPLef2q6KaqVUear4ST0zg82kNo_TdHkz5v%5D (reused)
Aug 14, 2022 10:23:50.532 [0x7ff1e5658140] ERROR - We got an error scanning in /YouTube/tubesync-downloads/video
Aug 14, 2022 10:23:50.532 [0x7ff1e5658140] WARN - Scanning the location /YouTube/tubesync-downloads/video did not complete
Aug 14, 2022 10:23:50.532 [0x7ff1e5658140] DEBUG - [HCl#7] HTTP requesting PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=99.000000
Aug 14, 2022 10:23:50.533 [0x7ff1df907b00] DEBUG - [HttpClient/HCl#7] HTTP/1.1 (0.0s) 200 response from PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=99.000000 (reused)
Aug 14, 2022 10:23:50.533 [0x7ff1e5658140] DEBUG - Since it was an incomplete scan, we are not going to whack missing media.
Aug 14, 2022 10:23:50.533 [0x7ff1e5658140] DEBUG - [HCl#8] HTTP requesting PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=100
Aug 14, 2022 10:23:50.534 [0x7ff1df907b00] DEBUG - [HttpClient/HCl#8] HTTP/1.1 (0.0s) 200 response from PUT http://127.0.0.1:32400/activities/8fd8e839-ca93-4627-84cb-b7c80a2d916a?percentComplete=100 (reused)
Aug 14, 2022 10:23:50.538 [0x7ff1e5658140] DEBUG - Refreshing section 41 of type: 2
Aug 14, 2022 10:23:50.539 [0x7ff1dfbccb00] DEBUG - Refreshing 0 IDs.

2600box avatar Aug 14 '22 10:08 2600box

There's a few things you can do to fix this. If you're like me the issue is that you have a file with a non-ascii encode-able character like 'Fūshin' (see the u?)

The code attempts to lookup the default system encoding and decodes based on that or falling back to utf-8 if nothing is present (unlikely) -- so as indicated by the traceback yours is defaulting to 'ascii'. Python determines this by either looking at LC_CTYPE or failing that not existing looking at LANG environment variable.

To see what you have available run the below on your box to see what you can use. root@plex:[~]: locale -a Output: C C.UTF-8 POSIX

If you see some utf-8 variant try setting your LANG to that. Mine was set to 'C' and changing it to C.UTF-8 fixed my issue.

Failing the above you can just hack apart the Absolute Series Scanner.py and remove all instances of sys.getfilesystemencoding() or 'utf-8' and just force the arg to be 'utf-8' provided you have it available on your host.

kkelly87 avatar Mar 12 '23 14:03 kkelly87