RuntimeError on Thread Creation During Content Scraping

[phromaj]

Description

The content scraping service fails when attempting to create a new thread, which leads to a RuntimeError indicating that a new thread can't be created at interpreter shutdown. This issue occurs during the scraping of sources for new content.

Steps to Reproduce

1. Initiate the scraping process for new content.
2. Service tries to start a new thread for a scrape task.

Expected Behavior

The service should successfully create a new thread and perform the scrape task without any errors.

Actual Behavior

The thread creation fails, and after a retry mechanism, it completely halts the scraping process with a RuntimeError.

Logs

[05/11/23 21:58:53] checking new content ... done
[05/11/23 21:58:53] scraping sources [torrentio,jackett] for query "REDACTED_TITLE" ... done
[05/11/23 21:58:53] error starting new thread (perhaps maximum number of threads reached), will retry in 5 seconds and exit if it fails again.
Exception in thread Thread-1 (threaded):
Traceback (most recent call last):
  File "/scraper/__init__.py", line 26, in scrape
    t.start()
  File "/usr/local/lib/python3.12/threading.py", line 971, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/threading.py", line 1052, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 989, in run
    self._target(*self._args, **self._kwargs)
  File "/ui/__init__.py", line 474, in threaded
    element.download(library=library)
  File "/content/classes.py", line 1201, in download
    self.Releases += scraper.scrape(self.query(title).replace(
  File "/scraper/__init__.py", line 30, in scrape
    t.start()
  File "/usr/local/lib/python3.12/threading.py", line 971, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown

Environment

• Docker Swarm

Additional Context

This problem may indicate a limit on the number of concurrent threads that can be created or perhaps other resource limitations in the Docker environment. Further investigation is needed to pinpoint the exact cause and determine a solution.

[carloss66]

I am having the same issue on Docker since version 2.95.

[08/11/23 08:09:20] scraping sources [torrentio] for query "poor.things.2023" ... done [08/11/23 08:09:20] error starting new thread (perhaps maximum number of threads reached), will retry in 5 seconds and exit if it fails again. After the error, plex_debrid crashes.

[IIIEII]

https://github.com/itsToggle/plex_debrid/issues/573 helped me, you just need to enable tty in you container configuration

[phromaj]

I tried to launch the service in Docker Swarm (which should also apply to Docker Compose) with the following configuration, but it did not work:

plex_debrid:
    image: itstoggle/plex_debrid
    volumes:
      - /home/XXXX/.config/appdata/plex_debrid:/config
    tty: true
    stdin_open: true
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/Paris

However, the command you provided does work. I hope the bug will be fixed soon. Thank you for your help.

[carloss66]

#573 helped me, you just need to enable tty in you container configuration

Thanks for the tip. I am running Docker in Unraid. I edited the container configuration and added -ti under Extra Parameters and that fixed the problem for me.