Unpackerr
unpackerr consuming insanely high CPU and IO
On Ubuntu 22 64bit I was running unpackerr_0.11.2-505_amd64.deb and saw in the logs that there was an error about not being able to work with the log file even after checking permissions and ownership. Realizing that was an old version, I archived the log file and upgraded to unpackerr_0.14.5-797_amd64.deb. It immediately started ramping up the CPU to 200-400% and was consuming all the IO until it pulled a beefy server to it's knees. I checked iotop and shortly after restarting unpackerr I saw that it had read over 80G in a very short time. Stopping unpackerr allowed my server to go back to normal. Any ideas what would cause this to happen?
Reopened issue. I only have 1 parallel process set and unpackerr keeps ramping up ridiculously high unlike the older version or when I ran this on a mac. I have to use cpulimit to limit the processor otherwise every few minutes unpackerr locks up the server for a few minutes at a time. There are active downloads in transmission however there zero just completed downloads that would explain this.
I forgot to turn on the switch for logging in the new config I made with your configurator so everything dumped to syslog. I extracted that and have attached it. I redid the config and turned on logging into it's own file. I can provide a new sample of that later.
Removing things from queue will likely alleviate the problem. This is a bit much.
02-02T09:34:00.562864-07:00 localhost unpackerr[2385360]: [INFO] 2025/02/02 09:34:00 [Radarr] Updated (http://127.0.0.1/radarr): 96 Items Queued, 96 Retrieved
2025-02-02T09:34:00.635148-07:00 localhost unpackerr[2385360]: [INFO] 2025/02/02 09:34:00 [Sonarr] Updated (http://127.0.0.1/sonarr): 956 Items Queued, 956 Retrieved
2025-02-02T09:34:03.590995-07:00 localhost unpackerr[2385360]: [INFO] 2025/02/02 09:34:03 [Lidarr] Updated (http://127.0.0.1/lidarr): 380 Items Queued, 380 Retrieved
I have no idea how those can be in the queue. I have items downloading SLOW from the lack of seeders however nothing is done and decompressing.
I'm not sure I understand what you mean. Go into Sonarr, click Activity -> Queue. Remove 900+ things.
I see items in that list I know i have downloaded from elsewhere because of failures etc. If I do that, it says it will remove them from the download client. Won't that destroy my rankings on the torrent trackers?
Sounds like you forgot to setup categories for your downloads. See questions 2 and 3. https://unpackerr.zip/docs/unpackerr/faq
I see what you're saying about the Activity. I removed all the NZBGet because they failed and I replaced them from other sources. I wish it would remove those when a scan shows they are complete. Anyway, I'll do some reading but immediately, I don't see a way to configure categories. AFAICT, that's set by Sonarr, Radarr, Lidarr etc.
I don't see a way to configure categories.
https://trash-guides.info/Downloaders/NZBGet/Paths-and-Categories/
I do have categories already configured in NZBGet but I don't have the specific paths for each category set, they use DestDir, because the *arrs set the download path. I don't have unpackerr set to monitor nzbs, only torrents.
You have 900 things in Sonarr queue. That's the problem. Those things get into queue based on the category you've chosen. You need to reduce the queue to less than 100 for things to be less chaotic.
I will look those over. Some are definitely wishlist items missing from a collection. If I were to disconnect unpackerr from sonarr, wouldn't unpackerr find those downloads and decompress them then sonarr scanning for completed downloads pick up the unpackerred items?
Sorry I don't understand your question. if you remove the sonar config from unpackerr it wont extract any of the things in Sonarr queue. How many of the 900 items are you expecting unpackerr to extract? Why are there incomplete "wishlist" items in queue? This is not making sense and I think you're using Sonarr incorrectly. 900 things in queue is the symptom of incorrect use/configuration, usually category related.
Some of the 900 were rare items that have been downloading then the seeders come on and go offline every so often and it takes a long time for those to finish. I can't just remove those from transmission otherwise I lose credit for seeding what I have. I have cleared out as many of those as I can. Sonarr/Radarr are well under 100. We'll see how it goes. Odd that the older version didn't have this problem.
I wasn't recommending removing them from Transmission, just Starr app queues.
I understood that but when removing them in the activity section, it appears to remove them from transmission too. Notice the dropdown is disabled.
Change the category on the item in Transmission to remove it from the queue in a Starr app.
That's one of the problems I've seen with Transmission, there is no category setting. It just has the download path. What you're saying works perfect with NZBGet.
Transmission on macOS calls them groups. I believe Transmission on Linux calls them labels.
Thanks, found the labels on Transmission. I'll have to see if there's a way that the *arrs can set these in Transmission. Even with the Activity list way down, it's still going crazy on CPU.
What is it doing that's making it go crazy? strace it? is there disk io? Need details and a log file.
Still seems to be the same problem.
[INFO] 2025/02/02 11:25:47 [Radarr] Updated (http://127.0.0.1/radarr): 97 Items Queued, 97 Retrieved
[INFO] 2025/02/02 11:25:48 [Sonarr] Updated (http://127.0.0.1/sonarr): 786 Items Queued, 786 Retrieved
[INFO] 2025/02/02 11:25:50 [Lidarr] Updated (http://127.0.0.1/lidarr): 380 Items Queued, 380 Retrieved
All of these things that say
Completed item still waiting need to be removed from queue.
Current counts- Sonarr: 20, Radarr: 13, Lidarr: 367. I'm working to reduce Lidarr right now. So, those counts in the logs are off. They've been updated now for about 20 min. Should I clear the log file and restart unpackerr?
Unknown is checked in all three *arrs. I did that earlier when you told me.
That was the most current. I had just downloaded it. Let me clear it out and restart unpackerr.
Perhaps tail the log to figure out if it's freezing and if that's the point at which memory pressure goes up.