Unpack stuck

[hypersucker]

Win10. Latest sabnzbd.

Every now and then an unpack process seems to get stuck somehow and holds up the whole queue.

A 3GB file takes 40+ minutes, but eventually finishes if I wait long enough. If I manually unpack that archive and delete the process from the processing queue sabnzbd then chugs along quite nicely again.

Only happens on 7zip.001 archive files. Never rars.

I disabled the 7zip switch in specials assuming it would then just use unrar to process 7zip archives but I was wrong :-)
I assume there is no way to force this?

Any way I can find out what specifically goes wrong in 7zip?

[safihre]

Enable Debug logging in the Status window and then after it happens again click Show Logging in the Status window.
It will tell you what it spent time on.
What storage is it writing to? Some NAS or usb drive?

[hypersucker]

I have +debug logging on permanently. But I'll wait until it happens again, prob easier to find then.

Currently it writes to a USB3 external drive. But it also happens when it writes to internal SATA drives. Never wrote directly to NAS.

[safihre]

We don't have timed logging for the 7zip output.. But might tell something.

[hypersucker]

I wouldn't mind that much since at some point it resolves but some times it takes so long that sab fills up the temp drive, pauses and never restarts once the queue is worked off.

Couldn't sabnzbd just check periodically if the drives (all, not just the destination drive) involved are avail again and restart on its own?

[safihre]

It can, you have to Enable it in Config Folders.

[hypersucker]

Huh, indeed there it is! Thanks

[hypersucker]

So it happened again. I realized it about 30min into the stuck file unpack.

I hit show logging but that file isn’t even listed in the log anymore (and neither are the next few files in the unpack queue).

How do I catch that file in the logs? Even if I’m lucky enough to actually see that stuck process in real time it’ll be at least 10min in since it takes a while to see that it is stuck.

[safihre]

In the folder that has your sabnzbd.ini there is also all the log backups of older log files. It stores 5 backups.

[hypersucker]

Found it. I'm not really great at this but it shows as completed in the logs at least 10min before it showed as completed in sabnzbd and the rest of the queue processed.

Also it looks to me as if sab sees it's completed and all is good (no missing stuff) and it still goes round and round to mention missing stuff. But again, not good at analyzing logs.

Can you have a quick look?

[safihre]

Yes you can send the logs to me at safihre@sabnzbd.org
But please also include the log from Status window as it includes an anonymous copy of your settings so I understand what's going on.

[hypersucker]

Done

[safihre]

You did not include the log from Status window, so I cannot see your settings.
But I already see the problem: it is spending many minutes deleting temporary files, which are only written in this amount when your cache is way too small or your download speed is much faster than your CPU.
It is not 7zip dependent at all.
So, what is your cache set to in Config General? 1G?
And maybe just limit your download speed to something manageable.

[hypersucker]

I’ve now sent the complete log incl. the ini data.

I have some doubts about your conclusion.

It NEVER happens with rar files (no matter the size) or with non-archives, only with 001+ 7zip files.

It only started to happen a few months ago, before this it ran smoothly for years and I have not changed settings or hardware.

Can I change to another release of 7z or should I change drive config re: temp files?

Also, while a 7z unpack is stuck in the unpack queue I can easily just manually unpack that archive (although using my installed version of 7z) and it’ll unpack smoothly and in record time, so I dont think it’s normal resources that are the problem.

I can also - using the same hardware - easily saturate all pipes and drives using parpar/rar/7z creating files and posting files without bottlenecks causing issues like this.

For me this is a dedicated sabnzbd/7z issue.

[safihre]

When you open the Status window it will have shown Speed Limited by Disk, after it happens again.

It seems something is causing a near complete lock of your disks, resulting in thousands of little files being written to disk to prevent memory overflow.

What is the output of the performance tests in the Status window (ran while the queue is paused)?

Yes, it could be something specific in the 7zip release, or it's something wrong with your disks, memory or drives.
We have seen this before.