Slow verifying and unpacking speeds after moving server and setting up atomic moves/hardlinks

[moostmartijn]

The old situation:
Docker with Sabnzbd and all the Starr apps on my Synology NAS. Atomic moves didn't work. Direct unpack enabled, but this slowed down the download speed alot.

New situation:
A Mac Mini M1 running Docker with Sabnzbd, all the Starr apps and Plex with mounts pointing to my Synology NAS. On the NAS I created the folder structure as in the Trash guide to allow atomic moves, which works. Direct unpack is disabled, full download speed possible.

So, importing a file from the Starr apps is almost instant, which is great! But now I have the problem that unpacking and verifying sometimes take alot of time.

Sabnzbd is connected to a VPN.


This one took more than 40 minutes, and veryifying/repairing took more than 30 minutes.


This one took 1 hour and 39 minutes to unpack. (I think Tdarr was busy transcoding during unpack)

What is causing the problem of this very slow unpacking and verifying? Can't my HDD's handle it?



I was really happy about the working atomic moves, but I did not experience the slow unpacking and verifying speeds in the old situation. Am I better off with having Sabnzbd on the same machine as the media and download folders? Or should I download to the Mac Mini, have it unpack and verify on this machine and transfer it to the NAS when it's done? (transfering to the NAS will be slow?)

Is there something else I can try to speed things up?

[sander]

SAB says a few important things:
"Download speed limited by CPU (4485x) Disk Speed (2154x)". Not good.
System performance (380.216) and disk speed (130+ MB/s) are quite good.
So the long times are strange to me. Or has SAB been running for a long time, and are these the statistics of a long time?

You can try a few things:
No "Atomic moves", just normal directories
everything (SAB and directories) on your MAC
everything (SAB and directories) on your Synology

[moostmartijn]

Thank you for your reply.

Sabnzbd was online for one day, but... I think Tdarr was running sometimes which capped the read/write speed of the disk or capped the LAN bandwith. That may have caused the very slow unpacking issues. But still, how long would unpacking normally take for the above two movies?

And would I benefit from having Sabnzbd on the same host as the media folder? Or is there any downside of running Docker containers on another host not aside the media folder?
The Mac Mini M1 has a far better CPU than the Synology, that's why I moved all of the containers to the Mac, but the media folders remain on the Synology NAS. Unfortunately the Mac does not have enough storage to have all the media on it.

In the coming days I will move from SHR to Raid 0 for faster read/write speeds and I will consider to move some Docker containers back to my Synology IF I would benefit from it.

[sander]

> But still, how long would unpacking normally take for the above two movies?

I have DirectUnpack on, so I can't check in my logfiles.
I did a unrar of a rar set leading to a 5GB file, and that took 1.5 minute on my slow VPS.
Note: repairing (if neede) takes much longer

Best setup: SAB and disk on same machine, fast CPU, fast disk
2nd best setup: SAB and Incomplete on one machine, and Complete on a FAST remote disk.

But ... you can experiment yourself. With everything on your MAC it should be superfast. Nice start point to see the impact of remote drives.

Use the the SAB 1GB and 10GB test download (via the Wrench) for the best results. Measurements:
- download and unpack time
- Wrench output like "Download speed limited by CPU (4485x) Disk Speed (2154x)".

[moostmartijn]

Unfortunately the download speed is about 1/3 of full speed (with VPN on) when direct unpack is enabled. But in the end it doesn't really matter maybe, not sure yet. Will have to do some testing to see which is faster.

I will experiment in the coming days. Thanks for the reply. I will find out what will be the fastest configuration for me.

[sander]

Then try without vpn