Enabling SSL is slowing down downloads significantly!

[cyxis]

Hey,

I've been using SABnzbd for quite some time now (without SSL).

Recently I've enabled SSL and experienced quite the slow down when it comes to downloading the default 10GB test file.

WITH SSL: ~230-240MB/s

WITHOUT SSL: ~280-300MB/s



If you open the image you can see when I disabled SSL and the speed went up.

As far as I can tell my limit is my disk throughput, here's a test I did while the 10 GB download was running (non-SSL):

Download speed limited by - Disk speed (530x)
System performance (Pystone) -212597 Intel(R) Xeon(R) Gold 5220R CPU @ 2.20GHz AVX2
Download folder speed - 292.6 MB/s /config/downloads
Complete folder speed - 347.5 MB/s /config/complete
Internet Bandwidth - 319.22 MB/s 2553.76 Mbps

Now checking online it seems that many people say that SSL shouldn't affect your performance THAT much (I would expect the same beyond the initial establishing of the connection). I was expecting a small hit of maximum 5% but this is quite a bit more. Also, since I'm limited by my disk shouldn't SSL calculation not be a factor at all?

Didn't see anything relevant in the debug logs either beyond it saying that SSL is being used.

Can someone advise me if there's anything I can do here or if my understanding is fundamentally incorrect?

Edit:
I'm currently testing with one provider (Eweka) with 25 connections.

[safihre]

You experience about a 15% slowdown, which is not too bad I would say.
There's just a bit more work to do with the extra ssl layer involved, having to do with how data packages are decoded in the ssl layer of openssl.
You could try playing with the number of connections, or the ssl cipher: https://sabnzbd.org/wiki/advanced/ssl-ciphers

[cyxis]

You experience about a 15% slowdown, which is not too bad I would say.
There's just a bit more work to do with the extra ssl layer involved, having to do with how data packages are decoded in the ssl layer of openssl.
You could try playing with the number of connections, or the ssl cipher

I see, didn't expect 15% to be within reason, thanks for clarifying.

Do you have an SSL cipher you can recommend that is a good middle ground to still have a decent amount of security while also not loosing too much speed?

[safihre]

Start with the suggestion in the link, then you can always increase it. Although, if you check online, with aes128 it would still take many years to decrypt, so that should be plenty in my opinion.

[cyxis]

Yep that works great! Got me to the 270-280MB/s ballpark.

Also I noticed disabling direct unpack removes the issue completely and I get around 330MB/s... but I suppose in the end it would amount to the same amount of time to get the file since it would first download everything and then unpack but maybe I'm wrong here.

Thanks a bunch for the help anyway!

[safihre]

Indeed you already diagnosed that the last limit is your disk speed. I have seen that speeds are more constant if you throttle things a tiny bit. For example, set the speed limit to 270. It will keep the speed graph more constant and leaves just a few more gaps for Direct Unpack to work.
When speed limit is left unlimited, you often see spiky beviour where the internal limiters trigger on and off. While the tops of the spikes might be higher, the final download total time is often faster if you forcefully set the speed limit just a bit below the max.

[sander]

@cyxis, out of curiosity & trying to understand your statistics:

Is this SABnzbd straight on OS (Linux) straight on Intel(R) Xeon(R) Gold 5220R? Or is there a layer like VM, VPS or docker involved?

Your Internet speed is 2553.76 Mbps: is this on a home connection (XGSPON?), or something in a datacenter?

Your disk /config with speed around 300 MB/s: is that a NVMe, or a spinning disk?

[cyxis]

Thanks @sahfihre, setting it to 270 was a good tip

@sander:

Is this SABnzbd straight on OS (Linux) straight on Intel(R) Xeon(R) Gold 5220R? Or is there a layer like VM, VPS or docker involved?

Straight on Linux and straight on said processor

Your Internet speed is 2553.76 Mbps: is this on a home connection (XGSPON?), or something in a datacenter?

Datacenter and it's in reality 10gbps but since I tested under load it shows less (if I test without anything downloading I get 1053.07 MB/s but it can fluctuate)

Your disk /config with speed around 300 MB/s: is that a NVMe, or a spinning disk?

I renamed it to "config" in reality it's my home directory but yes it's 300 MB/s (817.1 MB/s before you fill up the NVMEs cache) and yes it's an NVME

I use a shared hosting provider (seedhost.eu specifically) as a downloading/caching server for media files, so if someone requests something it gets downloaded as quick as possible (it's very important to get as much "Netflix" as possible). My actual streaming server then streams the downloaded files from the VPS. In the end they get uploaded to my archive using rclone.

It's 15 euro a month for a 1TB NVME which is fine for a caching/downloading server but I wouldn't use it for anything besides that nowadays, they limit you quite a bit in terms of what you can use.

ultra.cc is another provider I use but since they have slower NVMEs I don't use them for downloads.

Eventually I want to co-locate and put a RAID0 NVME array in a datacenter to get even higher speeds but I'm still choosing between DCs.

If you have any recommendations in the EU let me know currently I'm speaking with serverius since they have great bandwidth offers.

Also as a side question is there some downside of running it on docker in terms of performance? The only reason I didn't do that on the VPS is because they manage the deployment of apps but I was planning on running it through docker on my own server, let me know if there are downsides to that.

[sander]

> I use a shared hosting provider (seedhost.eu specifically)

A, a shared account? And "seed" makes me think it's a seedbox. So no VM, but sharing, and maybe limiting CPU and disk speed.

Quite slow speed for an NVMe. My i3 laptop with NVMe has 558.9 MB/s, as reported by SABnzbd. And more modern NVMe's should give 1000+ MB/s, AFAIK.

> Also as a side question is there some downside of running it on docker in terms of performance?

Yes, less performance with docker. I think it's caused by python on Docker. So not advisable for speed.

But ... depending on what you pay, overall not bad: 300 MB/s SABnzbd download speeds ... nice.

[cyxis]

Yep you’re correct it’s shared but I’ve been told I’m using most of their IOPS anyway although I suppose the neighbours take the other half of the performance (?)

Thanks for the docker tip too!