Can't quite max speed (CPU bottleneck?)

[houkouonchi]

I am having an issue where I can't quite max out my download speed while downloading with sabnzb. Also I wanted to state that sabNZB is great software and really enjoying it so far.

I appear to be getting between 1-1.5 megabytes/sec under my connections maximum speed. I believe this is caused by the yenc conversion or when things are being written to disk. I did notice that python's CPU usage is between 89-96 in top:

Code: Select all

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
17452 root      20   0  520m 207m 4512 S   95  0.9   4:25.50 python2.6

My connection should max out at around 18.2 megabytes/sec This is one of the faster results but it was mainly staying in the 16.X MB/sec area.

It does appear to be maxing at times but it appears some what spikey. You can see the download is not constant in gkrellm:



Compare the network graph to another screenshot of while I also did an HTTP download and the connection is a lot more constant near the maximum line speed:



Before sabnzb I was able to get my full connection speed with pan. That being said it did use more CPU usage but the speed was consistent. Pan used around 150-170% CPU usage.

[houkouonchi]

Ok, So I am not sure if this is because of the fact it had errors and the server was being slow or that caused additional CPU usage that caused the bottleneck. The file wouldn't repair as it had a lot of errors. I tried adding supplemental NZB just to play around with it and now its using less CPU usage and the download is much more stable/maxing my connection:

Code: Select all

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
17452 root      20   0  704m 425m 4668 S   78  1.8  24:36.40 python2.6

So not sure of the cause but when downloading files that aren't erroring out all the time I am seeing perfect maxing of download speed at less CPU usage than pan. Nice job guys =)

[shypike]

Did you install the yEnc package?
Did you set an article cache in Config->General?

[houkouonchi]

Did you install the yEnc package?
Did you set an article cache in Config->General?

Yes to yEnc but it didn't seem to make that large of a difference and yes I have the cache set to 300M.

[shypike]

Well, that's pretty much it.
Python makes our solution simply less efficient.
We do want to improve the speed, but after bugs are out and more features are in

[houkouonchi]

Well the actual bottleneck seemed to be the fact that the specific file I downloaded had errors and that was causing the slowdown. When I downloaded another file I was able to get the full speed this time (and less CPU usage to boot). Honestly I am pretty happy with the CPU usage.

[shypike]

Retries over multiple servers for missing articles is time consuming.
Especially when servers refuse to admit that the articles are missing.