high cpu load during download [NFS, Ubuntu 16.04]

[sander]

Some remarks:

- So it is only a problem on Ubuntu 16.04 (in VM), not in Ubuntu 14.04? If so, can I change the subject of this thread?
- Can you change both incomplete and complete to disk within the VM, and try again?
- A CPU load of 100% on a 8-core CPU does not have to be a problem. However, "Lost connection to SABnzbd.." message is a problem
- A python instance only uses one core. AFAIK that means that the high load overage can NOT be caused by Python.
- Did you read https://unix.stackexchange.com/question ... 970#130970 ? So 1) yes, load means waiting but not per se for the CPU and 2) "ps -e v | grep -e R -e D" is an ugly way to possible waiting PIDs

FWIW: I have Ubuntu 14.04.5 LTS in a VM, with CPU usage < 50% while downloading and unpacking:

Code: Select all

23081 sander    20   0  249972  39076   3928 S  0.3  7.7   2:01.08 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  249972  39232   3928 S  4.0  7.7   2:01.20 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  249972  39232   3928 S  1.0  7.7   2:01.23 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  249972  39232   3928 S  1.0  7.7   2:01.26 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  249972  39232   3928 S  1.3  7.7   2:01.30 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  249972  39232   3928 S  1.0  7.7   2:01.33 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  248808  37840   4104 S 13.6  7.5   2:01.74 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  268036  57052   4104 S 74.8 11.2   2:04.00 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  287616  76736   4104 S 34.8 15.1   2:05.05 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  315088 104072   4104 S 44.1 20.5   2:06.38 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  337356 126336   4104 S 38.4 24.9   2:07.54 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  361840 150804   4104 S 44.0 29.7   2:08.87 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+

...

23081 sander    20   0  431772 173640   4104 S 34.4 34.2   2:23.74 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  431772 173640   4104 S 29.5 34.2   2:24.63 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  431772 173640   4104 S 29.4 34.2   2:25.52 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  419644 163984   4104 S 38.7 32.3   2:26.69 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  419640 163980   4104 S 39.7 32.3   2:27.89 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  419640 163980   4104 S 28.8 32.3   2:28.76 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  419640 163980   4104 S 48.0 32.3   2:30.21 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  419640 163980   4104 S 22.5 32.3   2:30.89 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  369264 126388   4104 S 16.3 24.9   2:31.38 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  369264 126388   4104 S  0.7 24.9   2:31.40 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  369264 126388   4104 S  0.7 24.9   2:31.42 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+
23081 sander    20   0  369264 126388   4104 S  0.3 24.9   2:31.43 /usr/bin/python -OO /usr/bin/sabnzbdplus --daemon --pidfile /var/run/sabnz+

[scout1]

ok, spun up a 14.04 vm to test, patched it, installed sabnzbd and downloads to local complete/incomplete files without a problem

install nfs-common, change complete/incomplete paths... hangs...

try to let sabnzbd download/unpack to local complete/incomplete directories, then have sonarr rename/move it to nfs path... hangs...

so, it looks like it is an nfs issue... ugh :/ sorry for the goat rodeo, i know its not an sabnzbd issue now but if you have some insight I'd appreciate it, otherwise thanks again for the pointers and making sabnzbd!

[sander]

<snip> then have sonarr rename/move it to nfs path... hangs...

so, it looks like it is an nfs issue... ugh :/ sorry for the goat rodeo, i know its not an sabnzbd issue now but if you have some insight I'd appreciate it, otherwise thanks again for the pointers and making sabnzbd!

A simple rename/move to the NFS causes a hang? You could try to reproduce that from the CLI or Python.

Backend NFS mount is a Synology 1512+ NAS

Consider using SMB towards your Synology?

[scout1]

**SOLVED**

After hours of troubleshooting I finally pinned this down to a recent DSM update on my Synology DS1512+ NAS (In particular 6.0.1-7393 Update 2), I regressed it back to DSM 6.0.1-7393 and everything is working great now. Sorry for the foxhunt, I really appreciate the help! Thanks again!

[Roddyloo]

Hello,

I am having the same CPU load issue with my DS212+ running the latest DSM 6.0.1-7393 Update 2.
However I am not quite sure how to downgrade DSM as it seems impossible from the interface update window.

Could you redirect me to a fullproof tutorial ?

Thanks for the help.

[HorizonXP]

I seem to be having a similar issue.

Ubuntu 16.04 running in a VM. NFS shares mounted from my FreeNAS server. Permissions seem to be correct since I can read/write as the sabnzbd user.

I keep getting tempfile.mkstemp errors whenever I add any NZB to SABnzbd. I'm not sure what permission I've missed here.

Help appreciated. Thanks!

[shypike]

I keep getting tempfile.mkstemp errors whenever I add any NZB to SABnzbd. I'm not sure what permission I've missed here.

You should not have your admin and "incomplete" folders on a share, but a local drive.