joining at 36,000%, sysload at almost 10

Report & discuss bugs found in SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

Image

system is nearly unresponsive. I'm going to try to restart sabnzbd and see what that does, if that doesn't work i'll try a system reboot. Man... this version of sabnzbd is giving me all kinds of grief.

Version: 0.7.4
OS: ubuntu server 12.04
Install-type: repository
Skin (if applicable): plush
Firewall Software: linux
Are you using IPV6? no
Is the issue reproducible? not sure yet
Top
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

Re: joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

I think I may have found a memory leak:

Image

iotop shows low filesystem utilization across the board:

Image
Top
User avatar
sander
Release Testers
Release Testers
Posts: 9429
Joined: January 22nd, 2008, 2:22 pm

Re: joining at 36,000%, sysload at almost 10

Post by sander »

79.7% of your 8GB memory? That's impressive indeed.

If you stop SAB 0.7.4 and run SAB 0.7.3 on the same queue, does or doesn't it happen?
Top
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

Re: joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

Also, I've got these warnings:

Code: Select all

2012-10-29 00:06:32,533	ERROR:	[29/Oct/2012:00:06:32] Traceback (most recent call last): File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 79, in setapp s, h, b = self.get_response() File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 219, in get_response response = self.request.run(meth, path, qs, rproto, headers, rfile) File "/usr/share/sabnzbdplus/cherrypy/_cprequest.py", line 579, in run raise cherrypy.TimeoutError() TimeoutError
2012-10-29 00:06:32,531	ERROR:	[29/Oct/2012:00:06:32] Traceback (most recent call last): File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 79, in setapp s, h, b = self.get_response() File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 219, in get_response response = self.request.run(meth, path, qs, rproto, headers, rfile) File "/usr/share/sabnzbdplus/cherrypy/_cprequest.py", line 579, in run raise cherrypy.TimeoutError() TimeoutError
2012-10-29 00:06:32,529	ERROR:	[29/Oct/2012:00:06:32] Traceback (most recent call last): File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 79, in setapp s, h, b = self.get_response() File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 219, in get_response response = self.request.run(meth, path, qs, rproto, headers, rfile) File "/usr/share/sabnzbdplus/cherrypy/_cprequest.py", line 579, in run raise cherrypy.TimeoutError() TimeoutError
2012-10-29 00:06:32,491	ERROR:	[29/Oct/2012:00:06:32] Traceback (most recent call last): File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 79, in setapp s, h, b = self.get_response() File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 219, in get_response response = self.request.run(meth, path, qs, rproto, headers, rfile) File "/usr/share/sabnzbdplus/cherrypy/_cprequest.py", line 579, in run raise cherrypy.TimeoutError() TimeoutError
2012-10-29 00:06:31,777	ERROR:	[29/Oct/2012:00:06:31] Traceback (most recent call last): File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 79, in setapp s, h, b = self.get_response() File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 219, in get_response response = self.request.run(meth, path, qs, rproto, headers, rfile) File "/usr/share/sabnzbdplus/cherrypy/_cprequest.py", line 579, in run raise cherrypy.TimeoutError() TimeoutError
2012-10-29 00:06:31,361	ERROR:	[29/Oct/2012:00:06:31] Traceback (most recent call last): File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 79, in setapp s, h, b = self.get_response() File "/usr/share/sabnzbdplus/cherrypy/_cpwsgi.py", line 219, in get_response response = self.request.run(meth, path, qs, rproto, headers, rfile) File "/usr/share/sabnzbdplus/cherrypy/_cprequest.py", line 579, in run raise cherrypy.TimeoutError() TimeoutError
Top
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

Re: joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

sander wrote:79.7% of your 8GB memory? That's impressive indeed.

If you stop SAB 0.7.4 and run SAB 0.7.3 on the same queue, does or doesn't it happen?
Yeah, it's using a majority of the swap as well! The sysload from memory leak was so high that everything I tried to do took almost 5 minutes, which is really bad for a system with 8gb of DDR3 1600 and a quad core 3.3ghz processor.

How would you suggest I try 0.7.3? I'm using the version from the official repositories right now. Should I just sudo service sabnzbdplus stop then manually run a copy of 0.7.3 to test?
Top
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

Re: joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

Alright, so I had to reboot the entire computer because it was lagged down so hard I could barely type commands into the terminal over ssh. After everything came back up, the job that was giving the weirdness looks complete, and the queue is going again... sooo WTF. I'm pretty sure there's a bug here, but I just don't know how to reproduce it.

Image
Top
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

Re: joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

Checked it again this morning, and the system was basically frozen with massive RAM usage again. I can't even get into the web interface to see what's going on. I'm going to reboot again when I get home and post some logs here.
Top
User avatar
shypike
Administrator
Administrator
Posts: 19773
Joined: January 18th, 2008, 12:49 pm

Re: joining at 36,000%, sysload at almost 10

Post by shypike »

To what size did you set the article cache in Config->General?
Top
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

Re: joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

shypike wrote:To what size did you set the article cache in Config->General?
800M to reduce IOPS since it writes directly to a MDADM raid5
Top
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

Re: joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

Just reproduced the issue on a totally different nzb, "joining at 36000%" sysload and ram usage off the charts. I'm going to reboot and post my +debug logs in here.
Top
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

Re: joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

Upon rebooting, I'm watching the Ram info at the top left of sabnzbd, and it is going up like 2MB/sec...

I double checked my article cache limit, and I do have it set to 1G, but I have 8 gig of ram and the system only uses about 5-700mb without sabnzbd running, so I thought that was reasonable. I let it go for a minute, and according to the status page, it is sitting at 1.0GB (2910 articles) cached. Ram usage is stable, htop says I have between 2.2 and 2.3GB ram used at any given moment. I'm going to wait and see what happens as the jobs go on and update here.
Top
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

Re: joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

Now I have 2.6 gigs of ram used. It is raising at a slow and steady pace.
Top
User avatar
shypike
Administrator
Administrator
Posts: 19773
Joined: January 18th, 2008, 12:49 pm

Re: joining at 36,000%, sysload at almost 10

Post by shypike »

Is any download ever completed?
Looks more like an issue with the Python interpreter.
The quality varies and some have memory leakage problems.
Then again, we haven't seen issues with stock Ubuntu.
The behaviour that you are describing is rather absurd
and not recognized in any way by me. Nor are there comparable complaints on the forum.

I have two suggestions to test, simply to rule out potential issues.
Go to Config->Special and enable the option "use_pickle". After that, restart SABnzbd.
Remove the package python-yenc from your system. Requires also a restart.
You might want to try those separately.
Top
User avatar
sander
Release Testers
Release Testers
Posts: 9429
Joined: January 22nd, 2008, 2:22 pm

Re: joining at 36,000%, sysload at almost 10

Post by sander »

One additional advice from me: clean up your queue, restart SABnzbd and just download a 100MB post to see if that works.
Top
thomasmaerz
Newbie
Newbie
Posts: 22
Joined: September 12th, 2012, 2:47 pm

Re: joining at 36,000%, sysload at almost 10

Post by thomasmaerz »

Yes, downloads are completing. Even the ones that say joining at 36000% are complete after a reboot or service restart.

I just enabled use_pickle, and restarted the service from the gui.

at 2:52PM CST, sabnzbd gui reads memory usage as: V=3104M R=937M
Top
Post Reply

Return to “Bug Reports”