Page 2 of 2
Re: joining at 36,000%, sysload at almost 10
Posted: October 30th, 2012, 3:54 pm
by shypike
What about just disabling article caching altogether (set it to 0)?
Re: joining at 36,000%, sysload at almost 10
Posted: October 30th, 2012, 9:20 pm
by thomasmaerz
at 9:18PM, memory usage was at 1518MB. with the use_pickle enabled, but it wasn't downloading the whole time.
Just set article caching to 0, RAM is at V=2862M R=667M
never got around to removing yenc. what will that do? also, out of curiosity what does use_pickle do?
I would like to be able to use article caching to reduce disk thrashing since every time it writes, it's actuating 11 different disks for RAID.
Thanks for all your help so far. I really want to avoid reformatting this machine's OS, because I have quite a bit of time invested in this thing.
Re: joining at 36,000%, sysload at almost 10
Posted: October 31st, 2012, 12:48 am
by shypike
Use_pickle lets SABnzbd use a different library substituting one that
has shown to be leaky in some versions on some platforms.
Still, not so much that it explains this phenomenon.
Disabling the article cache is not the right solution,
you might as well set it to something like 300m again.
After trying without yEnc, you can try to set a maximum memory usage for SABnzbd.
Unfortunately I don't know how that works,
maybe our resident Ubuntu guru Sander can tell?
All in all, my guess is that Python's garbage collector is failing.
Re: joining at 36,000%, sysload at almost 10
Posted: October 31st, 2012, 1:05 am
by sander
Yes: see
https://forums.sabnzbd.org/viewtopic.ph ... =30#p34487 how to limit memory usage under Linux.
Use
Code: Select all
ulimit -d 555111 -m 555111 -v 555111
to limit the memory usage to 555 MB. Please let us know if that works and how SAB behaves for you under these constraints.
Re: joining at 36,000%, sysload at almost 10
Posted: October 31st, 2012, 1:20 am
by thomasmaerz
ok, so i did a distribution upgrade of Ubuntu Server 12.04 --> Ubuntu Server 12.10... sorry I got antsy. I know that's risky, but I just got to the point where I was ready to reformat the damn thing despite the fact that I have a lot of custom stuff implemented that I don't even know how I did it any more :p
I am not 100% right now but I think the issue has gone away after that. I was still seeing memory usage increasing over time with use_pickle enabled and cache limit 0 before the dist upgrade. I set both settings back to how I had them... I am going to let it go overnight and see how it does. Unfortunately, this doesn't help isolate the issue further if it is fixed, except that it is something in ubuntu 12.04 server, or at least something I installed/configured incorrectly/incompatibly in 12.04 server.