Page 1 of 1

Large queue slows downloads; interface unusable

Posted: September 11th, 2010, 10:03 am
by feld
Hi guys,

When I have a large queue the interface becomes pretty much unusable and I've also noticed that the download speed drops from 2+MB/s down to about 1MB/s. This is extremely strange. It often gets so bad that I can't even get it to shutdown or restart; it simply refuses to do anything and I have to kill -9 it. Also, it's not chewing up unusual amounts of memory and I've never hit swap on this machine.

Thoughts?


Version: 0.5.4 (final)
OS: FreeBSD 8.1 (Atom CPU, 4GB ram)
Install-type: ports
Skin : smpl
Firewall Software: pf
Are you using IPV6? yes
Is the issue reproducible? yes, just drop about 50 items in your queue. helps if it's going to be a lot. Recently the queue was 300GB.

Re: Large queue slows downloads; interface unusable

Posted: September 11th, 2010, 10:43 am
by shypike
Set the refresh rate to a higher value or use the Plush skin.
When your queue is empty, start up once with the --clean parameter
to remove all debris from the cache folder.

Re: Large queue slows downloads; interface unusable

Posted: September 11th, 2010, 11:16 am
by feld
Thanks for the reply. I also rebuilt python2.6 with HUGE_STACK_SIZE and PTH (gnu pth) support. My research shows they both should give a big increase in performance -- especially the stack size because FreeBSD's default stack size is pretty small. This seems to have lowered my CPU usage a lot for both Sabnzbd and Sickbeard. The web interface is much more responsive now. I'll try out your tweaks as well.

Re: Large queue slows downloads; interface unusable

Posted: September 12th, 2010, 8:00 am
by shypike
I learn everyday!
I wasn't aware that this was possible at all.
Do you have some more details, so that we can add the info to our Wiki?

Re: Large queue slows downloads; interface unusable

Posted: September 19th, 2010, 4:27 pm
by feld
shypike wrote: I learn everyday!
I wasn't aware that this was possible at all.
Do you have some more details, so that we can add the info to our Wiki?
They're just standard options when you build Python on FreeBSD right now. The stack size increase is because on FreeBSD the default stack size is really small (I don't know the details), and the GNU Portable Thread Library (pth) is because the python developers discovered something is wrong with the way threads work on FreeBSD and how Python works, and they think it's a bug in the OS. Telling it to use the well-known GNU Portable Thread Library is a workaround and helps to mitigate the problems. They're working towards a fix in the OS so this isn't necessary.