0.6.6 still has memory leak on Qnap TS-119P+

[Manni]

First of all thanks for fixing the pars download not following pause status in 0.6.6. It's great to have this working now.

I have had a memory leak issue for a while on a Qnap ts-119P+. I was waiting to see if 0.6.6 would fix this, but it doesn't.

I have a fairly large download queue, but even reducing it doesn't cure the memory leak, it just slows it down.

If the download queue is 9.5TB, SAB needs to be restarted after 24/48hours.

If you bring the queue down to 1.5Tb, it takes around 5 days until you get timeout errors and have to restart it.

What happens is SAB first eats all the physical memory, then eats all the virtual memory, until it freezes.

With 0.6.6, it starts taking up 124MB (total memory usage just after install is 160MB), and after a few hours it's already taking more than 255MB (Sab itself taking 192MB). This is with a 16MB cache.

I've reported this in the Qnap forums (http://forum.qnap.com/viewtopic.php?f=133&t=46900), but the maintainer (SL1000) can't do anything about this. He has kindly had a look at my config over SSH and couldn't see anything wrong. I have followd all the advice he forwarded from one of the Sab developpers (reducing the size of the download queue, reducing the size of the cache), without being able to solve this.

I've documented the issue when the queue was larger, but it's exactly the same problem with a smaller queue, it just takes longer for the memory to be eaten by SAB.

This is with a cache of 16MB and a queue reduced to 1.2TB over 35 jobs.

1) Can you reproduce this memory leak when the download queue is over 1TB?
2) Any chance this could be fixed?

Many thanks, and keep up the good work!

[shypike]

To be honest I don't test with queues as massive as this
and not at all with embedded devices.
It takes me months to download a few TByte.
There have been reports about some Python ports not doing garbage collection right
or otherwise not returning memory.
I am looking at ways to cut down memory usage for very large queues,
but currently that's not the biggest priority.
Is there some way on this OS to restrict memory use per process.
I have seen some reports on how to do that for Ubuntu.
Check out the Ubuntu/Debian board on this forum.

[Manni]

To be honest I don't test with queues as massive as this
and not at all with embedded devices.
It takes me months to download a few TByte.
There have been reports about some Python ports not doing garbage collection right
or otherwise not returning memory.
I am looking at ways to cut down memory usage for very large queues,
but currently that's not the biggest priority.
Is there some way on this OS to restrict memory use per process.
I have seen some reports on how to do that for Ubuntu.
Check out the Ubuntu/Debian board on this forum.

Thanks for the reply.

What I don't understand is why you need to have in memory the whole queue when a skin like plush is selected. Why not only load the number of jobs in the displayed page? That would solve the issue.

I thought that was the point of having pages in the skin, sothat you don't have to deal with the whole download queue all the time.

I don't think it's possible to specify a memory limit per process, or I assume SL1000 would have sugested it.

Ive decided to schedule a reboot every day as a temporary workaround, as it doesn't seem to be a priority, but it would be nice if you could find a way to fix this more permanently.

Thanks!

[shypike]

What I don't understand is why you need to have in memory the whole queue when a skin like plush is selected. Why not only load the number of jobs in the displayed page? That would solve the issue.

It's not all in memory. Believe me, if it would memory usage would explode.
Some admin per NZB needs to be kept in memory, but it might be possible to reduce it more.

What you describe is a memory leak, which is another phenomenon altogether.
At some point in time each NZB's admin goes into memory, when it's being downloaded.
The problem is that, in your case, it never leaves the memory.

[Manni]

Thanks for the explanation. I hope you'll find a solution.