Long 'Repair > Starting Repair' phase for large nzb's

[Zazz]

Version 0.6.4 on Ubuntu 11.04 'Natty'.

After downloading a nzb consisting of 20 50 file rarsets, I noticed the 'Repair > Staring Repair' message lingering for multiple minutes.
Some checking of the log files showed this to be related to the sabnzbd.newsunpack.build_filelists method. While there is a check_rar parameter, that only seems to apply to the joinables array, with every candidate for the rars array still being verified.
This would create a small but significant delay on a single 1000 file rar set, but there being 20 results in this operation being performed an additional 19 times.

Replacing (line 1217)

Code: Select all

rars = [f for f in filelist if RAR_RE.search(f) and is_rarfile(f)]

with

Code: Select all

if check_rar:
    rars = [f for f in filelist if RAR_RE.search(f) and is_rarfile(f)]
else:
    rars = [f for f in filelist if RAR_RE.search(f)]

solved the problem for me, but I haven't spent enough time with the sabnzbd code to know if this change would break anything.

A more solid solution would probably be to change the is_rarfile method or create a wrapper method to add some elementary caching of results.

As I'm running some unholy VMware ESXi + Solaris + ZFS + NFS + Ubuntu hybrid, I realize this might not be a problem in most other use cases, but I hope you can still take a look at it.

[shypike]

You are right, this implementation is inefficient.
But it's also true that this is only a problem in extreme cases
and even then it's adding minutes to hours, I guess.

I don't know if your fix is acceptable.
The code needs to cater for some very complex situations
dealing with whether rar segments need to be handled directly
by unrar or by par2-joining.
I'll look into this, but not with a high priority.