Unpack mystery

[vbsaltydog]

Ubuntu 10.0.4
Sab Repo 0.6.6
Unrar (rarlab 4.0.1)

I am going to skip all of the prefacing and get straight to the point. I have set the log level to the most verbose (2) and I have grep'd through a massive log file to get to the particular context which is the post processing of an nzb that I know unpacks properly on my macbook does fails to unpack on my Ubuntu media center. Here is the log data:

Code: Select all

Starting PostProcessing on secret-show-name_S01E01_720p_HDTV_x264-MOMENTUM => Repair:True, Unpack:True, Delete:True, Script:None, Cat:*
2011-07-31 22:26:39,545::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part11.rar OK
2011-07-31 22:26:39,545::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.sfv OK
2011-07-31 22:26:39,545::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.srr OK
2011-07-31 22:26:39,546::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part10.rar OK
2011-07-31 22:26:39,546::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part05.rar OK
2011-07-31 22:26:39,546::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part04.rar OK
2011-07-31 22:26:39,547::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part12.rar OK
2011-07-31 22:26:39,547::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part09.rar OK
2011-07-31 22:26:39,556::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part08.rar OK
2011-07-31 22:26:39,557::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part03.rar OK
2011-07-31 22:26:39,557::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part02.rar OK
2011-07-31 22:26:39,558::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part07.rar OK
2011-07-31 22:26:39,558::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part06.rar OK
2011-07-31 22:26:39,558::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e01.720p.hdtv.x264-momentum.part01.rar OK
2011-07-31 22:26:39,559::INFO::[newsunpack:756] Quick-check for secret-show-name.s01e01.720p.hdtv.x264-momentum is OK, skipping repair
2011-07-31 22:26:39,559::INFO::[newsunpack:813] Deleting /home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.par2
2011-07-31 22:26:39,560::INFO::[newsunpack:827] Deleting /home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.vol00+1.par2
2011-07-31 22:26:39,577::INFO::[postproc:296] Running unpack_magic on secret-show-name_S01E01_720p_HDTV_x264-MOMENTUM
2011-07-31 22:26:39,580::DEBUG::[newsunpack:1228] build_filelists(): joinables: []
2011-07-31 22:26:39,580::DEBUG::[newsunpack:1229] build_filelists(): zips: []
2011-07-31 22:26:39,580::DEBUG::[newsunpack:1230] build_filelists(): rars: ['/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part07.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part05.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part06.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part11.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part03.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part12.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part01.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part09.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part10.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part08.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part02.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E01.720p.HDTV.x264-MOMENTUM/secret-show-name.s01e01.720p.hdtv.x264-momentum.part04.rar']
2011-07-31 22:26:39,589::DEBUG::[newsunpack:1231] build_filelists(): ts: []
2011-07-31 22:26:39,590::INFO::[postproc:298] unpack_magic finished on secret-show-name_S01E01_720p_HDTV_x264-MOMENTUM
2011-07-31 22:31:51,515::DEBUG::[assembler:106] Got md5pack for set secret-show-name.s01e02.hdtv.xvid-momentum
2011-07-31 22:34:25,316::INFO::[postproc:237] Starting PostProcessing on secret-show-name_S01E02_HDTV_XviD-MOMENTUM => Repair:True, Unpack:True, Delete:True, Script:None, Cat:*
2011-07-31 22:34:25,332::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part05.rar OK
2011-07-31 22:34:25,332::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part04.rar OK
2011-07-31 22:34:25,340::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part07.rar OK
2011-07-31 22:34:25,341::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part06.rar OK
2011-07-31 22:34:25,341::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part01.rar OK
2011-07-31 22:34:25,341::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part03.rar OK
2011-07-31 22:34:25,342::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part02.rar OK
2011-07-31 22:34:25,342::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part12.rar OK
2011-07-31 22:34:25,342::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part13.rar OK
2011-07-31 22:34:25,349::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part10.rar OK
2011-07-31 22:34:25,349::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part11.rar OK
2011-07-31 22:34:25,349::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part09.rar OK
2011-07-31 22:34:25,350::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.part08.rar OK
2011-07-31 22:34:25,350::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.sfv OK
2011-07-31 22:34:25,350::DEBUG::[newsunpack:1256] Quick-check of file secret-show-name.s01e02.hdtv.xvid-momentum.srr OK
2011-07-31 22:34:25,350::INFO::[newsunpack:756] Quick-check for secret-show-name.s01e02.hdtv.xvid-momentum is OK, skipping repair
2011-07-31 22:34:25,351::INFO::[newsunpack:813] Deleting /home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.par2
2011-07-31 22:34:25,352::INFO::[newsunpack:827] Deleting /home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.vol00+01.par2
2011-07-31 22:34:25,364::INFO::[postproc:296] Running unpack_magic on secret-show-name_S01E02_HDTV_XviD-MOMENTUM
2011-07-31 22:34:25,375::DEBUG::[newsunpack:1228] build_filelists(): joinables: []
2011-07-31 22:34:25,376::DEBUG::[newsunpack:1229] build_filelists(): zips: []
2011-07-31 22:34:25,376::DEBUG::[newsunpack:1230] build_filelists(): rars: ['/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part08.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part03.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part04.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part06.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part02.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part05.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part10.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part11.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part01.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part13.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part07.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part12.rar', '/home/vbsaltydog/downloads/incomplete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM/secret-show-name.s01e02.hdtv.xvid-momentum.part09.rar']
2011-07-31 22:34:25,376::DEBUG::[newsunpack:1231] build_filelists(): ts: []
2011-07-31 22:34:25,377::INFO::[postproc:298] unpack_magic finished on secret-show-name_S01E02_HDTV_XviD-MOMENTUM

This data suggests that the rar files were unpacked successfully, there are no WARNING or ERROR lines in the log file and the Sab front end reports no errors but when I go to the complete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM folder, the rar files are there and there is no media file suggesting that the unpack did not work?

I have gone to the complete/secret-show-name.S01E02.HDTV.XviD-MOMENTUM folder and run

Code: Select all

unrar e the-first-ordered-rar-file.rar

and I get a read error during the unrar process. Something doesnt make sense when a manual unrar reports a fatal error but the Sab log file says everything went fine yet it did not because I have no media files.

Any suggestions?

Thank you

[shypike]

First of all, all the OK's mean that according to the par2 files all RAR files are OK.
Sometimes you still get CRC errors because the poster goofed, but that's not the issue here.
I don't know the internals of unrar, but I can think of just two causes for read errors.
1. problems in the file system
2. SABnzbd writes incomplete files (the verification is done "in-memory", so just before writing to disk).

The best check is to transfer all the RAR files to a Windows or OSX system and do the unpacking there.
If that succeeds, the problem is narrowed down to unrar (but file system issues are still possible).

There's one more test you can do.
Disable QuickCheck in Config->Switches.
If you do that, SABnzbd will run the par2 program instead of doing an in-memory check.
Par2 tests from the actual disk files and not from memory so it would expose incomplete files and/or file system errors.

[combspecialist]

I think the intial post needs a serious edit - we're not supposed to post details on what is being downloaded are we?

[vbsaltydog]

I do appreciate the concise, informative answer. I am reinstalling Ubuntu now after a test run with Mint 11.
I will report the results of the suggested troubleshooting in a bit.

I have the JCFP PPA enabled, should I ignore this and just pull Sab from the Ubuntu Main Repo?

[vbsaltydog]

I think the intial post needs a serious edit - we're not supposed to post details on what is being downloaded are we?

OP has been scrubbed of any download definitions.

[shypike]

I have the JCFP PPA enabled, should I ignore this and just pull Sab from the Ubuntu Main Repo?

The main repo is way behind and contains versions that we no longer support.

BTW: did you check the RAR files on another platform and were they correct?
Incompleteness of files after they've been QuickChecked, would be a serious problem.

[sander]

I have the JCFP PPA enabled, should I ignore this and just pull Sab from the Ubuntu Main Repo?

I'm on Ubuntu, and this is what I always do:

1) "sudo apt-get install sabnzbdplus" to get all dependencies (like yenc etc) in place.
2) Then download SABnzbd src from http://sourceforge.net/projects/sabnzbd ... abnzbdplus, unpack it in my home directory, and run it.

[vbsaltydog]

I have the JCFP PPA enabled, should I ignore this and just pull Sab from the Ubuntu Main Repo?

The main repo is way behind and contains versions that we no longer support.

I am back up on Ubuntu 10.0.4 with the PPA enabled and Sab installed so I am about to test your suggestions.

BTW: did you check the RAR files on another platform and were they correct?
Incompleteness of files after they've been QuickChecked, would be a serious problem.

I only tested it by pulling the same nzb on another platform to make sure the source was good and it was... repeatedly.
I am going to disable the quick checking now and test as you suggested.

[vbsaltydog]

I have the JCFP PPA enabled, should I ignore this and just pull Sab from the Ubuntu Main Repo?

I'm on Ubuntu, and this is what I always do:

1) "sudo apt-get install sabnzbdplus" to get all dependencies (like yenc etc) in place.
2) Then download SABnzbd src from http://sourceforge.net/projects/sabnzbd ... abnzbdplus, unpack it in my home directory, and run it.

How do you handle the init script since /etc/default/sabnzbdplus will launch the packaged version vs. your source version?

[vbsaltydog]

I just pulled a small test nzb (250mb) after disabling the "quick check" and I ended up with no errors and a directory containing avi files vs rar files, ...yes!

I am pulling the nzb that has been failing to unpack all day now for the real test.

Should I also enable the "fail on CRC errors" switch?

[vbsaltydog]

Well, while the end result wasn't the mkv file that I wanted, I did get the familiar steps of "Repairing" and "Verifying" before marking the nzb as failed which is what I am used to seeing and is a much better indicator of a failed nzb than no front end errors and a bunch of corrupt rar files in a completed directory so I am a happy camper.

Thanks to everyone and especially shypike for helping to restore a bit of my sanity.

[shypike]

I not happy with this.
I would like to know whether QuickCheck approves files which are not OK.
Can you email the NZB file in question to bugs at sabnzbd.org ?

[vbsaltydog]

Absolutely

[sander]

I have the JCFP PPA enabled, should I ignore this and just pull Sab from the Ubuntu Main Repo?

I'm on Ubuntu, and this is what I always do:

1) "sudo apt-get install sabnzbdplus" to get all dependencies (like yenc etc) in place.
2) Then download SABnzbd src from http://sourceforge.net/projects/sabnzbd ... abnzbdplus, unpack it in my home directory, and run it.

How do you handle the init script since /etc/default/sabnzbdplus will launch the packaged version vs. your source version?

I don't; I start SABnzbd once by hand (cd ~ ; cd SAB<tab> ; ./SAB<tab>) after a reboot.

[vbsaltydog]

Shypike,

I submitted the bug report with the attached nzb but FYI, I have run disk checking programs against my HD and found read errors so this may be the problem for the failure to unpack. The bug would persist where checking the file(s) in memory would pass testing regardless of any subsequent disk errors.

Thanks again for the help.