Repair says 100% but not working

[kanarie2007]

Hello,

First of all, my compliments to all the coders of this excellent piece of software.

I have SABnzbd-0.4.6 running on CentOS 5.2 linux and I can't get repairs to work.
The par2 executable works fine from the command line.

When a download is complete, sab checks the files and in case of corruption, downloads the correct amount of par2 repair files, but in the history I get:

Repairing : 100%, No post-processing because of failed verification.

The par2 files are correctly downloaded, but no repairs to the damaged files have been done.

When I execute 'par2 r filename.par2' from the command line (terminal) the files are scanned and repaired correctly and I can unpack the files.

Am I doing something wrong here?


All suggestions are very welcome.

Kanarie.

[shypike]

Which of the many variants of par2 are you using?
What does the signon say?
What did you fill in for par2 parameters on the page Config->Switches?

[kanarie2007]

I use par2cmdline version 0.4 and have no parameters given in Switches.

[shypike]

Did you happen to check the logging?
Could set logging to "debug" level (Config->Connections), download a small job (that has par2 files) and send the
zipped or rarred log files to bugs@sabnzbd.org?

None of the team members uses CentOS, so we have no experience with this plaform.
(There are just too many Linux distros  )

[kanarie2007]

CentOS = RHEL5, maybe someone is using RH? I compiled par2 from source and tested it on various par2 files, no problem.

I did some more testing and with some downloads it repairs perfect, but some don't.

Later this day I will turn on debug logging and start the failed download again. (takes some time, 5Gb on a 1Mbit link).

[kanarie2007]

I attached the log of the problem download (stripped to most relevant part).

The file is in gzip format, but I had to add the .log extention to upload, so please remove and rename to *.log.gz.

[switch]

Please don't post nzb files to the forum. Send it in an email to bugs@sabnzbd.org with this topic url in the email, Thanks.

[kanarie2007]

Sorry, didn't realize that posting nzb's was not allowed. Mailed the nzb and log to bugs@sabnzbd.org

[kanarie2007]

I tested again with differtent downloads. This problem occurs in about 30% of all repair attemps, so 70% of all repairs are working well.

[shypike]

This will be difficult to diagnose.
Please email the failing NZB's (or just the newzbin report numbers).

If I can find some time, I might set up a CentOS VMWare image.
Can you tell me what and how you got and installed the required software (python-,py-modeules, par2, unrar etc.)

[kanarie2007]

I already mailed the nzb, but you can search on binsearch by  xxxxx (rascalli edit .. no names of releases please)  (I know its for my kids).

The installation I did was the following (maybe useful in the wiki?):

Standard CentOS text-only server installation with webserver.

Enable the Dag repro by adding to /etc/yum.repros.d/CentOS-Base.repro:

[dag]
name=Dag RPM Repostory for Red Hat Enterprise Linux
baseurl=http://apt.sw.be/redhat/el$releasever/en/$basearch/dag
gpgcheck=1
enabled=1
gpgkey=http://dag.wieers.com/packages/RPM-GPG-KEY.dag.txt

Install available packages:
yum install gcc python-devel cc-devel libgcc gcc-c++ gcc-devel python-cheetah rar unrar zip unzip screen

In one of my previous posts I said I compiled par2 from source, but that wasn't true.
I remember now I downloaded par2 pre-compiled, as the source won't compile on my machine
(stops with: 'warning: ignoring packed attribute because of unpacked non-POD field').

According to my .bash_history I download the following version:

wget http://chuchusoft.com/par2_tbb/par2cmdl ... n32.tar.gz
Unpack and copy the par2 exec. to /usr/bin

make two symlinks for par2:
ls -s /usr/bin/par2 /usr/bin/par2repair
ls -s /usr/bin/par2 /usr/bin/par2verify

Then some packages from source:

wget http://sabnzbd.sourceforge.net/yenc-0.3.tar.gz
gzip -d yenc-0.3.tar.gz
tar xf yenc-0.3.tar
cd yenc-0.3
python setup.py install

wget http://feedparser.googlecode.com/files/ ... er-4.1.zip
mkdir feedparser-4.1
cd feedparser-4.1
unzip ../feedparser-4.1.zip
python setup.py install

create a normal user to run sab:
useradd -m sabnzbd
passwd sabnzbd (give the user a password)

su sabnzbd
cd ~
wget http://heanet.dl.sourceforge.net/source ... src.tar.gz
mkdir SABnzbd-0.4.6
cd SABnzbd-0.4.6

I made some dirs of course and started sab from screen, so I can monitor it and detach from a terminal:

screen -R
python SABnzbd.py -b0 -s 0.0.0.0:8080
(followed by a 'ctrl-a' 'd' to detach)

Maybe its not totally accurate, but this is what I remember and haven't tested it on a clean install. Perhaps someday I will to make a wiki article for sab on centos

[kanarie2007]

Just got another download with the same error. Search for 69110109014 and you will find it.

All files have downloaded, SAB says: Repairing : 100% - No post-processing because of failed verification

With a par2 v filename.par2 I see that 091.rar and 092.rar are damaged. (output stripped):
Target: "XxiBiTe Team Post.RK.part090.rar" - found.
Target: "XxiBiTe Team Post.RK.part091.rar" - damaged. Found 30 of 31 data blocks.
Target: "XxiBiTe Team Post.RK.part092.rar" - damaged. Found 30 of 31 data blocks.
Target: "XxiBiTe Team Post.RK.part094.rar" - found.

sab has tried to repair, because there are a 091.par.1 and 092.rar.1 file present. ls -al (filterd output):

-rw-rw-rw-  1 sabnzbd users 50000000 Jan 22 20:29 XxiBiTe Team Post.RK.part091.rar
-rw-rw-rw-  1 sabnzbd users 48848000 Jan 22 20:19 XxiBiTe Team Post.RK.part091.rar.1
-rw-rw-rw-  1 sabnzbd users 50000000 Jan 22 20:29 XxiBiTe Team Post.RK.part092.rar
-rw-rw-rw-  1 sabnzbd users 49232000 Jan 22 20:21 XxiBiTe Team Post.RK.part092.rar.1
-rw-rw-rw-  1 sabnzbd users 50000000 Jan 22 20:22 XxiBiTe Team Post.RK.part093.rar

The repaired files are indeed 50000000 bytes, but still corrupt.

Now I tested the following (made a copy of all files first):

Start repair on the already repaired files: repair success.

Start repair on the original files (deleted 091.rar, rn 091.rar.1 091.rar and so on): repair failed!
Output:
Target: "XxiBiTe Team Post.RK.part091.rar" - damaged. Found 29 of 31 data blocks.
Target: "XxiBiTe Team Post.RK.part092.rar" - damaged. Found 29 of 31 data blocks.
Target: "XxiBiTe Team Post.RK.part093.rar" - found.
Target: "XxiBiTe Team Post.RK.part094.rar" - found.

Scanning extra files:


Repair is required.
2 file(s) exist but are damaged.
92 file(s) are ok.
You have 2906 out of 2910 data blocks available.
You have 7 recovery blocks available.
Repair is possible.
You have an excess of 3 recovery blocks.
4 recovery blocks will be used to repair.

Computing Reed Solomon matrix.
Constructing: done.
Solving: done.

Wrote 100000000 bytes to disk

Verifying repaired files:

Target: "XxiBiTe Team Post.RK.part091.rar" - damaged. Found 30 of 31 data blocks.
Target: "XxiBiTe Team Post.RK.part092.rar" - damaged. Found 30 of 31 data blocks.
Repair Failed.

Start repair again and repair is successfull!

The problem must be in my par2 executable.

I tried again to compile par2cmdline, now with the par2cmdline-0.4-gcc4.patch applied from:
http://aur.archlinux.org/packages.php?d ... s=1&ID=741
Compiled with no problems.

Start the test all over again and par2 repaired correctly in one run:

Target: "XxiBiTe Team Post.RK.part090.rar" - found.
Target: "XxiBiTe Team Post.RK.part091.rar" - damaged. Found 29 of 31 data blocks.
Target: "XxiBiTe Team Post.RK.part092.rar" - damaged. Found 29 of 31 data blocks.
Target: "XxiBiTe Team Post.RK.part093.rar" - found.
Target: "XxiBiTe Team Post.RK.part094.rar" - found.

Scanning extra files:


Repair is required.
2 file(s) exist but are damaged.
92 file(s) are ok.
You have 2906 out of 2910 data blocks available.
You have 7 recovery blocks available.
Repair is possible.
You have an excess of 3 recovery blocks.
4 recovery blocks will be used to repair.

Computing Reed Solomon matrix.
Constructing: done.
Solving: done.

Wrote 100000000 bytes to disk

Verifying repaired files:

Target: "XxiBiTe Team Post.RK.part091.rar" - found.
Target: "XxiBiTe Team Post.RK.part092.rar" - found.

Repair complete.


Many thanks for your help, I now have a full functional sab installation on CentOS.

I hope this will help others who have the same (par2) problem.

[shypike]

Good news.
Happy downloading !

I got CentOS-5.2 running on VMWare now.
Needed the par2 patch to get it to compile at all.
Cannot download the job you mentioned (virtual disk is too small).

Maybe I'll write up a simple installation instruction for the Wiki.

[kanarie2007]

Yeah, is a big job...

Just finished a nice working init script for CentOS, maybe usefull for the Wiki:

Code: Select all

#! /bin/sh
#
# sabnzbd    This shell script takes care of starting and stopping
#            the SABnzbd software on RHEL and CentOS.
#
# chkconfig: - 90 10
# description: SABnzbd is a full automatic usenet downloader
#
# This script assumes SABnzbd is installed in the SABnzbd-versionnumber
# directory of the homedirectory the user running it.


# Source function libarary
. /etc/init.d/functions


USERNAME="sabnzbd"
HOST="localhost"
PORT="8080"
VERSION="0.4.6"


case "$1" in
    start)
        echo "Starting SABnzbd $VERSION."
        /usr/bin/sudo -u $USERNAME -H python /home/$USERNAME/SABnzbd-$VERSION/SABnzbd.py -b0 -s 0.0.0.0:$PORT -f /home/$USERNAME/.sabnzbd/sabnzbd.ini --browser 0 -d
        ;;
    stop)
        echo "Shutting down SABnzbd $VERSION."
        /usr/bin/wget -q --delete-after "http://$HOST:$PORT/sabnzbd/api?mode=shutdown"
        ;;
    status)
        CHECK=`netstat -l -n |grep $PORT|grep LISTEN|wc -l|cut -c1`
        if [ "$CHECK" == "1" ] ; then
            echo "SABnzbd is running on port $PORT."
        else
            echo "SABnzbd not running."
        fi
        ;;
    *)
        echo "Usage: $0 {start|stop|status}"
        exit 1
esac
exit 0

Place it in /etc/init.d
Supports service command to start, stop and status check and chkconfig to automatically start on boot.

[shypike]

Just put up the Wiki page.
http://sabnzbd.wikidot.com/install-centos5