Page 1 of 1

Article Failed

Posted: April 11th, 2016, 2:01 am
by flizer
Hey everyone, i recently switched around the way i use sab. I have a little Netgear NAS that has an ARM processor, so it is not the best at processing too much. So i decided to use a little intel NUC running debian to run Sab, Sonarr, etc. With .7, it was running great without errrors, after i updated to 1.0, i started getting these errors ( Loading /media/readynas/Videos/Unprocessed Downloads/incomplete/Avatar.The.Last.Airbender.S02E19.The.Guru.480p.DVDRip.DD2.0.x264-SA89-BUYMORE/__ADMIN__/SABnzbd_article_4MKWz4 failed). The files still download and work perfectly, yet i keep getting this error. I have the NAS mapped in the FSTAB to mount as //192.168.1.2/... cifs guest,uid=1000,gid=1000,iocharset=utf8, 0 0. Im not sure if its a permission issue at this point or what. any help would be greatly appreciated!

Re: Article Failed

Posted: April 11th, 2016, 2:50 am
by shypike
You get errors, but everything works normal?
It's just useless error messages?
Can you enable Debug logging and create a log for one (small) NZB download?

Re: Article Failed

Posted: April 19th, 2016, 11:00 pm
by flizer
shypike wrote:You get errors, but everything works normal?
It's just useless error messages?
Can you enable Debug logging and create a log for one (small) NZB download?
Sorry for the late Reply, ive been having issues with creating the logs, because i guess SAB doesn't instantly create them. Here is the best i could get so far, im not not even sure this is exactly what you wanted.

2016-04-19 20:36:20,291::INFO::[decoder:226] <Article: article=1456919470.28919.69@reader.usenetbucket.com, bytes=212100, partnum=69, art_id=None> => missing from all servers, discarding
2016-04-19 20:36:20,294::DEBUG::[__init__:867] Saving data for SABnzbd_nzo_BFp8Rg in /media/readynas/Videos/Unprocessed Downloads/incomplete/Game.Of.Thrones.s05e01.720p.BluRay.x264-DEMAND-Obfuscated/__ADMIN__
2016-04-19 20:36:20,295::INFO::[downloader:661] Thread 7@news.usenet.farm: Article 1456919470.28937.2@reader.usenetbucket.com missing (error=430)
2016-04-19 20:36:20,450::INFO::[__init__:941] Saving data for totals10.sab in /home/nuc/.sabnzbd/admin/totals10.sab
2016-04-19 20:36:21,154::INFO::[decoder:226] <Article: article=1456919470.28937.2@reader.usenetbucket.com, bytes=793158, partnum=2, art_id=None> => missing from all servers, discarding
2016-04-19 20:36:21,154::INFO::[downloader:661] Thread 16@news.newshosting.com: Article 1456919470.28984.12@reader.usenetbucket.com missing (error=430)
2016-04-19 20:36:21,157::INFO::[downloader:661] Thread 19@news.newshosting.com: Article

The saving data in _ADMIN_ is what seems to always fail, and yet the file downloads without issues.

Re: Article Failed

Posted: April 20th, 2016, 9:38 am
by shypike
If you have lots of "missing from all servers, discarding"
it means that a post has been removed from (most) Usenet servers.
This happens all the time.

The actual error you reported before, I don't see in the log.
We always recommend to use local storage for the "incomplete" part
and not networked storage because the latter is slower and often unreliable.
At least this is so for the kind of intense disk access needed during download
and especially repair by par2.
It is still weird that it only happens for the kind of files you listed.
This is internal admin which is dumped to disk to protect against crashes of power failure.
It may not harm the download process.
We may need to examine this further.

Re: Article Failed

Posted: April 24th, 2016, 9:45 pm
by flizer
So here is a picture of the error as it is displayed on SAB
Image

Some good new, i was able to find the error on the log!
2016-04-24 18:18:23,432::INFO::[assembler:88] Decoding /media/readynas/Videos/Unprocessed Downloads/incomplete/Game.Of.Thrones.S01E01.720p.BluRay.x264-Obfuscated-Obfuscated/27c140fd54b07a9a13687224df64a15059592756.part046.rar yenc
2016-04-24 18:18:23,434::ERROR::[__init__:887] Saving /media/readynas/Videos/Unprocessed Downloads/incomplete/Game.Of.Thrones.S01E01.720p.BluRay.x264-Obfuscated-Obfuscated/__ADMIN__/SABnzbd_nzo_ucUZyG failed
2016-04-24 18:18:23,435::INFO::[__init__:888] Traceback:
Traceback (most recent call last):
File "/usr/share/sabnzbdplus/sabnzbd/__init__.py", line 877, in save_data
pickler.dump(data)
RuntimeError: dictionary changed size during iteration
2016-04-24 18:18:23,473::INFO::[__init__:941] Saving data for totals10.sab in /home/nuc/.sabnzbd/admin/totals10.sab

2016-04-24 18:18:45,837::ERROR::[__init__:887] Saving /media/readynas/Videos/Unprocessed Downloads/incomplete/Game.Of.Thrones.S01E01.720p.BluRay.x264-Obfuscated-Obfuscated/__ADMIN__/SABnzbd_nzo_ucUZyG failed
2016-04-24 18:18:45,837::INFO::[__init__:888] Traceback:
Traceback (most recent call last):
File "/usr/share/sabnzbdplus/sabnzbd/__init__.py", line 877, in save_data
pickler.dump(data)


I can set it to store the incomplete downloads to a local drive, i set it to the NAS because the local disk only has 20gb of free space.

Re: Article Failed

Posted: April 25th, 2016, 1:25 am
by shypike

Code: Select all

RuntimeError: dictionary changed size during iteration
Hmm, this points to a race condition.
Meaning: some other part of SABnzbd is changing data while this part is saving it.
Tricky to solve, but we'll try.

I still doubt that this error is related to the missing articles.
Like I said before, the error is happening when SABnzbd writes the administration
for a completed file to disk. If that fails, it's not fatal because the data
is still in memory.
Of course the error message is irritating nonetheless.

Re: Article Failed

Posted: April 26th, 2016, 1:18 pm
by ALbino
I've always assumed those failed errors were common and everybody had them. I get them almost every day, hundreds over the course of a year, so I just clear them and move on. It's never seemed to affect any actual downloads so far as I can tell.

Since my log always stops like a day after starting SAB, I only have old ones right now. Example:

Code: Select all

2016-04-08 11:57:32,187::DEBUG::[newsunpack:576] Running unrar ['C:\\Program Files (x86)\\SABnzbd\\win\\unrar\\x64\\UnRAR.exe', 'x', '-idp', '-o-', '-or', '-ai', '-p-', 'E:\\Downloads\\incomplete\\Title.S01E03.720p.BluRay.x264-SHORTBREHD\\title.s01e03.720p.bluray.x264-shortbrehd.rar', 'E:\\Downloads\\_UNPACK_Title.S01E03.720p.BluRay.x264-SHORTBREHD/']
2016-04-08 11:57:32,220::ERROR::[__init__:894] Loading E:\Downloads\Title.S01E04.720p.BluRay.x264-SHORTBREHD\__ADMIN__\SABnzbd_article_ymzg8g failed
2016-04-08 11:57:32,221::INFO::[__init__:895] Traceback: 
Traceback (most recent call last):
  File "sabnzbd\__init__.pyo", line 892, in load_data
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'E:\\Downloads\\incomplete\\Title.S01E04.720p.BluRay.x264-SHORTBREHD\\__ADMIN__\\SABnzbd_article_ymzg8g'
2016-04-08 11:57:32,221::INFO::[assembler:181] <Article: article=1458449435.01413.1@reader.easyusenet.nl, bytes=793503, partnum=1, art_id=SABnzbd_article_ymzg8g> missing
This is Windows 7 using 0.7.20 FWIW.

Re: Article Failed

Posted: April 26th, 2016, 1:38 pm
by shypike
Your system may exhibit a specific timing that triggers the race condition that I mentioned.
We will try to resolve it in 1.0.2 or 1.1.0.

The other error is different.
It may mean that you're running two instances of SABnzbd or that another program (virus scanner?)
is accessing one of SABnzbd's files.

Re: Article Failed

Posted: April 26th, 2016, 2:48 pm
by ALbino
Of the things that might cause simultaneous file access I can think of a few possibilities: Virus scanner (Avast), Online Backup (Crashplan, though I don't have that folder backing up), and Everything Search which keeps a constant index of everything on my system (and which is amazing BTW).

Re: Article Failed

Posted: April 27th, 2016, 6:31 pm
by ALbino
I went ahead and added \incomplete\* to my exclusion list for both Avast and Everything Search. I'll let it run like this for a few days and report back if it helps alleviate the issue.

Re: Article Failed

Posted: May 8th, 2016, 11:52 pm
by ALbino
ALbino wrote:I went ahead and added \incomplete\* to my exclusion list for both Avast and Everything Search. I'll let it run like this for a few days and report back if it helps alleviate the issue.
Just an update: I haven't received the error once since I added those exclusions, so I'm thinking that indeed was the problem all this time.

Re: Article Failed

Posted: May 9th, 2016, 7:45 am
by shypike
Hopefully it's resolved now.
Thanks for the feedback.