Downloads Fluctuate Speeds, Failing

[rustifr]

Been a SAB/Sonarr user for quite some time, never had any issues until the last few weeks - definitely not a power user, but I’m familiar enough that i’ve tried a lot of obvious fixes and done some basic investigations into what might be wrong. Instead of writing a novel, i’m gonna try and summarize the issue and i’ll provide additional info as needed. Any help appreciated - logs available on request.

Version: 0.7.20
OS: OS X 10.11.3 (Mid 2011 Mac Mini 2.3ghz i5 2GB)
Install: OS X .app
Skin: Plush Gold
No Firewall
No IPV6 (?)
Reproducible (affects all downloads)
Wired directly via ethernet
Speedtest.net shows very healthy connection. No other speed issues on connection.

I’ve had consistent issues related to fluctuating download speeds, and odd inconsistencies between what SAB reports in the queue and what the finder reports in the folder.

What happens on every download is this:

1 - Add NZB manually, or my usual from Sonarr. All of these NZB’s are recent, between 1 day and 10 days old.
2 - Download commences quite normally at first. I’ll get pretty decent speeds - between 3 and 10 down.
3 - Transfers get to about 50%-75% complete, and then the speed starts bucking like crazy - dropping to 0, then usually it seems to fight to get back up and fluctuates between 15k-700k, maybe up to 1 or 2mb if i’m lucky. If i just sit and wait, eventually the transfer will reach 100%, and the speed will drop to 0kb with 0 time remaining.
4 - Download sits in the queue at 100%, nothing seems to happen for a LONG time. It always reports a small amount of missing articles.
5 - If I go check the supposedly 100% downloaded folder in the incomplete folder, and check the size - its WAY off. A 750mb tv show will report like 10-15mb. Oddly, If i monitor the size via Get Info, it shows that the size is slowly increasing - so even though SAB is reporting 100% complete, the finder is telling an entirely different story.
6 - Walk away, go to sleep, etc - give this thing like 10 hours to do its thing, and some of them will actually complete, and others fail.
7- My last successful download was a ~800mb file that my history says was downloaded in a matter of minutes, with a number of malformed and missing articles - though the timestamp on the completion means it took about 8 hours (started download around 1am, reported complete at around 9am).
8 - When SAB gets to this phase where a couple of 100% downloaded files are sitting in queue, it becomes sort of unresponsive - the web UI takes a long time to load and change pages. Trying to Force Disconnect, Reset, Repair, Shut Down - SAB seems unable to do so (logs even show the shutdown/restart command, but nothing happens) I have to go into Activity Monitor and Force Quit - OR - wait it out for the downloads to fail or succeed (8+ hours) for SAB to kind of get back to normal.


Warnings / Logs:

- Warnings: All I get is occasional Cherrypy timeouts, which I’m guessing is just related to serving the web UI and is irrelevant to the downloads themselves.
- Connections report '0' and a timeout during this waiting phase.
- Logs show SAB kind of freaking out over missing articles and CRC errors.
- Logs also show a vicious cycle of super long verifications (~45 mins) and even longer repairs (~1 hour), where downloads continuously bounce back and forth - verifying, failing, block fetching, verifying, failing again, etc etc. As mentioned, for a couple files last night, this went on for 8-10 HOURS, but eventually did result in success. I'm pretty used to needing repairs, happens often enough, but its usually taken care of pretty quickly, never been like this before a few weeks ago.


What i’ve already attempted to fix:

1 - Was really convinced it was just some BS happening with my Mac Mini, so the first thing I did was a total wipe, clean install of El Cap, and reinstall everything. No effect. That was irritating.
2 - Tried a ton of different servers and ports from Astraweb, ssl or not, and no real effect. (sometimes it seems like there is a brief momentary improvement after restarting, but quickly devolves into the same condition, though this could just be coincidence)
3 - Tried adding a more “legal” NZB (Ubuntu Server) - had the same issues. Actually failed.
4 - Increased cache size, no effect (though when i increased the cache to 1G, the Mac Mini really gets cranky and Activity Monitor shows SAB using 1GB+ of memory. Basically SAB is always using the maximum cache I give it, which is probably the way thats supposed to work - but I read a lot about memory leaks, etc so I’m not sure if SAB using 1GB+ of memory is normal or not…
5- Performed Traceroutes to various Astraweb servers - hard to read, but all non-eu based Astraweb servers get about 15 steps in and hangs up a little, steps 15-17 take a long time to fill in, though the times don’t seem significantly higher. The first 10 steps give me about ~20ms average, steps 11-17 jump up to ~90ms avg. Eu based servers give me basically same speed results (first half of steps around ~20, last half of steps around ~90) BUT for whatever reason it doesnt ‘hang up’, just kind of spills results very quickly and normally, while the non-EU servers consistently seem to hang up reporting results at around step 15.

My gut is that this IS NOT a SAB issue, but I want to rule it out at least and at most I am hoping someone here can point me in the right direction in regards to helping me sort out what IS wrong.

1 - Is my ISP (Time Warner Cable) all of a sudden screwing me after years of complacency? (how can I determine this?)
2 - Is Astraweb having issues (no evidence of others complaining lately)
3 - Is this a problem with the NZB’s i’m getting from my indexer? (Nzb.su?)
4 - Is this just crazy DMCA aggressiveness lately? (doesn’t explain the Ubuntu download failing…)
5 - If its not my ISP, and its not Astraweb, and i’ve completely wiped and reinstalled SAB/Sonarr - what’s left that could be wrong?

Thanks in advance for any info or help!

[shypike]

I need some time to analyse this.

[rustifr]

Thanks, let me know if you'd like to see logs.

The only thing new since I posted is that it's succeeded on a few more files, though it once again took about 8-10 hours to fully download, verify, fetch blocks, repair, and eventually move the file. (seems to really get caught up grinding in this verify->fetch->repair loop).

I got one more random mystery error in the log, but it hasn't returned: "Suspect Error in Downloader".

I tested one more thing, where I got worried that maybe the drive with the incoming folder was having issues, but Disk Utility reported no errors on it. I will say that this drive seems to become very janky and unresponsive while SAB is in the throes of post-processing / repairing, etc. Not sure if its a symptom or a cause.

[sander]

If you want to rule out possible throttling by your ISP Time Warner Cable, you could setup a VPN to a neutral VPN provider (so: not related to a usenet provider), and then test again.

[rustifr]

Hey - just wanted to circle back and close this up. So, despite the drive passing utility tests, it was in fact failing and causing the issues. Since replacing the drive, all issues have been solved and i'm back to normal. Never would have considered this, but thats what it was! Thanks!