Page 1 of 1
Issues with OpenSSL and Sabnzbd
Posted: March 18th, 2015, 11:59 pm
by poizun
Hey all,
Running into issues currently where sab isn't able to fetch NZBs via the URL. This includes things that are dropped in by other apps and also if I manually upload it by pointing it to a URL.
I can open the URL from my browser and obtain the NZB though.
I believe it has something to do with OpenSSL just judging by some log entries; there are some SSL handshake errors, but it's not limited to just one site/indexer.
Any ideas on how to solve this?
Re: Issues with OpenSSL and Sabnzbd
Posted: March 19th, 2015, 3:36 am
by shypike
poizun wrote:
Any ideas on how to solve this?
Not without more information.
Like, which operating system.
Re: Issues with OpenSSL and Sabnzbd
Posted: March 19th, 2015, 9:41 pm
by typo101
I was having a similar issue, and it certainly was OpenSSL related, specifically that SSLv3 is disabled by default in Python 2.7.9. I found two solutions: downgrade to python 2.7.8 or patch urlgrabber.
Here is my patch:
Code: Select all
--- urlgrabber.py.old 2015-03-19 22:40:59.643171308 -0400
+++ urlgrabber.py.new 2015-03-19 22:40:07.611581288 -0400
@@ -27,0 +28 @@
+import ssl
@@ -116,0 +118,3 @@
+ context = ssl.create_default_context()
+ context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ context.verify_mode = ssl.CERT_NONE
@@ -118 +122 @@
- opener = urllib.URLopener({})
+ opener = urllib.URLopener(context=context)
@@ -120 +124 @@
- opener = urllib.FancyURLopener({})
+ opener = urllib.FancyURLopener(context=context)
EDIT: I pasted the wrong patch. On the indexer I was using TLSv1 and SSLv23 work, but not TLSv1_1 or TLSv1_2. I meant to paste the patch that used TLSv1 because it is more secure than SSLv23
Re: Issues with OpenSSL and Sabnzbd
Posted: March 19th, 2015, 10:28 pm
by typo101
So it wasnt until after I posted (and even edited) that I realized the contradiction in my patch and the apparent reason for my patch. The fact is the indexer I was using supports a connection that doesn't use SSLv3, so I shouldn't have to patch the SSL context at all. The real problem was that my python environment wasn't finding the CA file. I am running FreeBSD 9.3 and the root cause was that security/ca_root_nss wasn not installing the crt file in the one place Python was looking: /etc/ssl/cert.pem
Before:
Code: Select all
Python 2.7.9 (default, Feb 26 2015, 02:55:37)
[GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.get_default_verify_paths()
DefaultVerifyPaths(cafile=None, capath='/etc/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/ssl/certs')
Fix:
Code: Select all
$ sudo ln -s /usr/local/share/certs/ca-root-nss.crt /etc/ssl/cert.pem
After
Code: Select all
Python 2.7.9 (default, Feb 26 2015, 02:55:37)
[GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.get_default_verify_paths()
DefaultVerifyPaths(cafile='/etc/ssl/cert.pem', capath='/etc/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/ssl/certs')
Re: Issues with OpenSSL and Sabnzbd
Posted: March 20th, 2015, 9:10 pm
by poizun
Hello,
I really appreciate the help! Apologies for being a little log-starved. I had intended to post a follow-up/edit my post after pulling some logs, but got side-tracked by my 1 year old.
Here are some logs.
sabnzbd.error.log:
Code: Select all
Exception in thread CP WSGIServer Thread-14:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/local/share/sabnzbdplus/cherrypy/wsgiserver/__init__.py", line 1295, in run
conn.communicate()
File "/usr/local/share/sabnzbdplus/cherrypy/wsgiserver/__init__.py", line 1212, in communicate
req.simple_response("408 Request Timeout")
File "/usr/local/share/sabnzbdplus/cherrypy/wsgiserver/__init__.py", line 612, in simple_response
self.wfile.sendall("".join(buf))
File "/usr/local/share/sabnzbdplus/cherrypy/wsgiserver/__init__.py", line 1126, in sendall
return self._safe_call(False, super(SSL_fileobject, self).sendall, *args, **kwargs)
File "/usr/local/share/sabnzbdplus/cherrypy/wsgiserver/__init__.py", line 1078, in _safe_call
return call(*args, **kwargs)
File "/usr/local/share/sabnzbdplus/cherrypy/wsgiserver/__init__.py", line 748, in sendall
bytes_sent = self.send(data)
File "/usr/local/share/sabnzbdplus/cherrypy/wsgiserver/__init__.py", line 1129, in send
return self._safe_call(False, super(SSL_fileobject, self).send, *args, **kwargs)
File "/usr/local/share/sabnzbdplus/cherrypy/wsgiserver/__init__.py", line 1108, in _safe_call
raise FatalSSLAlert(*e.args)
FatalSSLAlert: [('SSL routines', 'SSL23_WRITE', 'ssl handshake failure')]
sabnzbd.log:
Code: Select all
2015-03-20 22:08:57,566::INFO::[urlgrabber:116] Grabbing URL https://XXXXX
2015-03-20 22:08:57,815::INFO::[urlgrabber:199] Retry URL https://XXXXX
Re: Issues with OpenSSL and Sabnzbd
Posted: March 20th, 2015, 9:11 pm
by poizun
typo101 wrote:So it wasnt until after I posted (and even edited) that I realized the contradiction in my patch and the apparent reason for my patch. The fact is the indexer I was using supports a connection that doesn't use SSLv3, so I shouldn't have to patch the SSL context at all. The real problem was that my python environment wasn't finding the CA file. I am running FreeBSD 9.3 and the root cause was that security/ca_root_nss wasn not installing the crt file in the one place Python was looking: /etc/ssl/cert.pem
Before:
Code: Select all
Python 2.7.9 (default, Feb 26 2015, 02:55:37)
[GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.get_default_verify_paths()
DefaultVerifyPaths(cafile=None, capath='/etc/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/ssl/certs')
Fix:
Code: Select all
$ sudo ln -s /usr/local/share/certs/ca-root-nss.crt /etc/ssl/cert.pem
After
Code: Select all
Python 2.7.9 (default, Feb 26 2015, 02:55:37)
[GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.get_default_verify_paths()
DefaultVerifyPaths(cafile='/etc/ssl/cert.pem', capath='/etc/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/ssl/certs')
Thanks so much for your help! I tried the symbolic link, but that did not work. Maybe I'll try patching urlgrabber next...
Re: Issues with OpenSSL and Sabnzbd
Posted: March 20th, 2015, 9:21 pm
by poizun
typo101 wrote:I was having a similar issue, and it certainly was OpenSSL related, specifically that SSLv3 is disabled by default in Python 2.7.9. I found two solutions: downgrade to python 2.7.8 or patch urlgrabber.
Here is my patch:
Code: Select all
--- urlgrabber.py.old 2015-03-19 22:40:59.643171308 -0400
+++ urlgrabber.py.new 2015-03-19 22:40:07.611581288 -0400
@@ -27,0 +28 @@
+import ssl
@@ -116,0 +118,3 @@
+ context = ssl.create_default_context()
+ context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ context.verify_mode = ssl.CERT_NONE
@@ -118 +122 @@
- opener = urllib.URLopener({})
+ opener = urllib.URLopener(context=context)
@@ -120 +124 @@
- opener = urllib.FancyURLopener({})
+ opener = urllib.FancyURLopener(context=context)
EDIT: I pasted the wrong patch. On the indexer I was using TLSv1 and SSLv23 work, but not TLSv1_1 or TLSv1_2. I meant to paste the patch that used TLSv1 because it is more secure than SSLv23
typo101, I'm not much of a drinker, but I'd buy you a beer right now.
I did a manual installation of py-urlgrabber via:
Code: Select all
portmaster /usr/ports/www/py-urlgrabber
And now things are working perfectly! Thanks for the idea! You're awesome!
Re: Issues with OpenSSL and Sabnzbd
Posted: March 21st, 2015, 8:34 am
by poizun
OK some good news and some bad news.
The good news is that the issue went away with one of my indexers. The bad news is that it's still present with my other one.
Same errors. Spoke too soon. :/
Maybe something that might help is the error from a wget for the same URL:
Code: Select all
OpenSSL: error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)
Unable to establish SSL connection.
As usual, any help is greatly appreciated!
Re: Issues with OpenSSL and Sabnzbd
Posted: March 21st, 2015, 9:32 am
by shypike
Isn't this simply a matter of the site having an invalid certificate?
Re: Issues with OpenSSL and Sabnzbd
Posted: March 21st, 2015, 11:26 am
by poizun
shypike wrote:Isn't this simply a matter of the site having an invalid certificate?
So, I'm starting to piece things together. Their site disabled SSLv3 to avoid the POODLE issue, which I thought was pretty common. It seems like perhaps, it's not falling back to SSLv2 correctly.
I'm using FreeNAS and this is the standard current port of sabnzbd. I just used portmaster to install it (and reinstall a few times), but no dice.
Do you know if changing the Switch in sabnzbd's config to SSL type V2 instead of V23 would do anything? From the text in that area, it seems to only apply to the providers...
EDIT:
The Switch didn't do anything. On to exploring more options...
Re: Issues with OpenSSL and Sabnzbd
Posted: March 21st, 2015, 12:18 pm
by shypike
While working on 0.8.0, I added TLS1 as the default protocol for Usenet connections.
I discovered that older Ubuntu versions do not support it (at least not from Python).
When your operating system's Python/OpenSSL combo doens't support TLS1, then you have a problem.
SABnzbd doesn't specify a protocol for RSS and URL-based URL fetches,
so whatever is negotiated between the libraries and the server is OK.