Urlopen [errno -2] Python

I have a developed a part of code which I use from web scraping: link = 'http://www.cmegroup.com'+div.findAll('a')[3]['href'] user_agent = 'Mozilla/5.0' headers = {'User-Agent':use

Solution 1:

This looks like a DNS or network problem. If you run the same code for the same URL several times and it sometimes works but sometimes doesn't, the problem is probably not your code.

To debug the issue, you could do a try-except block around the statement and start pdb or ipdb (if installed) from there:

try:
    response = urllib2.urlopen(req)
except urllib2.URLError as ex:
    import pdb; pdb.set_trace()  # Use ipdb if installedelse:
    page = response.read()

Then you can take a look at the response, the status code, the exception trace etc...

(As a sidenote, if external dependencies are not a problem, I'd strongly recommend to use the requests package instead of urllib2.)