Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-04-05 15:30:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Detect if "url_data" contains proxy attributes before using them.

Browse source 
Fix proposed by @colwilson in issue #555.
This commit is contained in:
Nicolas Bigaouette 2014-11-12 09:58:30 -05:00
parent 27937e6f83
commit 4e56eceb35
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
linkcheck/cache/robots_txt.py vendored
View file

@ -58,7 +58,7 @@ class RobotsTxt (object):
return rp.can_fetch(self.useragent, url_data.url)
self.misses += 1
kwargs = dict(auth=url_data.auth, session=url_data.session)
if url_data.proxy:
if hasattr(url_data, "proxy") and hasattr(url_data, "proxy_type"):
kwargs["proxies"] = {url_data.proxytype: url_data.proxy}
rp = robotparser2.RobotFileParser(**kwargs)
rp.set_url(roboturl)