Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-05-01 19:34:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Fix robotstxt cache miss stats.

Browse source
This commit is contained in:
Bastian Kleineidam 2012-09-21 21:12:28 +02:00
parent f6b007f757
commit fba465e8e8
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
linkcheck/cache/robots_txt.py vendored
View file

@ -47,7 +47,7 @@ class RobotsTxt (object):
self.hits += 1
rp = self.cache[roboturl]
else:
self.misses = 1
self.misses += 1
rp = robotparser2.RobotFileParser(proxy=proxy, user=user,
password=password)
rp.set_url(roboturl)