Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-04-15 03:41:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Use GET in case a HEAD method does not succeed, even if robots.txt content checkes denied the page. This way proper check results are achieved (but the content is still not checked, so it's ok).

Browse source
This commit is contained in:
Bastian Kleineidam 2012-09-22 07:53:11 +02:00
parent cff97b9718
commit 1c59cb4d4c
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
linkcheck/checker/httpurl.py
View file

@ -259,7 +259,7 @@ class HttpUrl (internpaturl.InternPatternUrl, proxysupport.ProxySupport):
@return: True if checker should use GET, else False
@rtype: bool
"""
if self.method == "HEAD" and self.method_get_allowed:
if self.method == "HEAD":
# Some sites do not support HEAD requests, for example
# youtube sends a 404 with HEAD, 200 with GET. Doh.
# A 405 "Method not allowed" status should also use GET.