Except for tests that include non-unicode characters:
tests/test_po.py
tests/test_strformat.py
tests/test_url.py
tests/checker/test_error.py
tests/checker/test_news.py
norobots.html was used for testing <meta name="robots"
content="nofollow"> in local files until [1]. This commit reinstates
local file testing and adds an http test.
Checking is reported by checker.httpurl.HttpUrl.content_allows_robots().
[1] ce733ae7 ("Don't check for robots.txt directives in local html
files.", 2014-03-19)
pytest output before this change:
SKIPPED [3] tests/__init__.py:217: condition: True
SKIPPED [1] tests/checker/test_news.py:63: condition: True
SKIPPED [1] tests/checker/test_news.py:41: condition: True
SKIPPED [1] tests/checker/test_news.py:116: condition: True
SKIPPED [1] tests/checker/test_news.py:75: condition: True
After:
SKIPPED [3] tests/__init__.py: disabled for now until some stable news server comes up
SKIPPED [4] tests/checker/test_news.py: disabled for now until some stable news server comes up
Linkchecker's telnet://username:password@host:port URL verification logic is
- connect to host:port
- wait for 'login: ' to appear (with a 10 second timeout), send username
- wait for 'Password: ' to appear (with a 10 second timeout), send password
The test spawns a fake telnet server on localhost that never presented
the login/password prompts, forcing the 10 second timeout three times.
This commit makes the fake telnet server emit the expected prompts,
making the test pass in .2 seconds.
In cast we forget or somebody else wants to tackle this. After all, the
assertion error + traceback shows up at the end of the test run, and
it's not immediately clear which test is to blame for it!
I'm not entirely sure why the test is hanging, but this seems clear
enough:
- the test setup spawns a (non-daemon) background thread that runs
forever, or until it is told to quit by receiving a TCP packet on a
certain port
- the test teardown tries to tell the background thread to quit (which
doesn't work) and waits for that to happen
- as a result the entire test run hangs forever
This commit adds a timeout as an extra safety net so that the test run
will complete even if the clean shutdown procedure fails for some
reason.
tests/checker/test_all_parts.py:21: in <module>
import __init__ as init
E ModuleNotFoundError: No module named '__init__'
testWarning: cannot collect test class 'TestLogger' because it has a
__init__ constructor
I believe all these tests require the network, at least they seem to
fail if it's I run them without connecting my computer to the web.
I'm looking at this as part of packaging linkchecker for GNU Guix,
where the package is build and the tests are run in a isolated
environment, intentionally without network access, to avoid issues
with non-reproducible package builds.
Without this patch, tests would fail on IPv6 hosts with this
mysterious error:
```
_______________________________________________________________________ TestHttpMisc.test_html ________________________________________________________________________
tests/checker/test_http_misc.py:30: in test_html
self.obfuscate_test()
tests/checker/test_http_misc.py:51: in obfuscate_test
url = u"http://%s/" % iputil.obfuscate_ip(ip)
linkcheck/network/iputil.py:290: in obfuscate_ip
raise ValueError('Invalid IP value %r' % ip)
E ValueError: Invalid IP value '2a02:2e0:3fe:1001:7777:772e:2:85'
```
As it turns out, the test host (`www.heise.de`) does have an IPv6
record and our tests pass on Travis only because they do not have a
working IPv6 stack. I happen to have IPv6 at home and tests are broken
here, so add a quick workaround so tests pass again.
Ideally, we would not have to deal with this hack and would handle
"obfuscation" correctly, but I have yet to figure out what that test
actually does before fixing it properly.
The test unzipps a zip file with a weird-looking non-ASCII filename in it.
I don't think zip files specify the encoding for filenames. Different
unzip utilities may interpret the filename differently. Plus, the byte
representation of the unzipped filename may be different depending on
the filesystem charset.
To me it looks as if the filename is garbage encoded as valid UTF-8, and
the test expectation is to get it in latin-1 or something.
The HTML tag has two attributes with URLs:
<applet archive="file.html" src="file.css">
It would appear that the order in which these attributes are crawled
does not match the order in the result file.
Possibly the crawling order is non-deterministic, although I cannot
reproduce that. If that's the case, the fix would be to sort the
attributes in the crawler before following them, which means we want the
expected results sorted as well (and since 'archive' comes before 'src',
so file.html should come before file.css).
