Commit graph

403 commits

Author SHA1 Message Date
Marius Gedminas
8489730eac Print the names of the hanging tests
In cast we forget or somebody else wants to tackle this.  After all, the
assertion error + traceback shows up at the end of the test run, and
it's not immediately clear which test is to blame for it!
2019-04-26 00:57:21 +03:00
Marius Gedminas
e285b0f257 Wow this test _is_ actually very slow!
tox -e py27 -- tests/checker/test_telnet.py takes 30 seconds to
complete.  That seems excessive to me, but one thing at a time.
2019-04-26 00:23:51 +03:00
Marius Gedminas
e9fb9b01bf Fix a hanging test on Python 3
I'm not entirely sure why the test is hanging, but this seems clear
enough:

- the test setup spawns a (non-daemon) background thread that runs
  forever, or until it is told to quit by receiving a TCP packet on a
  certain port
- the test teardown tries to tell the background thread to quit (which
  doesn't work) and waits for that to happen
- as a result the entire test run hangs forever

This commit adds a timeout as an extra safety net so that the test run
will complete even if the clean shutdown procedure fails for some
reason.
2019-04-26 00:15:10 +03:00
anarcat
095c6c57d4
Merge pull request #252 from cjmayo/init
Make test_all_parts TestLogger import Python 3 compatible
2019-04-25 15:54:26 -04:00
Chris Mayo
5caa683123 Make test_all_parts TestLogger import Python 3 compatible
tests/checker/test_all_parts.py:21: in <module>
    import __init__ as init
E   ModuleNotFoundError: No module named '__init__'

testWarning: cannot collect test class 'TestLogger' because it has a
__init__ constructor
2019-04-25 20:28:21 +01:00
anarcat
243dedf3bc
Merge pull request #247 from cjmayo/robots37
Make TestRobotsTxt Python 3.7 compatible
2019-04-25 15:21:35 -04:00
anarcat
7767bc52fa
Merge pull request #216 from cjmayo/python3_06
{python3_06} Python3: fix tests init - exceptions and string
2019-04-25 15:20:55 -04:00
Petr Dlouhý
5e918cef53 Python3: fix tests init - exceptions and string 2019-04-25 19:35:09 +01:00
anarcat
4b3d91ffea
Merge pull request #245 from cjmayo/future_str
Import str as str_text from builtins when supporting transition
2019-04-24 10:59:04 -04:00
anarcat
bb0a1e1992
Merge pull request #242 from cjmayo/wummel
Update references to GitHub project from wummel to linkchecker
2019-04-24 10:58:15 -04:00
anarcat
8219b976ac
Merge pull request #223 from cjmayo/python3_13
{python3_13} Python3: fix imports in test_noproxy
2019-04-24 10:56:50 -04:00
anarcat
5916206f5f
Merge pull request #220 from cjmayo/python3_10
{python3_10} Python3: fix httpserver tests
2019-04-24 10:56:17 -04:00
Chris Mayo
8678feaa59 Make TestRobotsTxt Python 3.7 compatible
urllib.parse.quote() moved from RFC 2396 to RFC 3986 for quoting URL
strings. "~" is now included in the set of reserved characters.

https://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote
2019-04-22 19:50:32 +01:00
Chris Mayo
d8a52381f2 Import str as str_text from builtins when supporting transition
Expected to be removed when the project moves to Python 3 only.
2019-04-19 19:25:50 +01:00
EsuS
004632a99b Update references to GitHub project from wummel to linkchecker
Remove all mention of donations.
2019-04-18 19:59:52 +01:00
anarcat
9d57bee16f
Merge pull request #218 from cjmayo/python3_08
{python3_08} Python3: use str and basestring from builtins
2019-04-17 09:04:35 -04:00
Marius Gedminas
85cee2138d Fix TestFile results not always ordered as expected values
self = <tests.checker.test_file.TestFile testMethod=test_good_dir_space>

    def test_good_dir_space (self):
...
>       self.direct(url, resultlines, recursionlevel=2)

tests/checker/test_file.py:173:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/checker/__init__.py:260: in direct
    self.fail_unicode(text(os.linesep).join(l))
tests/checker/__init__.py:237: in fail_unicode
    self.fail(msg)
E   AssertionError: Differences found testing
2019-04-16 20:25:16 +01:00
Petr Dlouhý
8f4acc3168 Python3: use str and basestring from builtins 2019-04-16 20:08:29 +01:00
anarcat
1c82686e7a
Merge pull request #234 from cjmayo/python3_05b
{python3_05b} Python3: replace xrange
2019-04-15 10:29:59 -04:00
Petr Dlouhý
45d95289ab Python3: fix logging 2019-04-14 18:59:50 +01:00
Petr Dlouhý
f30d0b5888 Python3: replace xrange 2019-04-13 20:38:58 +01:00
Petr Dlouhý
295555ac38 Python3: fix imports in test_noproxy 2019-04-12 20:27:09 +01:00
Petr Dlouhý
af08b4905b Python3: fix httpserver tests 2019-04-11 20:37:49 +01:00
anarcat
75626d456a
Merge pull request #217 from cjmayo/python3_07
{python3_07} Python3: use BytesIO instead of StringIO
2019-04-11 11:48:45 -04:00
anarcat
4b90f7b4e5
Merge pull request #225 from cjmayo/python3_15
{python3_15} fixes for Python 3: fix test_internpat and test_news
2019-04-11 11:47:21 -04:00
anarcat
6b73320cdf
Merge pull request #224 from cjmayo/python3_14
{python3_14} fixes for Python 3: fix httpserver
2019-04-11 11:46:56 -04:00
anarcat
0d35cf959d
Merge pull request #221 from cjmayo/python3_11
{python3_11} Python3: fix permission mask in test_file
2019-04-11 11:46:28 -04:00
Petr Dlouhý
106d58c2da Python3: use BytesIO instead of StringIO 2019-04-09 20:09:35 +01:00
Petr Dlouhý
4211e8aecd fixes for Python 3: fix test_internpat and test_news 2019-04-09 20:09:35 +01:00
Petr Dlouhý
e8f6bc62c8 fixes for Python 3: fix httpserver 2019-04-09 20:09:35 +01:00
Petr Dlouhý
1e9fd51dfa Python3: fix permission mask in test_file 2019-04-09 20:09:35 +01:00
Petr Dlouhý
033f9fbdb3 Python3: mark bytes explicitly 2019-04-09 20:09:35 +01:00
Christopher Baines
f24c88a073
Mark more tests that require the network
I believe all these tests require the network, at least they seem to
fail if it's I run them without connecting my computer to the web.

I'm looking at this as part of packaging linkchecker for GNU Guix,
where the package is build and the tests are run in a isolated
environment, intentionally without network access, to avoid issues
with non-reproducible package builds.
2019-01-01 22:37:21 +00:00
Antoine Beaupré
ab7502b6ff
make tests pass on IPv6 hosts
Without this patch, tests would fail on IPv6 hosts with this
mysterious error:

```
_______________________________________________________________________ TestHttpMisc.test_html ________________________________________________________________________
tests/checker/test_http_misc.py:30: in test_html
    self.obfuscate_test()
tests/checker/test_http_misc.py:51: in obfuscate_test
    url = u"http://%s/" % iputil.obfuscate_ip(ip)
linkcheck/network/iputil.py:290: in obfuscate_ip
    raise ValueError('Invalid IP value %r' % ip)
E   ValueError: Invalid IP value '2a02:2e0:3fe:1001:7777:772e:2:85'
```

As it turns out, the test host (`www.heise.de`) does have an IPv6
record and our tests pass on Travis only because they do not have a
working IPv6 stack. I happen to have IPv6 at home and tests are broken
here, so add a quick workaround so tests pass again.

Ideally, we would not have to deal with this hack and would handle
"obfuscation" correctly, but I have yet to figure out what that test
actually does before fixing it properly.
2018-04-11 19:42:30 -04:00
Marius Gedminas
6f55f446ae Load cookies from the --cookiefile correctly
requests.cookies.merge_cookies() requires a dict or a CookieJar as the second argument.
We've been passing lists of Cookie objects instead.

Fixes #62, harder this time.
2018-03-16 13:23:26 +02:00
Marius Gedminas
01b5dd619e Regression test for --cookiefile bug 2018-03-16 10:23:04 +02:00
anarcat
22449abb91
Merge pull request #126 from PetrDlouhy/tests-linenumbers
Test for linenumbers and other parts of url_data
2018-02-12 14:25:53 -05:00
anarcat
e2f3ae78a3
Merge pull request #121 from PetrDlouhy/tests-parser-divided
Execute parser test by parametrized
2018-02-12 14:25:20 -05:00
Petr Dlouhý
d6f39b4e1a Python3: use file descriptors 2018-01-19 09:52:43 +01:00
Petr Dlouhý
1cdc974e6d Python3: fix prints 2018-01-19 09:52:43 +01:00
Petr Dlouhý
c1ab81627e test of correct logging of all parts in url_data 2018-01-14 17:17:07 +01:00
Petr Dlouhý
0a13fae3b4 remove third party packages and use them as dependency 2018-01-09 23:25:27 +01:00
Petr Dlouhý
99b18eee6d execude parser test by parametrized 2018-01-09 23:15:09 +01:00
Philipp Hahn
1368643a50 Fix fragment identifier quoting
According to <https://tools.ietf.org/html/rfc3986>:
 fragment    = *( pchar / "/" / "?" )
 pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"
 unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"
 pct-encoded = "%" HEXDIG HEXDIG
 sub-delims  = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "="

Fixes #96
2017-11-10 08:03:03 -05:00
Petr Dlouhý
f5100138ff fix tests that fail because of changed linkchecker output 2017-02-14 10:59:38 +01:00
Petr Dlouhý
3b8fe41206 add tests for urlqueue 2017-02-14 10:23:32 +01:00
Graham Seaman
233e7dcf68 Allow wayback-format urls without affecting atom 'feed' urls 2017-02-09 11:43:45 +00:00
Marius Gedminas
743a5f31cb Crawl HTML attributes in deterministic order
Fixes #17.
2017-02-01 19:19:53 +02:00
Marius Gedminas
a825b9d901 Mark the non-deterministic test as xfail 2017-02-01 18:57:40 +02:00
Marius Gedminas
02869ea076 Mark TestFile.test_directory_listing as known to fail
The test unzipps a zip file with a weird-looking non-ASCII filename in it.
I don't think zip files specify the encoding for filenames.  Different
unzip utilities may interpret the filename differently.  Plus, the byte
representation of the unzipped filename may be different depending on
the filesystem charset.

To me it looks as if the filename is garbage encoded as valid UTF-8, and
the test expectation is to get it in latin-1 or something.
2017-02-01 18:45:05 +02:00