Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-05-22 05:11:52 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
7192 commits 1 branch 0 tags 24 MiB
Commit graph

634 commits

Author SHA1 Message Date
Chris Mayo
d91a328224 Remove strformat.unicode_safe() and strformat.url_unicode_split() 
All strings support Unicode in Python 3.
 2020-07-07 17:25:28 +01:00
Chris Mayo
d66e64460c Remove unused code from strformat.py 2020-06-18 19:31:00 +01:00
Chris Mayo
18d6eeae76 Ensure PO files are opened as UTF-8 in test_gtranslator() 2020-06-09 19:47:24 +01:00
Chris Mayo
74d449f8ac Test po files as strings and check po files have been found 2020-06-05 16:59:46 +01:00
Chris Mayo
4330b8a59e Replace codecs.open() with open() 2020-06-05 16:59:46 +01:00
Chris Mayo
d591fedb60 Remove unused updater code that supports linkchecker-gui 
pip provides update support for linkchecker.
 2020-06-05 16:05:25 +01:00
Chris Mayo
a6b1eb45b1 Convert to Python 3 super() 2020-06-03 20:06:36 +01:00
Chris Mayo
5df8aa085c Convert space-separated strings in tests/ 2020-05-29 19:40:46 +01:00
Chris Mayo
c71cfcbea4 Tidy TestClamav.testInfected() acceptable_responses 2020-05-29 19:40:46 +01:00
Chris Mayo
5ee8d8e1ea Add trailing comma to single dict list in TestLoginUrl.visit_loginurl() 2020-05-29 19:40:46 +01:00
Chris Mayo
a534be0b50 Remove unnecessary character match in regexp in TestLogger.normalize() 2020-05-29 19:40:46 +01:00
Chris Mayo
be53c4a659 Remove unnecessary commas before closing brackets in tests/ 2020-05-29 19:40:46 +01:00
Chris Mayo
87039913b2 Fix remaining flake8 violations in tests/ 
tests/test_clamav.py:58:89: E501 line too long (90 > 88 characters)
tests/test_containers.py:38:9: F841 local variable 'dummy' is assigned to but never used
tests/test_dummy.py:35:9: F841 local variable 'dummy' is assigned to but never used
tests/test_ftpparse.py:94:89: E501 line too long (96 > 88 characters)
tests/test_url.py:128:89: E501 line too long (130 > 88 characters)
tests/test_strformat.py:62:9: E741 ambiguous variable name 'l'
tests/test_strformat.py:136:9: E731 do not assign a lambda expression, use a def
tests/checker/ftpserver.py:94:9: E722 do not use bare 'except'
tests/checker/httpserver.py:55:39: E231 missing whitespace after ','
tests/checker/httpserver.py:224:9: E722 do not use bare 'except'
tests/checker/telnetserver.py:84:9: E722 do not use bare 'except'
tests/checker/__init__.py:71:89: E501 line too long (119 > 88 characters)
tests/checker/__init__.py:292:13: E741 ambiguous variable name 'l'
tests/checker/test_http_misc.py:30:1: W293 blank line contains whitespace
tests/checker/test_https.py:21:1: F401 'tests.need_network' imported but unused
tests/checker/test_news.py:35:1: E302 expected 2 blank lines, found 1
 2020-05-28 20:29:13 +01:00
Chris Mayo
165c51aeea Run black on tests/ 2020-05-28 20:29:13 +01:00
Chris Mayo
6f126a54d2 Add coverage for parser.sitemap.parse_sitemapindex() 2020-05-27 20:02:03 +01:00
Chris Mayo
f6e182f0e4 Mark TestFile.test_html_url_quote as need_network 
Else without the internet the test fails, eventually, with:

warning No MX mail host for users.sourceforge.net found
 2020-05-25 19:55:28 +01:00
Chris Mayo
d3c9618b1b TestHttps.test_https doesn't need the internet now 
A result of changes introduced in:

dee4be4b ("Enable https checking using a test server", 2019-11-11)
 2020-05-25 19:55:28 +01:00
Chris Mayo
32689ea230 Enable as many TestHttp html tests as possible without the internet 2020-05-25 19:55:28 +01:00
Chris Mayo
313a14ff0d Remove instances of Python 2 unicode 2020-05-24 19:14:47 +01:00
Marius Gedminas
d0169c46d4
Merge pull request #348 from weshaggard/HandleRateLimiting 
Turn status code 429 into warning instead of failure
 2020-05-24 16:16:56 +03:00
Chris Mayo
d611564cb0 Add a test for an empty html file accessed over http 2020-05-23 20:01:24 +01:00
Marius Gedminas
f268a90cfb
Merge branch 'master' into HandleRateLimiting 2020-05-23 14:15:52 +03:00
Marius Gedminas
5bd1fb4e36 Fix internal error on empty HTML files 
When BeautifulSoup finds an empty file on disk, it sets
original_encoding to None.  It doesn't matter what encoding we pick for
empty files, so let's just pick one.

I don't know if there are any circumstances where BeautifulSoup might
set the encoding to None for a non-empty file.

Closes #392.
 2020-05-21 19:01:33 +03:00
Chris Mayo
96e1c00ff7 TestLogger diff output is all Unicode in Python 3 2020-05-20 19:58:44 +01:00
Chris Mayo
71eaf9a982 Remove str_text from tests/ 2020-05-19 19:56:42 +01:00
Chris Mayo
a127902607 Replace str_text in asserts 2020-05-19 19:56:42 +01:00
Chris Mayo
12fd59057e Remove duplicate tests from test_strformat.py 2020-05-17 20:10:28 +01:00
Chris Mayo
339d293326 Convert tests/test_po.py to UTF-8 2020-05-17 20:10:28 +01:00
Chris Mayo
04465530c4 Use HttpServerTest.get_url() 2020-05-17 20:10:28 +01:00
Chris Mayo
58dbe1f282 Remove unused import pytest from tests/checker/test_http.py 
pytest.mark.xfail() removed in:
743a5f31 ("Crawl HTML attributes in deterministic order", 2017-02-01)
 2020-05-17 20:10:28 +01:00
Chris Mayo
79eafee826 Add a test for VirusCheck 2020-05-17 19:04:49 +01:00
Chris Mayo
a15a2833ca Remove spaces after names in class method definitions 
And also nested functions.

This is a PEP 8 convention, E211.
 2020-05-16 20:19:42 +01:00
Chris Mayo
1663e10fe7 Remove spaces after names in function definitions 
This is a PEP 8 convention, E211.
 2020-05-16 20:19:42 +01:00
Chris Mayo
fc11d08968 Remove spaces after names in class definitions 2020-05-16 20:19:42 +01:00
Chris Mayo
1416a08119 On Python 3 no need to convert os.linesep to a string 2020-05-16 17:02:01 +01:00
Chris Mayo
10552a79c7 Remove LinkCheckTest.fail_unicode() 
No need to encode Python 3 strings before output.
 2020-05-16 17:02:00 +01:00
Chris Mayo
9f95d06a39 Remove Python 2 test.test_support import 2020-05-16 16:26:38 +01:00
Chris Mayo
f8c9faec1b Remove Python 2 cStringIO imports 2020-05-15 19:37:04 +01:00
Chris Mayo
bda9612273 Make html.escape Python 3 only 2020-05-14 20:15:28 +01:00
Chris Mayo
42de609f8e Make urllib imports Python 3 only 2020-05-14 20:15:28 +01:00
Chris Mayo
08ddf658bc
Merge pull request #366 from cjmayo/userorpwd 
Support login forms with user and/or password
 2020-05-13 19:37:44 +01:00
Chris Mayo
736c893707
Merge pull request #377 from cjmayo/tidyten3 
Remove u string prefixes
 2020-05-13 19:36:54 +01:00
Chris Mayo
00c4a30386 Add user and password only loginurl tests 2020-05-13 19:32:29 +01:00
Chris Mayo
31a9f68c46
Merge pull request #367 from cjmayo/loginurl 
Add test for loginurl
 2020-05-12 20:08:57 +01:00
Chris Mayo
44e81d27dd Remove inheriting object 
All Python 3 classes are new-style.
 2020-05-08 10:45:31 +01:00
Chris Mayo
b0ea72e8c1 Remove # -*- coding: lines 
Except for tests that include non-unicode characters:

tests/test_po.py
tests/test_strformat.py
tests/test_url.py
tests/checker/test_error.py
tests/checker/test_news.py
 2020-05-08 10:45:31 +01:00
Chris Mayo
4d3e5abcfa Remove u string prefixes 2020-04-30 20:11:59 +01:00
anarcat
ab476fa4bf
Merge pull request #364 from cjmayo/parser5 
Stop using HTML handlers and improve login form error handling
 2020-04-30 09:28:48 -04:00
Chris Mayo
1d1d9c3bde Add testing for variants of the robots meta directive 2020-04-29 20:14:10 +01:00
Chris Mayo
9eed070a73 Stop using HTML handlers 
LinkFinder is the only remaining HTML handler therefore no need for
htmlsoup.process_soup() as an independent function or TagFinder as a
base class.
 2020-04-29 20:07:00 +01:00
Chris Mayo
a1433767e5 Replace HtmlPrettyPrinter with pretty_print_html() 2020-04-29 20:07:00 +01:00
Chris Mayo
0361d9e0e8 Remove encoding and default fd from HtmlPrettyPrinter 
Neither are used.
 2020-04-29 20:07:00 +01:00
Chris Mayo
4ffdbf2406 Replace MetaRobotsFinder using BeautifulSoup.find() 2020-04-29 20:07:00 +01:00
Chris Mayo
8fc0dcc055 Make matching login form credentials case-sensitive 
The keys of the form.data dictionary are case-sensitive and therefore a
KeyError was possible if the configured values are not identical to
the input element name attributes.
 2020-04-27 18:06:29 +01:00
Chris Mayo
7a6ef938cc Rename htmlutil.formsearch to htmlutil.loginformsearch 
Make it clear that this module has only one specific use.
 2020-04-27 18:06:29 +01:00
anarcat
183d483074
Merge pull request #365 from cjmayo/tidyten1 
Remove use of the future package
 2020-04-26 12:02:30 -04:00
Chris Mayo
3b8af403be Add test for loginurl 
A new cgi-bin directory is created to identify the scripts to be run by
http.server.CGIHTTPRequestHandler.
 2020-04-19 19:05:55 +01:00
Chris Mayo
56b8c9f7ab Add tests for <meta name="robots" content="nofollow"> 
norobots.html was used for testing <meta name="robots"
content="nofollow"> in local files until [1]. This commit reinstates
local file testing and adds an http test.

Checking is reported by checker.httpurl.HttpUrl.content_allows_robots().

[1] ce733ae7 ("Don't check for robots.txt directives in local html
files.", 2014-03-19)
 2020-04-18 20:30:46 +01:00
Chris Mayo
d189445a8e LinkFinder does not raise StopParse 2020-04-18 20:30:46 +01:00
Chris Mayo
ee6628a831 Move HtmlParser/htmlsax.py to htmlutil/htmlsoup.py 
Remove one subpackage and some import lines where htmlutil.linkparse is
also being used.
 2020-04-18 20:30:45 +01:00
Chris Mayo
a83fbb56c0 Remove from __future__ imports 2020-04-15 19:49:16 +01:00
Chris Mayo
f5e7f3a382 Remove use of the future package 
It was providing Python 2 compatibility.
 2020-04-15 19:49:16 +01:00
Chris Mayo
0795e3c1b4 Replace Parser class using BeautifulSoup.find_all() 2020-04-10 13:51:09 +01:00
Chris Mayo
eb3cf28baa Remove support for start_end_element() callback 
The LinkFinder handler start_end_element() callback does nothing apart
from call start_element().
 2020-04-10 13:51:09 +01:00
Chris Mayo
c9f17e92b9 Remove support for end_element() callback 2020-04-10 13:51:09 +01:00
Chris Mayo
48b590cf8b Replace FormFinder using BeautifulSoup.find_all() 
FormFinder was the only handler that used an end_element() callback and
was therefore a blocker to moving the Parser class to use
BeautifulSoup.find_all()

FormFinder was a specialised handler used to parse a login form at
the start of a session if the user had configured authentication
credentials.
 2020-04-10 13:51:05 +01:00
Chris Mayo
974915cc4f Remove encoding from Parser 
Only used by the test and an attribute of the soup object.
 2020-04-08 20:03:35 +01:00
Chris Mayo
02e1c389b2 Remove parser flush() and reset() 
Remnants of the feed() interface.
 2020-04-08 20:03:35 +01:00
Chris Mayo
3771dd9136 Use parser.feed_soup() instead of parser.feed() 
Markup is not being passed in pieces to the parser, so simplify the
interface and reduce the state further.
 2020-04-08 20:03:35 +01:00
Chris Mayo
9d8d251d06 Replace Parser lineno() and column() methods 
Stop storing this data in Parser object state.
 2020-04-08 20:03:35 +01:00
Chris Mayo
514210199d Add tests for search_form 2020-04-07 19:24:34 +01:00
Chris Mayo
036b900ffc Remove unused linkcheck.containers classes 2020-04-03 19:24:08 +01:00
Chris Mayo
3ff3d72492 Use BeautifulSoup element attrs directly 2020-04-03 19:24:08 +01:00
Wes Haggard
5c3978ac58 Update http test to handle new 429 behavior 2020-04-02 14:37:42 -07:00
Chris Mayo
28701e291a Remove use of Python 2 unicode() and related u prefixes 
Several instances for MS Windows left unchanged.
 2020-04-01 19:39:50 +01:00
anarcat
cf4e6bb235
Merge pull request #351 from cjmayo/tagsonly 
Remove support for non-Tag elements from Parser
 2020-04-01 12:17:18 -04:00
Chris Mayo
9fc651e82b Remove Python 2 compatibility from parser tests 2020-03-31 20:10:35 +01:00
Chris Mayo
ffa6ac457f Remove support for non-Tag elements from Parser 
This change is made because the linkchecker handlers only process
Tags.

The test HtmlPrettyPrinter handler is updated to output element text
because its support for non-Tag elements has been removed. This results
in a number of the existing tests still passing.
 2020-03-31 20:10:35 +01:00
Chris Mayo
0ee4414a60 Replace memoized with functools.lru_cache 2020-03-31 19:46:31 +01:00
Chris Mayo
1255119ca8 Move HtmlPrinter and HtmlPrettyPrinter into tests 2020-03-30 19:32:30 +01:00
Chris Mayo
f743be57e8 Remove unused functions from linkcheck.HtmlParser 
resolve_entities() unused since:
2c000683 ("Remove unused linkcheck.htmlutil.linkname module",
2020-03-30)

set_doctype(), set_encoding() unused since:
51a06d8a ("Remove home-cooked htmlparser and use BeautifulSoup",
2019-07-22)
 2020-03-30 19:32:18 +01:00
Chris Mayo
2c000683e1 Remove unused linkcheck.htmlutil.linkname module 
Unused since:
d6d48b48 ("html parser: use name instead of peeking", 2019-07-22)
 2020-03-30 19:31:11 +01:00
Chris Mayo
ecd06776ab Fix TypeError when checking https link and test 
File "/usr/lib/python3.7/site-packages/linkcheck/httputil.py", line 68, in asn1_generaltime_to_seconds
    line: res = datetime.strptime(timestr, timeformat + 'Z')
    locals:
      res = <local> None
      datetime = <global> <class 'datetime.datetime'>
      datetime.strptime = <global> <built-in method strptime of type object at 0x7fa39064dda0>
      timestr = <local> b'20191106202117Z'
      timeformat = <local> '%Y%m%d%H%M%S'
TypeError: strptime() argument 1 must be str, not bytes

pyOpenSSL OpenSSL.crypto.X509.get_notAfter() returns bytes:
https://www.pyopenssl.org/en/stable/api/crypto.html#OpenSSL.crypto.X509.get_notAfter
 2019-11-11 20:12:25 +00:00
Chris Mayo
dee4be4b1d Enable https checking using a test server 
Verification has to be turned off because we are using a
self-signed certificate.
 2019-11-11 20:12:25 +00:00
Chris Mayo
2f16152dc8 Improve test failure diff 
Some url lines were missing a url prefix while others had a double url
prefix. diff was reporting more url lines as changed than actually had.
Improve formatting by removing newlines from control lines and adding
headings.

Before:

E   AssertionError: http://localhost:46031/tests/checker/data/sitemap.xml
E   ---
E
E   +++
E
E   @@ -1,4 +1,8 @@
E
E   -url http://localhost:46031/tests/checker/data/sitemap.xml
E   +http://www.example.com/
E   +cache key http://www.example.com/
E   +real url http://www.example.com/
E   +valid
E   +url url http://localhost:46031/tests/checker/data/sitemap.xml
E    cache key http://localhost:46031/tests/checker/data/sitemap.xml
E    real url http://localhost:46031/tests/checker/data/sitemap.xml
E    valid

After:

E   AssertionError: http://localhost:44021/tests/checker/data/sitemap.xml
E   --- expected
E   +++ result
E   @@ -2,3 +2,7 @@
E    cache key http://localhost:44021/tests/checker/data/sitemap.xml
E    real url http://localhost:44021/tests/checker/data/sitemap.xml
E    valid
E   +url http://www.example.com/
E   +cache key http://www.example.com/
E   +real url http://www.example.com/
E   +valid
 2019-10-29 20:03:08 +00:00
Chris Mayo
ec8b6e09f0 Fix XmlTagUrlParser and make Python 3 compatible 
URLs within a sitemap file were not being captured.
 2019-10-28 19:20:05 +00:00
Marius Gedminas
8bdd402aed
Merge pull request #333 from linkchecker/fix-clamav-on-py3 
Fix test_clamav.py on Python 3
 2019-10-25 16:16:23 +03:00
Marius Gedminas
5b2b3613ec
Merge pull request #330 from linkchecker/fix-sitemap 
Fix sitemap parser
 2019-10-25 16:15:55 +03:00
Marius Gedminas
f9766a2049 Python 3: fix bytes vs strings in viruscheck plugin 
Socket communication deals with bytes.

There are probably remaining issues with the viruscheck plugin on
Python 3, we just can't see them because the code is not fully covered
with tests.
 2019-10-25 14:24:07 +03:00
Marius Gedminas
606ece0308 Explain why these tests are being skipped 
pytest output before this change:

    SKIPPED [3] tests/__init__.py:217: condition: True
    SKIPPED [1] tests/checker/test_news.py:63: condition: True
    SKIPPED [1] tests/checker/test_news.py:41: condition: True
    SKIPPED [1] tests/checker/test_news.py:116: condition: True
    SKIPPED [1] tests/checker/test_news.py:75: condition: True

After:

    SKIPPED [3] tests/__init__.py: disabled for now until some stable news server comes up
    SKIPPED [4] tests/checker/test_news.py: disabled for now until some stable news server comes up
 2019-10-23 17:35:31 +03:00
Marius Gedminas
87b504785c Add a regression test for the sitemap parser 2019-10-23 17:30:10 +03:00
Marius Gedminas
c6de64978c
Merge pull request #325 from linkchecker/type-error-in-robot-parser 
Fix TypeError: string arg required in content_allows_robots()
 2019-10-22 18:07:31 +03:00
Marius Gedminas
7e94e542b3 Enable clamav integration tests on Travis CI 2019-10-22 17:04:09 +03:00
Marius Gedminas
58b0d5aaae Fix TypeError: string arg required in content_allows_robots() 
See #323 an #317.
 2019-10-22 14:13:45 +03:00
Marius Gedminas
6a9ab5ae44 Add a failing test 2019-10-22 14:13:45 +03:00
Marius Gedminas
84dbb5d603 Fix TypeError: string arg required in find_links() 
Fixes #317.
 2019-10-21 17:47:46 +03:00
Marius Gedminas
a4967fe92c Add a regression test for issue #317 
The important bit was making the `file_test` helper not ignore internal
errors.
 2019-10-21 17:45:18 +03:00
Chris Mayo
c7a32d67fe Remove unused code from network subpackage 2019-10-19 10:27:34 +01:00
Chris Mayo
74d5c68094 Add new tests for URL quoting 2019-10-05 19:38:57 +01:00
Chris Mayo
b7ec71d8cc Always use utf-8 encoding when quoting 2019-10-05 19:38:57 +01:00
Chris Mayo
5bb4524a63 Update strformat.ascii_safe() because paths are now strings 2019-10-05 19:38:57 +01:00
Chris Mayo
646e138166 Pass encoding when unquoting 
Else non-UTF-8 codes are misinterpreted:

>>> from urllib import parse
>>> parse.unquote("%FF")
'�'
>>> parse.unquote("%FF", "latin1")
'ÿ'
 2019-10-05 19:38:57 +01:00
Chris Mayo
30df69c158 Improve pretty printed comments 2019-10-05 19:38:57 +01:00
Chris Mayo
607328d5c5 Support Beautiful Soup line numbers 2019-10-05 19:38:57 +01:00
anarcat
bae4282c92
Merge pull request #307 from cjmayo/cgi_escape 
Replace deprecated cgi.escape
 2019-09-18 10:16:58 -04:00
Chris Mayo
53cd9475b5 Replace deprecated cgi.escape 
html provided for Python 2 by future
https://python-future.org/compatible_idioms.html#html-escaping-and-entities
 2019-09-17 20:25:05 +01:00
Petr Dlouhý
1b41df4af3 Python3: fix test error message 2019-09-17 20:20:46 +01:00
anarcat
1590408a65
Merge pull request #306 from cjmayo/python3_49 
{python3_49} enable and fix remaining bookmark tests
 2019-09-16 15:18:26 -04:00
anarcat
2b18ff0a5f
Merge pull request #301 from cjmayo/python3_44 
{python3_44} Python3: fixes for httpserver
 2019-09-16 15:16:21 -04:00
Petr Dlouhý
eaa7131523 enable and fix remaining bookmark tests 
biplist module preferred for reading Safari bookmarks in
bookmarks/safari.py so install it for tox testing.
 2019-09-16 20:08:01 +01:00
Petr Dlouhý
030cf8321a Python3: fixes for httpserver 2019-09-15 19:49:33 +01:00
Petr Dlouhý
a2e67af7b4 fixes for Python 3: fix telneturl 2019-09-15 19:49:18 +01:00
anarcat
fe39db4fbf
Merge pull request #287 from cjmayo/python3_36 
{python3_36} fixes for Python 3 + Travis test: fix cgi
 2019-09-14 11:50:53 -04:00
Petr Dlouhý
36465112d0 fixes for Python 3 + Travis test: fix cgi 2019-09-13 19:46:13 +01:00
Petr Dlouhý
8a294be95f Python3: fix robotparser 2019-09-11 20:04:26 +01:00
Marius Gedminas
0d58a39376 Fix failing test 
http://www.heise.de/ now does a redirect to HTTPS instead of denying our
crawl via robots.txt.

Fixes #269.
 2019-09-04 14:04:07 +03:00
Petr Dlouhý
69d426b36f fix parser encoding tests after change of parser 
UnicodeDammit input has to be non-unicode to trigger character set
detection.
 2019-07-22 19:59:37 +01:00
Petr Dlouhý
b5111453d8 change test_parse encoding to UTF-8 2019-07-22 19:59:37 +01:00
Petr Dlouhý
2c3c794e52 fix http test after parser change 2019-07-22 19:59:37 +01:00
Petr Dlouhý
0089349760 fix parser tests after parser change 2019-07-22 19:59:37 +01:00
Petr Dlouhý
d6d48b4814 html parser: use name instead of peeking 2019-07-22 19:59:37 +01:00
Petr Dlouhý
d1844a526e add charset tests 2019-07-22 19:59:37 +01:00
Marius Gedminas
947b108f9e Make test_telnet.py fast 
Linkchecker's telnet://username:password@host:port URL verification logic is

- connect to host:port
- wait for 'login: ' to appear (with a 10 second timeout), send username
- wait for 'Password: ' to appear (with a 10 second timeout), send password

The test spawns a fake telnet server on localhost that never presented
the login/password prompts, forcing the 10 second timeout three times.

This commit makes the fake telnet server emit the expected prompts,
making the test pass in .2 seconds.
 2019-04-27 21:52:33 +03:00
Marius Gedminas
3a7c2a9823
Merge pull request #255 from linkchecker/stop-threads-more-reliably 
Stop threads more reliably
 2019-04-27 21:51:34 +03:00
Marius Gedminas
068e9bae8d Stop the telnet server threads more reliably 
Instead of speaking text-based protocols over TCP we can use
threading.Event() objects to indicate the desire for the server thread
to quit.
 2019-04-26 01:10:36 +03:00
Marius Gedminas
8489730eac Print the names of the hanging tests 
In cast we forget or somebody else wants to tackle this.  After all, the
assertion error + traceback shows up at the end of the test run, and
it's not immediately clear which test is to blame for it!
 2019-04-26 00:57:21 +03:00
Marius Gedminas
e285b0f257 Wow this test _is_ actually very slow! 
tox -e py27 -- tests/checker/test_telnet.py takes 30 seconds to
complete.  That seems excessive to me, but one thing at a time.
 2019-04-26 00:23:51 +03:00
Marius Gedminas
e9fb9b01bf Fix a hanging test on Python 3 
I'm not entirely sure why the test is hanging, but this seems clear
enough:

- the test setup spawns a (non-daemon) background thread that runs
  forever, or until it is told to quit by receiving a TCP packet on a
  certain port
- the test teardown tries to tell the background thread to quit (which
  doesn't work) and waits for that to happen
- as a result the entire test run hangs forever

This commit adds a timeout as an extra safety net so that the test run
will complete even if the clean shutdown procedure fails for some
reason.
 2019-04-26 00:15:10 +03:00
anarcat
b65e0f9d4c
Merge pull request #244 from cjmayo/fixes 
Fix mistakes in changes to test_dummy.py and test_updater.py in 8f4acc31
 2019-04-25 16:20:44 -04:00
anarcat
59fe9ed876
Merge pull request #228 from cjmayo/python3_18 
{python3_18} Python3: fix unicode in urlbase
 2019-04-25 16:17:00 -04:00
anarcat
70f0bbf225
Merge pull request #250 from cjmayo/ftpserver 
Get FtpServerTest working by updating to current pyftpdlib API
 2019-04-25 16:16:33 -04:00
anarcat
095c6c57d4
Merge pull request #252 from cjmayo/init 
Make test_all_parts TestLogger import Python 3 compatible
 2019-04-25 15:54:26 -04:00
Chris Mayo
5caa683123 Make test_all_parts TestLogger import Python 3 compatible 
tests/checker/test_all_parts.py:21: in <module>
    import __init__ as init
E   ModuleNotFoundError: No module named '__init__'

testWarning: cannot collect test class 'TestLogger' because it has a
__init__ constructor
 2019-04-25 20:28:21 +01:00
anarcat
243dedf3bc
Merge pull request #247 from cjmayo/robots37 
Make TestRobotsTxt Python 3.7 compatible
 2019-04-25 15:21:35 -04:00
anarcat
7767bc52fa
Merge pull request #216 from cjmayo/python3_06 
{python3_06} Python3: fix tests init - exceptions and string
 2019-04-25 15:20:55 -04:00
Petr Dlouhý
b3881ce3b5 Python3: fix urlbase, strformat and others 2019-04-25 19:57:45 +01:00
Petr Dlouhý
5e918cef53 Python3: fix tests init - exceptions and string 2019-04-25 19:35:09 +01:00
anarcat
4b3d91ffea
Merge pull request #245 from cjmayo/future_str 
Import str as str_text from builtins when supporting transition
 2019-04-24 10:59:04 -04:00
anarcat
bb0a1e1992
Merge pull request #242 from cjmayo/wummel 
Update references to GitHub project from wummel to linkchecker
 2019-04-24 10:58:15 -04:00
anarcat
8219b976ac
Merge pull request #223 from cjmayo/python3_13 
{python3_13} Python3: fix imports in test_noproxy
 2019-04-24 10:56:50 -04:00
anarcat
5916206f5f
Merge pull request #220 from cjmayo/python3_10 
{python3_10} Python3: fix httpserver tests
 2019-04-24 10:56:17 -04:00
Chris Mayo
8678feaa59 Make TestRobotsTxt Python 3.7 compatible 
urllib.parse.quote() moved from RFC 2396 to RFC 3986 for quoting URL
strings. "~" is now included in the set of reserved characters.

https://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote
 2019-04-22 19:50:32 +01:00
Chris Mayo
64e9392fb9 Get FtpServerTest working by updating to current pyftpdlib API 2019-04-22 19:34:46 +01:00
Chris Mayo
d8a52381f2 Import str as str_text from builtins when supporting transition 
Expected to be removed when the project moves to Python 3 only.
 2019-04-19 19:25:50 +01:00
Chris Mayo
0031bbdccc Fix mistakes in changes to test_dummy.py and test_updater.py in 8f4acc31 2019-04-19 19:22:38 +01:00
EsuS
004632a99b Update references to GitHub project from wummel to linkchecker 
Remove all mention of donations.
 2019-04-18 19:59:52 +01:00
anarcat
9d57bee16f
Merge pull request #218 from cjmayo/python3_08 
{python3_08} Python3: use str and basestring from builtins
 2019-04-17 09:04:35 -04:00
Marius Gedminas
85cee2138d Fix TestFile results not always ordered as expected values 
self = <tests.checker.test_file.TestFile testMethod=test_good_dir_space>

    def test_good_dir_space (self):
...
>       self.direct(url, resultlines, recursionlevel=2)

tests/checker/test_file.py:173:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/checker/__init__.py:260: in direct
    self.fail_unicode(text(os.linesep).join(l))
tests/checker/__init__.py:237: in fail_unicode
    self.fail(msg)
E   AssertionError: Differences found testing
 2019-04-16 20:25:16 +01:00
Petr Dlouhý
8f4acc3168 Python3: use str and basestring from builtins 2019-04-16 20:08:29 +01:00
anarcat
1c82686e7a
Merge pull request #234 from cjmayo/python3_05b 
{python3_05b} Python3: replace xrange
 2019-04-15 10:29:59 -04:00