Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-05-20 20:31:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
6895 commits 1 branch 0 tags 24 MiB
Commit graph

312 commits

Author SHA1 Message Date
anarcat
70f0bbf225
Merge pull request #250 from cjmayo/ftpserver 
Get FtpServerTest working by updating to current pyftpdlib API
 2019-04-25 16:16:33 -04:00
Chris Mayo
5caa683123 Make test_all_parts TestLogger import Python 3 compatible 
tests/checker/test_all_parts.py:21: in <module>
    import __init__ as init
E   ModuleNotFoundError: No module named '__init__'

testWarning: cannot collect test class 'TestLogger' because it has a
__init__ constructor
 2019-04-25 20:28:21 +01:00
Petr Dlouhý
b3881ce3b5 Python3: fix urlbase, strformat and others 2019-04-25 19:57:45 +01:00
anarcat
8219b976ac
Merge pull request #223 from cjmayo/python3_13 
{python3_13} Python3: fix imports in test_noproxy
 2019-04-24 10:56:50 -04:00
anarcat
5916206f5f
Merge pull request #220 from cjmayo/python3_10 
{python3_10} Python3: fix httpserver tests
 2019-04-24 10:56:17 -04:00
Chris Mayo
64e9392fb9 Get FtpServerTest working by updating to current pyftpdlib API 2019-04-22 19:34:46 +01:00
Marius Gedminas
85cee2138d Fix TestFile results not always ordered as expected values 
self = <tests.checker.test_file.TestFile testMethod=test_good_dir_space>

    def test_good_dir_space (self):
...
>       self.direct(url, resultlines, recursionlevel=2)

tests/checker/test_file.py:173:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/checker/__init__.py:260: in direct
    self.fail_unicode(text(os.linesep).join(l))
tests/checker/__init__.py:237: in fail_unicode
    self.fail(msg)
E   AssertionError: Differences found testing
 2019-04-16 20:25:16 +01:00
Petr Dlouhý
295555ac38 Python3: fix imports in test_noproxy 2019-04-12 20:27:09 +01:00
Petr Dlouhý
af08b4905b Python3: fix httpserver tests 2019-04-11 20:37:49 +01:00
anarcat
4b90f7b4e5
Merge pull request #225 from cjmayo/python3_15 
{python3_15} fixes for Python 3: fix test_internpat and test_news
 2019-04-11 11:47:21 -04:00
anarcat
6b73320cdf
Merge pull request #224 from cjmayo/python3_14 
{python3_14} fixes for Python 3: fix httpserver
 2019-04-11 11:46:56 -04:00
Petr Dlouhý
4211e8aecd fixes for Python 3: fix test_internpat and test_news 2019-04-09 20:09:35 +01:00
Petr Dlouhý
e8f6bc62c8 fixes for Python 3: fix httpserver 2019-04-09 20:09:35 +01:00
Petr Dlouhý
1e9fd51dfa Python3: fix permission mask in test_file 2019-04-09 20:09:35 +01:00
Christopher Baines
f24c88a073
Mark more tests that require the network 
I believe all these tests require the network, at least they seem to
fail if it's I run them without connecting my computer to the web.

I'm looking at this as part of packaging linkchecker for GNU Guix,
where the package is build and the tests are run in a isolated
environment, intentionally without network access, to avoid issues
with non-reproducible package builds.
 2019-01-01 22:37:21 +00:00
Antoine Beaupré
ab7502b6ff
make tests pass on IPv6 hosts 
Without this patch, tests would fail on IPv6 hosts with this
mysterious error:

```
_______________________________________________________________________ TestHttpMisc.test_html ________________________________________________________________________
tests/checker/test_http_misc.py:30: in test_html
    self.obfuscate_test()
tests/checker/test_http_misc.py:51: in obfuscate_test
    url = u"http://%s/" % iputil.obfuscate_ip(ip)
linkcheck/network/iputil.py:290: in obfuscate_ip
    raise ValueError('Invalid IP value %r' % ip)
E   ValueError: Invalid IP value '2a02:2e0:3fe:1001:7777:772e:2:85'
```

As it turns out, the test host (`www.heise.de`) does have an IPv6
record and our tests pass on Travis only because they do not have a
working IPv6 stack. I happen to have IPv6 at home and tests are broken
here, so add a quick workaround so tests pass again.

Ideally, we would not have to deal with this hack and would handle
"obfuscation" correctly, but I have yet to figure out what that test
actually does before fixing it properly.
 2018-04-11 19:42:30 -04:00
Petr Dlouhý
c1ab81627e test of correct logging of all parts in url_data 2018-01-14 17:17:07 +01:00
Petr Dlouhý
0a13fae3b4 remove third party packages and use them as dependency 2018-01-09 23:25:27 +01:00
Philipp Hahn
1368643a50 Fix fragment identifier quoting 
According to <https://tools.ietf.org/html/rfc3986>:
 fragment    = *( pchar / "/" / "?" )
 pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"
 unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"
 pct-encoded = "%" HEXDIG HEXDIG
 sub-delims  = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "="

Fixes #96
 2017-11-10 08:03:03 -05:00
Petr Dlouhý
f5100138ff fix tests that fail because of changed linkchecker output 2017-02-14 10:59:38 +01:00
Marius Gedminas
743a5f31cb Crawl HTML attributes in deterministic order 
Fixes #17.
 2017-02-01 19:19:53 +02:00
Marius Gedminas
a825b9d901 Mark the non-deterministic test as xfail 2017-02-01 18:57:40 +02:00
Marius Gedminas
02869ea076 Mark TestFile.test_directory_listing as known to fail 
The test unzipps a zip file with a weird-looking non-ASCII filename in it.
I don't think zip files specify the encoding for filenames.  Different
unzip utilities may interpret the filename differently.  Plus, the byte
representation of the unzipped filename may be different depending on
the filesystem charset.

To me it looks as if the filename is garbage encoded as valid UTF-8, and
the test expectation is to get it in latin-1 or something.
 2017-02-01 18:45:05 +02:00
Marius Gedminas
cffea5fcbd Mark TestHttps.test_https as known to fail 
This test depends on the way http://amazon.com/ works.  I don't think
that's a good idea.
 2017-02-01 18:44:21 +02:00
Marius Gedminas
f4ec7531c1 Fix TestHttp.test_html 
The HTML tag has two attributes with URLs:

  <applet archive="file.html" src="file.css">

It would appear that the order in which these attributes are crawled
does not match the order in the result file.

Possibly the crawling order is non-deterministic, although I cannot
reproduce that.  If that's the case, the fix would be to sort the
attributes in the crawler before following them, which means we want the
expected results sorted as well (and since 'archive' comes before 'src',
so file.html should come before file.css).
 2017-02-01 18:41:47 +02:00
Bastian Kleineidam
88c060699d Fix tests 2016-01-19 22:05:15 +01:00
Bastian Kleineidam
914995b5fc Use example.com for tests. 2016-01-19 12:17:08 +01:00
Vadim Khohlov
d4352fc828 Added plugin for parsing and checking links in Markdown files 2014-11-11 15:35:18 +02:00
Bastian Kleineidam
7239cd1b76 Add test for itms-services URL. 2014-09-05 21:37:33 +02:00
Bastian Kleineidam
0fa7ed2699 Fix empty URL handling. 2014-07-03 23:34:40 +02:00
Bastian Kleineidam
cde261c009 Parse Refresh: and Content-Location: header values for URLs. 2014-07-01 20:16:43 +02:00
Bastian Kleineidam
b152ce7a6e Add PDF test and fix page number. 2014-04-29 18:53:24 +02:00
Bastian Kleineidam
7baa2f0b1b Fix http_link check and add a basic auth check. 2014-04-10 18:06:15 +02:00
Bastian Kleineidam
6caf654031 Parse Link: heaaders. 2014-04-10 17:50:55 +02:00
Bastian Kleineidam
a8623bc0bc Display SSL info on redirects. 2014-03-26 07:16:03 +01:00
Bastian Kleineidam
9cd67dfcb2 More SSL message work. 2014-03-20 20:24:57 +01:00
Bastian Kleineidam
9a7ad3a84f Print SSL cipher info for https URLs. 2014-03-19 17:02:34 +01:00
Bastian Kleineidam
ce733ae76b Don't check for robots.txt directives in local html files. 2014-03-19 16:33:22 +01:00
Bastian Kleineidam
9be667b52a Do not warn about missing addresses on mailto links that have subjects. 2014-03-18 23:27:59 +01:00
Bastian Kleineidam
fc73c6ca6e Log number of checked unique URLs. 2014-03-14 23:46:17 +01:00
Bastian Kleineidam
bca226c293 Fix assertion checking external links; fix tests 2014-03-10 18:23:44 +01:00
Bastian Kleineidam
6b334dc79b Fix URL result caching. 2014-03-08 19:35:10 +01:00
Bastian Kleineidam
fab2c2da98 Improve content type setting. 2014-03-05 20:12:19 +01:00
Bastian Kleineidam
ef13a3fce1 Implement sitemap and sitemap index parsing. 2014-03-05 19:26:37 +01:00
Bastian Kleineidam
b17211f162 Set for release. 2014-03-04 21:36:24 +01:00
Bastian Kleineidam
978b24f2d7 Merge branch 'caching' 2014-03-04 07:21:42 +01:00
Bastian Kleineidam
f1076c8813 Increase url-too-long warning. 2014-03-03 23:31:04 +01:00
Bastian Kleineidam
82f81241fd Check all links and add better caching. 2014-03-03 23:29:45 +01:00
Bastian Kleineidam
cc21f8f3d2 Add missing import. 2014-03-02 20:01:55 +01:00
Bastian Kleineidam
b8175e2357 Disable news test. 2014-03-02 20:01:36 +01:00
Bastian Kleineidam
924c6285d2 Fix some tests 2014-03-02 07:45:04 +01:00
Bastian Kleineidam
7b34be590b Introduce check plugins, use Python requests for http/s connections, and some code cleanups and improvements. 2014-03-01 00:12:34 +01:00
Bastian Kleineidam
c806be5c15 Updated copyright 2014-01-08 22:33:04 +01:00
Bastian Kleineidam
e0a2558b2b Updated copyright. 2013-12-24 07:13:16 +01:00
Bastian Kleineidam
5736987b60 Refactor output loggers. 2013-12-11 18:41:55 +01:00
Bastian Kleineidam
b363945052 Adjust example.com/org tests. This seems to change every now and then. 2013-12-04 19:13:18 +01:00
Bastian Kleineidam
023da7c993 Remove the duplicate URL content check. 2013-12-04 19:12:40 +01:00
Bastian Kleineidam
c966fe6b24 Remove the http-wrong-redirect warning 2013-04-11 18:33:19 +02:00
Bastian Kleineidam
35bc79dd90 Updated copyright. 2013-01-25 21:14:27 +01:00
Bastian Kleineidam
a86e36e5d3 Fix test cases for example.com redirection. 2013-01-23 19:42:29 +01:00
Bastian Kleineidam
e6ad32c028 Catch UnicodeError for invalid host names. 2013-01-23 19:42:29 +01:00
Bastian Kleineidam
4dad2aa33c Support dns-prefetch URLs. 2013-01-17 20:41:09 +01:00
Bastian Kleineidam
03f2e19cfd Fix html tests. 2013-01-17 20:40:51 +01:00
Bastian Kleineidam
aaf35c0f4a Added Word test. 2013-01-09 23:02:47 +01:00
Bastian Kleineidam
e91c2edf7e Test all http response codes. 2012-11-13 18:11:25 +01:00
Bastian Kleineidam
7bd58af106 Updated copyright. 2012-11-07 18:07:00 +01:00
Bastian Kleineidam
faa052fa99 Fix test case. 2012-11-07 18:07:00 +01:00
Bastian Kleineidam
f9a7f5ef96 Restrict local file checking. 2012-11-07 18:07:00 +01:00
Bastian Kleineidam
eabaa41bd2 Do not check duplicate URLs. 2012-11-06 21:34:22 +01:00
Bastian Kleineidam
bc6cf5de34 Start local telnet server for tests. 2012-10-30 17:44:00 +01:00
Bastian Kleineidam
e594ca3c39 Improved documentation. 2012-10-30 17:44:00 +01:00
Bastian Kleineidam
a77a5dddfd Fix sporadic test failures with a dummy directory listing. 2012-10-15 14:36:27 +02:00
Bastian Kleineidam
aa2960e889 Fix content check. 2012-10-10 12:26:33 +02:00
Bastian Kleineidam
871508ef5d Add docs and updated copyright. 2012-10-10 06:53:16 +02:00
Bastian Kleineidam
81ca9a08d4 Fix typos. 2012-10-04 19:49:54 +02:00
Bastian Kleineidam
6f6608525e Split mail tests. 2012-10-01 20:11:59 +02:00
Bastian Kleineidam
6f5e55fd3b Code cleanup. 2012-10-01 10:43:20 +02:00
Bastian Kleineidam
1b3b040be5 Fix check result order. 2012-10-01 10:28:42 +02:00
Bastian Kleineidam
5a12ccf8d0 Fix anchor test result ordering. 2012-09-30 22:02:29 +02:00
Bastian Kleineidam
2479c53e6c Use a free port number in ftp tests for local server. 2012-09-29 19:22:12 +02:00
Bastian Kleineidam
cff97b9718 Updated copyright. 2012-09-21 21:13:00 +02:00
Bastian Kleineidam
1c2a66ffaf Refactor http tests into multiple files. 2012-09-21 20:34:05 +02:00
Bastian Kleineidam
a03090c20f Optimize intern/extern pattern parsing. 2012-09-20 20:19:13 +02:00
Bastian Kleineidam
aee515d406 Fix tests. 2012-09-18 09:17:08 +02:00
Bastian Kleineidam
58cbe4b152 Updated copyright. 2012-09-17 21:03:52 +02:00
Bastian Kleineidam
4e59056ee7 Warn about duplicate URL contents. 2012-09-17 19:49:50 +02:00
Bastian Kleineidam
99bf8aa940 Updated copyright. 2012-09-17 16:09:55 +02:00
Bastian Kleineidam
c3a6603987 Replace URL with example.org. 2012-09-17 16:07:06 +02:00
Bastian Kleineidam
cb71f483a5 Warn about too long URLs. 2012-09-17 16:00:23 +02:00
Bastian Kleineidam
93f3683ac1 Fix tests. 2012-09-02 23:22:01 +02:00
Bastian Kleineidam
07f7be5cf3 Remove slashdot from tests. 2012-08-23 23:56:50 +02:00
Bastian Kleineidam
14326710bc Updated copyright. 2012-08-23 16:46:06 +02:00
Bastian Kleineidam
e252bbf623 Remove Amazon quirk because the default behaviour handles this now. 2012-08-23 05:36:51 +02:00
Bastian Kleineidam
ecef16b2c9 Support WML sites. 2012-08-22 22:43:14 +02:00
Bastian Kleineidam
76f57dc4ad Updated copyright. 2012-08-14 20:37:24 +02:00
Bastian Kleineidam
564ae6479f Fix tests. 2012-08-13 18:01:59 +02:00
Bastian Kleineidam
fe82c380c3 Updated test data. 2012-08-12 11:15:36 +02:00
Bastian Kleineidam
1d7e93fe62 Updated http check data. 2012-08-12 10:55:25 +02:00
Bastian Kleineidam
b0e5c7fc59 Ignore feed: URLs. 2012-06-27 21:32:03 +02:00
Bastian Kleineidam
0fd1a78378 Always compare encoded anchor names. 2012-06-27 20:59:53 +02:00
Bastian Kleineidam
363ccc0121 Check <object codebase=...> as normal URL. 2012-06-23 14:28:32 +02:00
Bastian Kleineidam
3dd35c57a8 Rename wrong module name. 2012-06-20 21:43:25 +02:00
Bastian Kleineidam
dcf886860f Fix binary safari bookmark test. 2012-06-20 21:40:14 +02:00
Bastian Kleineidam
f107092a8a Fix handling of user/password info in URLs. 2012-06-10 22:07:42 +02:00
Bastian Kleineidam
db95fce77e Ignore PHP processing instructions in local files. 2012-06-10 14:02:01 +02:00
Bastian Kleineidam
7bb5dac321 Updated copyright 2012-04-23 21:33:59 +02:00
Bastian Kleineidam
cd6ee8a1bc Fix checker test cases for non-english locales. 2012-04-23 20:56:33 +02:00
Bastian Kleineidam
4c9fd8d488 Cache real url. 2012-03-14 21:12:13 +01:00
Bastian Kleineidam
187a94312b Updated copyright. 2012-03-09 11:16:18 +01:00
Bastian Kleineidam
626bd3e249 Fix obfuscated IP address check. 2012-03-09 10:08:04 +01:00
Bastian Kleineidam
71f5ee42c8 Updated copyright. 2012-01-29 17:18:28 +01:00
Bastian Kleineidam
a0581cc2a1 Ignore steam:// URIs. 2012-01-10 19:37:19 +01:00
Bastian Kleineidam
fb979b4f3c Add test for archive attribute support. 2011-12-30 12:36:22 +01:00
Bastian Kleineidam
4a04ff3224 Add html5 tests. 2011-12-30 12:30:07 +01:00
Bastian Kleineidam
8848e34d20 Add missing copyright. 2011-12-25 11:17:48 +01:00
Bastian Kleineidam
dff425710d More Freshmeat/Freecode replacements. 2011-12-25 09:06:18 +01:00
Bastian Kleineidam
9c40078ab9 Run obfuscated IP and unicode directory listing test only on linux. 2011-12-18 08:12:23 +01:00
Bastian Kleineidam
0e29ebde2a Run obfuscated IP test only on linux. 2011-12-18 08:10:33 +01:00
Bastian Kleineidam
2aea1ef1e5 Fix localhost tests. 2011-12-17 20:31:47 +01:00
Bastian Kleineidam
ef8d9b629f Use dynamic port number in results. 2011-12-17 19:13:14 +01:00
Bastian Kleineidam
f8ef9cca6a Test cleanup. 2011-12-17 16:39:21 +01:00
Bastian Kleineidam
3d9958dfbb Parse Safari bookmark files. 2011-12-17 16:38:25 +01:00
Bastian Kleineidam
09d9264470 Updated copyright. 2011-08-04 20:40:49 +02:00
Bastian Kleineidam
210b45d1e4 Removed FastCGI script. 2011-07-25 21:37:35 +02:00
Bastian Kleineidam
430e1db78d Do not use fixed port for HTTP server. 2011-05-28 19:24:38 +02:00
Bastian Kleineidam
6a544f2d69 Only allow redirections to FTP, HTTP and HTTPS URLs. 2011-04-19 07:01:55 +02:00
Bastian Kleineidam
82e5ba8ce6 Add warning tag attribute in XML loggers. 2011-03-15 13:42:21 +01:00
Bastian Kleineidam
f4f921384e Updated copyright 2011-03-13 07:52:18 +01:00
Bastian Kleineidam
ae109ed994 Correct conversion between URL and filename paths. 2011-03-11 10:38:17 +01:00
Bastian Kleineidam
420c21c2de Strip leading and trailing whitespace from URLs. 2011-03-07 12:33:09 +01:00
Bastian Kleineidam
633206342b Improved documentation. 2011-03-07 12:30:40 +01:00
Bastian Kleineidam
e638a2fe6d Updated copyright and translations. Added some missing docstrings. 2011-02-17 07:38:02 +01:00
Bastian Kleineidam
0d4377d1ba Support Google Chrome Bookmark files. 2011-02-15 18:26:00 +01:00
Bastian Kleineidam
85f3690068 Updated copyright. 2011-02-11 14:00:31 +01:00
Bastian Kleineidam
f11dcdf09f Fix tests with new example.com redirect. 2011-02-08 21:30:42 +01:00
Bastian Kleineidam
d011d1524c Parse PHP files recursively. 2010-12-28 17:11:29 +01:00
Bastian Kleineidam
0d8a583e39 Fix internal pattern for file URLs (regression from commit 90e0f4e) 2010-12-21 21:10:31 +01:00
Bastian Kleineidam
90e0f4e5cc Detect filenames with spaces as internal links. 2010-12-21 07:05:12 +01:00
Bastian Kleineidam
9876fd91a5 Fix test cases after anchor caching bugfix. 2010-12-20 20:37:23 +01:00
Bastian Kleineidam
0f1d3ed513 Disable news tests for now. 2010-12-19 17:15:09 +01:00
Bastian Kleineidam
6811448bc7 Split Telnet tests. 2010-11-22 20:30:47 +01:00
Bastian Kleineidam
19cfef0d33 Fix redirection test. 2010-11-22 19:52:21 +01:00
Bastian Kleineidam
26fc98b90c Fix usage of EnvironmentVarGuard. 2010-11-22 08:06:18 +01:00
Bastian Kleineidam
03034ddc1c Updated copyright 2010-11-21 11:25:07 +01:00
Bastian Kleineidam
017a1087ba Remove unneeded __future__ import 2010-11-21 10:45:30 +01:00
Bastian Kleineidam
eef778ed8e Use new EnvironmentVarGuard() for testing. 2010-11-21 10:44:54 +01:00
Bastian Kleineidam
17ce930611 Ignore irc:// URLs. 2010-11-10 19:56:31 +01:00
Bastian Kleineidam
c6ba688c70 Use new NTTP news server. 2010-11-06 17:27:23 +01:00
Bastian Kleineidam
532a4cf4e9 Revert amazon test changes since they fixed their servers. 2010-11-06 13:43:18 +01:00
Bastian Kleineidam
94e2225acc Updated tests for Amazon https server. 2010-11-05 12:27:09 +01:00