Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-04-29 18:44:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
7085 commits 1 branch 0 tags 24 MiB
Commit graph

333 commits

Author SHA1 Message Date
Chris Mayo
c3d0787654 Replace deprecated http.server.CGIHTTPRequestHandler for TestLoginUrl 
Scheduled for removal in Python 3.15.
 2023-10-16 19:23:43 +01:00
Chris Mayo
40c39c27e6 Fix TestMisc.test_itms_services() on Python 3.12 
E   -cache key itms-services:?action=download-manifest&url=http://www.example.com/
E   -real url itms-services:?action=download-manifest&url=http://www.example.com/
E   +cache key itms-services://?action=download-manifest&url=http://www.example.com/
E   +real url itms-services://?action=download-manifest&url=http://www.example.com/
E    valid

gh-104139: Teach urllib.parse.unsplit() to retain the "//" when assembling
itms-services://?action=generate-bugs style Apple Platform Deployment [2] URLs. [1]

[1]
https://docs.python.org/3.12/whatsnew/changelog.html

[2] "Distribute proprietary in-house apps to Apple devices":
https://support.apple.com/en-gb/guide/deployment/depce7cefc4d/web
 2023-09-06 19:22:04 +01:00
Chris Mayo
0faccf2ab3
Merge pull request #752 from cjmayo/deprecated_modules 
Remove support for nntp and telnet
 2023-09-04 19:22:38 +01:00
Chris Mayo
b3429c4759 Remove support for nntp and telnet 
Python is dropping nntplib and telnetlib.
 2023-08-28 19:24:57 +01:00
Chris Mayo
beaf9399f8 Elevate redirection to a warning tagged http-redirected 
Include the HTTP status code and reason in the message.
 2023-08-28 19:22:24 +01:00
Chris Mayo
1cb7f3d78a Test on GitHub with httpbin from a container 
Avoid failures due to the web service.
 2023-06-05 19:30:59 +01:00
Chris Mayo
4b06485a05 Fix FTP checker 
In Python 2 StringIO could accept either Unicode or 8-bit strings.
Similar change made for HttpUrl:
06fdd78f9 ("Python3: fix TypeError in HttpUrl.read_content()",
2019-09-15)

Non-existent FtpUrl.max_size introduced in:
7b34be590 ("Introduce check plugins, use Python requests for http/s
connections, and some code cleanups and improvements.", 2014-03-01)

Additional self.direct() not added in:
f107092a8 ("Fix handling of user/password info in URLs.", 2012-06-10)
 2023-04-17 19:24:22 +01:00
Chris Mayo
1fcc0e2b4c Add FTP tests for listing and download 2023-04-17 19:24:22 +01:00
Chris Mayo
a7b9ac42be Fail on internal errors in LinkCheckTest.direct() 
Copied from LinkCheckTest.file_test().
 2023-04-17 19:24:22 +01:00
Chris Mayo
a673a26409 Replace deprecated cgi module in loginurl tests 2023-01-20 19:22:23 +00:00
Chris Mayo
b6bc366af0 Run pyupgrade --py37-plus x 2 2022-11-08 19:21:29 +00:00
Chris Mayo
0bb1576887 Run pyupgrade --py37-plus --keep-percent-format 2022-11-08 19:21:29 +00:00
Chris Mayo
78536c578a Fix tests failing when run with pytest 
TypeError: 'NoneType' object is not callable

As per:
2cbff492 ("Fix http tests failing with pytest due to missing _()", 2022-10-03)
 2022-10-17 19:26:53 +01:00
Chris Mayo
bbb8096df5 Add @need_network to test_no_error() in test_ignoreerrors.py 
Needs network access for DNS:

warning No MX mail host for example.com found.
 2022-10-05 19:27:13 +01:00
Chris Mayo
354ea933ca
Merge pull request #673 from cjmayo/sitemap 
Fix sitemap output with multiple threads
 2022-10-05 19:20:40 +01:00
Nathan Arthur
2d1bf6ef98 Add tests for encoded anchors for file: and http: 
I started with a test of urlencoded anchors, assuming at the URL might
have a urlencoded anchor, but the actual anchor in the HTML would NOT be
urlencoded.
 2022-10-03 19:33:05 +01:00
Nathan Arthur
5398fd2406 Add an anchor test for multiple inter-connected files 2022-10-03 19:33:05 +01:00
Nathan Arthur
c221afdab5 Enable AnchorCheck to be used with local files 
[I] discovered that fileurl.py was stripping the anchors from url_data,
which breaks AnchorCheck. So I stopped it from doing that, and
tried to fix up all the places that were assuming the url would map to a
filesystem file. The tests all pass, but I'm not 100% sure I caught all
the cases, or fixed them correctly.
 2022-10-03 19:33:05 +01:00
Chris Mayo
2cbff49221 Fix http tests failing with pytest due to missing _() 
TypeError: 'NoneType' object is not callable

Ensure LinkCheckTest.setUp() is called to initialise translations.
 2022-10-03 19:33:05 +01:00
Chris Mayo
54bcefd7d7 Revert "Disable AnchorCheck plugin" 
This reverts commit 0356524369.
 2022-10-03 19:33:05 +01:00
Chris Mayo
009f22e9b6 Remove outdated comment in TestLogger 
Configuration.init_logging() removed in:
eaa8a963 ("Refactor logging configuration.", 2014-05-10)
 2022-09-30 19:22:17 +01:00
Lukas Pirl
8c959589c3
add option to ignore specific errors for specific URLs 2022-09-25 22:52:04 +02:00
Chris Mayo
3c7fb5b571 Fix checking directory containing Unicode filenames 
Non-Unicode filenames are not supported.

sys.platform has not returned "linux2" since Python 3.3.
 2022-09-05 19:28:40 +01:00
Chris Mayo
1abd9ea10e Skip tests in TestFile rather than silently returning 2022-09-05 19:28:40 +01:00
Chris Mayo
d6936ceb91 Add warning url-content-type-unparseable 2022-09-02 19:29:11 +01:00
Kian-Meng Ang
a70ea9ea14 Fix typos 
Found via `codespell ./linkcheck/ ./tests ./doc/man/en -L bu,noone,fo,pres,shttp`
 2022-09-02 17:20:02 +08:00
Chris Mayo
b0878dd7e8 Require TestFile.test_directory_listing to succeed 2021-12-15 19:36:53 +00:00
Chris Mayo
76815bcf47 Don't guess the URL for files that end in .html 
Fixes:
linkchecker ftp.html
failing looking for ftp://ftp.html
 2021-12-13 19:31:13 +00:00
Chris Mayo
fe5a34c68f Remove linkcheck.checker.proxysupport 
Set up the requests.Session() with the complete proxy configuration
to fix a problem with using an HTTP server as an HTTPS proxy and
potential redirection issues.

Requests handles no_proxy.
 2021-12-13 19:25:23 +00:00
Chris Mayo
0356524369 Disable AnchorCheck plugin 
Can't be relied on. Multiple reports of expected results not returned.

https://github.com/linkchecker/linkchecker/issues/542
https://github.com/linkchecker/linkchecker/issues/555
https://github.com/linkchecker/linkchecker/issues/568

Previously a fix was needed just to get the tests working:
0912e8a2c ("Don't strip the URL fragment from cache key if using AnchorCheck", 2020-07-27)

After:
eaa538c81 ("don't check one url multiple times", 2016-11-09)
 2021-11-29 19:35:34 +00:00
Chris Mayo
ef60e9dcd6 Enable certificate verification during https test 2021-11-22 19:27:18 +00:00
Chris Mayo
bb4102da5a Replace deprecated ssl.wrap_socket() in tests 2021-11-22 19:27:18 +00:00
Chris Mayo
7c2036b68c Drop support for Beautiful Soup < 4.8.1 
The minimum version supported was already 4.8.0 because of the use
of multi_valued_attributes [1].

Test support for < 4.8.1 is the only code that needs removing [2].

[1] 3ff3d724 ("Use BeautifulSoup element attrs directly", 2020-04-03)
[2] 607328d5 ("Support Beautiful Soup line numbers", 2019-10-05)
 2021-01-28 19:20:24 +00:00
Chris Mayo
e922dd0224 Stop using biplist 
plistlib has supported binary files since Python 3.4.
 2020-10-12 19:55:46 +01:00
Chris Mayo
9891fc3f70 Python 3.9 adds suport for HTTP status code 103 EARLY_HINTS 2020-09-14 19:55:05 +01:00
Chris Mayo
b1faef93c3
Merge pull request #495 from cjmayo/mswindows 
MS Windows Python 3.7 and MS Store compatibility
 2020-09-01 19:46:44 +01:00
Chris Mayo
314ec085a3
Merge pull request #462 from cjmayo/anchor 
Fix anchor checking
 2020-09-01 19:39:29 +01:00
Chris Mayo
89613d56f2 Replace the use of Python internal test.support 
Its use is discourged and it is not present in the MS Store version of
Python.
 2020-08-29 16:57:57 +01:00
Chris Mayo
1390c9cd7e
Merge pull request #489 from cjmayo/urlsplit 
Replace deprecated urllib.parse.split functions
 2020-08-29 16:44:56 +01:00
Chris Mayo
2de25d54fd Rename blacklist to failures 
Continue to support blacklist for the time being, with deprecation
warnings.
 2020-08-23 17:19:26 +01:00
Chris Mayo
737c61cd67
Merge pull request #484 from cjmayo/issuetests 
Tests of img srcset and invalid host name
 2020-08-22 16:32:03 +01:00
Chris Mayo
f99f15c349 Add a test for UrlBase.build_url() 2020-08-22 16:28:53 +01:00
Chris Mayo
24c2f4ac39 Add test for invalid host name in content 
Tests code added in:
d5690203 ("Fix critical exception when parsing a URL with a ]", 2020-08-08)
 2020-08-15 17:04:41 +01:00
Chris Mayo
8c804c35a5 Detect sitemaps that do not start with an XML declaration 2020-08-11 19:35:56 +01:00
Chris Mayo
a7eacd6200 Add a test for a page with links to anchors 
Query and fragment URL parts for filesystem URLs are ignored, therefore
test over http.
 2020-07-27 19:22:32 +01:00
Chris Mayo
4330b8a59e Replace codecs.open() with open() 2020-06-05 16:59:46 +01:00
Chris Mayo
a6b1eb45b1 Convert to Python 3 super() 2020-06-03 20:06:36 +01:00
Chris Mayo
5df8aa085c Convert space-separated strings in tests/ 2020-05-29 19:40:46 +01:00
Chris Mayo
5ee8d8e1ea Add trailing comma to single dict list in TestLoginUrl.visit_loginurl() 2020-05-29 19:40:46 +01:00
Chris Mayo
a534be0b50 Remove unnecessary character match in regexp in TestLogger.normalize() 2020-05-29 19:40:46 +01:00