Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-04-02 06:00:39 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
6347 commits 1 branch 0 tags 24 MiB
Commit graph

6347 commits

This branch
This branch
All branches
Author SHA1 Message Date
Marius Gedminas
dcafa2df75
Avoid u-prefixed strings 
linkchecker is Python 3 only, all strings are unicode.
 2020-05-24 14:50:07 +03:00
Marius Gedminas
f268a90cfb
Merge branch 'master' into HandleRateLimiting 2020-05-23 14:15:52 +03:00
anarcat
b1e8137da2
Merge pull request #410 from cjmayo/install 
Installation of test data, standardise Markdown extension, remove linkchecker.desktop
 2020-05-22 20:07:15 -04:00
Chris Mayo
df79e9b196 Add missing test data and Markdown documentation to the distribution 2020-05-22 19:43:57 +01:00
Chris Mayo
c60887cc63 Rename .mdwn files to .md 
- RFC 7763 file extensions are .md and .markdown
- Consistent with other documentation files
 2020-05-22 19:43:57 +01:00
Chris Mayo
87f0c31928 Remove linkchecker.desktop 
- Not useful for a command-line application
- Refers to an icon with a generic name, that is not installed
 2020-05-22 19:43:57 +01:00
Marius Gedminas
6dffacf17f
Merge pull request #409 from linkchecker/fix-login-timeouts 
Make sure login form fetching uses a timeout and sends User-Agent
 2020-05-22 21:40:48 +03:00
anarcat
2256a6e889
Merge pull request #408 from linkchecker/fix-timeouts 
Make sure fetching robots.txt uses the configured timeout
 2020-05-22 14:29:12 -04:00
Marius Gedminas
b0435b3d47 Make sure login form fetching uses a timeout 
Also resolve an XXX comment about the User-Agent header (which is
configured in new_request_session), but add a couple of XXX comments
about using proxy and possibly disabling TLS certificate checking.
 2020-05-22 11:19:51 +03:00
Marius Gedminas
4f3fe5e1c3 Make sure fetching robots.txt uses the configured timeout 
Closes #396.
 2020-05-22 10:53:33 +03:00
Marius Gedminas
639ba0dba2
Merge pull request #406 from linkchecker/fix-empty-file-problem 
Fix internal error on empty HTML files
 2020-05-21 19:57:46 +03:00
Marius Gedminas
c60d7c66e4 Clarify the decision to fall back to Latin-1 2020-05-21 19:35:39 +03:00
Marius Gedminas
5bd1fb4e36 Fix internal error on empty HTML files 
When BeautifulSoup finds an empty file on disk, it sets
original_encoding to None.  It doesn't matter what encoding we pick for
empty files, so let's just pick one.

I don't know if there are any circumstances where BeautifulSoup might
set the encoding to None for a non-empty file.

Closes #392.
 2020-05-21 19:01:33 +03:00
Marius Gedminas
fd3ab13470
Merge pull request #397 from linkchecker/doc-389 
do not require ssh to clone from source
 2020-05-21 18:28:20 +03:00
anarcat
a226b4e406
Merge pull request #405 from cjmayo/tidyten13 
Remove encoding of TestLogger diff  and url in Checker.check_url_data()
 2020-05-21 08:56:08 -04:00
Chris Mayo
6cfc8eeb49 Replace threading.Thread.setName() with setting the name property 
As recommended in:

https://docs.python.org/3.5/library/threading.html#threading.Thread.setName
 2020-05-20 19:58:44 +01:00
Chris Mayo
42eba19a7d No need to encode url in Checker.check_url_data() 
Was causing b'' in log messages e.g. CheckThread-b'http:...
 2020-05-20 19:58:44 +01:00
Chris Mayo
96e1c00ff7 TestLogger diff output is all Unicode in Python 3 2020-05-20 19:58:44 +01:00
Chris Mayo
768952e111
Merge pull request #403 from cjmayo/tidyten12 
Remove "from builtins import str as str_text"
 2020-05-20 19:38:14 +01:00
Marius Gedminas
1ab45c2e60
Merge pull request #402 from linkchecker/flake8 
Add a 'tox -e flake8' and a Travis CI job
 2020-05-19 23:09:03 +03:00
Chris Mayo
71eaf9a982 Remove str_text from tests/ 2020-05-19 19:56:42 +01:00
Chris Mayo
28f4587dfa Remove str_text from fileutil.py, strformat.py and url.py 2020-05-19 19:56:42 +01:00
Chris Mayo
ebcc3c4961 Remove str_text from plugins/ 2020-05-19 19:56:42 +01:00
Chris Mayo
1c14583535 Remove str_text from logger/ 2020-05-19 19:56:42 +01:00
Chris Mayo
6bddd4ac60 Remove str_text from checker/ 2020-05-19 19:56:42 +01:00
Chris Mayo
a127902607 Replace str_text in asserts 2020-05-19 19:56:42 +01:00
Chris Mayo
7490804e2c
Merge pull request #395 from cjmayo/tidyten11 
Remove unused code from linkcheck/fileutil.py
 2020-05-19 19:45:08 +01:00
Marius Gedminas
7a43abe6d6 Add a flake8 job to the Travis matrix 
https://docs.travis-ci.com/user/build-matrix/#rows-that-are-allowed-to-fail
suggests that this might not work.
 2020-05-19 19:27:40 +03:00
Marius Gedminas
72e7c600f3 Add a 'tox -e flake8' 2020-05-19 19:24:22 +03:00
anarcat
8183b7feb8
Update doc/install.txt 
Co-authored-by: Marius Gedminas <marius@gedmin.as>
 2020-05-19 12:05:21 -04:00
Marius Gedminas
391bd5882a
Merge pull request #394 from gbabin/fix-translations-encoding 
Fix translations encoding (issue #165)
 2020-05-19 18:53:06 +03:00
Marius Gedminas
e6e969f975
Merge pull request #391 from linkchecker/dev-version 
Bump version in git to 10.0.0.dev0
 2020-05-19 18:49:34 +03:00
Antoine Beaupré
2c9bc4e7ba
do not require ssh to clone from source 
Closes: #389
 2020-05-19 11:04:15 -04:00
Chris Mayo
690605c519 Remove unused code from linkcheck/fileutil.py 2020-05-18 19:29:55 +01:00
Chris Mayo
ce0a151c8a
Merge pull request #393 from cjmayo/tidyten10 
Assorted test tidy-ups
 2020-05-18 19:16:44 +01:00
Guillaume Babin
1cda677bac
Make po/Makefile version-agnostic 
Co-authored-by: Marius Gedminas <marius@gedmin.as>
 2020-05-18 18:24:36 +02:00
gbabin
fc763be5a2 Make translations 2020-05-18 13:33:12 +02:00
gbabin
ede0b149fd Fix .mo makefile 2020-05-18 13:32:28 +02:00
gbabin
d339c61602 Fix #165 translations encoding 2020-05-18 12:58:45 +02:00
Chris Mayo
12fd59057e Remove duplicate tests from test_strformat.py 2020-05-17 20:10:28 +01:00
Chris Mayo
339d293326 Convert tests/test_po.py to UTF-8 2020-05-17 20:10:28 +01:00
Chris Mayo
04465530c4 Use HttpServerTest.get_url() 2020-05-17 20:10:28 +01:00
Chris Mayo
58dbe1f282 Remove unused import pytest from tests/checker/test_http.py 
pytest.mark.xfail() removed in:
743a5f31 ("Crawl HTML attributes in deterministic order", 2017-02-01)
 2020-05-17 20:10:28 +01:00
Marius Gedminas
5317347e54 Avoid distutils.version.StrictVersion 
distutils.version is old code that predates PEP 440.  We could add a
dependency on https://packaging.pypa.io/en/latest/version/, but meh.
 2020-05-17 21:12:43 +03:00
Chris Mayo
52dc90255a
Merge pull request #390 from cjmayo/viruscheck 
Add a test for VirusCheck
 2020-05-17 19:08:49 +01:00
Chris Mayo
79eafee826 Add a test for VirusCheck 2020-05-17 19:04:49 +01:00
Marius Gedminas
05c02da2b0 Bump version in git to 10.0.0.dev0 
It is confusing to have different versions of the code self-identify
with the same version number.  In my experience it's always best to
increment the version number and add a .dev0 suffix right after making a
release.  When it's time to make a new release, you remove the .dev0,
commit, tag that commit, then make second commit that bumps the version
and adds .dev0 back.

This way only releases identify themselves as "version X.Y.Z" with no
.dev0 suffix and it's immediatelly apparent when you've got a prerelease
installed from git.
 2020-05-17 20:06:50 +03:00
Marius Gedminas
bb53aaa621 Fix viruscheck plugin 
The clamav interface needs bytes, not unicode.

It would be nice if we had tests for this code.
 2020-05-17 17:50:11 +01:00
Chris Mayo
1e35530f9d
Merge pull request #388 from cjmayo/tidyten9 
Remove spaces after names
 2020-05-17 17:34:04 +01:00
Chris Mayo
a15a2833ca Remove spaces after names in class method definitions 
And also nested functions.

This is a PEP 8 convention, E211.
 2020-05-16 20:19:42 +01:00