Commit graph

55 commits

Author SHA1 Message Date
WhizSid
6bd7bbf51f
feat: Support relative URLs (#15) 2020-10-21 01:31:06 +02:00
Paweł Romanowski
0790afdbf2 Un-ignore test_timeout, upgrade wiremock to 0.3 2020-10-18 11:44:00 +02:00
Matthias
f9fe11b078
Merge pull request #2 from u5surf/issue-1
Support exponential backoff in check_real #1
2020-10-18 00:46:50 +02:00
Matthias
9ac2176b32
Merge pull request #10 from pawroman/fix_clippy_warnings
Actions improvements: add rustfmt and clippy checks, run tests
2020-10-18 00:31:28 +02:00
Paweł Romanowski
e175558376 Add --exclude-all-private flag and cli integration test 2020-10-17 10:01:06 +02:00
Paweł Romanowski
b2ada4746c Introduce cargo fmt and clippy checks, fix all clippy warnings 2020-10-16 14:35:38 +02:00
Paweł Romanowski
c043776b77 Use website_url function in checker tests
This makes the test code shorter and more readable.
2020-10-16 13:59:44 +02:00
Paweł Romanowski
69e18785f9 Implement exclude private URLs feature
The exclusion is currently based on IP addresses, as specified by inputs.
We support IPv4 and IPv6, where possible using the current stable stdlib
(as of Rust 1.47.0).

Note that we could go one step further and resolve all URIs using DNS
and then exclude-filter the private IPs.
2020-10-16 13:59:42 +02:00
Paweł Romanowski
cd6cf4add4 Add Uri::host_ip method and tests
This lets extract the host IP address, if defined for a website.
Mail addresses are not supported.
2020-10-16 13:55:43 +02:00
Paweł Romanowski
ba278743ed Simplify println invocation 2020-10-16 13:55:43 +02:00
Matthias
92b9ed11ca
Remove workaround for type recursion 2020-10-15 01:58:35 +02:00
Xiaochuan Yu
e80cb70a98 default true flags don't work? 2020-10-10 11:24:56 -04:00
Xiaochuan Yu
df54ce1eef Add progress bar 2020-10-10 00:39:06 -04:00
u5surf
fb860c52d2 Support exponential backoff in check_real #1 2020-10-03 06:34:30 +09:00
Matthias Endler
d1683bba32 Add e-mail checking support 2020-08-23 23:22:48 +02:00
Matthias Endler
16649a1d22 Use timeout instead of connect_timeout 2020-08-22 00:41:24 +02:00
Matthias Endler
c953528fb7 Add connection timeout 2020-08-22 00:36:03 +02:00
Matthias Endler
3650c673df Move to hubcaps. Allow defining accepted status codes 2020-08-18 01:17:26 +02:00
Matthias Endler
17139cfd3b Add escaping test 2020-08-17 20:19:36 +02:00
Matthias Endler
e224dcb5c7 Support head requests 2020-08-14 17:36:43 +02:00
Matthias Endler
25a31eacd6 Add support for custom request headers 2020-08-14 15:24:41 +02:00
Matthias Endler
b6a96c0c0f Clean up error code handling 2020-08-14 11:48:55 +02:00
Matthias Endler
bdd83128eb Refactor options 2020-08-14 11:43:45 +02:00
Matthias Endler
fbf6f09482 Refactor collector 2020-08-14 11:38:41 +02:00
Matthias Endler
391144b2ff Add globbing support 2020-08-14 02:33:04 +02:00
Matthias Endler
e758056f60 Add support for scheme (e.g. HTTPS) 2020-08-14 01:54:05 +02:00
Matthias Endler
8356ef1d03 Add support for website input 2020-08-14 01:14:47 +02:00
Matthias Endler
bbfe6a8531 rename summary function 2020-08-13 23:16:00 +02:00
Matthias Endler
b9757c505c Refactor link collection 2020-08-13 23:15:24 +02:00
Matthias Endler
47f1e306ab Support multiple file inputs 2020-08-13 23:01:30 +02:00
Matthias Endler
cca984017a Add support for changing number of threads 2020-08-13 19:58:41 +02:00
Matthias Endler
bb5268fbf1 Add statistics 2020-08-13 19:58:25 +02:00
Matthias Endler
184e263a44 Remove Github token from env var 2020-08-13 19:57:48 +02:00
Matthias Endler
96ea6c7a5c Add support for ignoring certificate errors 2020-08-12 23:38:21 +02:00
Matthias Endler
cd79f72d2d Make user-agent configurable 2020-08-12 13:10:15 +02:00
Matthias Endler
156f2b03c2 Make redirects configurable 2020-08-12 12:59:15 +02:00
Matthias Endler
1566a99647 Use CheckStatus enum for more fine-grained control over check results 2020-08-12 12:36:05 +02:00
Matthias Endler
d4a3b09790 Add support for excluding URLs 2020-08-11 22:48:50 +02:00
Matthias Endler
1d235b578b pico-args -> gumdrop
Needed multi-value args (with values accumulated into a vec)
for excluded urls
2020-08-11 22:18:37 +02:00
Matthias Endler
ad8033a90f Info output 2020-08-11 22:06:18 +02:00
Matthias Endler
3fc369b3fa Make things blazing fast thanks to async 2020-08-11 16:13:10 +02:00
Matthias Endler
262b18e6b8 Rewrite link checking into iterator
This is a preparation for easier async execution
2020-08-09 23:36:14 +02:00
Matthias Endler
8588d2eade Update tests 2020-08-09 23:16:23 +02:00
Matthias Endler
6e0f559b25 Switch to linkify to cover non MD links 2020-08-09 23:12:25 +02:00
Matthias Endler
fb517dab03 Add tests for extract 2020-08-09 23:09:27 +02:00
Matthias Endler
883b68ce66 Clippy 2020-08-09 22:50:17 +02:00
Matthias Endler
46e30f081b Move tests 2020-08-09 22:49:14 +02:00
Matthias Endler
23ba1f2e11 Formatting 2020-08-09 22:48:02 +02:00
Matthias Endler
bc615c9bfb Split up code into modules 2020-08-09 22:47:39 +02:00
Matthias Endler
7c51a24c44 Print instead of info 2020-08-09 22:43:25 +02:00