Hopiu/lychee
1
0
Fork 0
mirror of https://github.com/Hopiu/lychee.git synced 2026-04-21 05:31:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
146 commits 1 branch 0 tags 5.1 MiB
Commit graph

64 commits

Author SHA1 Message Date
Paweł Romanowski
1f787613d4
Add support for reading from stdin and make input handling more robust (closes #26) 
* Adds a `skip_missing` flag
* Adds an `Input` enum to handle different types of inputs
 2020-12-02 23:28:37 +01:00
Matthias Endler
68b574d85d Make README.md the default when no inputs are given 
That used to be the case according to the docs,
but somehow we broke it while introducing some
changes.
 2020-11-25 10:40:36 +01:00
Matthias
8294e47307
Properly announce lychee user agent (#38) 2020-11-25 10:11:02 +01:00
Matthias
b0f7a805ef
Use builder pattern and channels (fixes #12) (#33) 
This implements a basic builder for the Checker struct as discussed in #12.
It is using derive_builder and uses a custom build method to instantiate the more elaborate fields like reqwest::Client.
It also adds deadpool and tokio::mpsc as dependencies to handle a pool of clients to query websites.
 2020-11-24 21:30:06 +01:00
Matthias Endler
d0b7a64d0a Refactor and add documentation 2020-11-10 00:03:50 +01:00
Paweł Romanowski
326683f4eb
Make GITHUB_TOKEN optional (#22) 
* Make GITHUB_TOKEN optional

This also makes the token possible to pass in from CLI args.

* Add missing test fixture file

* Normalize exit codes and GitHub checking behavior

The exit code is now defined as 1 for unexpected or config errors,
and 2 for link check failures.

GitHub checking behavior has been tweaked to generate errors if
a GitHub-specific check cannot be performed because of a missing
token.

* Remove short flag for github token
 2020-10-26 23:31:31 +01:00
Milton Fabian Bastidas Guerra
2bf62e7709
Add support for basic auth #18 (#20) 
[Issue #18](https://github.com/hello-rust/lychee/issues/18)
* Add headers crate to type headers and create auth header
* Add cmd param basic-auth to set property to the main
* Add simple test to test if with auth headres is no broken

Signed-off-by: FabianBG <f4b4g3@gmail.com>
 2020-10-26 09:23:45 +01:00
Matthias
f0e4c3adc1
Add support for include patterns (#23) 
In one or more `include` arguments are specified, only check the URLs that match the patterns.
In case `exclude` arguments are also
specified, make an exception from the
excluded URLs if they also match the
`include` patterns.
 2020-10-25 13:41:06 +01:00
Alexander Krantz
3a12b3e220
Configuration file (lychee.toml) (#16) 2020-10-21 02:10:25 +02:00
WhizSid
6bd7bbf51f
feat: Support relative URLs (#15) 2020-10-21 01:31:06 +02:00
Paweł Romanowski
0790afdbf2 Un-ignore test_timeout, upgrade wiremock to 0.3 2020-10-18 11:44:00 +02:00
Matthias
f9fe11b078
Merge pull request #2 from u5surf/issue-1 
Support exponential backoff in check_real #1
 2020-10-18 00:46:50 +02:00
Matthias
9ac2176b32
Merge pull request #10 from pawroman/fix_clippy_warnings 
Actions improvements: add rustfmt and clippy checks, run tests
 2020-10-18 00:31:28 +02:00
Paweł Romanowski
e175558376 Add --exclude-all-private flag and cli integration test 2020-10-17 10:01:06 +02:00
Paweł Romanowski
b2ada4746c Introduce cargo fmt and clippy checks, fix all clippy warnings 2020-10-16 14:35:38 +02:00
Paweł Romanowski
c043776b77 Use website_url function in checker tests 
This makes the test code shorter and more readable.
 2020-10-16 13:59:44 +02:00
Paweł Romanowski
69e18785f9 Implement exclude private URLs feature 
The exclusion is currently based on IP addresses, as specified by inputs.
We support IPv4 and IPv6, where possible using the current stable stdlib
(as of Rust 1.47.0).

Note that we could go one step further and resolve all URIs using DNS
and then exclude-filter the private IPs.
 2020-10-16 13:59:42 +02:00
Paweł Romanowski
cd6cf4add4 Add Uri::host_ip method and tests 
This lets extract the host IP address, if defined for a website.
Mail addresses are not supported.
 2020-10-16 13:55:43 +02:00
Paweł Romanowski
ba278743ed Simplify println invocation 2020-10-16 13:55:43 +02:00
Matthias
92b9ed11ca
Remove workaround for type recursion 2020-10-15 01:58:35 +02:00
Xiaochuan Yu
e80cb70a98 default true flags don't work? 2020-10-10 11:24:56 -04:00
Xiaochuan Yu
df54ce1eef Add progress bar 2020-10-10 00:39:06 -04:00
u5surf
fb860c52d2 Support exponential backoff in check_real #1 2020-10-03 06:34:30 +09:00
Matthias Endler
d1683bba32 Add e-mail checking support 2020-08-23 23:22:48 +02:00
Matthias Endler
16649a1d22 Use timeout instead of connect_timeout 2020-08-22 00:41:24 +02:00
Matthias Endler
c953528fb7 Add connection timeout 2020-08-22 00:36:03 +02:00
Matthias Endler
3650c673df Move to hubcaps. Allow defining accepted status codes 2020-08-18 01:17:26 +02:00
Matthias Endler
17139cfd3b Add escaping test 2020-08-17 20:19:36 +02:00
Matthias Endler
e224dcb5c7 Support head requests 2020-08-14 17:36:43 +02:00
Matthias Endler
25a31eacd6 Add support for custom request headers 2020-08-14 15:24:41 +02:00
Matthias Endler
b6a96c0c0f Clean up error code handling 2020-08-14 11:48:55 +02:00
Matthias Endler
bdd83128eb Refactor options 2020-08-14 11:43:45 +02:00
Matthias Endler
fbf6f09482 Refactor collector 2020-08-14 11:38:41 +02:00
Matthias Endler
391144b2ff Add globbing support 2020-08-14 02:33:04 +02:00
Matthias Endler
e758056f60 Add support for scheme (e.g. HTTPS) 2020-08-14 01:54:05 +02:00
Matthias Endler
8356ef1d03 Add support for website input 2020-08-14 01:14:47 +02:00
Matthias Endler
bbfe6a8531 rename summary function 2020-08-13 23:16:00 +02:00
Matthias Endler
b9757c505c Refactor link collection 2020-08-13 23:15:24 +02:00
Matthias Endler
47f1e306ab Support multiple file inputs 2020-08-13 23:01:30 +02:00
Matthias Endler
cca984017a Add support for changing number of threads 2020-08-13 19:58:41 +02:00
Matthias Endler
bb5268fbf1 Add statistics 2020-08-13 19:58:25 +02:00
Matthias Endler
184e263a44 Remove Github token from env var 2020-08-13 19:57:48 +02:00
Matthias Endler
96ea6c7a5c Add support for ignoring certificate errors 2020-08-12 23:38:21 +02:00
Matthias Endler
cd79f72d2d Make user-agent configurable 2020-08-12 13:10:15 +02:00
Matthias Endler
156f2b03c2 Make redirects configurable 2020-08-12 12:59:15 +02:00
Matthias Endler
1566a99647 Use CheckStatus enum for more fine-grained control over check results 2020-08-12 12:36:05 +02:00
Matthias Endler
d4a3b09790 Add support for excluding URLs 2020-08-11 22:48:50 +02:00
Matthias Endler
1d235b578b pico-args -> gumdrop 
Needed multi-value args (with values accumulated into a vec)
for excluded urls
 2020-08-11 22:18:37 +02:00
Matthias Endler
ad8033a90f Info output 2020-08-11 22:06:18 +02:00
Matthias Endler
3fc369b3fa Make things blazing fast thanks to async 2020-08-11 16:13:10 +02:00