Commit graph

48 commits

Author SHA1 Message Date
Matthias
b0f7a805ef
Use builder pattern and channels (fixes #12) (#33)
This implements a basic builder for the Checker struct as discussed in #12.
It is using derive_builder and uses a custom build method to instantiate the more elaborate fields like reqwest::Client.
It also adds deadpool and tokio::mpsc as dependencies to handle a pool of clients to query websites.
2020-11-24 21:30:06 +01:00
Matthias Endler
d0b7a64d0a Refactor and add documentation 2020-11-10 00:03:50 +01:00
Paweł Romanowski
326683f4eb
Make GITHUB_TOKEN optional (#22)
* Make GITHUB_TOKEN optional

This also makes the token possible to pass in from CLI args.

* Add missing test fixture file

* Normalize exit codes and GitHub checking behavior

The exit code is now defined as 1 for unexpected or config errors,
and 2 for link check failures.

GitHub checking behavior has been tweaked to generate errors if
a GitHub-specific check cannot be performed because of a missing
token.

* Remove short flag for github token
2020-10-26 23:31:31 +01:00
Milton Fabian Bastidas Guerra
2bf62e7709
Add support for basic auth #18 (#20)
[Issue #18](https://github.com/hello-rust/lychee/issues/18)
* Add headers crate to type headers and create auth header
* Add cmd param basic-auth to set property to the main
* Add simple test to test if with auth headres is no broken

Signed-off-by: FabianBG <f4b4g3@gmail.com>
2020-10-26 09:23:45 +01:00
Matthias
f0e4c3adc1
Add support for include patterns (#23)
In one or more `include` arguments are specified, only check the URLs that match the patterns.
In case `exclude` arguments are also
specified, make an exception from the
excluded URLs if they also match the
`include` patterns.
2020-10-25 13:41:06 +01:00
Alexander Krantz
3a12b3e220
Configuration file (lychee.toml) (#16) 2020-10-21 02:10:25 +02:00
WhizSid
6bd7bbf51f
feat: Support relative URLs (#15) 2020-10-21 01:31:06 +02:00
Matthias
9ac2176b32
Merge pull request #10 from pawroman/fix_clippy_warnings
Actions improvements: add rustfmt and clippy checks, run tests
2020-10-18 00:31:28 +02:00
Paweł Romanowski
b2ada4746c Introduce cargo fmt and clippy checks, fix all clippy warnings 2020-10-16 14:35:38 +02:00
Paweł Romanowski
69e18785f9 Implement exclude private URLs feature
The exclusion is currently based on IP addresses, as specified by inputs.
We support IPv4 and IPv6, where possible using the current stable stdlib
(as of Rust 1.47.0).

Note that we could go one step further and resolve all URIs using DNS
and then exclude-filter the private IPs.
2020-10-16 13:59:42 +02:00
Paweł Romanowski
ba278743ed Simplify println invocation 2020-10-16 13:55:43 +02:00
Matthias
92b9ed11ca
Remove workaround for type recursion 2020-10-15 01:58:35 +02:00
Xiaochuan Yu
df54ce1eef Add progress bar 2020-10-10 00:39:06 -04:00
Matthias Endler
d1683bba32 Add e-mail checking support 2020-08-23 23:22:48 +02:00
Matthias Endler
16649a1d22 Use timeout instead of connect_timeout 2020-08-22 00:41:24 +02:00
Matthias Endler
c953528fb7 Add connection timeout 2020-08-22 00:36:03 +02:00
Matthias Endler
3650c673df Move to hubcaps. Allow defining accepted status codes 2020-08-18 01:17:26 +02:00
Matthias Endler
e224dcb5c7 Support head requests 2020-08-14 17:36:43 +02:00
Matthias Endler
25a31eacd6 Add support for custom request headers 2020-08-14 15:24:41 +02:00
Matthias Endler
b6a96c0c0f Clean up error code handling 2020-08-14 11:48:55 +02:00
Matthias Endler
bdd83128eb Refactor options 2020-08-14 11:43:45 +02:00
Matthias Endler
fbf6f09482 Refactor collector 2020-08-14 11:38:41 +02:00
Matthias Endler
391144b2ff Add globbing support 2020-08-14 02:33:04 +02:00
Matthias Endler
e758056f60 Add support for scheme (e.g. HTTPS) 2020-08-14 01:54:05 +02:00
Matthias Endler
8356ef1d03 Add support for website input 2020-08-14 01:14:47 +02:00
Matthias Endler
bbfe6a8531 rename summary function 2020-08-13 23:16:00 +02:00
Matthias Endler
b9757c505c Refactor link collection 2020-08-13 23:15:24 +02:00
Matthias Endler
47f1e306ab Support multiple file inputs 2020-08-13 23:01:30 +02:00
Matthias Endler
cca984017a Add support for changing number of threads 2020-08-13 19:58:41 +02:00
Matthias Endler
bb5268fbf1 Add statistics 2020-08-13 19:58:25 +02:00
Matthias Endler
96ea6c7a5c Add support for ignoring certificate errors 2020-08-12 23:38:21 +02:00
Matthias Endler
cd79f72d2d Make user-agent configurable 2020-08-12 13:10:15 +02:00
Matthias Endler
156f2b03c2 Make redirects configurable 2020-08-12 12:59:15 +02:00
Matthias Endler
1566a99647 Use CheckStatus enum for more fine-grained control over check results 2020-08-12 12:36:05 +02:00
Matthias Endler
d4a3b09790 Add support for excluding URLs 2020-08-11 22:48:50 +02:00
Matthias Endler
1d235b578b pico-args -> gumdrop
Needed multi-value args (with values accumulated into a vec)
for excluded urls
2020-08-11 22:18:37 +02:00
Matthias Endler
3fc369b3fa Make things blazing fast thanks to async 2020-08-11 16:13:10 +02:00
Matthias Endler
262b18e6b8 Rewrite link checking into iterator
This is a preparation for easier async execution
2020-08-09 23:36:14 +02:00
Matthias Endler
883b68ce66 Clippy 2020-08-09 22:50:17 +02:00
Matthias Endler
46e30f081b Move tests 2020-08-09 22:49:14 +02:00
Matthias Endler
23ba1f2e11 Formatting 2020-08-09 22:48:02 +02:00
Matthias Endler
bc615c9bfb Split up code into modules 2020-08-09 22:47:39 +02:00
Matthias Endler
7c51a24c44 Print instead of info 2020-08-09 22:43:25 +02:00
Matthias Endler
5876f494f3 Only request unique URLs 2020-08-08 00:09:10 +02:00
Matthias Endler
dc7af2d74e Clean up error handling and configure reqwests 2020-08-08 00:06:17 +02:00
Matthias Endler
a58b3e1232 Add logging and proper URL parsing 2020-08-07 19:00:21 +02:00
Matthias Endler
8885a63f82 check normal link first and only if it fails use Github api 2020-08-05 01:44:16 +02:00
Matthias Endler
2a93ce8093 Extract from hello-rust/show repository 2020-08-05 00:32:37 +02:00