2020-08-04 23:06:27 +00:00
|
|
|
|
|
|
|
|
|
|
|
2020-08-07 22:10:30 +00:00
|
|
|
|
|
|
|
|
|
|
|
2020-11-05 23:22:14 +00:00
|
|
|
|
A fast, async, resource-friendly link checker written in Rust.
|
2020-10-26 22:31:31 +00:00
|
|
|
|
For GitHub links, it can optionally use a `GITHUB_TOKEN` to avoid getting blocked by the rate
|
2020-08-14 13:24:41 +00:00
|
|
|
|
limiter.
|
2020-08-04 23:06:27 +00:00
|
|
|
|
|
2020-10-20 08:40:45 +00:00
|
|
|
|
|
2020-08-17 18:14:46 +00:00
|
|
|
|
|
2020-10-18 22:09:53 +00:00
|
|
|
|
## Features
|
|
|
|
|
|
|
2020-10-20 22:21:11 +00:00
|
|
|
|
This comparison is made on a best-effort basis. Please create a PR to fix outdated information.
|
|
|
|
|
|
|
|
|
|
|
|
| | lychee | awesome_bot | muffet | broken-link-checker | linkinator | linkchecker | markdown-link-check | fink |
|
|
|
|
|
|
| -------------------- | ------ | ----------- | ------ | ------------------- | ---------- | ----------- | ------------------- | ---- |
|
|
|
|
|
|
| Language | Rust | Ruby | Go | JS | TypeScript | Python | JS | PHP |
|
|
|
|
|
|
| Async/Parallel | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
|
2020-10-21 10:44:12 +00:00
|
|
|
|
| Static binary | ✔️ | ✖️ | ✔️ | ✖️ | ✖️ | ️ ✖️ | ✖️ | ✖️ |
|
2020-10-20 22:21:11 +00:00
|
|
|
|
| Markdown files | ✔️ | ✔️ | ✖️ | ✖️ | ✖️ | ✖️ | ️ ✔️ | ✖️ |
|
|
|
|
|
|
| HTML files | ✔️ | ✖️ | ✖️ | ✔️ | ✔️ | ✖️ | ✖️ | ✖️ |
|
|
|
|
|
|
| Txt files | ✔️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ |
|
|
|
|
|
|
| Website support | ✔️ | ✖️ | ✔️ | ✔️ | ✔️ | ✔️ | ✖️ | ✔️ |
|
2020-10-21 10:46:24 +00:00
|
|
|
|
| Chunked encodings | ✔️ | **?** | **?** | **?** | **?** | ✖️ | ✔️ | ✔️ |
|
|
|
|
|
|
| GZIP compression | ✔️ | **?** | **?** | ✔️ | **?** | ✔️ | **?** | ✖️ |
|
2020-10-26 08:25:00 +00:00
|
|
|
|
| Basic Auth | ✔️ | ✖️ | ✖️ | ✔️ | ✖️ | ✔️ | ✖️ | ✖️ |
|
2020-10-20 22:21:11 +00:00
|
|
|
|
| Custom user agent | ✔️ | ✖️ | ✖️ | ✔️ | ✖️ | ✔️ | ✖️ | ✖️ |
|
2020-10-21 11:03:55 +00:00
|
|
|
|
| Relative URLs | ✔️ | ✔️ | ✖️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
|
2020-10-21 10:46:24 +00:00
|
|
|
|
| Skip relative URLs | ✔️ | ✖️ | ✖️ | **?** | ✖️ | ✖️ | ✖️ | ✖️ |
|
2020-10-25 12:41:06 +00:00
|
|
|
|
| Include patterns | ✔️️ | ✔️ | ✖️ | ✔️ | ✖️ | ✖️ | ✖️ | ✖️ |
|
2020-10-20 22:21:11 +00:00
|
|
|
|
| Exclude patterns | ✔️ | ✖️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
|
|
|
|
|
|
| Handle redirects | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
|
|
|
|
|
|
| Ignore insecure SSL | ✔️ | ✔️ | ✔️ | ✖️ | ✖️ | ✔️ | ✖️ | ✔️ |
|
|
|
|
|
|
| File globbing | ✔️ | ✔️ | ✖️ | ✖️ | ✔️ | ✖️ | ✔️ | ✖️ |
|
|
|
|
|
|
| Limit scheme | ✔️ | ✖️ | ✖️ | ✔️ | ✖️ | ✔️ | ✖️ | ✖️ |
|
|
|
|
|
|
| [Custom headers] | ✔️ | ✖️ | ✔️ | ✖️ | ✖️ | ✖️ | ✔️ | ✔️ |
|
2020-10-21 10:46:24 +00:00
|
|
|
|
| Summary | ✔️ | ✔️ | ✔️ | **?** | ✔️ | ✔️ | ✖️ | ✔️ |
|
2020-10-20 22:21:11 +00:00
|
|
|
|
| `HEAD` requests | ✔️ | ✔️ | ✖️ | ✔️ | ✔️ | ✔️ | ✖️ | ✖️ |
|
2020-10-21 10:46:24 +00:00
|
|
|
|
| Colored output | ✔️ | **?** | ✔️ | **?** | ✔️ | ✔️ | ✖️ | ✔️ |
|
2020-10-20 22:21:11 +00:00
|
|
|
|
| [Filter status code] | ✔️ | ✔️ | ✖️ | ✖️ | ✖️ | ✖️ | ✔️ | ✖️ |
|
|
|
|
|
|
| Custom timeout | ✔️ | ✔️ | ✔️ | ✖️ | ✔️ | ✔️ | ✖️ | ✔️ |
|
|
|
|
|
|
| E-mail links | ✔️ | ✖️ | ✖️ | ✖️ | ✖️ | ✔️ | ✖️ | ✖️ |
|
|
|
|
|
|
| Progress bar | ✔️ | ✔️ | ✖️ | ✖️ | ✖️ | ✔️ | ✔️ | ✔️ |
|
|
|
|
|
|
| Retry and backoff | ✔️ | ✖️ | ✖️ | ✖️ | ✔️ | ✖️ | ✔️ | ✖️ |
|
|
|
|
|
|
| Skip private domains | ✔️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ |
|
|
|
|
|
|
| [Use as lib] | ✖️ | ✔️ | ✖️ | ✔️ | ✔️ | ✖️ | ✔️ | ✖️ |
|
|
|
|
|
|
| Quiet mode | ✔️ | ✖️ | ✖️ | ✖️ | ✔️ | ✔️ | ✔️ | ✔️ |
|
2020-10-20 22:29:47 +00:00
|
|
|
|
| Amazing lychee logo | ✔️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ |
|
2020-10-21 10:41:53 +00:00
|
|
|
|
| Config file | ✔️ | ✖️ | ✖️ | ✖️ | ✔️ | ✔️ | ✔️ | ✖️ |
|
2020-10-20 08:40:45 +00:00
|
|
|
|
|
2020-11-05 23:22:14 +00:00
|
|
|
|
## Planned features. Please help out!
|
2020-10-20 08:40:45 +00:00
|
|
|
|
|
|
|
|
|
|
- report output in HTML, SQL, CSV, XML, JSON, YAML... format
|
|
|
|
|
|
- report extended statistics: request latency
|
|
|
|
|
|
- recursion
|
2020-08-26 13:31:29 +00:00
|
|
|
|
- use colored output (https://crates.io/crates/colored)
|
2020-10-20 08:40:45 +00:00
|
|
|
|
- skip duplicate urls
|
2020-10-20 22:21:11 +00:00
|
|
|
|
- request throttling
|
2020-08-19 23:08:04 +00:00
|
|
|
|
|
|
|
|
|
|
## Users
|
|
|
|
|
|
|
2020-10-20 21:13:45 +00:00
|
|
|
|
- https://github.com/analysis-tools-dev/static-analysis (soon)
|
|
|
|
|
|
- https://github.com/mre/idiomatic-rust (soon)
|
2020-08-17 18:14:46 +00:00
|
|
|
|
|
2020-08-04 23:06:27 +00:00
|
|
|
|
## How?
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
cargo install lychee
|
|
|
|
|
|
```
|
|
|
|
|
|
|
2020-10-26 22:31:31 +00:00
|
|
|
|
Optional (to avoid being rate limited for GitHub links): set an environment variable with your token
|
|
|
|
|
|
like so `GITHUB_TOKEN=xxxx`, or use the `--github-token` CLI option. This can also be set in the
|
|
|
|
|
|
config file.
|
2020-08-04 23:06:27 +00:00
|
|
|
|
|
2020-10-20 08:40:45 +00:00
|
|
|
|
Run it inside a repository with a `README.md` or specify a file with
|
2020-08-04 23:44:16 +00:00
|
|
|
|
|
|
|
|
|
|
```
|
2020-08-13 21:01:30 +00:00
|
|
|
|
lychee <yourfile>
|
2020-08-04 23:44:16 +00:00
|
|
|
|
```
|
2020-08-04 23:06:27 +00:00
|
|
|
|
|
2020-10-26 22:31:31 +00:00
|
|
|
|
### CLI exit codes
|
|
|
|
|
|
|
|
|
|
|
|
- `0` for success (all links checked successfully or excluded/skipped as configured)
|
|
|
|
|
|
- `1` for any unexpected runtime failures or config errors
|
|
|
|
|
|
- `2` for link check failures (if any non-excluded link failed the check)
|
|
|
|
|
|
|
2020-08-14 13:24:41 +00:00
|
|
|
|
## Comparison
|
|
|
|
|
|
|
|
|
|
|
|
Collecting other link checkers here to crush them in comparison. :P
|
|
|
|
|
|
|
|
|
|
|
|
- https://github.com/dkhamsing/awesome_bot
|
2020-08-17 18:14:46 +00:00
|
|
|
|
- https://github.com/tcort/markdown-link-check
|
|
|
|
|
|
- https://github.com/raviqqe/liche
|
|
|
|
|
|
- https://github.com/raviqqe/muffet
|
2020-10-17 22:15:11 +00:00
|
|
|
|
- https://github.com/stevenvachon/broken-link-checker
|
|
|
|
|
|
- https://github.com/JustinBeckwith/linkinator
|
|
|
|
|
|
- https://github.com/linkchecker/linkchecker
|
|
|
|
|
|
- https://github.com/dantleech/fink
|
|
|
|
|
|
- https://github.com/bartdag/pylinkvalidator
|
|
|
|
|
|
- https://github.com/victoriadrake/hydra-link-checker
|
2020-08-14 13:24:41 +00:00
|
|
|
|
|
2020-08-04 23:06:27 +00:00
|
|
|
|
## Thanks
|
|
|
|
|
|
|
2020-08-04 23:44:16 +00:00
|
|
|
|
...to my Github sponsors and Patreon sponsors for supporting these projects. If
|
|
|
|
|
|
you want to help out as well, [go here](https://github.com/sponsors/mre/).
|
2020-10-20 08:40:45 +00:00
|
|
|
|
|
|
|
|
|
|
[custom headers]: https://github.com/rust-lang/crates.io/issues/788)
|
2020-10-20 22:21:11 +00:00
|
|
|
|
[filter status code]: https://github.com/tcort/markdown-link-check/issues/94
|
|
|
|
|
|
[skip private domains]: https://github.com/appscodelabs/liche/blob/a5102b0bf90203b467a4f3b4597d22cd83d94f99/url_checker.go
|
|
|
|
|
|
[use as lib]: https://github.com/raviqqe/liche/issues/13
|
