Hopiu/lychee
1
0
Fork 0
mirror of https://github.com/Hopiu/lychee.git synced 2026-04-07 15:01:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
⚡ Fast, async, stream-based link checker written in Rust. Finds broken URLs and mail addresses inside Markdown, HTML, reStructuredText, websites and more!
101 commits 1 branch 0 tags 5.1 MiB
Rust 98.6%
Dockerfile 0.7%
Nix 0.3%
Makefile 0.3%
Shell 0.1%
Find a file
Matthias 93a2e93999
Formatting
2020-10-20 12:19:33 +02:00
.github/workflows Use more descriptive name for cargo test action 2020-10-16 14:47:54 +02:00
assets Add comparison table and recording of demo run (#17) 2020-10-20 10:40:45 +02:00
fixtures Add --exclude-all-private flag and cli integration test 2020-10-17 10:01:06 +02:00
src Un-ignore test_timeout, upgrade wiremock to 0.3 2020-10-18 11:44:00 +02:00
tests Add comment 2020-10-17 10:04:24 +02:00
.gitignore Extract from hello-rust/show repository 2020-08-05 00:32:37 +02:00
Cargo.lock Un-ignore test_timeout, upgrade wiremock to 0.3 2020-10-18 11:44:00 +02:00
Cargo.toml Un-ignore test_timeout, upgrade wiremock to 0.3 2020-10-18 11:44:00 +02:00
README.md Formatting 2020-10-20 12:19:33 +02:00

README.md

lychee

Rust

...because who says I can't write yet another link checker?

What?

This thing was created from Hello Rust Episode 10. It's a link checker that treats Github links specially by using a GITHUB_TOKEN to avoid getting blocked by the rate limiter.

Lychee demo

Why?

The existing link checkers were not flexible enough for my use-case. lychee runs all requests fully asynchronously and has a low memory/CPU footprint.

Features

lychee awesome_bot muffet broken-link-checker linkinator
Language Rust Ruby Go JS TypeScript
Static binary ✔️ ✖️ ✔️ ✖️ ✖️
Async/Parallel ✔️ ✔️ ✔️ ✔️ ✔️
Markdown support ✔️ ✔️ ✖️ ✖️ ✖️
HTML support ✔️ ✖️ ✖️ ✔️ ✔️
Plaintext support ✔️ ✖️ ✖️ ✖️ ✖️
Website support ✔️ ✖️ ✔️ ✔️ ✔️
Chunked encodings ✔️ ? ? ? ?
GZIP compression ✔️ ? ? ✔️ ?
Basic Auth ✖️ ✖️ ✖️ ✔️ ✖️
Custom user agent ✔️ ✖️ ✖️ ✔️ ✖️
Relative URLs ✖️ ✔️ ✖️ ✔️ ✔️
Skip relative URLs ✔️ ✖️ ✖️ ? ✖️
Include patterns ✖️ ✔️ ✖️ ✔️ ✖️
Exclude patterns ✔️ ✖️ ✔️ ✔️ ✔️
Handle redirects ✔️ ✔️ ✔️ ✔️ ✔️
Ignore SSL ✔️ ✔️ ✔️ ✖️ ✖️
File globbing ✔️ ✔️ ✖️ ✖️ ✔️
Limit scheme (e.g. only HTTPS) ✔️ ✖️ ✖️ ✔️ ✖️
[Custom headers] ✔️ ✖️ ✔️ ✖️ ✖️
Summary ✔️ ✔️ ✔️ ? ✔️
HEAD requests ✔️ ✔️ ✖️ ✔️ ✔️
Colored output ✔️ ? ✔️ ? ✔️
[Filter on status code] ✔️ ✔️ ✖️ ✖️ ✖️
Custom request timeout ✔️ ✔️ ✔️ ✖️ ✔️
E-mail links ✔️ ✖️ ✖️ ✖️ ✖️
Progress bar ✔️ ✔️ ✖️ ✖️ ✖️
Retry and backoff ✔️ ✖️ ✖️ ✖️ ✔️
Exclude private domains ✔️ ✖️ ✖️ ✖️ ✖️
[Usable as a library] ✖️ ✔️ ✖️ ✔️ ✔️
Silent mode ✔️ ✖️ ✖️ ✖️ ✔️

Planned features:

  • lychee.toml
  • report output in HTML, SQL, CSV, XML, JSON, YAML... format
  • report extended statistics: request latency
  • recursion
  • use colored output (https://crates.io/crates/colored)
  • skip duplicate urls

Users

How?

cargo install lychee

Set an environment variable with your token like so GITHUB_TOKEN=xxxx.

Run it inside a repository with a README.md or specify a file with

lychee <yourfile>

Comparison

Collecting other link checkers here to crush them in comparison. :P

Thanks

...to my Github sponsors and Patreon sponsors for supporting these projects. If you want to help out as well, go here.

[custom headers]: https://github.com/rust-lang/crates.io/issues/788) [filter on status code]: https://github.com/tcort/markdown-link-check/issues/94 [exclude private domains]: a5102b0bf9/url_checker.go [usable as library]: https://github.com/raviqqe/liche/issues/13