Fast, async, stream-based link checker written in Rust. Finds broken URLs and mail addresses inside Markdown, HTML, reStructuredText, websites and more!
Find a file
2020-08-14 01:54:05 +02:00
.github/workflows Create rust.yml 2020-08-08 00:09:48 +02:00
assets Add logging and proper URL parsing 2020-08-07 19:00:21 +02:00
fixtures Add more links 2020-08-09 22:43:11 +02:00
src Add support for scheme (e.g. HTTPS) 2020-08-14 01:54:05 +02:00
.gitignore Extract from hello-rust/show repository 2020-08-05 00:32:37 +02:00
Cargo.lock Pin dependencies 2020-08-13 09:43:06 +02:00
Cargo.toml Pin dependencies 2020-08-13 09:43:06 +02:00
README.md Add support for scheme (e.g. HTTPS) 2020-08-14 01:54:05 +02:00

lychee

Rust

...because who says I can't write yet another link checker.

What?

This thing was created from Hello Rust Episode 10. It's a link checker that treats Github links specially by using a GITHUB_TOKEN to avoid getting blocked less by the rate limiter.

Why?

The existing link checkers were not flexible enough for my use-case. lychee can...

  • run requests concurrently (fully async Rust)
  • handle links inside Markdown, HTML, and other documents
  • handle chunked encodings
  • handle gzip compression
  • fake user agents (required for some firewalls)
  • skip non-links like anchors or relative URLs
  • exclude some websites with regular expressions
  • handle a configurable number of redirects
  • disguise as a different user agent (like curl)
  • optionally ignore SSL certificate errors
  • run with a low memory/CPU footprint
  • check multiple files at once
  • support checking links from any website URL
  • limit scheme (e.g. only check HTTPS links with "https")

SOON:

  • automatically retry and backoff
  • check relative and absolute paths
  • support input files using wildcards
  • set timeout for HTTP requests in seconds. Disabled by default.
  • accept custom headers (see https://github.com/rust-lang/crates.io/issues/788)
  • use HEAD requests instead of GET to avoid network I/O
  • show the progress
  • show final status

How?

cargo install lychee

Set an environment variable with your token like so GITHUB_TOKEN=xxxx.

Run it inside a repository with a README.md or specify a different Markdown file with

lychee <yourfile>

Thanks

...to my Github sponsors and Patreon sponsors for supporting these projects. If you want to help out as well, go here.