Hopiu/lychee
1
0
Fork 0
mirror of https://github.com/Hopiu/lychee.git synced 2026-04-16 11:20:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
⚡ Fast, async, stream-based link checker written in Rust. Finds broken URLs and mail addresses inside Markdown, HTML, reStructuredText, websites and more!
100 commits 1 branch 0 tags 5.1 MiB
Rust 98.6%
Dockerfile 0.7%
Nix 0.3%
Makefile 0.3%
Shell 0.1%
Find a file
Matthias 114a36866f
Formatting
2020-10-20 12:18:28 +02:00
.github/workflows Use more descriptive name for cargo test action 2020-10-16 14:47:54 +02:00
assets Add comparison table and recording of demo run (#17) 2020-10-20 10:40:45 +02:00
fixtures Add --exclude-all-private flag and cli integration test 2020-10-17 10:01:06 +02:00
src Un-ignore test_timeout, upgrade wiremock to 0.3 2020-10-18 11:44:00 +02:00
tests Add comment 2020-10-17 10:04:24 +02:00
.gitignore Extract from hello-rust/show repository 2020-08-05 00:32:37 +02:00
Cargo.lock Un-ignore test_timeout, upgrade wiremock to 0.3 2020-10-18 11:44:00 +02:00
Cargo.toml Un-ignore test_timeout, upgrade wiremock to 0.3 2020-10-18 11:44:00 +02:00
README.md Formatting 2020-10-20 12:18:28 +02:00

README.md

lychee

Rust

...because who says I can't write yet another link checker?

What?

This thing was created from Hello Rust Episode 10. It's a link checker that treats Github links specially by using a GITHUB_TOKEN to avoid getting blocked by the rate limiter.

Lychee demo

Why?

The existing link checkers were not flexible enough for my use-case. lychee runs all requests fully asynchronously and has a low memory/CPU footprint.

Features

lychee awesome_bot muffet broken-link-checker linkinator
Language Rust Ruby Go JS TypeScript
Static binary ☑ ✖️ ☑ ✖️ ✖️
Async/Parallel ☑ ☑ ☑ ☑ ☑
Markdown support ☑ ☑ ✖️ ✖️ ✖️
HTML support ☑ ✖️ ✖️ ☑ ☑
Plaintext support ☑ ✖️ ✖️ ✖️ ✖️
Website support ☑ ✖️ ☑ ☑ ☑
Chunked encodings ☑ ? ? ? ?
GZIP compression ☑ ? ? ☑ ?
Basic Auth ✖️ ✖️ ✖️ ☑ ✖️
Custom user agent ☑ ✖️ ✖️ ☑ ✖️
Relative URLs ✖️ ☑ ✖️ ☑ ☑
Skip relative URLs ☑ ✖️ ✖️ ? ✖️
Include patterns ✖️ ☑ ✖️ ☑ ✖️
Exclude patterns ☑ ✖️ ☑ ☑ ☑
Handle redirects ☑ ☑ ☑ ☑ ☑
Ignore SSL ☑ ☑ ☑ ✖️ ✖️
File globbing ☑ ☑ ✖️ ✖️ ☑
Limit scheme (e.g. only HTTPS) ☑ ✖️ ✖️ ☑ ✖️
[Custom headers] ☑ ✖️ ☑ ✖️ ✖️
Summary ☑ ☑ ☑ ? ☑
HEAD requests ☑ ☑ ✖️ ☑ ☑
Colored output ☑ ? ☑ ? ☑
[Filter on status code] ☑ ☑ ✖️ ✖️ ✖️
Custom request timeout ☑ ☑ ☑ ✖️ ☑
E-mail links ☑ ✖️ ✖️ ✖️ ✖️
Progress bar ☑ ☑ ✖️ ✖️ ✖️
Retry and backoff ☑ ✖️ ✖️ ✖️ ☑
Exclude private domains ☑ ✖️ ✖️ ✖️ ✖️
[Usable as a library] ✖️ ☑ ✖️ ☑ ☑
Silent mode ☑ ✖️ ✖️ ✖️ ☑

Planned features:

  • lychee.toml
  • report output in HTML, SQL, CSV, XML, JSON, YAML... format
  • report extended statistics: request latency
  • recursion
  • use colored output (https://crates.io/crates/colored)
  • skip duplicate urls

Users

How?

cargo install lychee

Set an environment variable with your token like so GITHUB_TOKEN=xxxx.

Run it inside a repository with a README.md or specify a file with

lychee <yourfile>

Comparison

Collecting other link checkers here to crush them in comparison. :P

Thanks

...to my Github sponsors and Patreon sponsors for supporting these projects. If you want to help out as well, go here.

[custom headers]: https://github.com/rust-lang/crates.io/issues/788) [filter on status code]: https://github.com/tcort/markdown-link-check/issues/94 [exclude private domains]: a5102b0bf9/url_checker.go [usable as library]: https://github.com/raviqqe/liche/issues/13