Hopiu/lychee
1
0
Fork 0
mirror of https://github.com/Hopiu/lychee.git synced 2026-04-03 21:20:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
631 commits 1 branch 0 tags 5.1 MiB
Commit graph

631 commits

This branch
This branch
All branches
Author SHA1 Message Date
dependabot[bot]
d4b6e8ddbf
Bump serde_json from 1.0.78 to 1.0.79 (#511) 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.78...v1.0.79)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-02-18 10:58:21 +01:00
dependabot[bot]
2114406235
Bump futures from 0.3.19 to 0.3.21 (#493) 
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.19 to 0.3.21.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.19...0.3.21)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-02-18 10:58:01 +01:00
Matthias
d821168f24
Only login to Dockerhub on image push (#515) 2022-02-18 10:49:47 +01:00
Matthias
812663d832
Prevent flaky tests (#514) 
Move from example.org to example.com, which seems to be more permissive for testing
 2022-02-18 10:29:49 +01:00
Lucius Hu
6d56c6b55c
Replace plain String with SecretString for GitHub token (#509) 
This commit changed the type of `lychee-lib::ClientBuilder::github_token` from
`String` to `secrecy::SecretString` to fortify the secret management within our
program.

Note that this won't affect TOML configuration of `lychee-bin` because
`serde::Deserialize` is still implemented for `SecretString`.
 2022-02-13 13:53:46 +01:00
Matthias
47df7780fe
Use captured identifiers in format strings (#507) 
Makes for arguably cleaner-looking code.
The downside is that the MSRV is 1.58
https://blog.rust-lang.org/2022/01/13/Rust-1.58.0.html

Given that nobody uses lychee as a library yet
and we have precompiled binaries, it's an acceptable
tradeoff.
My little research revealed that this is a much-liked
feature: https://twitter.com/matthiasendler/status/1483895557621960715
 2022-02-12 10:51:52 +01:00
Lucius Hu
53c41b03d8
replace hubcaps by octocrab (#502) 
This commit replaced `hubcaps` by `octocrab`, which has more downloads per month
and receives more frequent release updates.

The caveats are:

1. When instantiating the API client, `octocrab` doesn't offer you a way to
specify custom user-agent. But I would argue that, at least presently, this
doesn't seem to cause issues.
2. `octocrab` doesn't export as much details of its error types as `hubcaps`
does. So we will have fewer control on the display of the error message. But I
would also argue that this is not really important. Though we should do more
tests to make sure the error looks good enough.

* hide implementation details in error message

Co-authored-by: Lucius Hu <lebensterben@users.noreply.github.com>
 2022-02-11 23:43:47 +01:00
Matthias
0a38c26f0d
Shrink down repo Docker image 2022-02-11 16:16:24 +01:00
Matthias
97e89d7f7b
Multiarch for main Docker workflow (#506) 2022-02-11 16:12:07 +01:00
Matthias
71f3877933
Start pushing on non-pull request 2022-02-11 09:45:40 +01:00
Matthias
12f4134f77
Add Docker semver tagging for new multiarch image (#503) 2022-02-11 09:40:08 +01:00
Lucius Hu
476a048350
lychee-lib::client reworked (#500) 
This commit mainly added or improved documentation for `lychee-lib::client`
module.

But it also contains a few API changes:

- `ClientBuilder::client()` now consumes itself instead of taking a reference.
  This helps to avoid a few unnecessary clones.
- `ClientBuilder::build_filter()` was a private function and is inlined to avoid
  unnecessary clones.
- Added a new crate-scoped function `Uri::set_scheme()`.

* added notes on deprecated site-local network

Co-authored-by: Lucius Hu <lebensterben@users.noreply.github.com>
 2022-02-10 00:04:48 +01:00
Lucius Hu
5921fd248a
Update license files (#497) 
- The date in MIT license files have been updated to 2022
- Each of the benchmark and example crates are theoretically
  a separate package in Cargo's sense. So license files are
  added for them as well.

Co-authored-by: Lucius Hu <lebensterben@users.noreply.github.com>
 2022-02-08 10:59:54 +01:00
Lucius Hu
f9d83018bb
update rust-toolchain to .toml format (#496) 
rust-toolchain is phasing out in favour of rust-toolchain.toml

Co-authored-by: Lucius Hu <lebensterben@users.noreply.github.com>
 2022-02-08 10:56:19 +01:00
Matthias
9d738fb3f5
Fix default config (#491) 
The default configuration was broken since the
introduction of caching and specifically `max_cache_age`.
This fixes deserialization and config merging for
the case where this key is missing from the config.
 2022-02-07 23:17:50 +01:00
Markus Unterwaditzer
d8305f7f53
fix constant updating of progressbar (#488) 
* fix constant updating of progressbar

In other issues I've already lamented how slow lychee is when used
without `-n`. This fixes an issue where without `-n`, lychee would take
1 minute instead of 4 seconds to check sentry-docs.

* fix values
 2022-02-07 23:15:26 +01:00
dependabot[bot]
71ed0d9d90
Bump dashmap from 5.0.0 to 5.1.0 (#492) 
Bumps [dashmap](https://github.com/xacrimon/dashmap) from 5.0.0 to 5.1.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases)
- [Commits](https://github.com/xacrimon/dashmap/compare/v5.0.0...v5.1.0)

---
updated-dependencies:
- dependency-name: dashmap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-02-07 23:03:26 +01:00
Markus Unterwaditzer
68d09f7e5b
Add html5gum as alternative link extractor (#480) 
html5gum is a HTML parser that offers lower-level control over which tokens actually get created and are tracked. As such, the extractor doesn't allocate anything tokens it doesn't care about. On some benchmarks it provides a substantial performance boost. The old parser, html5ever is still available by setting the `LYCHEE_USE_HTML5EVER=1` env var.
 2022-02-07 22:54:47 +01:00
Lucius Hu
6bf8c1fe39
lychee-bin: replace lazy_static by const_format (#495) 
This commit replaced the use of `lazy_static` by
`const_format` in `lychee-bin`.

Currently `lazy_static` is used to generate static
String at runtime. With `const_format` we can instead
make constant String at compile time.

Co-authored-by: Lucius Hu <lebensterben@users.noreply.github.com>
 2022-02-07 22:45:17 +01:00
dependabot[bot]
eacc0bdd02
Bump tracing-subscriber from 0.3.7 to 0.3.8 (#494) 
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.3.7 to 0.3.8.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.7...tracing-subscriber-0.3.8)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-02-07 13:29:42 +01:00
Matthias
89106d11aa
Update rust.yml 2022-02-06 15:51:18 +01:00
Matthias
0b9e9ecf57
Uppercase Release action 2022-02-06 15:50:24 +01:00
Matthias
58b17b2252
Update link to troubleshooting guide 2022-02-04 12:05:59 +01:00
Matthias
bf00b2b7f1
Move troubleshooting guide to docs 2022-02-04 12:05:28 +01:00
dependabot[bot]
1f3abce671
Bump pretty_assertions from 1.0.0 to 1.1.0 (#487) 
Bumps [pretty_assertions](https://github.com/colin-kiegel/rust-pretty-assertions) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/colin-kiegel/rust-pretty-assertions/releases)
- [Changelog](https://github.com/colin-kiegel/rust-pretty-assertions/blob/main/CHANGELOG.md)
- [Commits](https://github.com/colin-kiegel/rust-pretty-assertions/compare/v1.0.0...v1.1.0)

---
updated-dependencies:
- dependency-name: pretty_assertions
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-02-01 13:42:58 +01:00
dependabot[bot]
129c314f51
Bump headers from 0.3.5 to 0.3.6 (#485) 
Bumps [headers](https://github.com/hyperium/headers) from 0.3.5 to 0.3.6.
- [Release notes](https://github.com/hyperium/headers/releases)
- [Commits](https://github.com/hyperium/headers/compare/headers-v0.3.5...headers-v0.3.6)

---
updated-dependencies:
- dependency-name: headers
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-31 19:11:42 +01:00
dependabot[bot]
a8d1359df4
Bump tokio from 1.15.0 to 1.16.1 (#482) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.15.0 to 1.16.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.15.0...tokio-1.16.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-28 13:23:02 +01:00
Matthias
6635863746
Add Alpine page for benchmark; refactor code (#481) 2022-01-27 23:42:06 +01:00
dependabot[bot]
8e31b234d3
Bump check-if-email-exists from 0.8.25 to 0.8.26 (#479) 
Bumps [check-if-email-exists](https://github.com/reacherhq/check-if-email-exists) from 0.8.25 to 0.8.26.
- [Release notes](https://github.com/reacherhq/check-if-email-exists/releases)
- [Changelog](https://github.com/reacherhq/check-if-email-exists/blob/master/CHANGELOG.md)
- [Commits](https://github.com/reacherhq/check-if-email-exists/compare/v0.8.25...v0.8.26)

---
updated-dependencies:
- dependency-name: check-if-email-exists
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-27 15:02:05 +01:00
Matthias
04f9b7c23b
Add additional parameters for Docker usage example 2022-01-27 12:43:34 +01:00
dependabot[bot]
54551d6184
Bump tracing-subscriber from 0.3.6 to 0.3.7 (#477) 
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.6...tracing-subscriber-0.3.7)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-26 16:35:19 +01:00
dependabot[bot]
d963c10fed
Bump serde_json from 1.0.75 to 1.0.78 (#475) 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.75 to 1.0.78.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.75...v1.0.78)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-25 13:49:18 +01:00
dependabot[bot]
761766cf9b
Bump anyhow from 1.0.52 to 1.0.53 (#476) 
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.52 to 1.0.53.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.52...1.0.53)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-24 19:30:36 +01:00
Matthias
97b06230fc
Add missing Github exclusions; sort entries (#473) 2022-01-21 23:54:59 +01:00
dependabot[bot]
84eff209ff
Bump cached from 0.29.0 to 0.30.0 (#472) 
Bumps [cached](https://github.com/jaemk/cached) from 0.29.0 to 0.30.0.
- [Release notes](https://github.com/jaemk/cached/releases)
- [Changelog](https://github.com/jaemk/cached/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jaemk/cached/commits)

---
updated-dependencies:
- dependency-name: cached
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-21 21:50:20 +01:00
dependabot[bot]
9082865e24
Bump pulldown-cmark from 0.9.0 to 0.9.1 (#468) 
Bumps [pulldown-cmark](https://github.com/raphlinus/pulldown-cmark) from 0.9.0 to 0.9.1.
- [Release notes](https://github.com/raphlinus/pulldown-cmark/releases)
- [Commits](https://github.com/raphlinus/pulldown-cmark/compare/v0.9.0...v0.9.1)

---
updated-dependencies:
- dependency-name: pulldown-cmark
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-18 14:44:45 +01:00
dependabot[bot]
b9358aacf3
Bump structopt from 0.3.25 to 0.3.26 (#469) 
Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.25...v0.3.26)

---
updated-dependencies:
- dependency-name: structopt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-18 14:44:35 +01:00
dependabot[bot]
dcbac47b73
Bump tracing-subscriber from 0.3.5 to 0.3.6 (#467) 
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.3.5 to 0.3.6.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.5...tracing-subscriber-0.3.6)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-17 13:22:23 +01:00
dependabot[bot]
ea00edc339
Bump serde_json from 1.0.74 to 1.0.75 (#466) 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.74 to 1.0.75.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.74...v1.0.75)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-17 13:22:11 +01:00
Matthias
5802ae912c
Fix bugs in extractor; reduce allocs (#464) 
When URLs couldn't be extracted from a tag,
we ran a plaintext search, but never added the
newly found urls to the vec of extracted urls.

Also tried to make the code a little more idiomatic
 2022-01-16 02:13:38 +01:00
Matthias
6e757fa20e
Add more information about mail errors (#463) 2022-01-14 22:22:53 +01:00
Matthias
4630216c30 Add description for max-cache-age flag 2022-01-14 16:55:56 +01:00
Matthias
994aadf6a1
Simplify error messages (#462) 
Using pattern matching to make the hubcaps and reqwest error messages a little shorter and (subjectively) more readable.
 2022-01-14 15:26:13 +01:00
Matthias
ac490f9c53
Add caching functionality (v2) (#443) 
A while ago, caching was removed due to some issues (see #349).
This is a new implementation with the following improvements:

 * Architecture: The new implementation is decoupled from the collector, which was a major issue in the last version.    Now the collector has a single responsibility: collecting links. This also avoids race-conditions when running multiple collect_links instances, which probably was an issue before.
* Performance: Uses DashMap under the hood, which was noticeably faster than Mutex<HashMap> in my tests.
* Simplicity: The cache format is a CSV file with two columns: URI and status. I decided to create a new struct called CacheStatus for serialization, because trying to serialize the error kinds in Status turned out to be a bit of a nightmare and at this point I don't think it's worth the pain (and probably isn't idiomatic either).

This is an optional feature. Caching only gets used if the `--cache` flag is set.
 2022-01-14 15:25:51 +01:00
dependabot[bot]
69805fa73e Bump assert_cmd from 2.0.3 to 2.0.4 
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd) from 2.0.3 to 2.0.4.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases)
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.3...v2.0.4)

---
updated-dependencies:
- dependency-name: assert_cmd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-14 15:16:26 +01:00
Matthias
76c96b0cfd Ignore .lycheecache 2022-01-14 12:15:08 +01:00
dependabot[bot]
80fb20cca5 Bump cached from 0.28.0 to 0.29.0 
Bumps [cached](https://github.com/jaemk/cached) from 0.28.0 to 0.29.0.
- [Release notes](https://github.com/jaemk/cached/releases)
- [Changelog](https://github.com/jaemk/cached/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jaemk/cached/commits)

---
updated-dependencies:
- dependency-name: cached
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-13 13:38:48 +01:00
Matthias
92b939eb2f
Update README.md 2022-01-13 13:11:55 +01:00
Matthias
0e4973eda6 Avoid flagging lychee as HTML 2022-01-13 01:07:33 +01:00
dependabot[bot]
93f486ec3a Bump assert_cmd from 2.0.2 to 2.0.3 
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd) from 2.0.2 to 2.0.3.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases)
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.2...v2.0.3)

---
updated-dependencies:
- dependency-name: assert_cmd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-12 22:56:12 +01:00