Hopiu/lychee
1
0
Fork 0
mirror of https://github.com/Hopiu/lychee.git synced 2026-04-12 09:20:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
1109 commits 1 branch 0 tags 5.1 MiB
Commit graph

408 commits

Author SHA1 Message Date
Matthias Endler
55797071b0
Fix nested URL extraction in verbatim elements (#988) 
Skipping URLs in verbatim elements didn't take nested
elements into consideration, which were not verbatim.

For instance, the following HTML snippet would yield
`https://example.com` in non-verbatim mode, even if
it is nested inside a verbatim `<pre>` element:

```html
<pre><a href="https://example.com">link</a></pre>
```

This commit fixes the behavior for both `html5gum` and
`html5ever`.

Note that nested verbatim elements of the same kind
still are not handled correctly.

For instance,  the following HTML snippet would still yield
`https://example.com`:

```html
<pre>
  <pre></pre>
  <a href="https://example.com">link</a>
</pre>
```

The reason is that we currently only keep track of a single
verbatim element and not a stack of elements, which we
would need to unwind and resolve the situation.

Fixes https://github.com/lycheeverse/lychee/issues/986.
 2023-03-11 15:18:25 +01:00
Matthias Endler
2255ad9286
Better retry handling (#981) 
Previously, lychee would blindly retry all requests,
no matter if the request error was transient or fatal.

Taking a lesson from https://github.com/TrueLayer/reqwest-middleware,
we can be more granular about the error behavior.
This PR adds their retry logic to lychee, reducing the number of
unnecessary requests significantly.

I also made some ergonomic changes to the client, which should not
affect its behavior.
 2023-03-10 22:36:45 +01:00
Matthias Endler
30e2a2b62b
Fix --max-redirects (#987) 
Having more than the max number of redirects
caused lychee to abort the requests, but did not
lead to an error.

Related: https://github.com/lycheeverse/lychee-action/issues/164
 2023-03-10 15:15:37 +01:00
dependabot[bot]
7ffb9eac10
Bump serde from 1.0.153 to 1.0.154 
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.153 to 1.0.154.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.153...v1.0.154)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-09 12:58:27 +00:00
dependabot[bot]
a511793c5c
Bump serde from 1.0.152 to 1.0.153 
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.152 to 1.0.153.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.152...v1.0.153)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-08 12:58:21 +00:00
dependabot[bot]
ce7d99d88c
Bump csv from 1.2.0 to 1.2.1 
Bumps [csv](https://github.com/BurntSushi/rust-csv) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/BurntSushi/rust-csv/releases)
- [Commits](https://github.com/BurntSushi/rust-csv/compare/1.2.0...1.2.1)

---
updated-dependencies:
- dependency-name: csv
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-07 12:58:08 +00:00
dependabot[bot]
244f013f22
Bump serde_json from 1.0.93 to 1.0.94 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.93 to 1.0.94.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.93...v1.0.94)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-06 12:59:31 +00:00
Matthias
c9edb7f809 Split up quirks and skip twitter check 
It's flaky on Github
 2023-03-03 12:13:09 +01:00
Matthias
51628213d6 Explicit quirks output for test 2023-03-03 12:13:09 +01:00
Matthias
9eb3149a69 Custom config handling to spot errors when passing invalid config and ignoring errors loading missing default conf 2023-03-03 12:13:09 +01:00
Matthias
6c133493e9 Revert "Don't ignore file-not-found errors when loading config" 
This reverts commit 9ade4502a27cb3776c5fb39cdad7666ab854a373.
 2023-03-03 12:13:09 +01:00
Matthias
19976cd9e7 Add test for missing and example config file and helper methods 2023-03-03 12:13:09 +01:00
Matthias
387766322d Don't ignore file-not-found errors when loading config 
This is no longer necessary ever since 712bdfa8cb
 2023-03-03 12:13:09 +01:00
Matthias
86f13609e6 Put lycheecache tests into separate subfolders to avoid race 2023-03-03 12:13:09 +01:00
Matthias
17937537f8 Ignored URLs don't lead to failing exit code 2023-03-03 12:13:09 +01:00
Matthias
c549213bfe Fix test_skip_cache_unsupported 2023-03-03 12:13:09 +01:00
Matthias
f64c60aac0 Fix cache test 2023-03-03 12:13:09 +01:00
Matthias
09d0064e69 Split up test 2023-03-03 12:13:09 +01:00
Matthias
9b75da60a6 Ignore remap test 
https://github.com/robinst/linkify/pull/58
 2023-03-03 12:13:09 +01:00
Matthias
4306150e56 Excluded URLs are no longer cached 
See https://github.com/lycheeverse/lychee/pull/692
 2023-03-03 12:13:09 +01:00
Matthias
a7f7b989ef fix multiple_exclude_files test 2023-03-03 12:13:09 +01:00
Matthias
46d238ba93 fix url output 2023-03-03 12:13:09 +01:00
Matthias
e125d45a8e Multiple arguments get handled differently in clap in comparison to structopt 
We should document that change
 2023-03-03 12:13:09 +01:00
Matthias
1f62590a02 Fix expected json output 2023-03-03 12:13:09 +01:00
Matthias
ac13e5d16e Fix CLI tests 2023-03-03 12:13:09 +01:00
dependabot[bot]
74ce563c5d
Bump tokio from 1.25.0 to 1.26.0 (#974) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.25.0...tokio-1.26.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-02 17:06:33 +01:00
dependabot[bot]
85ae5cf1de
Bump clap from 4.1.7 to 4.1.8 
Bumps [clap](https://github.com/clap-rs/clap) from 4.1.7 to 4.1.8.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.1.7...v4.1.8)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-01 13:00:44 +00:00
dependabot[bot]
4f130b4979
Bump clap from 4.1.6 to 4.1.7 
Bumps [clap](https://github.com/clap-rs/clap) from 4.1.6 to 4.1.7.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.1.6...v4.1.7)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-28 12:58:39 +00:00
dependabot[bot]
bed7dc7310
Bump tempfile from 3.3.0 to 3.4.0 (#967) 
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/commits)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-27 15:58:26 +01:00
Matthias
7e0b9e2c68 Update verbosity docs 
Thanks to @MichaIng for mentioning the issue and providing a fix.
 2023-02-25 15:44:43 +01:00
Matthias Endler
7874195bbb
Customize verbosity (#956) 2023-02-24 23:53:09 +01:00
Matthias Endler
b653a0a1ec
Fix cached 200 status code handling (#958) 
* Fix cached 200 status code handling

Assert that code 200 never needs to be explicitly accepted for cached response
to match the behavior of uncached checks

* Bump version to v0.11.1
 2023-02-23 00:25:53 +01:00
Matthias Endler
123742a8b4
Bump to v0.11.0 (#957) 2023-02-22 21:35:38 +01:00
dependabot[bot]
fb1e820a60
Bump tokio-stream from 0.1.11 to 0.1.12 
Bumps [tokio-stream](https://github.com/tokio-rs/tokio) from 0.1.11 to 0.1.12.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.11...tokio-stream-0.1.12)

---
updated-dependencies:
- dependency-name: tokio-stream
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-20 12:26:17 +00:00
dependabot[bot]
98247cd864
Bump http from 0.2.8 to 0.2.9 
Bumps [http](https://github.com/hyperium/http) from 0.2.8 to 0.2.9.
- [Release notes](https://github.com/hyperium/http/releases)
- [Changelog](https://github.com/hyperium/http/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/http/compare/v0.2.8...v0.2.9)

---
updated-dependencies:
- dependency-name: http
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-20 12:17:04 +00:00
dependabot[bot]
626b2a5b2d
Bump clap from 4.1.4 to 4.1.6 
Bumps [clap](https://github.com/clap-rs/clap) from 4.1.4 to 4.1.6.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.1.4...v4.1.6)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-16 12:58:04 +00:00
dependabot[bot]
e5ad976c68
Bump once_cell from 1.17.0 to 1.17.1 
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.17.0 to 1.17.1.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.17.0...v1.17.1)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-15 12:58:23 +00:00
dependabot[bot]
340e093f0b
Bump csv from 1.1.6 to 1.2.0 (#950) 
Bumps [csv](https://github.com/BurntSushi/rust-csv) from 1.1.6 to 1.2.0.
- [Release notes](https://github.com/BurntSushi/rust-csv/releases)
- [Commits](https://github.com/BurntSushi/rust-csv/compare/1.1.6...1.2.0)

---
updated-dependencies:
- dependency-name: csv
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-14 15:08:48 +01:00
dependabot[bot]
a3c9e94beb
Bump serde_json from 1.0.92 to 1.0.93 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.92 to 1.0.93.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.92...v1.0.93)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-09 12:02:13 +00:00
dependabot[bot]
cb9622c411
Bump toml from 0.7.1 to 0.7.2 
Bumps [toml](https://github.com/toml-rs/toml) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.1...toml-v0.7.2)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-08 12:01:51 +00:00
dependabot[bot]
02b566a0cd
Merge pull request #943 from lycheeverse/dependabot/cargo/serde_json-1.0.92 2023-02-06 12:08:12 +00:00
dependabot[bot]
c3dad91356
Bump serde_json from 1.0.91 to 1.0.92 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.91 to 1.0.92.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.91...v1.0.92)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-06 12:03:49 +00:00
dependabot[bot]
096f5b61cd
Bump anyhow from 1.0.68 to 1.0.69 
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.68 to 1.0.69.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.68...1.0.69)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-06 12:03:30 +00:00
dependabot[bot]
6b79242c84
Bump uuid from 1.2.2 to 1.3.0 (#939) 
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.2.2 to 1.3.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.2.2...1.3.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-01 21:21:16 +01:00
dependabot[bot]
05bb15f3a6
Merge pull request #937 from lycheeverse/dependabot/cargo/futures-0.3.26 2023-01-31 12:07:27 +00:00
dependabot[bot]
709aac2a01
Bump toml from 0.7.0 to 0.7.1 
Bumps [toml](https://github.com/toml-rs/toml) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.0...toml-v0.7.1)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-01-31 12:02:40 +00:00
dependabot[bot]
128e19d567
Bump futures from 0.3.25 to 0.3.26 
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-01-31 12:02:19 +00:00
dependabot[bot]
d8e4940dbe
Bump toml from 0.5.11 to 0.7.0 (#933) 
* Bump toml from 0.5.11 to 0.7.0

Bumps [toml](https://github.com/toml-rs/toml) from 0.5.11 to 0.7.0.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.5.11...toml-v0.7.0)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Introduce new let...else syntax

* Update config file loading for latest toml crate version

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthias <matthias-endler@gmx.net>
Co-authored-by: Matthias Endler <matthias@endler.dev>
 2023-01-30 15:12:34 +01:00
dependabot[bot]
7de4e3f0f0
Bump tokio from 1.24.2 to 1.25.0 (#935) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-30 14:30:20 +01:00
dependabot[bot]
c08ce1dc1a
Bump clap from 4.1.3 to 4.1.4 
Bumps [clap](https://github.com/clap-rs/clap) from 4.1.3 to 4.1.4.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.1.3...v4.1.4)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-01-25 12:02:59 +00:00