Commit graph

1117 commits

Author SHA1 Message Date
dependabot[bot]
c4d05bc5cb
Merge pull request #996 from lycheeverse/dependabot/cargo/toml-0.7.3 2023-03-14 13:03:24 +00:00
dependabot[bot]
6c4026b286
Bump toml from 0.7.2 to 0.7.3
Bumps [toml](https://github.com/toml-rs/toml) from 0.7.2 to 0.7.3.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.2...toml-v0.7.3)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-14 12:59:18 +00:00
dependabot[bot]
b83851e2ed
Bump criterion from 27642b4 to 2f53607 (#994)
Bumps [criterion](https://github.com/bheisler/criterion.rs) from `27642b4` to `2f53607`.
- [Release notes](https://github.com/bheisler/criterion.rs/releases)
- [Commits](27642b4768...2f53607378)

---
updated-dependencies:
- dependency-name: criterion
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-13 14:54:46 +01:00
dependabot[bot]
594eb8d797
Merge pull request #991 from lycheeverse/dependabot/cargo/futures-0.3.27 2023-03-13 13:26:07 +00:00
dependabot[bot]
8be84547c5
Bump futures from 0.3.26 to 0.3.27
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.26 to 0.3.27.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.26...0.3.27)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-13 13:20:47 +00:00
dependabot[bot]
ccc19035d1
Merge pull request #993 from lycheeverse/dependabot/cargo/serde-1.0.155 2023-03-13 13:04:40 +00:00
dependabot[bot]
15c711bf39
Merge pull request #992 from lycheeverse/dependabot/cargo/hyper-0.14.25 2023-03-13 13:03:44 +00:00
dependabot[bot]
6c126bb15a
Bump serde from 1.0.154 to 1.0.155
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.154 to 1.0.155.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.154...v1.0.155)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-13 12:59:15 +00:00
dependabot[bot]
ff087cbed3
Bump hyper from 0.14.24 to 0.14.25
Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.24 to 0.14.25.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/v0.14.25/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.24...v0.14.25)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-13 12:59:04 +00:00
Matthias Endler
55797071b0
Fix nested URL extraction in verbatim elements (#988)
Skipping URLs in verbatim elements didn't take nested
elements into consideration, which were not verbatim.

For instance, the following HTML snippet would yield
`https://example.com` in non-verbatim mode, even if
it is nested inside a verbatim `<pre>` element:

```html
<pre><a href="https://example.com">link</a></pre>
```

This commit fixes the behavior for both `html5gum` and
`html5ever`.

Note that nested verbatim elements of the same kind
still are not handled correctly.

For instance,  the following HTML snippet would still yield
`https://example.com`:

```html
<pre>
  <pre></pre>
  <a href="https://example.com">link</a>
</pre>
```

The reason is that we currently only keep track of a single
verbatim element and not a stack of elements, which we
would need to unwind and resolve the situation.

Fixes https://github.com/lycheeverse/lychee/issues/986.
2023-03-11 15:18:25 +01:00
Matthias Endler
2255ad9286
Better retry handling (#981)
Previously, lychee would blindly retry all requests,
no matter if the request error was transient or fatal.

Taking a lesson from https://github.com/TrueLayer/reqwest-middleware,
we can be more granular about the error behavior.
This PR adds their retry logic to lychee, reducing the number of
unnecessary requests significantly.

I also made some ergonomic changes to the client, which should not
affect its behavior.
2023-03-10 22:36:45 +01:00
Matthias Endler
30e2a2b62b
Fix --max-redirects (#987)
Having more than the max number of redirects
caused lychee to abort the requests, but did not
lead to an error.

Related: https://github.com/lycheeverse/lychee-action/issues/164
2023-03-10 15:15:37 +01:00
dependabot[bot]
ac189306ef Bump typed-builder from 0.13.0 to 0.14.0
Bumps [typed-builder](https://github.com/idanarye/rust-typed-builder) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/idanarye/rust-typed-builder/releases)
- [Changelog](https://github.com/idanarye/rust-typed-builder/blob/master/CHANGELOG.md)
- [Commits](https://github.com/idanarye/rust-typed-builder/commits)

---
updated-dependencies:
- dependency-name: typed-builder
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-09 19:47:18 +01:00
dependabot[bot]
add15af2aa
Merge pull request #982 from lycheeverse/dependabot/cargo/serde-1.0.154 2023-03-09 13:05:12 +00:00
dependabot[bot]
7ffb9eac10
Bump serde from 1.0.153 to 1.0.154
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.153 to 1.0.154.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.153...v1.0.154)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-09 12:58:27 +00:00
dependabot[bot]
12f3dcbc6d Bump typed-builder from 0.12.0 to 0.13.0
Bumps [typed-builder](https://github.com/idanarye/rust-typed-builder) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/idanarye/rust-typed-builder/releases)
- [Changelog](https://github.com/idanarye/rust-typed-builder/blob/master/CHANGELOG.md)
- [Commits](https://github.com/idanarye/rust-typed-builder/commits)

---
updated-dependencies:
- dependency-name: typed-builder
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-09 01:31:49 +01:00
dependabot[bot]
688d864671
Merge pull request #980 from lycheeverse/dependabot/cargo/serde-1.0.153 2023-03-08 13:05:07 +00:00
dependabot[bot]
a511793c5c
Bump serde from 1.0.152 to 1.0.153
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.152 to 1.0.153.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.152...v1.0.153)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-08 12:58:21 +00:00
Matthias Endler
fea200b512
Update FUNDING.yml 2023-03-08 00:49:50 +01:00
dependabot[bot]
684668f780
Merge pull request #979 from lycheeverse/dependabot/cargo/csv-1.2.1 2023-03-07 13:04:58 +00:00
dependabot[bot]
ce7d99d88c
Bump csv from 1.2.0 to 1.2.1
Bumps [csv](https://github.com/BurntSushi/rust-csv) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/BurntSushi/rust-csv/releases)
- [Commits](https://github.com/BurntSushi/rust-csv/compare/1.2.0...1.2.1)

---
updated-dependencies:
- dependency-name: csv
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-07 12:58:08 +00:00
dependabot[bot]
46ed6f45ad
Merge pull request #976 from lycheeverse/dependabot/cargo/thiserror-1.0.39 2023-03-06 13:05:53 +00:00
dependabot[bot]
e8b4df4afc
Merge pull request #977 from lycheeverse/dependabot/cargo/serde_json-1.0.94 2023-03-06 13:05:30 +00:00
dependabot[bot]
244f013f22
Bump serde_json from 1.0.93 to 1.0.94
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.93 to 1.0.94.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.93...v1.0.94)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-06 12:59:31 +00:00
dependabot[bot]
d17624e0fa
Bump thiserror from 1.0.38 to 1.0.39
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.38 to 1.0.39.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.38...1.0.39)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-06 12:59:22 +00:00
Matthias
c9edb7f809 Split up quirks and skip twitter check
It's flaky on Github
2023-03-03 12:13:09 +01:00
Matthias
51628213d6 Explicit quirks output for test 2023-03-03 12:13:09 +01:00
Matthias
9eb3149a69 Custom config handling to spot errors when passing invalid config and ignoring errors loading missing default conf 2023-03-03 12:13:09 +01:00
Matthias
08d71e6196 Add new test config, which covers all tests 2023-03-03 12:13:09 +01:00
Matthias
49d1d3accb update help message 2023-03-03 12:13:09 +01:00
Matthias
6c133493e9 Revert "Don't ignore file-not-found errors when loading config"
This reverts commit 9ade4502a27cb3776c5fb39cdad7666ab854a373.
2023-03-03 12:13:09 +01:00
Matthias
08466ad59b Ignore config smoketest output report file 2023-03-03 12:13:09 +01:00
Matthias
19976cd9e7 Add test for missing and example config file and helper methods 2023-03-03 12:13:09 +01:00
Matthias
387766322d Don't ignore file-not-found errors when loading config
This is no longer necessary ever since 712bdfa8cb
2023-03-03 12:13:09 +01:00
Matthias
257ecd7bc0 disable all features flag (breaks example domain tests) 2023-03-03 12:13:09 +01:00
Matthias
c47f0b8c53 Use nextest for make test 2023-03-03 12:13:09 +01:00
Matthias
86f13609e6 Put lycheecache tests into separate subfolders to avoid race 2023-03-03 12:13:09 +01:00
Matthias
388bd20673 Fix tests after address is no longer a verbatim element 2023-03-03 12:13:09 +01:00
Matthias
17937537f8 Ignored URLs don't lead to failing exit code 2023-03-03 12:13:09 +01:00
Matthias
c549213bfe Fix test_skip_cache_unsupported 2023-03-03 12:13:09 +01:00
Matthias
f64c60aac0 Fix cache test 2023-03-03 12:13:09 +01:00
Matthias
09d0064e69 Split up test 2023-03-03 12:13:09 +01:00
Matthias
9b75da60a6 Ignore remap test
https://github.com/robinst/linkify/pull/58
2023-03-03 12:13:09 +01:00
Matthias
4306150e56 Excluded URLs are no longer cached
See https://github.com/lycheeverse/lychee/pull/692
2023-03-03 12:13:09 +01:00
Matthias
a7f7b989ef fix multiple_exclude_files test 2023-03-03 12:13:09 +01:00
Matthias
46d238ba93 fix url output 2023-03-03 12:13:09 +01:00
Matthias
e125d45a8e Multiple arguments get handled differently in clap in comparison to structopt
We should document that change
2023-03-03 12:13:09 +01:00
Matthias
59ddc1e27d Fix url input handling without scheme 2023-03-03 12:13:09 +01:00
Matthias
1f62590a02 Fix expected json output 2023-03-03 12:13:09 +01:00
Matthias
91c8bc3ef5 Temporarily move to criterion/master, which supports nextest 2023-03-03 12:13:09 +01:00