Hopiu/lychee
1
0
Fork 0
mirror of https://github.com/Hopiu/lychee.git synced 2026-04-01 20:20:32 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
483 commits 1 branch 0 tags 5.1 MiB
Commit graph

483 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matthias
175342baf4 Merge branch 'master' of github.com:lycheeverse/lychee 2021-10-09 21:17:41 +02:00
Matthias
bdcd6f87bf Make error message for broken file links more understandable 2021-10-09 21:17:37 +02:00
Matthias
56726f41fc
Add back connection pool (#355) 2021-10-08 13:08:44 +02:00
MichaIng
961f12e58e
Remove cache from collector and remove custom reqwest client pool 
* Reqwest comes with its own request pool, so there's no need in adding
another layer of indirection. This also gets rid of a lot of allocs.
* Remove cache from collector
* Improve error handling and documentation
* Add back test for request caching in single file

Signed-off-by: MichaIng <micha@dietpi.com>
Co-authored-by: Matthias <matthias-endler@gmx.net>
 2021-10-07 18:07:18 +02:00
Matthias
a7f809612d
Refactor extractor (#354) 
This avoids sending URLs back and forth between the different parsers.
Also, it should allow for future optimizations to reduce allocs.
 2021-10-07 12:51:02 +02:00
dependabot[bot]
59a7e3af03
Bump tracing-subscriber from 0.2.24 to 0.2.25 (#353) 
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.24 to 0.2.25.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.24...tracing-subscriber-0.2.25)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-06 14:56:30 +02:00
dependabot[bot]
ee1f26c44a
Bump check-if-email-exists from 0.8.24 to 0.8.25 (#352) 
Bumps [check-if-email-exists](https://github.com/reacherhq/check-if-email-exists) from 0.8.24 to 0.8.25.
- [Release notes](https://github.com/reacherhq/check-if-email-exists/releases)
- [Changelog](https://github.com/reacherhq/check-if-email-exists/blob/master/CHANGELOG.md)
- [Commits](https://github.com/reacherhq/check-if-email-exists/compare/v0.8.24...v0.8.25)

---
updated-dependencies:
- dependency-name: check-if-email-exists
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-06 14:56:20 +02:00
MichaIng
b648b5e914
Imply "localhost" when loopback IPs are excluded (#351) 
as "localhost" is usually mapped via "hosts" file to a loopback IP address.

Resolves: https://github.com/lycheeverse/lychee/issues/319

Signed-off-by: MichaIng <micha@dietpi.com>
 2021-10-06 11:33:23 +02:00
Matthias
251332efe2
Cache absolute_path to decrease allocations (#346) 
* Cache `absolute_path` to decrease allocations

While profiling local file handling, I noticed that resolving paths was taking a
significant amount of time. It also caused quite a few allocations.
By caching the path and using a constant value for the current
directory, we can reduce the number of allocs by quite a lot.
For example, when testing on the sentry documentation, we do 50,4%
less allocations in total now. That's just a single test-case of course,
but it's probably also helping in many other cases as well.

* Defer to_string for attr.value to reduce allocs
* Use Tendrils instead of Strings for parsing (another ~1.5% less allocs)
* Move option parsing code into separate module
* Handle base dir more correctly
* Temporarily disable dry run
 2021-10-05 01:37:43 +02:00
Daniel Doubrovkine (dB.)
df12f31f83
Fix: typo. (#350) 2021-10-02 23:53:14 +02:00
dependabot[bot]
aadce95e35
Bump pretty_assertions from 0.7.2 to 1.0.0 (#347) 
Bumps [pretty_assertions](https://github.com/colin-kiegel/rust-pretty-assertions) from 0.7.2 to 1.0.0.
- [Release notes](https://github.com/colin-kiegel/rust-pretty-assertions/releases)
- [Changelog](https://github.com/colin-kiegel/rust-pretty-assertions/blob/main/CHANGELOG.md)
- [Commits](https://github.com/colin-kiegel/rust-pretty-assertions/compare/v0.7.2...v1.0.0)

---
updated-dependencies:
- dependency-name: pretty_assertions
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-29 00:56:15 +02:00
dependabot[bot]
6848b20546
Bump tokio from 1.11.0 to 1.12.0 (#343) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.11.0 to 1.12.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.11.0...tokio-1.12.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-22 14:28:59 +02:00
dependabot[bot]
61fa23099b
Bump http from 0.2.4 to 0.2.5 (#344) 
Bumps [http](https://github.com/hyperium/http) from 0.2.4 to 0.2.5.
- [Release notes](https://github.com/hyperium/http/releases)
- [Changelog](https://github.com/hyperium/http/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/http/compare/v0.2.4...v0.2.5)

---
updated-dependencies:
- dependency-name: http
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-22 14:27:23 +02:00
dependabot[bot]
7f17ffb9b1
Bump openssl-sys from 0.9.63 to 0.9.67 (#342) 
Bumps [openssl-sys](https://github.com/sfackler/rust-openssl) from 0.9.63 to 0.9.67.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-sys-v0.9.63...openssl-sys-v0.9.67)

---
updated-dependencies:
- dependency-name: openssl-sys
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-22 14:27:14 +02:00
dependabot[bot]
bfa17b978e
Bump tracing-subscriber from 0.2.23 to 0.2.24 (#341) 
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.23 to 0.2.24.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.23...tracing-subscriber-0.2.24)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-20 16:17:28 +02:00
Matthias
f2d7abbc29
Fix broken pipe when dumping links (#339) 
When piping the output of lychee's `--dump` output to another program,
we can run into issues with broken pipes as described in
https://github.com/rust-lang/rust/issues/46016
and https://gabebw.com/blog/2019/10/13/learning-rust-by-candlelight
To avoid this, we use the underlying writeln macro and check the
returned `ErrorKind`.
 2021-09-20 12:12:35 +02:00
Matthias
94cae4e9e5
Skip creating issues on forks (#340) 2021-09-20 12:11:31 +02:00
Matthias
b7bbcf69e9
Don't try to publish Docker image on fork (#332) 
* Don't try to publish Docker image on fork

https://github.com/lycheeverse/lychee/issues/321#issuecomment-919161530

* Move if condition to up to job
 2021-09-20 11:13:54 +02:00
dependabot[bot]
8759db5a7a
Bump tracing-subscriber from 0.2.22 to 0.2.23 (#334) 
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.22 to 0.2.23.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.22...tracing-subscriber-0.2.23)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-20 11:13:41 +02:00
Matthias
3b41c4c375
Silently ignore absolute paths without base (fixes #320) (#338) 2021-09-20 11:13:30 +02:00
Matthias
712bdfa8cb
Make inputs required (show help if not provided) (#329) 2021-09-16 16:40:38 +02:00
Matthias
dd667a656c
Undo Docker documentation from CI 
It's currently not supported to update docs with a Dockerhub token. See https://github.com/lycheeverse/lychee/pull/325.
 2021-09-16 15:54:14 +02:00
dependabot[bot]
d83d36788f
Bump serde_json from 1.0.67 to 1.0.68 (#331) 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.67 to 1.0.68.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.67...v1.0.68)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-16 15:37:12 +02:00
dependabot[bot]
d24511217f
Bump check-if-email-exists from 0.8.23 to 0.8.24 (#323) 
Bumps [check-if-email-exists](https://github.com/reacherhq/check-if-email-exists) from 0.8.23 to 0.8.24.
- [Release notes](https://github.com/reacherhq/check-if-email-exists/releases)
- [Changelog](https://github.com/reacherhq/check-if-email-exists/blob/master/CHANGELOG.md)
- [Commits](https://github.com/reacherhq/check-if-email-exists/compare/v0.8.23...v0.8.24)

---
updated-dependencies:
- dependency-name: check-if-email-exists
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-14 15:49:02 +02:00
dependabot[bot]
408f1de03c
Bump anyhow from 1.0.40 to 1.0.44 (#322) 
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.40 to 1.0.44.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.40...1.0.44)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-14 15:48:51 +02:00
faust
e23c6c55d0
Add a description on DockerHub registry (#325) 2021-09-14 15:47:58 +02:00
faust
c11b46d943
Optimize Dockerfile (#326) 
- Remove trailing white spaces

- Improve size (3MB removed in /var/cache/debconf) +
  `no-install-recommends`
  See:
  ```console
  docker run --rm -i -v /var/run/docker.sock:/var/run/docker.sock \
    -v $(pwd):/mnt -w /mnt wagoodman/dive:latest \
    --ci lycheeverse/lychee:latest
  ```

- Add cleaning mechanism (even if not necessary with this base image)
  See: https://github.com/debuerreotype/debuerreotype/blob/master/scripts/debuerreotype-minimizing-config

- DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.
  See: https://github.com/hadolint/hadolint/wiki/DL3059

- DL3020 error: Use COPY instead of ADD for files and folders
  See: https://github.com/hadolint/hadolint/wiki/DL3020
 2021-09-14 15:07:37 +02:00
dependabot[bot]
f7b3495fa1
Bump tracing-subscriber from 0.2.20 to 0.2.22 (#328) 
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.20 to 0.2.22.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.20...tracing-subscriber-0.2.22)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-14 14:55:10 +02:00
Matthias
21ea0fd033
Add support for tokio-console (#318) 
This allows troubleshooting and improving async Rust code.
It is an optional feature that is still
experimental (but can be quite helpful)
 2021-09-12 18:10:23 +02:00
Matthias
9b5fc399ed
Add support for local files #262 2021-09-09 19:50:37 +02:00
Matthias
de55fbd178 Add TODO for fixing URL encoding for paths 2021-09-09 19:31:49 +02:00
Matthias
d7436575eb formatting 2021-09-09 14:43:40 +02:00
Matthias
2a4170eade Add test for + encoding 2021-09-09 14:42:09 +02:00
Matthias
a41e81c988
Merge branch 'master' into local-files 2021-09-09 01:51:38 +02:00
Matthias
a1acf7b0d0 Reintegrate master 2021-09-09 01:49:25 +02:00
Matthias
93948d7367 Avoid double-encoding already encoded destination paths 
E.g. `web%20site` becomes `web site`.
That's because Url::from_file_path will encode the full URL in the end.
This behavior cannot be configured.
See https://github.com/lycheeverse/lychee/pull/262#issuecomment-915245411
 2021-09-09 01:44:10 +02:00
Matthias
a75cae54b1 Add failing test 2021-09-09 01:17:56 +02:00
MichaIng
16c6dcf4d6
Include build dependencies in cargo install section (#317) 
* Include build dependencies in cargo install section

Additionally remove a misplaced "use" in front of the comparison table and align the code spaces within the table to enhance readability of the code (no effect on visual Markdown).

Signed-off-by: MichaIng <micha@dietpi.com>

* DPkg -> dpkg

It is spelled lowercase, even at the beginning of a sentence according to the manpage. https://man7.org/linux/man-pages/man1/dpkg.1.html

Co-authored-by: Matthias <matthias-endler@gmx.net>
 2021-09-09 00:48:17 +02:00
Matthias
e188e72fc0
Don't panic on unknown status codes (#234) 2021-09-08 01:14:20 +02:00
Matthias
24ea2482d3 Update docs 2021-09-08 01:08:59 +02:00
dependabot[bot]
fa920d24b8
Bump assert_cmd from 1.0.4 to 2.0.1 (#314) 
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd) from 1.0.4 to 2.0.1.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases)
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v1.0.4...v2.0.1)

---
updated-dependencies:
- dependency-name: assert_cmd
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-08 00:37:21 +02:00
Matthias
f3fe46a4d6 Merge branch 'master' of github.com:lycheeverse/lychee into local-files 2021-09-08 00:35:41 +02:00
Matthias
ffab0343fc Revert refactor for removing params and fragments 
The refactored version was not equivalent. It could not handle
fragments containing a question mark.
See 67268ed598 (r703400238)
 2021-09-08 00:29:30 +02:00
Matthias
1246fa564c
Don't exlude mail on exclude-all-private (#316) 2021-09-08 00:21:00 +02:00
Matthias
67268ed598 Clean up params and fragment handling 2021-09-07 13:02:39 +02:00
Matthias
0c5dcf3aa3 whoops 2021-09-07 01:05:41 +02:00
Matthias
8353ab1071 Update docs 2021-09-07 00:53:42 +02:00
Matthias
a28f932fb2 Fix wildcard test 2021-09-07 00:41:07 +02:00
Matthias
4827ecf6bd Fix clippy warnings 2021-09-07 00:22:06 +02:00
Matthias
5d0b95271d Remove anchor from file links 2021-09-07 00:20:09 +02:00