Hopiu/lychee
1
0
Fork 0
mirror of https://github.com/Hopiu/lychee.git synced 2026-04-05 22:10:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
1695 commits 1 branch 0 tags 5.1 MiB
Commit graph

265 commits

Author SHA1 Message Date
dependabot[bot]
c5dfb45c9d
Bump the dependencies group with 4 updates (#1664) 
* Bump the dependencies group with 4 updates

Bumps the dependencies group with 4 updates: [log](https://github.com/rust-lang/log), [reqwest](https://github.com/seanmonstar/reqwest), [tempfile](https://github.com/Stebalien/tempfile) and [typed-builder](https://github.com/idanarye/rust-typed-builder).


Updates `log` from 0.4.26 to 0.4.27
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.26...0.4.27)

Updates `reqwest` from 0.12.14 to 0.12.15
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.12.14...v0.12.15)

Updates `tempfile` from 3.19.0 to 3.19.1
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.19.0...v3.19.1)

Updates `typed-builder` from 0.20.1 to 0.21.0
- [Release notes](https://github.com/idanarye/rust-typed-builder/releases)
- [Changelog](https://github.com/idanarye/rust-typed-builder/blob/master/CHANGELOG.md)
- [Commits](https://github.com/idanarye/rust-typed-builder/compare/v0.20.1...v0.21.0)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: typed-builder
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix lint

* fix two more lints

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthias <matthias@endler.dev>
 2025-03-24 16:30:44 +01:00
Matthias Endler
85263dfa33
Add support for custom file extensions in link checking. (#1559) 
- Update lychee-bin/src/options.rs
- Refactor file extension handling to use FileExtensions type for improved clarity and functionality
- Refactor file extension parsing to use FromStr implementation for improved clarity
- order derive and comment
- Add plaintext extension handling to FileType
- Set default value for file extensions in Config and implement Display for FileExtensions
- Add default file extensions to README documentation
- Implement TryFrom for FileExtensions and update input handling to use try_into
- Add plaintext extension handling to FileType and fix test

---------

Co-authored-by: Thomas Zahner <thomas.zahner@protonmail.ch>
 2025-03-02 17:47:05 +01:00
dependabot[bot]
98f3e4c188
Bump the dependencies group across 1 directory with 21 updates (#1643) 
* Bump the dependencies group across 1 directory with 21 updates

Bumps the dependencies group with 21 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [anyhow](https://github.com/dtolnay/anyhow) | `1.0.95` | `1.0.96` |
| [clap](https://github.com/clap-rs/clap) | `4.5.23` | `4.5.30` |
| [indicatif](https://github.com/console-rs/indicatif) | `0.17.9` | `0.17.11` |
| [log](https://github.com/rust-lang/log) | `0.4.22` | `0.4.26` |
| [once_cell](https://github.com/matklad/once_cell) | `1.20.2` | `1.20.3` |
| [openssl-sys](https://github.com/sfackler/rust-openssl) | `0.9.104` | `0.9.106` |
| [ring](https://github.com/ctz/ring) | `0.17.8` | `0.17.11` |
| [serde](https://github.com/serde-rs/serde) | `1.0.217` | `1.0.218` |
| [serde_json](https://github.com/serde-rs/json) | `1.0.134` | `1.0.139` |
| [strum](https://github.com/Peternator7/strum) | `0.26.3` | `0.27.1` |
| [tabled](https://github.com/zhiburt/tabled) | `0.17.0` | `0.18.0` |
| [tokio](https://github.com/tokio-rs/tokio) | `1.42.0` | `1.43.0` |
| [toml](https://github.com/toml-rs/toml) | `0.8.19` | `0.8.20` |
| [tempfile](https://github.com/Stebalien/tempfile) | `3.15.0` | `3.17.1` |
| [uuid](https://github.com/uuid-rs/uuid) | `1.11.0` | `1.14.0` |
| [async-trait](https://github.com/dtolnay/async-trait) | `0.1.84` | `0.1.86` |
| [html5ever](https://github.com/servo/html5ever) | `0.29.0` | `0.29.1` |
| [hyper](https://github.com/hyperium/hyper) | `1.5.2` | `1.6.0` |
| [octocrab](https://github.com/XAMPPRocky/octocrab) | `0.42.1` | `0.43.0` |
| [pulldown-cmark](https://github.com/raphlinus/pulldown-cmark) | `0.12.2` | `0.13.0` |
| [thiserror](https://github.com/dtolnay/thiserror) | `2.0.9` | `2.0.11` |



Updates `anyhow` from 1.0.95 to 1.0.96
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.95...1.0.96)

Updates `clap` from 4.5.23 to 4.5.30
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.23...clap_complete-v4.5.30)

Updates `indicatif` from 0.17.9 to 0.17.11
- [Release notes](https://github.com/console-rs/indicatif/releases)
- [Commits](https://github.com/console-rs/indicatif/compare/0.17.9...0.17.11)

Updates `log` from 0.4.22 to 0.4.26
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.22...0.4.26)

Updates `once_cell` from 1.20.2 to 1.20.3
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.20.2...v1.20.3)

Updates `openssl-sys` from 0.9.104 to 0.9.106
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-sys-v0.9.104...openssl-sys-v0.9.106)

Updates `ring` from 0.17.8 to 0.17.11
- [Release notes](https://github.com/ctz/ring/releases)
- [Commits](https://github.com/ctz/ring/compare/0.17.8...0.17.11)

Updates `serde` from 1.0.217 to 1.0.218
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.217...v1.0.218)

Updates `serde_json` from 1.0.134 to 1.0.139
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.134...v1.0.139)

Updates `strum` from 0.26.3 to 0.27.1
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/compare/v0.26.3...v0.27.1)

Updates `tabled` from 0.17.0 to 0.18.0
- [Changelog](https://github.com/zhiburt/tabled/blob/master/CHANGELOG.md)
- [Commits](https://github.com/zhiburt/tabled/commits)

Updates `tokio` from 1.42.0 to 1.43.0
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.42.0...tokio-1.43.0)

Updates `toml` from 0.8.19 to 0.8.20
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.8.19...toml-v0.8.20)

Updates `tempfile` from 3.15.0 to 3.17.1
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.15.0...v3.17.1)

Updates `uuid` from 1.11.0 to 1.14.0
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.11.0...v1.14.0)

Updates `async-trait` from 0.1.84 to 0.1.86
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.84...0.1.86)

Updates `html5ever` from 0.29.0 to 0.29.1
- [Commits](https://github.com/servo/html5ever/commits)

Updates `hyper` from 1.5.2 to 1.6.0
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v1.5.2...v1.6.0)

Updates `octocrab` from 0.42.1 to 0.43.0
- [Release notes](https://github.com/XAMPPRocky/octocrab/releases)
- [Changelog](https://github.com/XAMPPRocky/octocrab/blob/main/CHANGELOG.md)
- [Commits](https://github.com/XAMPPRocky/octocrab/compare/v0.42.1...v0.43.0)

Updates `pulldown-cmark` from 0.12.2 to 0.13.0
- [Release notes](https://github.com/raphlinus/pulldown-cmark/releases)
- [Commits](https://github.com/raphlinus/pulldown-cmark/compare/v0.12.2...v0.13.0)

Updates `thiserror` from 2.0.9 to 2.0.11
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/2.0.9...2.0.11)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: indicatif
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: openssl-sys
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: ring
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: strum
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: tabled
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: html5ever
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: octocrab
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: pulldown-cmark
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update markup5ever to 0.14.1

* feat: add support for Wiki links in markdown extraction

* Temporarily resolve test issue

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthias <matthias@endler.dev>
 2025-02-24 14:18:06 +01:00
MichaIng
d3d7f6a56b
fix: do not fail on empty # and #top fragments (#1609) 
The empty "#" and "#top" fragments are always valid without related HTML element. Browser will scroll to the top of the page. Hence lychee must not fail on those.

Credits go to @thiru-appitap for initial attempt and helping to find missing parts of the implementation.

Solves: https://github.com/lycheeverse/lychee/issues/1599

Signed-off-by: MichaIng <micha@dietpi.com>
 2025-02-06 15:09:59 +01:00
Matthias Endler
971ee67bc3
Fix new clippy lints (#1625) 2025-02-06 14:51:44 +01:00
Trask Stalnaker
6d0e94c799
Introduce --root-dir (#1576) 
* windows

* Introduce --root-path

* lint

* lint

* Simplification

* Add unit tests

* Add integration test

* Sync docs

* Add missing comment to make CI happy

* Revert one of the Windows-specific changes because causing a test failure

* Support both options at the same time

* Revert a comment change that is no longer applicable

* Remove unused code

* Fix and simplification

* Integration test both at the same time

* Unit tests both at the same time

* Remove now redundant comment

* Revert windows-specific change, seems not needed after recent changes

* Use Collector::default()

* extract method and unit tests

* clippy

* clippy: &Option<A> -> Option<&A>

* Remove outdated comment

* Rename --root-path to --root-dir

* Restrict --root-dir to absolute paths for now

* Move root dir check
 2024-12-13 14:36:33 +01:00
Trask Stalnaker
034fd129a9
Fix retries (#1573) 2024-11-27 22:58:32 +01:00
dependabot[bot]
2d35dccd34
Bump the dependencies group with 4 updates (#1566) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthias <matthias@endler.dev>
 2024-11-12 23:51:39 +01:00
Matthias Endler
9dc42176fa
Rename fail_map to error_map for improved clarity in response statistics (#1560) 
Fixes #1446
 2024-11-08 09:02:33 +01:00
Matthias Endler
f4a35ff580
Add quirks support for youtube-nocookie.com and youtube embed URLs (#1563) 2024-11-08 06:15:26 +01:00
Matthias Endler
6b53695be6
Support underscores in Markdown URLs (#1555) 2024-11-07 14:54:42 +01:00
dependabot[bot]
6e3219eb55
Bump the dependencies group across 1 directory with 7 updates (#1552) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthias <matthias@endler.dev>
 2024-11-07 00:36:45 +01:00
Matthias Endler
71564344de
Fix: Bring back error output for links (#1553) 
With the last lychee release, we simplified the status output for links.

While this reduced the visual noise, it also accidentally caused the source of errors to not be printed anymore. This change brings back the additional error information as part of the final report output. Furthermore, it shows the error information in the progress output if verbose mode is activated.

Fixes #1487
 2024-11-07 00:22:50 +01:00
autoantwort
98015907f2
Ignore casing when processing markdown fragments + check for percent encoded ancors (#1535) 
We must also check the fragment before it is percent-decoded as required by the HTML standard.

Fixes https://github.com/lycheeverse/lychee/issues/1467
 2024-10-28 09:21:13 +01:00
Matthias Endler
e43086c2e9
Fix skipping of email addresses in stylesheets (#1546) 2024-10-27 01:32:11 +02:00
Matthias Endler
3094bbca33
Add support for relative links (#1489) 
This commit introduces several improvements to the file checking process and URI handling:

- Extract file checking logic into separate `Checker` structs (`FileChecker`, `WebsiteChecker`, `MailChecker`)
- Improve handling of relative and absolute file paths
- Enhance URI parsing and creation from file paths
- Refactor `create_request` function for better clarity and error handling

These changes provide better support for resolving relative links, handling different base URLs, and working with file paths.

Fixes https://github.com/lycheeverse/lychee/issues/1296 and https://github.com/lycheeverse/lychee/issues/1480
 2024-10-26 04:07:37 +02:00
Matthias Endler
87d5b56e4f
Box Octocrab error as it is too large (#1543) 2024-10-26 03:31:24 +02:00
xlai89
7484a1ff6c
Don't check prefix attribute (#1536) 2024-10-20 22:56:46 +02:00
Damien Mathieu
f0ebac29a2
Allow excluding cache based on status code (#1403) 
This introduces an option `--cache-exclude-status`, which allows specifying a range of HTTP status codes which will be ignored from the cache.

Closes #1400.
 2024-10-14 02:41:56 +02:00
Marko Galevski
2a9f11a289
Ignore textContent links in html nodes (#1528) 
This fixes issue #1462 by removing plaintext URI parsing
in html5ever and pruning attribute-less URIs in html5gum
 2024-10-14 00:59:55 +02:00
Matthias Endler
913cf717a4
feat: Add tests for dns-prefetch (#1522) 
* Exclude `rel=dns-prefetch` links

Resolves #1499

* Add tests for dns-prefetch

---------

Co-authored-by: wackget <136205263+wackget@users.noreply.github.com>
 2024-10-12 01:45:35 +02:00
wackget
e398325bb0
Exclude rel=dns-prefetch links (#1520) 
Resolves #1499
 2024-10-12 01:37:54 +02:00
Matthias Endler
7014765988
Improve docs for fragment checker 2024-10-08 14:55:16 +02:00
Matthias Endler
11adc09725
Don't check preconnect links (#1187) 
Preconnect links are used to establish a server connection without loading a
specific resource yet. Not always do these links point to a URL that should
return a 200, and they are not user-facing, i.e. they don't show up in the
final rendered version of a page.

Therefore, we should not check them at all; not even in `--include-verbatim`
mode, as they might not point to a valid resource.

This turned out to require a significant overhaul of the html5gum extractor
to handle random attribute ordering correctly. Changes to the html5gum extractor:

* Refactor HTML link extractor for improved performance and maintainability
- Replace Vec<u8> with String for better readability and manipulation
- Introduce Element struct to encapsulate element-related data
- Use `HashMap<String, String>` for current_attributes for efficient lookups
- Add verbatim_stack to properly handle nested verbatim elements
- Remove unsafe code where possible, using String::from_utf8_lossy
- Improve attribute handling with `HashMap` entry API and prioritize `srcset`
- Simplify logic and consolidate verbatim element handling
- Enhance encapsulation in `LinkExtractor` struct
- Improve overall performance with more efficient data structures
- Increase flexibility for future feature additions or modifications

Fixes #897
 2024-10-07 22:36:16 +02:00
dependabot[bot]
b9ad685fea
Bump the dependencies group across 1 directory with 8 updates (#1509) 
* Bump the dependencies group across 1 directory with 8 updates

Bumps the dependencies group with 8 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [clap](https://github.com/clap-rs/clap) | `4.5.17` | `4.5.18` |
| [regex](https://github.com/rust-lang/regex) | `1.10.6` | `1.11.0` |
| [secrecy](https://github.com/iqlusioninc/crates) | `0.8.0` | `0.10.2` |
| [tempfile](https://github.com/Stebalien/tempfile) | `3.12.0` | `3.13.0` |
| [async-trait](https://github.com/dtolnay/async-trait) | `0.1.82` | `0.1.83` |
| [octocrab](https://github.com/XAMPPRocky/octocrab) | `0.39.0` | `0.40.0` |
| [thiserror](https://github.com/dtolnay/thiserror) | `1.0.63` | `1.0.64` |
| [rstest](https://github.com/la10736/rstest) | `0.22.0` | `0.23.0` |



Updates `clap` from 4.5.17 to 4.5.18
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.17...clap_complete-v4.5.18)

Updates `regex` from 1.10.6 to 1.11.0
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.10.6...1.11.0)

Updates `secrecy` from 0.8.0 to 0.10.2
- [Commits](https://github.com/iqlusioninc/crates/commits)

Updates `tempfile` from 3.12.0 to 3.13.0
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.12.0...v3.13.0)

Updates `async-trait` from 0.1.82 to 0.1.83
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.82...0.1.83)

Updates `octocrab` from 0.39.0 to 0.40.0
- [Release notes](https://github.com/XAMPPRocky/octocrab/releases)
- [Changelog](https://github.com/XAMPPRocky/octocrab/blob/main/CHANGELOG.md)
- [Commits](https://github.com/XAMPPRocky/octocrab/compare/v0.39.0...v0.40.0)

Updates `thiserror` from 1.0.63 to 1.0.64
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.63...1.0.64)

Updates `rstest` from 0.22.0 to 0.23.0
- [Release notes](https://github.com/la10736/rstest/releases)
- [Changelog](https://github.com/la10736/rstest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/la10736/rstest/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: secrecy
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: octocrab
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: rstest
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* Refactor personal token cloning in ClientBuilder

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthias <matthias@endler.dev>
 2024-10-01 13:10:00 +02:00
Thomas Zahner
99148af84a Disable pedantic clippy lint 2024-09-22 19:09:35 +02:00
Thomas Zahner
6075b4c87e Skip ignored and hidden files by default 2024-09-22 19:09:35 +02:00
Thomas Zahner
cf1420817f Remove is_symlink check, because it's mutually exclusive to is_file 2024-09-22 19:09:35 +02:00
Thomas Zahner
5f7b3c42ea Handle path exclusions when walking directories 2024-09-22 19:09:35 +02:00
Thomas Zahner
bd187ba0d9 Update tests 2024-09-22 19:09:35 +02:00
Thomas Zahner
6444e27a84 Make gitignored files configurable and disable by default 2024-09-22 19:09:35 +02:00
Thomas Zahner
41c1b971c7 Replace jwalk with ignore 2024-09-22 19:09:35 +02:00
Thomas Zahner
7fcf66c492
Extend compact format (#1497) 
* Show unknowns and timeouts in compact format
* Clippy: make functions const
 2024-09-09 18:33:18 +02:00
dependabot[bot]
0a53e920ed
Bump the dependencies group with 6 updates (#1486) 
* Bump the dependencies group with 6 updates

Bumps the dependencies group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [clap](https://github.com/clap-rs/clap) | `4.5.13` | `4.5.15` |
| [serde](https://github.com/serde-rs/serde) | `1.0.204` | `1.0.206` |
| [serde_json](https://github.com/serde-rs/json) | `1.0.122` | `1.0.124` |
| [assert_cmd](https://github.com/assert-rs/assert_cmd) | `2.0.15` | `2.0.16` |
| [tempfile](https://github.com/Stebalien/tempfile) | `3.11.0` | `3.12.0` |
| [html5ever](https://github.com/servo/html5ever) | `0.27.0` | `0.28.0` |


Updates `clap` from 4.5.13 to 4.5.15
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.13...v4.5.15)

Updates `serde` from 1.0.204 to 1.0.206
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.204...v1.0.206)

Updates `serde_json` from 1.0.122 to 1.0.124
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.122...v1.0.124)

Updates `assert_cmd` from 2.0.15 to 2.0.16
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.15...v2.0.16)

Updates `tempfile` from 3.11.0 to 3.12.0
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/commits)

Updates `html5ever` from 0.27.0 to 0.28.0
- [Commits](https://github.com/servo/html5ever/commits)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: assert_cmd
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: html5ever
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix compile error

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthias <matthias@endler.dev>
 2024-08-19 12:43:58 +02:00
Matthias Endler
138cd4dd53
test: fail when interpreting md footnote as link (#1479) 
* test: fail when interpreting md footnote as link

* Fix footnote link parsing
 2024-08-11 12:51:46 +02:00
Brian
8df3b99d8c
Fix: Windows drive paths misidentified as URLs (#1460) 
Co-authored-by: Matthias <matthias@endler.dev>
 2024-08-06 18:04:13 +02:00
Hugo McNally
4bb8a61545
Updated pulldown-cmark dependency and fixed maths parsing (#1473) 
* Update pulldown-cmark version to 0.11.0
* Fix markdown math parsing
* Fix lints
* Disable flaky wayback test

---------

Co-authored-by: Matthias <matthias@endler.dev>
 2024-08-06 15:43:34 +02:00
Matthias Endler
dedc554eda
Add response formatter; refactor stats formatter (#1398) 
This adds support for formatting responses in different ways.

For now, the options are:

* `plain`: No color, basic formatting
* `color`: Color, indented formatting (default)
* `emoji`: Fancy mode with emoji icons

Fixes #546
Related to #271
 2024-06-14 19:47:52 +02:00
Matthias Endler
cc7acfb0e0
Extend documentation for RequestChain (#1442) 2024-06-14 12:51:13 +02:00
Johannes Schindelin
8c6eee9b5f
Add a way to handle "pretty URLs", i.e. URIs without .html extension (#1422) 
In many circumstances (GitHub Pages, Apache configured with MultiViews,
etc), web servers process URIs by appending the `.html` file extension
when no file is found at the path specified by the URI but a `.html`
file corresponding to that path _is_ found.

To allow Lychee to use the fast, offline method of checking such files
locally via the `file://` scheme, let's handle this scenario gracefully
by adding the `--fallback-extensions=html` option.

Note: This new option can take a list of file extensions to use; The
first one for which a corresponding file is found is then used.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
 2024-06-11 16:11:24 +02:00
Thomas Zahner
255164ce25
Don't trim mailto: prefix when converting a Uri into a string (#1438) 2024-06-10 17:11:15 +02:00
n4n5
c3f7fe7ad4
Exclude tel scheme from being checked (#1429) 2024-05-19 20:31:38 +02:00
Johannes Schindelin
975901d470
Fix clippy errors (#1423) 
* Enclose Markdown links in brackets

The current clippy version (v0.1.78) says "you should put bare URLs
between `<`/`>` or make a proper Markdown link" and refers to
https://rust-lang.github.io/rust-clippy/master/index.html#doc_markdown

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

* Enclose documentation item in backticks

Clippy v0.1.78 complains about the IPv6 network mask, insisting that it
is missing backticks. So backticks it gets.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

* Avoid error claiming `Add(usize)` is dead code

Clippy v0.1.78 identifies this as dead code. However, further down in
the same file, there is clearly a user:

  impl Handler<Result, Result> for Add {

This might be yet another incarnation of
https://github.com/rust-lang/rust/issues/56750

Let's just mark it as intentionally dead-code, even if this is untrue,
to make clippy happy again.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

---------

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
 2024-05-13 19:07:02 +02:00
John Bampton
0956ec6c38
Fix spelling and remove unneeded trailing whitespace (#1417) 2024-04-26 08:22:44 +02:00
John Bampton
7be088bbfc
Fix spelling; Github -> GitHub (#1416) 2024-04-25 22:44:24 +02:00
Thomas Zahner
25a3eb1a3a
Chain visibility (#1415) 
* Make chain public
* Make function on chain public
* Add must_use attribute
* Make RequestChain type public
* Add chain usage example
 2024-04-25 15:31:03 +02:00
Matthias Endler
fc85695d21
Gracefully handle invalid URIs (#1414) 
With the upgrade to `reqwest` 0.12, we can finally handle a long-standing
issue, when Urls could not be parsed to Uris. Previously, we would panic, but
we can now handle that situation gracefully and return an error instead.

I've also renamed `Status::is_failure` to `Status::is_error`, because the
notion of failures no longer exists in the codebase and we use the term "error"
consistently throughout the codebase instead. This is technically a breaking
change in the API, but it's fine since we have not released a stable version
yet.

More information about the URI parsing issue:
- https://github.com/lycheeverse/lychee/issues/539
- https://github.com/seanmonstar/reqwest/issues/668
 2024-04-25 15:29:36 +02:00
Thomas Zahner
e0b4c73987 Adapt to breaking changes & revert to pulldown-cmark 0.9 2024-04-25 08:48:11 +02:00
Thomas Zahner
f2b1c29bd4 Rename chain to handle 2024-04-22 14:07:17 +02:00
Thomas Zahner
ddcca65e72 Rename Chainable to Handler 2024-04-22 11:03:26 +02:00