Commit graph

255 commits

Author SHA1 Message Date
Bastian Kleineidam
6437f08277 Display downloaded bytes. 2014-03-14 21:06:10 +01:00
Bastian Kleineidam
6b334dc79b Fix URL result caching. 2014-03-08 19:35:10 +01:00
Bastian Kleineidam
0113f06406 Enable arbitrary output encodings in CSV output. See #467 2014-03-06 22:40:52 +01:00
Bastian Kleineidam
82f81241fd Check all links and add better caching. 2014-03-03 23:29:45 +01:00
Bastian Kleineidam
eb7e52c0e2 -o none sets exit code now 2014-03-01 15:31:39 +01:00
Bastian Kleineidam
f7f5001256 Add missing column name to SQL insert statement. 2014-03-01 12:03:33 +01:00
Bastian Kleineidam
7b34be590b Introduce check plugins, use Python requests for http/s connections, and some code cleanups and improvements. 2014-03-01 00:12:34 +01:00
Bastian Kleineidam
c806be5c15 Updated copyright 2014-01-08 22:33:04 +01:00
Bastian Kleineidam
e0a2558b2b Updated copyright. 2013-12-24 07:13:16 +01:00
Bastian Kleineidam
5736987b60 Refactor output loggers. 2013-12-11 18:41:55 +01:00
Alper Kokmen
4b3e78cac0 Fix ISO formatting for modified datetime.
This change will make sure that format_modified returns datetime value
in ISO 8601 format. See W3C documentation at
http://www.w3.org/TR/NOTE-datetime.

Since ```modified``` is parsed and then converted to UTC after it's
extracted from HTTP response, it's safe to assume that format_modified
will always format UTC datetime values.

Instead of ```isoformat``` method which omits timezone information for
UTC values, ```strftime``` with a specific format (that ends with Z)
will be used.
2013-09-02 15:38:54 -07:00
Bastian Kleineidam
b0c2a90b94 Updated copyright. 2012-11-07 18:08:44 +01:00
Bastian Kleineidam
eabaa41bd2 Do not check duplicate URLs. 2012-11-06 21:34:22 +01:00
Bastian Kleineidam
0c20ef5de4 Strip console characters only from line text. 2012-10-10 12:27:08 +02:00
Bastian Kleineidam
e1e80b7dd5 Remove addrinfo cache. 2012-10-10 10:54:58 +02:00
Bastian Kleineidam
20be0f2519 Strip control chars from logger output. 2012-10-10 10:54:30 +02:00
Bastian Kleineidam
03a5d476b3 Use URL name if title is empty. 2012-10-09 21:04:54 +02:00
Bastian Kleineidam
cbc3bcb0d3 Sitemap logger fixes. 2012-09-23 23:20:21 +02:00
Bastian Kleineidam
1f3034b5f5 Sitemap logger fixes. 2012-09-23 20:59:38 +02:00
Bastian Kleineidam
18a200d85f Fix tests. 2012-09-19 11:05:26 +02:00
Bastian Kleineidam
b8f8bdf5fc Fix last modified formatting. 2012-09-19 10:09:19 +02:00
Bastian Kleineidam
75719b34f6 Updated copyright. 2012-09-19 09:17:25 +02:00
Bastian Kleineidam
71fba0f8b7 Log all valid URLs in sitemap loggers. 2012-09-19 09:17:08 +02:00
Bastian Kleineidam
3a352631ba Add modified field to loggers. 2012-09-18 12:12:00 +02:00
Bastian Kleineidam
932a07a9cf Added XML sitemap logger. 2012-09-18 09:16:34 +02:00
Bastian Kleineidam
99bf8aa940 Updated copyright. 2012-09-17 16:09:55 +02:00
Bastian Kleineidam
03667a4ec9 Print warning tags in text output. 2012-09-17 15:29:04 +02:00
Bastian Kleineidam
6e1841cf1f Print download and cache statistics. 2012-09-17 15:23:25 +02:00
Bastian Kleineidam
e98f15933f Stop checking of all output loggers have been deactivated. 2012-09-14 22:36:59 +02:00
Bastian Kleineidam
86f1c74006 Close loggers properly on I/O errors. 2012-09-14 22:09:18 +02:00
Bastian Kleineidam
451a520943 Prevent double color stream proxying. 2012-08-10 19:43:33 +02:00
Bastian Kleineidam
580ab74f0e Updated german translation. 2012-08-09 20:43:31 +02:00
Bastian Kleineidam
979d7f13d3 Updated copyright. 2012-06-20 21:40:21 +02:00
Bastian Kleineidam
1e13a4f8fc Add donation url to info blurb. 2012-06-20 00:37:39 +02:00
Bastian Kleineidam
eb30191bb0 Add copyright and missing docs. 2012-06-20 00:30:52 +02:00
Bastian Kleineidam
a6eaae2c38 Implement abstract method for logger. 2012-06-20 00:15:45 +02:00
Bastian Kleineidam
2dfe9d4b4f Use abstract base class for loggers. 2012-06-19 23:27:26 +02:00
Bastian Kleineidam
c7ced2445b Ensure correct encoding when writing non-ascii CSV output. 2012-04-22 17:52:47 +02:00
Bastian Kleineidam
3d831c1adb Updated copyright. 2012-04-11 22:23:43 +02:00
Bastian Kleineidam
ae0bd406d4 Do not encode CSV outro output comment. 2012-04-11 20:43:46 +02:00
Bastian Kleineidam
e9420d77db Updated copyright. 2012-03-31 09:24:08 +02:00
Bastian Kleineidam
b48812f612 Encode comments in CSV logger. 2012-03-31 09:17:49 +02:00
Bastian Kleineidam
1c26c14b64 Set copyright year and add missing docstrings. 2011-12-25 08:45:27 +01:00
Bastian Kleineidam
21532a70ec Return with non-zero return value when internal program errors occurred. 2011-12-14 22:54:26 +01:00
Bastian Kleineidam
91dce84c59 Fix sqlify for multiline contents. 2011-10-18 14:40:33 +02:00
Bastian Kleineidam
6e69f8f3b1 Fix sql logging output. 2011-10-09 19:02:23 +02:00
Bastian Kleineidam
581da4a9c6 Fix misnamed variable. 2011-05-14 20:21:39 +02:00
Bastian Kleineidam
50fc4ab566 Colorize result in text logger. 2011-05-14 09:36:21 +02:00
Bastian Kleineidam
7365170564 Updated copyright. 2011-04-12 09:13:39 +02:00
Bastian Kleineidam
84f6d56a49 Print level in loggers xml, csv and sql. 2011-04-09 10:51:03 +02:00