1.13.1 "Old men in new cars" (released xx.xx.xxxx) * fixed RPM generation by adding the generated config file to the installed files list Type: bugfix Changed: setup.py * mention to remove old versions when upgrading in the documentation Type: documentation Changed: doc/upgrading.txt, doc/install.txt 1.13.0 "The Butterfly Effect" (released 1.9.2004) * lots of internal code restructuring Type: code cleanup Changed: a lot * If checking revealed errors (or warnings with --warnings), the command line client exits with a non-zero exit status. Type: feature Closes: SF bug 1013191 Changed: linkchecker, linkcheck/checker/consumer.py * Specify the HTML doctype and charset in HTML output. Type: feature Closes: SF bug 1014283 Changed: linkcheck/logger/html.py * Fix endless loop on broken urls with non-empty anchor. Type: bugfix Changed: linkcheck/checker/httpurl.py * For news: or nntp: urls, entries in ~/.netrc are now ignored. You should give instead username/password info in the configuration file or on the command line. Type: bugfix Changed: linkcheck/checker/nntpurl.py * The HTML output shows now HTML and CSS validation links for the parent URL of invalid links. Type: feature Changed: linkcheck/logger/html.py * The status is now printed as default, it can be supressed with the new --no-status option. Type: feature Changed: linkchecker * The default recursion level is now infinite. Type: feature Changed: linkchecker * The 'outside of domain filter' is no more a warning but an informational message. A warning is inappropriate since the user is in full control over what links are extern or intern. Type: feature Closes: SF bug 1013206 Changed: linkcheck/urlbase.py * Renamed the --strict option to --extern-strict-all. Type: feature Changed: linkchecker * a new cache and queueing algorithm makes sure that no URL is checked twice. Type: feature Changed: linkcheck/checker/cache.py * the given user/password authententication is now also used to get robots.txt files. Type: feature Changed: linkcheck/robotparser2.py, linkcheck/checker/cache.py 1.12.3 "The Princess Bride" (released 27.5.2004) * fall back to GET on bad status line of a HEAD request Type: bugfix Changed: linkcheck/HttpUrlData.py * really fall back to GET with Zope servers; fixes infinite loop Type: bugfix Changed: linkcheck/HttpUrlData.py * better error msg on BadStatusLine error Type: feature Changed: linkcheck/UrlData.py * updated optcomplete to newest upstream Type: feature Changed: linkcheck/optcomplete.py * also quote query parts of urls Type: bugfix Changed: linkcheck/{HttpUrlData, url}.py * - preserve the order in which HTML attributes have been parsed - cope with trailing space in HTML comments Type: feature Changed: linkcheck/parser/{__init__.py,htmllex.l} Added: linkcheck/containers.py * rework anchor fallback Type: bugfix Changed: linkcheck/HttpUrlData.py * move contentAllowsRobot check to end of recursion check to avoid unnecessary GET request Type: bugfix Changed: linkcheck/UrlData.py 1.12.2 (release 4.4.2004) * use XmlUtils instead of xmlify for XML quoting Type: code cleanup Added: linkcheck/XmlUtils.py Changed: linkcheck/StringUtil.py, linkcheck/log/XMLLogger.py * don't require a value anymore with the --version option Type: bugfix Changed: linkchecker * before putting url data objects in the queue, check if they have correct syntax and are not already cached Type: optimization Changed: linkcheck/{UrlData,Config}.py * every once in a while, remove all already cached urls from the incoming queue. This action is reported when --status is given. Type: optimization Changed: linkcheck/Config.py * both changes above result in significant performance improvements when checking large websites, since a majority of the links tend to be navigation links to already-cached pages. Type: note * updated examples and put them before options in the man page for easier reading Type: documentation Changed: linkchecker, linkchecker.1 * added contact url and email to the HTTP User-Agent string, which gets us more accepted by some bot-blocking software; also see http://www.livejournal.com/bots/ Type: feature Changed: linkcheck/Config.py * only check robots.txt for http connections Type: bugfix Changed: linkcheck/{Http,}UrlData.py Closes: SF bug 928895 * updated regression tests Type: feature Changed: test/test_*.py, Makefile Added: test/run.sh * preserve the order in which HTML attributes have been parsed Type: feature Changed: linkcheck/parser/{__init__.py,htmllex.l} * handle and correct missing start quotes in HTML attributes Type: feature Changed: linkcheck/parser/htmllex.l * full parsing of .css files Type: feature Changed: linkcheck/{Http,}UrlData.py, linkcheck/linkparse.py * removed Gilman news draft Type: feature Removed: draft-gilman-news-url-00.txt 1.12.1 (release 21.2.2004) * raise IncompleteRead instead of ValueError on malformed chunked HTTP data Changed: linkcheck/httplib2.py * catch errors earlier in recursion check Changed: linkcheck/UrlData.py * quote url and parent url in log output Changed: linkcheck/log/*.py Added: linkcheck/url.py 1.12.0 (release 31.1.2004) * added LRU.setdefault function Changed: linkcheck/LRU.py Closes: SF bug 885916 * Added Mac OS X as supported platform (version 10.3 is known to work) Changed: README, INSTALL * HTML parser objects are now subclassable and collectable by the cyclic garbage collector Changed: linkcheck/parser/htmlparse.y * made some minor parser fixes for attribute scanning and JavaScript Changed: linkcheck/parser/htmllex.l * include the optcomplete module for bash autocompletion Added: linkcheck/optcomplete.py, linkcheck-completion Changed: MANIFEST.in, setup.py * print out nicer error message for unknown host names Changed: linkcheck/UrlData.py * added new logger type "none" printing out nothing which is handy for cron scripts. Changed: linkchecker, linkcheck/Config.py, linkcheck/log/__init__.py Added: linkcheck/log/NoneLogger.py * the -F file output option disables console output now Changed: linkchecker * added an example cron script Added: linkcheck-cron.sh Changed: MANIFEST.in, setup.py * only warn about missing anchor support servers when the url has actually an anchor Changed: linkcheck/HttpUrlData.py * always fall back to HTTP GET request when HEAD gave an error to cope with servers not supporting HEAD requests Changed: linkcheck/HttpUrlData.py, FAQ 1.10.3 (release 10.1.2004) * use the optparser module for command line parsing Changed: linkchecker, po/*.po * use Set() instead of hashmap Changed: linkcheck/Config.py * fix mime-type checking to allow parsing of .css stylesheets Changed: linkcheck/HttpUrlData.py * honor HTML meta tags for robots, ie. Changed: linkcheck/UrlData.py, linkcheck/linkparse.py * much less aggressive thread acquiring, this fixes the 100% CPU usage from the previous version Changed: linkcheck/Threader.py 1.10.2 (release 3.1.2004) * fixed CGI safe_url pattern, it was too strict Changed: linkcheck/lc_cgi.py * replace backticks with repr() or %r Changed: all .py files containing backticks, and po/*.po * make windows DNS nameserver parsing more robust Changed: linkcheck/DNS/Base.py Closes: SF bugs 863227,864383 * only cache used data, not the whole url object Changed: linkcheck/{Http,}UrlData.py * limit cached data Changed: linkcheck/{UrlData,Config}.py Added: linkcheck/LRU.py Closes: SF bug 864516 * use dummy_threading module and get rid of the _NoThreads functions Changed: linkchecker, linkcheck/{Config,Threader}.py, test/test_*.py * set default connection timeout to 60 seconds Changed: linkcheck/__init__.py * new option --status print regular messages about number of checked urls and urls still to check Changed: linkchecker, linkcheck/{__init__,Config}.py 1.10.1 (release 19.12.2003) * added Mandrake .spec file from Chris Green Added: linkchecker.spec Changed: MANIFEST.in * print last-modified date for http and https links in infos Changed: linkcheck/HttpUrlData.py * add detailed installation instructions for Windows Changed: INSTALL Closes: SF bug 857748 * updated the DNS nameserver config parse routines Changed: linkcheck/DNS/Base.py Added: linkcheck/DNS/winreg.py Removed: linkcheck/DNS/win32dns.py * fix https support test Changed: linkcheck/HttpUrlData.py 1.10.0 (released 7.12.2003) * catch httplib errors in robotparser Changed: linkcheck/robotparser2.py Closes: SF bug 836864 * - infinite recursion option with negative value works now - initialize self.urlparts to avoid crash when reading cached http urls - with --strict option do not add any automatic filters if the user gave his own on the command line Changed: linkcheck/UrlData.py 1.9.5 (released 31.10.2003) * Add Zope to servers with broken HEAD support, adjusted the FAQ Changed: linkcheck/HttpUrlData.py, FAQ Closes: SF bug 833419 * Disable psyco usage, it is causing infinite loops (this is a known issue with psyco); and it is disabling ctrl-c interrupts (this is also a known issue in psyco) Changed: linkchecker * use internal debug logger Changed: linkcheck/robotparser2.py * do not hardcode Accept-Encoding header in HTTP request Added: linkcheck/httplib2.py Changed: linkcheck/robotparser2.py 1.9.4 (released 22.10.2003) * parse CSS stylesheet files and check included urls, for example background images Changed: linkcheck/{File,Http,Ftp,}UrlData.py, linkcheck/linkparser.py * try to use psyco for the commandline linkchecker script Changed: linkchecker * when decompression of compressed HTML pages fails, assume the page is not compressed Changed: linkcheck/{robotparser2,HttpUrlData}.py 1.9.3 (released 16.10.2003) * re-added an updated robot parser which uses urllib2 and can decode compressed transfer encodings. Added: linkcheck/robotparser2.py * more restrictive url validity checking when running in CGI mode Changed: linkcheck/lc_cgi.py * accept more Windows path specifications, like file://C:\Dokume~1\test.html Changed: linkcheck/FileUrlData.py 1.9.2 * parser fixes: - do not #include , fixes build on some FreeBSD, Windows and Solaris/SunOS platforms - ignore first leading invalid backslash in a=\"b\" attributes Changed: linkcheck/parser/htmllex.{l,c} * add full script path to linkchecker on windows systems Changed: linkchecker.bat * fix generation of Linkchecker_Readme.txt under windows systems Changed: setup.py 1.9.1 * add documentation how to change the default C compiler Changed: INSTALL * fixed blacklist logging Changed: linkcheck/log/BlacklistLogger.py * removed unused imports Changed: linkcheck/*.py * parser fixes: - fixed parsing of end tags with trailing garbage - fixed parsing of script single comment lines Changed: linkcheck/parser/htmllex.l 1.9.0 * Require Python 2.3 - removed timeoutsocket.py and robotparser.py, using upstream - use True/False for boolean values - use csv module - use new-style classes Closes: SF bug 784977 Changed: a lot * update po makefiles and tools Changed po/* * start CGI output immediately Changed: lc.cgi, lc.fcgi, lc.sz_fcgi, linkcheck/lc_cgi.py Closes: SF bug 784331 1.8.22 * allow colons in HTML attribute names, used for namespaces Changed: linkcheck/parser/htmllex.l * fix match of intern patterns with --denyallow enabled Changed: linkcheck/UrlData.py * s/intern/internal/ and s/extern/external/ in the documentation Changed: linkchecker, linkchecker.1, FAQ * rename column "column" to "col" in SQL output, since "column" is a reserved keyword. Thanks Garvin Hicking for the hint. Changed: linkcheck/log/SQLLogger.py, create.sql * handle HTTP redirects to a non-http url Changed: linkcheck/{Http,}UrlData.py Closes: SF bug 784372 1.8.21 * detect recursive redirections; the maximum of five redirections is still there though * after every HTTP 301 or 302 redirection, check the URL cache again Closes: SF bug 776851 * put all HTTP 301 redirection answers also in the url cache as aliases of the original url. this could mess up some redirection warnings (ie warn about redirection when there is none), but it is more network efficient. 1.8.20 * fix setting of domain in set_intern_url Changed: linkcheck/UrlData.py * - parse JS strings and comments - accept "". Changed files: linkcheck/UrlData.py, linkchecker 1.8.17 * fix parsing of missing end tag in "" Changed files: linkcheck/parser/htmllex.l * fix entity resolving in parsed html links Closes: SF bug #749543 Changed files: linkcheck/StringUtil.py 1.8.16 * also look at id attributes on anchor check (Closes SF Bug #741131) Changed files: linkcheck/{linkparser,UrlData}.py * minor parser cleanups Changed files: linkcheck/parser/* 1.8.15 * Fix compile errors with C variable declarations in HTML parser. Thanks to Fazal Majid Changed files: linkcheck/parser/htmlparse.[yc] 1.8.14 * fix old bug in redirects not using the full url. This resulted in errors like (-2, "Name or service not known") Changed files: linkcheck/HttpUrlData.py Closes: SF Bug #729007 * only remove anchors on IIS servers (other servers are doing quite well with anchors... can you spell A-p-a-c-h-e ?) Changed files: linkcheck/{HttpUrlData, UrlData}.py * Parser changes: - correctly propagate and display parsing errors - really cope with missing ">" end tags Changed files: linkcheck/parser/html{lex.l, parse.y}, linkcheck/linkparse.py, linkcheck/UrlData.py * quote urls before a request Changed files: linkcheck/HttpUrlData.py 1.8.13 * fix typo in manpage Changed files: linkchecker.1 * remove anchor from HEAD and GET requests Changed files: linkcheck/{HttpUrlData, UrlData}.py 1.8.12 * convert urlparts to list also on redirect Changed files: linkcheck/HttpUrlData.py 1.8.11 * catch httplib.error exceptions Changed files: linkcheck/HttpUrlData.py * override interactive password question in robotparser.py Changed files: linkcheck/robotparser.py * switch to urllib2.py as default url connect. Changed files: linkcheck/UrlData.py * recompile html parser with flex 2.5.31 Changed files: linkcheck/parser/{htmllex.c,Makefile} 1.8.10 * new option --no-anchor-caching Changed files: linkchecker, linkcheck/{Config.py, UrlData.py}, FAQ * quote empty attribute arguments Changed files: linkcheck/parser/htmllex.[lc] 1.8.9 * recompile with bison 1.875a Changed files: linkcheck/parser/htmlparse.[ch] * remove stpcpy declaration, fixes compile error on RedHat 7.x Changed files: linkcheck/parser/htmlsax.h * clarify keyboard interrupt warning to wait for active connections to finish Changed files: linkcheck/__init__.py * resolve &#XXX; number entity references Changed files: linkcheck/{StringUtil.py,linkname.py} 1.8.8 * All amazon servers block HEAD requests with timeouts. Use GET as a workaround, but issue a warning. Changed files: linkcheck/HttpUrlData.py * restrict CGI access to localhost per default Changed files: lc.cgi, lc.fcgi, lc.sz_fcgi, linkcheck/lc_cgi.py 1.8.7 * #define YY_NO_UNISTD_H on Windows systems, fixes build error with Visual Studio compiler Changed files: setup.py * use python2.2 headers for parser compile, not 2.1. Changed files: linkcheck/parser/Makefile 1.8.6 * include a fixed robotparser.py (from Python 2.2 CVS maint branch) 1.8.5 * fix config.warn to warn Changed files: linkcheck/__init.py * parser changes: o recognise "" HTML comments (seen at Eonline) o recognise "" HTML comments (seen at www.nba.com) o rebuild with flex 2.5.27 Changed files: linkcheck/parser/htmllex.[lc] * added another url exclusion example to the FAQ numerate questions and answers Changed files: FAQ * fix linkchecker exceptions Changed files: linkcheck/{Ftp,Mailto,Nntp,Telnet,}UrlData.py, linkcheck/__init__.py 1.8.4 * Improve error message for failing htmlsax module import Changed files: linkcheck/parser/htmllib.py * Regenerate parser with new bison 1.875 Changed files: linkcheck/parser/htmlparser.c * Some CVS files were not the same as their local counterpart. Something went wrong. Anyway, I re-committed them. Changed files: a lot .py files 1.8.3 * add missing imports for StringUtil in log classes, defer i18n of log field names (used for CGI scripts) Changed files: linkcheck/log/*.py * fixed wrong debug level comparison from > to >= Changed files: linkcheck/Config.py * JavaScript checks in the CGI scripts Changed files: lconline/lc_cgi.html.* Added files: lconline/check.js * Updated documentation with a link restriction example Changed files: linkchecker, linkchecker.1, FAQ * Updated po/pygettext.py to version 1.5, cleaned up some gettext usages. * updated i18n Added files: linkcheck/i18n.py Changed files: all .py files using i18n * Recognise "= 2.2.1, remove httplib. Changed files: setup.py, INSTALL, linkchecker * Add again python-dns, the Debian package maintainer is unresponsive Added files: linkcheck/DNS/*.py Changed files: INSTALL, setup.py * You must now use named constants for ANSII color codes Changed files: linkcheckerrc, linkcheck/log/ColoredLogger.py * Release RedHat 8.0 rpm packages. Changed files: setup.py, MANIFEST.in * remove --robots-txt from manpage, fix HTZP->HTTP typo Changed files: linkchecker.1 1.7.1 * Fix memory leak in HTML parser flushing error path Changed files: htmlparse.y * add custom line and column tracking in parser Changed files: htmllex.l, htmlparse.y, htmlsax.h, htmllib.py * Use column tracking in urldata classes Changed files: UrlData.py, FileUrlData,py, FtpUrlData.py, HostCheckingUrlData.py * Use column tracking in logger classes Changed files: StandardLogger.py CVSLogger.py, ColoredLogger.py, HtmlLogger.py, SqlLogger.py 1.7.0 * Added new HTML parser written in C as a Python extension module. It is faster and it is more fault tolerant. Of course, this means I cannot provide .exe installers any more since the distutils dont provide cross-compilation. 1.6.7 * Removed check for tags codebase attribute, but honor it when checking applet links * Handle tags archive attribute as a comma separated list Closes: SF bug #636802 * Fix a nasty bug in tag searching, which ignored tags with more than one link attribute in it. * Fix concatenation with relative base urls by first joining the parent url. * New commandline option --profile to write profile data. * Add httplib.py from Python CVS 2.1 maintenance branch, which has the skip_host keyword argument I am using now. 1.6.6 * Use the new HTTPConnection/HTTPResponse interface of httplib Closes: SF bug #634679 Changed files: linkcheck/HTTPUrlData.py, linkcheck/HTTPSUrlData.py * Updated the ftp online test Changed files: test/output/test_ftp 1.6.5 * Catch the maximum recursion limit error while parsing links and print an error message instead of bailing out. Changed files: linkcheck/UrlData.py * Fixed Ctrl-C only interrupting one single thread, not the whole program. Changed files: linkcheck/UrlData.py, linkcheck/__init__.py * HTML syntax cleanup and relative cgi form url for the cgi scripts Changed files: lconline/*.html 1.6.4 * Support for ftp proxies Changed files: linkcheck/FtpUrlData.py, linkcheck/HttpUrlData.py Added files: linkcheck/ProxyUrlData.py * Updated german translation 1.6.3: * Generate md5sum checksums for distributed files Changed files: Makefile * use "startswith" string method instead of a regex Changed files: linkchecker, linkcheck/UrlData.py * Add a note about supported languages, updated the documentation. Changed files: README, linkchecker, FAQ * Remove --robots-txt option from documentation, it is per default enabled and you cannot disable it from the command line. Changed files: linkchecker, po/*.po * fix --extern argument creation Changed files: linkchecker, linkcheck/UrlData.py * Print help if PyDNS module is not installed Changed files: linkcheck/UrlData.py * Print information if a proxy was used. Changed files: linkcheck/HttpUrlData.py * Updated german documentation Changed files: po/de.po * Oops, an FTP proxy is not used. Will make it in the next release. Changed files: linkcheck/FtpUrlData.py * Default socket timeout is now 30 seconds (10 was too short) 1.6.2: * Warn about unknown Content-Encodings. Dont parse HTML in this case. * Support deflate content encoding (snatched from Debians reportbug) * Add appropriate Accept-Encoding header to HTTP request. * Updated german translations 1.6.1: * FileUrlData.py: remove searching for links in text files, this is error prone. Just handle *.html and Opera Bookmarks. * Make separate ChangeLog from debian/changelog. For previous changes, see debian/changelog. * Default socket timeout is now 10 seconds * updated linkcheck/timeoutsocket.py to newest version * updated README and INSTALL * s/User-agent/User-Agent/, use same case as other browsers