2004-01-28 22:33:11 +00:00
|
|
|
- use resolve_entities in scanner
|
|
|
|
|
|
2004-01-27 23:17:36 +00:00
|
|
|
- after a failed HEAD request, try always GET and see if that fails
|
|
|
|
|
too
|
|
|
|
|
|
2003-01-05 02:01:50 +00:00
|
|
|
- replace the debug() function with the logging module
|
|
|
|
|
we'll see how we can insert multiple debug levels into this thing
|
2003-08-11 12:50:12 +00:00
|
|
|
|
2003-04-29 14:55:42 +00:00
|
|
|
include some web check and/or spider features:
|
|
|
|
|
- warn if overall size of page (including images/flash/etc.) is too big
|
2004-01-03 21:19:00 +00:00
|
|
|
right now, the page size is only html content
|
2003-04-29 14:55:42 +00:00
|
|
|
- save downloaded pages
|
