- after a failed HEAD request, try always GET and see if that fails
  too

- replace the debug() function with the logging module
  we'll see how we can insert multiple debug levels into this thing

include some web check and/or spider features:
- warn if overall size of page (including images/flash/etc.) is too big
  right now, the page size is only html content
- save downloaded pages
