Wilbur -- the free file finder

This started out as a simple link check for some of my websites.
I was bored one day and started messing with it -- the result 
being the "messy" code before you. :)
It is time to sit down and think about it for a bit before 
it moves on, that is the stage I am at now.


to install: nothing fancy -- can be run from wherever you
extracted it to.

Operation:

on startup the program reads a file called project.txt in the
current directory. This file just contains a path to where
the other configuration files are located. this will also be
the path where any downloaded files are saved to.

other config files:

dltypes.txt  -> the kinds of files you want to download
ignoretypes.txt -> ignore links containing any of these strings -- good for 
filtering know "dead" sites etc
interesting.txt -> links containing these strings are saved out for later perusal
by you in a file called checkthis.txt

two files are generated while the program is running

resume.txt -- contains the current state of the "to check" que. if you shut
down in the middle of a run, you can just reload this list and start right where
you left off.

skipped.txt -- if the que gets full (20,000 links hardcoded right now) new links are
save to this file and NOT added to the que.

Enjoy!