Using a scripting language like Ruby, Perl or (my favourite) Python, the data from just about any site can be extracted. URLs can be constructed by the script to download today's pages. This data can then be stored in a database.

Some sites are harder to do than others, but it's really not rocket science if you know what you are doing. Only problem is sites that change their layout will most likely break the script, and it will need fixing.

If you google rent a coder or similar you'll find a plethora of sites where freelance coders will quote you a price for the job. They would almost certainly do it cheaper than I could