Crawl The Whole Web

Posted on April 1, 2008 | Leave a comment

Interesting Question – What kind of resources are required for doing a crawl of the whole web? (I’m just looking for ballpark numbers — servers, bandwidth, cost, etc.)

Related posts:

  1. Why Open Source Software / Free Software (OSS/FS)? Look at the Numbers!
This entry was posted in Old Blog. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>