Part-press part technical article about implementing massive web crawlers in Java

Become.com's Web Crawler: A Massively Scaled Java Technology Application