I’m preparing a chapter that describes our research at Santa Clara University. While reading, I found a paper with a [nowadays] funny comment about the search engine infrastructure of Alta Vista.
The paper says:
Alta Vista’s crawler called the Scooter, runs on a 1.5 GB memory, 30 GB RAID disk, 4x 533 MHz AlphaServer 4100-5/300 with 1 GB/s I/O bandwidth. Scooter connects to the indexing engine Vista, which is a 2 GB memory, 180 GB RAID disk, 2x 533 MHz AlphaServer 4100-5/300.
The most funny part is the statement
The query engine is even more impressive, […].
Ok. This made my day. My virtual machines get more resources today. Time seems to pass quickly, the paper is from 1999.
Chakrabarti, S., Van den Berg, M., & Dom, B. (1999). Focused crawling: a new approach to topic-specific Web resource discovery. Computer Networks,31(11), 1623-1640.