# robots.txt web.archive.org 02/14/2005 User-agent: * index, follow Disallow: /cgi-bin/ SITEMAP: http://www.info.url-indexer.com/sitemap.xml