# User-agent: * Disallow: /stats Disallow: /logs Disallow: /rating Disallow: /foo.html User-agent: Slurp Crawl-delay: 5 User-agent: Googlebot Disallow: /*diary.xml$ User-agent: msnbot Crawl-delay: 5 Disallow: /*diary.xml$ user-agent: msrbot Disallow: / User-agent: ShopWiki Disallow: / User-agent: WebImages Disallow: /