# robots.txt http://www.vintagegermanmotorcycles.com/ User-agent: * Disallow: /sale/ Disallow: /img/ Disallow: /sold/ Disallow: /tests/ User-agent: webmirror User-agent: webcopy User-agent: vobsub User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT; MS Search 4.0 Robot) Disallow: / User-agent: Slurp User-agent: MSNBot Crawl-delay: 5 User-agent: Googlebot User-agent: Googlebot-Mobile User-agent: Googlebot-Image Allow: / Disallow: