#robots.txt to disallow robots downloading specified pages User-agent: SBIder Disallow: / User-agent: * Disallow: /cgi-bin/ Disallow: /mail/ Disallow: /tmp/ Disallow: /a/Contact.html Disallow: /GOYA/ Disallow: /b/ Disallow: /x/ Disallow: /007/ab01.htm Disallow: /007/pr01.html Disallow: /007/pr02.html Disallow: /007/pr03.html