User-agent: naverbot Disallow: / User-agent: DKIMRepBot Disallow: / User-agent: DBLBot Disallow: / User-agent: dotbot Disallow: / User-agent: WebAlta Crawler Disallow: / User-agent: WebAlta Crawler/2.0 Disallow: / User-agent: webalta Disallow: / User-agent: Jyxobot/1 Disallow: / User-agent: CazoodleBot/CazoodleBot-0.1 Disallow: / User-agent: McBot/ Disallow: / User-agent: lanshanbot/1.0 Disallow: / User-agent: StackRambler Disallow: / User-agent: ConveraCrawler Disallow: / User-agent: KSbot Disallow: / User-agent: NextGenSearchBot Disallow: / User-agent: NutchCVS Disallow: / User-agent: Shim-Crawler Disallow: / User-agent: linksmanager_bot Disallow: / User-agent: linksmanager Disallow: / User-agent: noxtrumbot Disallow: / User-agent: Steeler/3.1 Disallow: / User-agent: SBIder Disallow: / User-agent: voyager/1.0 Disallow: / User-agent: genieBot Disallow: / User-agent: SkorkaBot/1.0 Disallow: / User-agent: AnanziSpider5 Disallow: / User-agent: Wells Search II Disallow: / User-agent: Port Huron Labs Disallow: / User-agent: Spiderman Disallow: / User-agent: Wget/1.9.1 Disallow: / User-agent: Wget/1.10.2 Disallow: / User-agent: Wget/1.8.2 Disallow: / User-agent: Wget/1.9+cvs-stable (Red Hat modified) Disallow: / User-agent: Wget Disallow: / User-agent: Twiceler Disallow: / User-agent: AuditBot Disallow: / User-agent: larbin Disallow: / User-agent: TurnitinBot Disallow: / User-agent: ichiro Disallow: / User-agent: aipbot Disallow: / User-agent: Baiduspider Disallow: / User-agent: http://www.almaden.ibm.com/cs/crawler Disallow: / User-agent: IRLbot Disallow: / User-agent: LinkWalker Disallow: / User-agent: Mj12bot Disallow: / User-agent: archive.org_bot Disallow: / User-agent: DiamondBot Disallow: / User-agent: oBot Disallow: / User-agent: BruinBot Disallow: / User-agent: LocalcomBot Disallow: / User-agent: OmniExplorer_Bot Disallow: / User-agent: Ultraseek Disallow: / User-agent: sogou spider Disallow: / User-agent: MQBOT/Nutch-0.9-dev Disallow: / User-agent: * Disallow: /cgi-bin/ Disallow: /dwnld/ Disallow: /xml/ Disallow: /cp/ Disallow: /counter/