# Disallow these bots User-Agent: Accoona Disallow: / User-Agent: hl_ftien_spider Disallow: / User-Agent: Accoona-AI-Agent Disallow: / User-agent: NetResearchServer Disallow: / User-agent: TurnitinBot Disallow: / User-agent: psbot Disallow: / User-agent: ia_archiver Disallow: / User-agent: boitho.com-dc Disallow: / User-agent: szukacz Disallow: / User-agent: IRLbot Disallow: / User-agent: sohu-search Disallow: / User-Agent: OmniExplorer_Bot Disallow: / User-Agent: DTAAgent Disallow: / # PARTIAL access (All Spiders) User-agent: * Disallow: /404.htm Disallow: /next.html Disallow: /help/ Disallow: /Art/Jscripts/ Disallow: /Art/images/ Disallow: /Art/thumbs/ Disallow: /AudioVisual/ Disallow: /Bob-the-Builder/ Disallow: /Business/Jscripts/ Disallow: /Church/Jscripts/ Disallow: /Doctor/ Disallow: /error/ Disallow: /Flash/ Disallow: /Forms/ Disallow: /Family-Pix/ Disallow: /Forum/ Disallow: /Guest/ Disallow: /help/ Disallow: /History/Coverack/Views/ Disallow: /History/Coverack/Ships/ Disallow: /History/Coverack/LifeBoat/ Disallow: /History/Diggens/message.html Disallow: /History/Downloads/ Disallow: /History/images/ Disallow: /History/Jscripts/ Disallow: /History/message.html Disallow: /Interiors/jscript/ Disallow: /images/ Disallow: /John-Smith/ Disallow: /Jscripts/ Disallow: /Parish-Hall/images/ Disallow: /Parish-Hall/Flash/ Disallow: /Parish-Hall/Jscripts/ Disallow: /Personal/Jscripts/ Disallow: /Youth/Jscripts/ Disallow: /aspnet_client/ Disallow: /cgi-bin/ Disallow: /personalise.html Disallow: /search/ Disallow: /secure/ Disallow: /stats/ Disallow: /Fishing/ Disallow: /uploads/