User-agent: * Allow: / User-agent: googlebot-image Disallow: User-agent: googlebot Disallow: User-agent: googlebot-mobile Disallow: User-agent: slurp Disallow: User-agent: teoma Disallow: User-agent: twiceler Disallow: User-agent: gigabot Disallow: User-agent: scrubby Disallow: User-agent: robozilla Disallow: User-agent: nutch Disallow: User-agent: ia_archiver Disallow: User-agent: baiduspider Disallow: User-agent: naverbot Disallow: User-agent: yeti Disallow: User-agent: yahoo-mmcrawler Disallow: User-agent: psbot Disallow: User-agent: asterias Disallow: User-agent: yahoo-blogs/v3.9 Disallow: User-agent: yahoo-slurp Disallow: User-agent: netcraft Disallow: User-agent: surveyBot Disallow: User-agent: alexa Disallow: User-agent: bspider Disallow: User-agent: yandex bot Disallow: User-agent: emailCollector Disallow: User-agent: MFC_Tear_Sample Disallow: User-agent: MSRBOT Disallow: User-agent: MicrosoftPrototypeCrawler Disallow: User-agent: mozilla/4 Disallow: User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95) Disallow: User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95) Disallow: User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98) Disallow: User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT) Disallow: User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP) Disallow: User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000) Disallow: User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows ME) Disallow: User-agent: mozilla/5 Disallow: User-agent: Scooter Disallow: User-agent: Mediapartners-Google Disallow: User-agent: msnbot-media/1.0 Disallow: User-agent: ArchitextSpider Disallow: User-agent: BBot Disallow: User-agent: AskJeeves Disallow: User-agent: Google AdSense Disallow: User-agent: Inktomi Slurp Disallow: User-agent: Internet Shinchakubin Disallow: User-agent: larbin Disallow: User-agent: LinkChecker Disallow: User-agent: LinkWalker Disallow: User-agent: Motor Disallow: User-agent: MSIECrawler Disallow: User-agent: Speedy Spider Disallow: User-agent: spider Disallow: User-agent: crawl Disallow: User-agent: robot Disallow: User-agent: The Python Robot Disallow: User-agent: Turn It In Disallow: User-agent: Voila Disallow: User-agent: Web Core / Roots Disallow: User-agent: Yandex bot Disallow: User-agent: IsraBot Disallow: User-agent: Orthogaffe Disallow: User-agent: UbiCrawler Disallow: User-agent: DOC Disallow: User-agent: Zao Disallow: User-agent: sitecheck.internetseer.com Disallow: User-agent: Zealbot Disallow: User-agent: SiteSnagger Disallow: User-agent: WebStripper Disallow: User-agent: Fetch Disallow: User-agent: Offline Explorer Disallow: User-agent: Teleport Disallow: User-agent: TeleportPro Disallow: User-agent: linko Disallow: User-agent: HTTrack Disallow: User-agent: Microsoft.URL.Control Disallow: User-agent: Xenu Disallow: User-agent: libwww Disallow: User-agent: ZyBORG Disallow: User-agent: Download Ninja Disallow: User-agent: wget Disallow: User-agent: grub-client Disallow: User-agent: k2spider Disallow: User-agent: WebReaper Disallow: User-agent: Teoma Disallow: User-agent: uTorrent/1500 Disallow: User-agent: 123spider-Bot Disallow: User-agent: Yeti Disallow: User-agent: Science Traveller International 1X/1.0 Disallow: User-agent: AIBOT/2.1 Disallow: User-agent: 2Bone_LinkChecker Disallow: User-agent: 8484 Boston Project v 1.0 Disallow: User-agent: AbachoBOT Disallow: User-agent: ABCdatos BotLink/5.xx.xxx #BBL Disallow: User-agent: BotLink Disallow: User-agent: Aberja Checkomat Disallow: User-agent: abot/0.1 Disallow: User-agent: About/0.1libwww-perl/5.47 Disallow: User-agent: Libby_1.1 Disallow: User-agent: Accelatech RSSCrawler/0.4 Disallow: User-agent: accoona Disallow: User-agent: Accoona-AI-Agent Disallow: User-agent: Ace Explorer Disallow: User-agent: Ack Disallow: User-agent: Acme.Spider Disallow: User-agent: AcoiRobot Disallow: User-agent: Acoon Robot Disallow: User-agent: Acorn/Nutch Disallow: User-agent: Activeworlds Disallow: User-agent: Ad Muncher Disallow: User-agent: Adaxas Disallow: User-agent: MFHttpScan Disallow: User-agent: AESOP_com_SpiderMan Disallow: User-agent: agadine/1.x.x Disallow: User-agent: ahoy Disallow: User-agent: AideRSS/1.0 Disallow: User-agent: Aladin Disallow: User-agent: Aleksika Disallow: User-agent: IAArchiver Disallow: User-agent: AlkalineBOT Disallow: User-agent: Allesklar/0.1 Disallow: User-agent: Alligator Disallow: User-agent: WebBug Disallow: User-agent: amzn_assoc Disallow: User-agent: aranhabot Disallow: User-agent: AmfibiBOT Disallow: User-agent: amibot Disallow: User-agent: emcspider Disallow: User-agent: AnnoMille Disallow: User-agent: Anonymizer/1.1 Disallow: User-agent: Anonymouse Disallow: User-agent: AnswerBus Disallow: User-agent: AnswerChase PROve x.0 Disallow: User-agent: PROve AnswerBot 4.0 Disallow: User-agent: ANTFresco Disallow: User-agent: anthill Disallow: User-agent: antibot-V1.1.5/i586-linux-2.2 Disallow: User-agent: AnzwersCrawl/2.0 Disallow: User-agent: Sqworm Disallow: User-agent: A-Online Disallow: User-agent: Apexoo Disallow: User-agent: Aplix HTTP/1.0.1 Disallow: User-agent: Aport Disallow: User-agent: ArabyBot Disallow: User-agent: Arachnophilia Disallow: User-agent: arale Disallow: User-agent: araneo Disallow: User-agent: AraybOt Disallow: User-agent: aretha Disallow: User-agent: X-Crawler Disallow: User-agent: ariadne Disallow: User-agent: www.arianna.it Disallow: User-agent: Arikus_Spider Disallow: User-agent: arks Disallow: User-agent: ASAHA Search Engine Turkey Disallow: User-agent: Asahina-Antenna Disallow: User-agent: Ask.24x Disallow: User-agent: AskAboutOil/0.06-rcp Disallow: User-agent: asked/Nutch-0.8 Disallow: User-agent: ASpider Disallow: User-agent: ASPSeek Disallow: User-agent: ASSORT Disallow: User-agent: AtlocalBot Disallow: User-agent: ATN_Worldwide Disallow: User-agent: Atomz Disallow: User-agent: atSpider/1.0 Disallow: User-agent: Attentio/Nutch-0.9-dev Disallow: User-agent: augurfind Disallow: User-agent: auresys Disallow: User-agent: autoemailspider Disallow: User-agent: autohttp Disallow: User-agent: AV Fetch Disallow: User-agent: AVSearch-1.0 Disallow: User-agent: AWeb Disallow: User-agent: axadine Disallow: User-agent: AxmoRobot Disallow: User-agent: FastBug Disallow: User-agent: Azureus Disallow: User-agent: BaboomBot Disallow: User-agent: backrub Disallow: User-agent: BackStreet Disallow: User-agent: BaiDuSpider Disallow: User-agent: Bloodhound/Nutch-0.9 Disallow: User-agent: BanBots Disallow: User-agent: Barca Disallow: User-agent: BarraHomeCrawler Disallow: User-agent: spider.batsch.com Disallow: User-agent: BigBrother Disallow: User-agent: bbot Disallow: User-agent: bCentral Disallow: User-agent: ImageWalker Disallow: User-agent: BDNcentral Crawler v2.3 Disallow: User-agent: BeamMachine/0.5 Disallow: User-agent: beautybot/1.0 Disallow: User-agent: BebopBot/2.5.1 Disallow: User-agent: bigbrother Disallow: User-agent: Big Brother Disallow: User-agent: BigCliqueBOT/1.03-dev Disallow: User-agent: BIGLOTRON Disallow: User-agent: Bigsearch.ca/Nutch-0.9-dev Disallow: User-agent: Bilbo/2.3b-UNIX Disallow: User-agent: BilgiBetaBot/0.8-dev Disallow: User-agent: Custom Disallow: User-agent: Bitacle bot/1.1 Disallow: User-agent: Biyubi Disallow: User-agent: Bjaaland Disallow: User-agent: SBL-BOT Disallow: User-agent: BlackWidow Disallow: User-agent: Blaiz-Bee Disallow: User-agent: BlitzBOT@tricus.net Disallow: User-agent: BlockNote Disallow: User-agent: Naamah Disallow: User-agent: BlogBridge Disallow: User-agent: BlogBot Disallow: User-agent: Bloglines Disallow: User-agent: Bloglines-Images Disallow: User-agent: BlogsNowBot Disallow: User-agent: blogWatcher_Spider Disallow: User-agent: BlogzIce Disallow: User-agent: Ukonline Disallow: User-agent: bluefish Disallow: User-agent: BMCLIENT Disallow: User-agent: BMLAUNCHER Disallow: User-agent: BoardReader Favicon Fetcher Disallow: User-agent: Bobby Disallow: User-agent: boitho.com-dc/0.xx Disallow: User-agent: Bookdog Disallow: User-agent: borg-bot/0.9 Disallow: User-agent: boxseabot Disallow: User-agent: BDFetch Disallow: User-agent: BravoBrian Disallow: User-agent: timboBot Disallow: User-agent: brightnet Disallow: User-agent: BruinBot Disallow: User-agent: BSDSeek/1.0 Disallow: User-agent: Btbot Disallow: User-agent: BuildCMS Disallow: User-agent: Bulkfeeds Disallow: User-agent: BullsEye  Disallow: User-agent: BunnySlippers  Disallow: User-agent: Norbert the Spider Disallow: User-agent: BurstFindCrawler Disallow: User-agent: Buscaplus Robi Disallow: User-agent: bdcindexer_2.6.2 Disallow: User-agent: www.business-socket.com registry verify/1.x  Disallow: User-agent: BW-C-2.0  Disallow: User-agent: cactvschemistryspider Disallow: User-agent: WebQL  Disallow: User-agent: calif Disallow: User-agent: CamelHttpStream Disallow: User-agent: Cassandra Disallow: User-agent: Catall Disallow: User-agent: CazoodleBot/CazoodleBot-0.1 Disallow: User-agent: ccubee/x.x  Disallow: User-agent: CE-Preload  Disallow: User-agent:parallelContextFocusCrawler1.1parallelContextFocusCrawler1.1 Disallow: User-agent: cfetch Disallow: User-agent: CFNetwork Disallow: User-agent: cg-eye Disallow: User-agent: Checkbot Disallow: User-agent: CheckLinks Disallow: User-agent: CheckWeb  Disallow: User-agent: Shim Disallow: User-agent: Chilkat/1.0.0 Disallow: User-agent: searchbot admin@google.com  Disallow: User-agent: ChristCRAWLER Disallow: User-agent: CHttpClient Disallow: User-agent: churl Disallow: User-agent: cienciaficcion Disallow: User-agent: CipinetBot Disallow: User-agent: CJ Spider Disallow: User-agent: CJB.NET Disallow: User-agent: ClariaBot Disallow: User-agent: DiamondBot  Disallow: User-agent: OliverPerry  Disallow: User-agent: Claymont Disallow: User-agent: Clushbot Disallow: User-agent: CMC/0.01 Disallow: User-agent: COAST WebMaster Pro/4.x.x.xx Disallow: User-agent: oBot Disallow: User-agent: CoBITSProbe  Disallow: User-agent: Cocoal.icio.us/1.0 Disallow: User-agent: MSRBOT  Disallow: User-agent: ColdFusion  Disallow: User-agent: collage.cgi/1.82  Disallow: User-agent: Collective Disallow: User-agent: combine Disallow: User-agent: cometrics-bot Disallow: User-agent: Crawler Disallow: User-agent: robi Disallow: User-agent: Comrite Disallow: User-agent: confuzzledbot Disallow: User-agent: ContactBot Disallow: User-agent: ContentSmartz  Disallow: User-agent: contype  Disallow: User-agent: Convera Disallow: User-agent: CoolBot Disallow: User-agent: CoralWebPrx Disallow: User-agent: voyager Disallow: User-agent: carleson Disallow: User-agent: CoteoNutchCrawler Disallow: User-agent: CougarSearch Disallow: User-agent: CrawlConvera Disallow: User-agent: Crawllybot Disallow: User-agent: CreativeCommons/0.06-dev Disallow: User-agent: CrocCrawler Disallow: User-agent: csci_b659 Disallow: User-agent: Cuam Ver0.050bx  Disallow: User-agent: Cuasarbot Disallow: User-agent: curl Disallow: User-agent: CurryGuide Disallow: User-agent: cusco Disallow: User-agent: Custo Disallow: User-agent: Cyberdog Disallow: User-agent: CyberSpyder Disallow: User-agent: DataCha0s/2.0  Disallow: User-agent: Web Image Collector  Disallow: User-agent: DataparkSearch Disallow: User-agent: DataSpear Disallow: User-agent: RaBot Disallow: User-agent: ELI/20070402:2.0 Disallow: User-agent: Talkro Web-Shot/1.0 Disallow: User-agent: daypopbot Disallow: User-agent: dbDig Disallow: User-agent: DBrowse 1.4b  Disallow: User-agent: DC-Sakura/x.xx  Disallow: User-agent: dCSbot/1.1  Disallow: User-agent: DDD  Disallow: User-agent: dds explorer Disallow: User-agent: online link validator Disallow: User-agent: deepak-USC/ISI  Disallow: User-agent: DeepIndex  Disallow: User-agent: del.icio.us-thumbnails/1.0 Mozilla/5.0 Disallow: User-agent: DeleGate Disallow: User-agent: Denmex Disallow: User-agent: RedCell/0.1 Disallow: User-agent: desertrealm, desert realm Disallow: User-agent: DevComponents.com Disallow: User-agent: deweb Disallow: User-agent: DiaGem/1.1 Disallow: User-agent: blindekuh Disallow: User-agent: DIE-KRAEHE- META-SEARCH-ENGINE Disallow: User-agent: dienstspider Disallow: User-agent: digger Disallow: User-agent: Digger/1.0 JDK/1.3.0rc3  Disallow: User-agent: Digimarc WebReader Disallow: User-agent: DIIbot Disallow: User-agent: Pompos Disallow: User-agent: grabber Disallow: User-agent: DISCo Pump x.x  Disallow: User-agent: disco/Nutch-0.9 Disallow: User-agent: DittoSpyder  Disallow: User-agent: DeadLinkCheck Disallow: User-agent: dlman  Disallow: User-agent: dnabot Disallow: User-agent: DocZilla Disallow: User-agent: dodgebot Disallow: User-agent: Jack  Disallow: User-agent: DonutP Disallow: User-agent: Doubanbot Disallow: User-agent: www.doweb.co.uk Disallow: User-agent: Download Demon Disallow: User-agent: DownloadDirect.1.0 Disallow: User-agent: Download Druid Disallow: User-agent: Download Express Disallow: User-agent: Download Master Disallow: User-agent: Download Ninja 3.0 Disallow: User-agent: Download Wonder Disallow: User-agent: Download.exe(1.1) Disallow: User-agent: Download-Tipp Linkcheck Disallow: User-agent: EyeCatcher Disallow: User-agent: Dr.Web ® Disallow: User-agent: DragonBot Disallow: User-agent: Dragonfly Disallow: User-agent: DreamPassport/3.0 Disallow: User-agent: Drecombot/1.0 Disallow: User-agent: Drupal Disallow: User-agent: DSurf15a 01 Disallow: User-agent: DSurf15a 71 Disallow: User-agent: DSurf15a 81  Disallow: User-agent: DSurf15a VA  Disallow: User-agent: dtSearchSpider  Disallow: User-agent: DTAAgent  Disallow: User-agent: Dual Proxy  Disallow: User-agent: Dumbot Disallow: User-agent: dwcp Disallow: User-agent: EARTHCOM.info Disallow: User-agent: ebiness Disallow: User-agent: EBrowse Disallow: User-agent: eCatch Disallow: User-agent: EchO!/2.0  Disallow: User-agent: ecollector Disallow: User-agent: EgotoBot Disallow: User-agent: Maintenance Disallow: User-agent: ejupiter.com  Disallow: User-agent: elfinbot Disallow: User-agent: elfbot/1.0 Disallow: User-agent: ELinks Disallow: User-agent: NextGenSearchBot Disallow: User-agent: Emacs-w3/v[0-9\.]+ Disallow: User-agent: EmailSiphon  Disallow: User-agent: EmailWolf Disallow: User-agent: EmeraldShield.com WebBot  Disallow: User-agent: EMPAS_ROBOT  Disallow: User-agent: endo/1.0 Disallow: User-agent: Enfish Tracker  Disallow: User-agent: Search/1.0 Disallow: User-agent: Enterprise_Search/1.0  Disallow: User-agent: envolk Disallow: User-agent: EroCrawler  Disallow: User-agent: esculapio Disallow: User-agent: e-sense Disallow: User-agent: ESISmartSpider  Disallow: User-agent: e-SocietyRobot Disallow: User-agent: esther Disallow: User-agent: eStyleSearch Disallow: User-agent: ESurf15a 15  Disallow: User-agent: EuripBot Disallow: User-agent: Arachnoidea Disallow: User-agent: EvaalSE Disallow: User-agent: eventax Disallow: User-agent: nt Evliya Celebi v0.151 - http://ilker.ulak.net.tr Disallow: User-agent: ExactSearch  Disallow: User-agent: eseek-larbin Disallow: User-agent: ExactSeek Crawler Disallow: User-agent: Exabot Disallow: User-agent: Exabot-Images/1.0  Disallow: User-agent: Excalibur Disallow: User-agent: ArchitextSpider  Disallow: User-agent: Execrawl Disallow: User-agent: Extreme Picture Finder  Disallow: User-agent: The Expert HTML Source Viewer Disallow: User-agent: Expired Domain Sleuth  Disallow: User-agent: Express WebPictures Disallow: User-agent: ExtractorPro  Disallow: User-agent: Factbot 1.09 (see http://www.factbites.com/webmasters.php)  Disallow: User-agent: FaEdit/2.0.x  Disallow: User-agent: FairAd Client  Disallow: User-agent: FARK.com link verifier  Disallow: User-agent: Fast Disallow: User-agent: fastbot Disallow: User-agent: fastcrawler Disallow: User-agent: FavIconizer  Disallow: User-agent: favo.eu Disallow: User-agent: FavOrg  Disallow: User-agent: Favorites Sweeper Disallow: User-agent: Faxobot Disallow: User-agent: FDM Disallow: User-agent: Feed Seeker Bot Disallow: User-agent: Feed::Find/0.0x  Disallow: User-agent: Feed24 Disallow: User-agent: Feedable Disallow: User-agent: FeedChecker Disallow: User-agent: FeedForAll rss2html.php v2  Disallow: User-agent: BlogMap Disallow: User-agent: Feedshow Disallow: User-agent: FeedshowOnline Disallow: User-agent: Feedster Disallow: User-agent: FeedZcollector Disallow: User-agent: FELIX IDE Disallow: User-agent: fetch libfetch Disallow: User-agent: NewsGator FetchLinks Disallow: User-agent: ESI Disallow: User-agent: fido Disallow: User-agent: Filangy Disallow: User-agent: fileboost Disallow: User-agent: FileHeap! file downloader Disallow: User-agent: FileHound Disallow: User-agent: JobSpider_BA Disallow: User-agent: FindAnISP.com_ISP_Finder_v99a  Disallow: User-agent: RoboPal Disallow: User-agent: DIIbot/1.2  Disallow: User-agent: FineBot  Disallow: User-agent: Finjan-prefetch  Disallow: User-agent: Firefly Disallow: User-agent: KIT-Fireball/2.0  Disallow: User-agent: FirstGov.gov Disallow: User-agent: firstsbot  Disallow: User-agent: Fish-Search-Robot Disallow: User-agent: Flapbot Disallow: User-agent: FLATARTS_FAVICO  Disallow: User-agent: flatlandbot Disallow: User-agent: FlickBot 2.0 RPT-HTTPClient/0.3-3  Disallow: User-agent: FDSE Disallow: User-agent: flyindex.net 1.0/http://www.flyindex.net  Disallow: User-agent: fly/6.01 libwww/4.0D  Disallow: User-agent: FocusedSampler Disallow: User-agent: Fooky.com/ScorpionBot/ScoutOut Disallow: User-agent: fouineur Disallow: User-agent: Franklin Locator 1.8  Disallow: User-agent: Freecrawl Disallow: User-agent: FreeFind.com-SiteSearchEngine Disallow: User-agent: FFC Trap Door Spider  Disallow: User-agent: FreshDownload Disallow: User-agent: FreshNotes Disallow: User-agent: FSurf15a 01  Disallow: User-agent: Full Web Bot 0416B  Disallow: User-agent: Full Web Bot 0516B  Disallow: User-agent: Full Web Bot 2816B  Disallow: User-agent: FunnelWeb-1.0 Disallow: User-agent: FuseBulb Disallow: User-agent: FyberSpider Disallow: User-agent: Gagglebot  Disallow: User-agent: Gaisbot Disallow: User-agent: GalaxyBot Disallow: User-agent: Gallent Disallow: User-agent: gamekitbot Disallow: User-agent: GameSpyHTTP/1.0  Disallow: User-agent: gammaSpider Disallow: User-agent: GammaSpider Disallow: User-agent: gazz Disallow: User-agent: gcreep Disallow: User-agent: geckobot  Disallow: User-agent: generic_crawler Disallow: User-agent: genieBot Disallow: User-agent: GeoBot Disallow: User-agent: GeonaBot Disallow: User-agent: geourl Disallow: User-agent: GeoURLBot Disallow: User-agent: getbot Disallow: User-agent: GetRight/4.5xx Disallow: User-agent: GetRight Disallow: User-agent: Getterrobo-Plus Disallow: User-agent: GetURL.rexx v1.05 Disallow: User-agent: GetWeb Disallow: User-agent: MicroBaz Disallow: User-agent: gigabaz Disallow: User-agent: Gigabot Disallow: User-agent: GigabotSiteSearch Disallow: User-agent: RAMPyBot Disallow: User-agent: Ocelli/1.x Disallow: User-agent: GNODSPIDER Disallow: User-agent: Go!Zilla Disallow: User-agent: GOFORITBOT Disallow: User-agent: Goblin Disallow: User-agent: GoldenFeed Disallow: User-agent: Goldfire Disallow: User-agent: golem Disallow: User-agent: gonzo1 Disallow: User-agent: mogimogi Disallow: User-agent: moget Disallow: User-agent: ichiro Disallow: User-agent: Goofer Disallow: User-agent: Feedfetcher-Google Disallow: User-agent: gsa-crawler Disallow: User-agent: Nokia-WAPToolkit Disallow: User-agent: Big Fish Disallow: User-agent: Go-Ahead-Got-It Disallow: User-agent: GrapeFX Disallow: User-agent: grapnel Disallow: User-agent: monkeyagent Disallow: User-agent: GreatNews Disallow: User-agent: GreenBrowser Disallow: User-agent: gridwell Disallow: User-agent: griffon Disallow: User-agent: GrigorBot Disallow: User-agent: Gromit Disallow: User-agent: grub Disallow: User-agent: LEIA Disallow: User-agent: GSiteCrawler Disallow: User-agent: Guestbook Auto Submitter Disallow: User-agent: gulper Disallow: User-agent: Gungho Disallow: User-agent: GurujiBot Disallow: User-agent: Hämähäkki Disallow: User-agent: hambot Disallow: User-agent: HappyFunBot Disallow: User-agent: harvest Disallow: User-agent: Harvest-NG Disallow: User-agent: Haste Disallow: User-agent: Hatena Disallow: User-agent: havIndex Disallow: User-agent: Helix Disallow: User-agent: archive.org_bot Disallow: User-agent: InternetArchive Disallow: User-agent: hgrepurl Disallow: User-agent: AITCSRobot/1.1 Disallow: User-agent: HiDownload Disallow: User-agent: Hippias Disallow: User-agent: Hitwise Disallow: User-agent: octopus Disallow: User-agent: HLoader Disallow: User-agent: HomePageSearch Disallow: User-agent: Homerbot Disallow: User-agent: nt Hometown Spider Pro Disallow: User-agent: Honda-Search Disallow: User-agent: HooWWWer Disallow: User-agent: Toutatis Disallow: User-agent: HotJava Disallow: User-agent: Hotzonu Disallow: User-agent: HPL/Nutch Disallow: User-agent: htdig Disallow: User-agent: Html Link Disallow: User-agent: HTMLParser Disallow: User-agent: pitkow Disallow: User-agent: HTML2JPG Disallow: User-agent: HTMLgobble v2.2 Disallow: User-agent: HTTP Retriever Disallow: User-agent: ParaSite Disallow: User-agent: HTTP::Lite Disallow: User-agent: HTTPResume Disallow: User-agent: httpunit Disallow: User-agent: Hybrid Disallow: User-agent: HyperEstraier Disallow: User-agent: hyperdecontextualizer Disallow: User-agent: irobot Disallow: User-agent: i1searchbot Disallow: User-agent: iajabot Disallow: User-agent: iaskspider2 Disallow: User-agent: iaskspider Disallow: User-agent: LotusDiscovery Disallow: User-agent: t IBM_Planetwide Disallow: User-agent: almaden Disallow: User-agent: iCab Disallow: User-agent: ICC-Crawler Disallow: User-agent: BlogSearch Disallow: User-agent: IconSurf Disallow: User-agent: ICRA_label_spider Disallow: User-agent: icsbot-0.1 Disallow: User-agent: iearthworm Disallow: User-agent: IEFav172Free Disallow: User-agent: iGetter Disallow: User-agent: IlTrovatore Disallow: User-agent: ilial/Nutch Disallow: User-agent: IlseBot Disallow: User-agent: INGRID Disallow: User-agent: image.kapsi.net/1.0 Disallow: User-agent: Imagelock Disallow: User-agent: ImageVisu/ Disallow: User-agent: Incutio Disallow: User-agent: NetResearchServer Disallow: User-agent: IncyWincy/1.0b1 Disallow: User-agent: IncyWincy Disallow: User-agent: IndexTheWeb Disallow: User-agent: Industry Program Disallow: User-agent: vb wininet Disallow: User-agent: Inet library Disallow: User-agent: InetURL Disallow: User-agent: infoConveraCrawler Disallow: User-agent: InfoFly Disallow: User-agent: InfoLink Disallow: User-agent: INFOMINE Disallow: User-agent: DataFountains Disallow: User-agent: InfoNaviRobot Disallow: User-agent: Informant Disallow: User-agent: Sidewinder Disallow: User-agent: InfoSeek Robot 1.0 Disallow: User-agent: Ultraseek Disallow: User-agent: Infoseek Sidewinder Disallow: User-agent: InfoSpiders Disallow: User-agent: INGRID/0.1 Disallow: User-agent: URL_Spider_Pro Disallow: User-agent: InnerpriseBot Disallow: User-agent: Rational SiteCheck Disallow: User-agent: xyro Disallow: User-agent: Jeode Disallow: User-agent: Insitor Disallow: User-agent: Insitornaut Disallow: User-agent: inspectorwww Disallow: User-agent: IAGENT/1.0 Disallow: User-agent: Blogpulse Disallow: User-agent: Interarchy Disallow: User-agent: Internet Cruiser Robot Disallow: User-agent: InternetLinkAgent Disallow: User-agent: sharp-info-agent Disallow: User-agent: intraVnews Disallow: User-agent: IP*Works! Disallow: User-agent: IP2LocationBot Disallow: User-agent: IP2MapBot Disallow: User-agent: IpselonBot Disallow: User-agent: Iria Disallow: User-agent: IRLbot Disallow: User-agent: Iron33 Disallow: User-agent: IrssiUrlLog Disallow: User-agent: Irvine Disallow: User-agent: iSiloX Disallow: User-agent: IsraeliSearch/1.0 Disallow: User-agent: istarthere Disallow: User-agent: isurf Disallow: User-agent: iVia Disallow: User-agent: IWAgent Disallow: User-agent: Jakarta Commons-HttpClient Disallow: User-agent: Jambot Disallow: User-agent: ideare Disallow: User-agent: SearchTone2.0 Disallow: User-agent: JavaBee Disallow: User-agent: Jayde Disallow: User-agent: JBot Disallow: User-agent: jBrowser Disallow: User-agent: JCheckLinks Disallow: User-agent: jcrawler Disallow: User-agent: JetCar Disallow: User-agent: Jetbot Disallow: User-agent: Jigsaw Disallow: User-agent: jobo Disallow: User-agent: Jobot/0.1alpha libwww-perl/4.0 Disallow: User-agent: JoeBot/x.x Disallow: User-agent: JordoMedia Disallow: User-agent: Journster Disallow: User-agent: J-PHONE Disallow: User-agent: JRTwine Disallow: User-agent: jumpstation Disallow: User-agent: Jyxobot Disallow: User-agent: KakleBot Disallow: User-agent: Kapere Disallow: User-agent: Katipo/1.0 Disallow: User-agent: Kazehakase Disallow: User-agent: KDD-Explorer Disallow: User-agent: Kenjin Disallow: User-agent: EasyDL Disallow: User-agent: KFSW-Bot Disallow: User-agent: kilroy Disallow: User-agent: kinjabot Disallow: User-agent: kinja-imagebot Disallow: User-agent: KIT-Fireball Disallow: User-agent: Klondike Disallow: User-agent: K-Meleon Disallow: User-agent: KnowItAll Disallow: User-agent: Knowledge Disallow: User-agent: Ksbot Disallow: User-agent: ko_yappo_robot Disallow: User-agent: Mariner/5.1b Disallow: User-agent: Kontiki Disallow: User-agent: Krugle Disallow: User-agent: kuloko-bot Disallow: User-agent: kulturarw3 Disallow: User-agent: KummHttp/1.1 Disallow: User-agent: label-grabber Disallow: User-agent: Lachesis Disallow: User-agent: lanshanbot Disallow: User-agent: LapozzBot/1.4 Disallow: User-agent: IPiumBot Disallow: User-agent: lawinfo-crawler Disallow: User-agent: lcabotAccept Disallow: User-agent: LECodeChecker Disallow: User-agent: LeechGet Disallow: User-agent: legs Disallow: User-agent: LetsCrawl Disallow: User-agent: LexiBot Disallow: User-agent: Mata Hari/2.00 Disallow: User-agent: LNSpiderguy Disallow: User-agent: libcurl-agent/1.0 Disallow: User-agent: LibertyW Disallow: User-agent: ActiveBookmark Disallow: User-agent: libWeb/clsHTTP Disallow: User-agent: Liferea Disallow: User-agent: LightningDownload Disallow: User-agent: Lincoln State Web Browser Disallow: User-agent: Link Valet Disallow: User-agent: Linkidator Disallow: User-agent: link_check3 Disallow: User-agent: LinkAlarm Disallow: User-agent: Linkbot Disallow: User-agent: LinkCheck Disallow: User-agent: BeebwareDirectory Disallow: User-agent: LinkLint-checkonly Disallow: User-agent: linknzbot Disallow: User-agent: AgentName/0.1 Disallow: User-agent: LinkPimpin Disallow: User-agent: Links Disallow: User-agent: Links4US-Crawler Disallow: User-agent: LinkScan Disallow: User-agent: LinksManager Disallow: User-agent: LinkSonar Disallow: User-agent: LinkSweeper Disallow: User-agent: ListBidBot Disallow: User-agent: LjSEEK Picture-Bot Disallow: User-agent: Llaut/1.0 Disallow: User-agent: LMQueueBot/0.2 Disallow: User-agent: lmspider Disallow: User-agent: LocalcomBot Disallow: User-agent: LocalBot/1.0 Disallow: User-agent: Lockon Disallow: User-agent: Lockstep Disallow: User-agent: logo_gif_crawler Disallow: User-agent: Seeker.lookseek.com Disallow: User-agent: MantraAgent Disallow: User-agent: MARTINI Disallow: User-agent: smartwit.com Disallow: User-agent: luchs Disallow: User-agent: Lunascape Disallow: User-agent: Lycos/x.x Disallow: User-agent: Mac Finder Disallow: User-agent: macworm Disallow: User-agent: MagicWML/1.0 Disallow: User-agent: Mag-Net Disallow: User-agent: Magpie/1.0 Disallow: User-agent: MagpieRSS Disallow: User-agent: Mail.Ru/1.0 Disallow: User-agent: MJ12bot Disallow: User-agent: mammoth Disallow: User-agent: MapoftheInternet Disallow: User-agent: OSSProxy Disallow: User-agent: Marvin v0.3 Disallow: User-agent: marvin Disallow: User-agent: SurfMaster Disallow: User-agent: MasterSeek Disallow: User-agent: mattie Disallow: User-agent: Spider/maxbot.com Disallow: User-agent: maxomobot/dev-20051201 Disallow: User-agent: McBot/5.001 Disallow: User-agent: MediaCrawler-1.0 Disallow: User-agent: mediafox Disallow: User-agent: Spinne Disallow: User-agent: MegaSheep Disallow: User-agent: Mercator Disallow: User-agent: Mercator-Scrub Disallow: User-agent: MerzScope Disallow: User-agent: RRC (crawler_admin@bigfoot.com) Disallow: User-agent: flunky Disallow: User-agent: Metaeuro Disallow: User-agent: MetaGer_PreChecker0.1 Disallow: User-agent: MetagerBot/0.8-dev Disallow: User-agent: MetaGer-LinkChecker Disallow: User-agent: XRL Disallow: User-agent: Mqbot Disallow: User-agent: Metaspinner Disallow: User-agent: metatagsdir Disallow: User-agent: Intelix Disallow: User-agent: MindCrawler Disallow: User-agent: Mindjet Disallow: User-agent: miniRank/1.6 Disallow: User-agent: minibot Disallow: User-agent: MiracleAlphaTest Disallow: User-agent: ExperimentalHenrytheMiragoRobot Disallow: User-agent: HeinrichderMiragoRobot Disallow: User-agent: HenrytheMiragoRobot Disallow: User-agent: HenriLeRobotMirago Disallow: User-agent: Missauga Locate Disallow: User-agent: Missouri College Browse Disallow: User-agent: Mister Pix II Disallow: User-agent: Misterbot-Nutch/0.7.1 Disallow: User-agent: Miva Disallow: User-agent: Felix - Mixcat Crawler (+http://mixcat.com) Disallow: User-agent: Morris - Mixcat Disallow: User-agent: Mizzu Labs 2.2 Disallow: User-agent: MnogoSearch Disallow: User-agent: udmsearch Disallow: User-agent: Mo College 1.9 Disallow: User-agent: moiNAG Disallow: User-agent: MojeekBot Disallow: User-agent: MOMspider/1.00 libwww-perl/0.40 Disallow: User-agent: monogol Disallow: User-agent: Monster Disallow: User-agent: MoonBrowser Disallow: User-agent: Moreoverbot Disallow: User-agent: holmes Disallow: User-agent: Motoricerca-Robots.txt-Checker/1.0 Disallow: User-agent: mozDex Disallow: User-agent: Mozi! Disallow: User-agent: Mouse-House/7.4 Disallow: User-agent: msnbot Disallow: User-agent: multiBlocker Disallow: User-agent: multicrawler Disallow: User-agent: MultiText Disallow: User-agent: muncher Disallow: User-agent: muninn Disallow: User-agent: MuscatFerret Disallow: User-agent: MVAClient Disallow: User-agent: MwdSearch Disallow: User-agent: Snappy/1.1 Disallow: User-agent: myDaemon Disallow: User-agent: Mylinea.com Disallow: User-agent: UIowaCrawler/1.0 Disallow: User-agent: NP Disallow: User-agent: aipbot Disallow: User-agent: naoFavicon4IE/1.xx Disallow: User-agent: NASA Search 1.0 Disallow: User-agent: NationalDirectory-WebSpider Disallow: User-agent: dloader Disallow: User-agent: NABOT/5.0 Disallow: User-agent: NaverBot-1.0 Disallow: User-agent: nabot_1.0 Disallow: User-agent: Cowbot Disallow: User-agent: NavissoBot Disallow: User-agent: NDSpider/1.5 Disallow: User-agent: Nebullabot/2.2 Disallow: User-agent: NEC Research Agent Disallow: User-agent: NEC-Hayek Disallow: User-agent: NEC-MeshExplorer Disallow: User-agent: Nederland.zoek Disallow: User-agent: Francis Disallow: User-agent: NetAnts/1.2x Disallow: User-agent: netcarta Disallow: User-agent: NETCOMplete Disallow: User-agent: NetinfoBot Disallow: User-agent: NetLookout Disallow: User-agent: Netluchs/0.8-dev Disallow: User-agent: NetMechanic Disallow: User-agent: NetNoseCrawler Disallow: User-agent: Netprospector Disallow: User-agent: NetPumper/x.xx Disallow: User-agent: NetScoop Disallow: User-agent: NSPlayer/10.0.0.xxxx WMFSDK/10.0 Disallow: User-agent: NetWhatCrawler/0.06-dev Disallow: User-agent: SmartDownload Disallow: User-agent: NetZippy Disallow: User-agent: NeuralBot/0.2 Disallow: User-agent: newscan-online Disallow: User-agent: newsearchengine Disallow: User-agent: NetNewsWire Disallow: User-agent: NewsGatorOnline Disallow: User-agent: findlinks/x.xxx Disallow: User-agent: NextopiaBOT Disallow: User-agent: NG-Search Disallow: User-agent: NHSEWalker/3.0 Disallow: User-agent: nicebot Disallow: User-agent: NICO Disallow: User-agent: Nikita the Spider Disallow: User-agent: NITLE Blog Spider Disallow: User-agent: Nitro Disallow: User-agent: Noago Spider Disallow: User-agent: Nocilla Disallow: User-agent: NokodoBot/1.x Disallow: User-agent: Nomad-V2.x Disallow: User-agent: gulliver Disallow: User-agent: Gulliver Disallow: User-agent: noxtrumbot/1.0 Disallow: User-agent: noyona_0_1 Disallow: User-agent: Nsauditor/1.x Disallow: User-agent: CheckUrl Disallow: User-agent: nttdirectory_robot/0.9 Disallow: User-agent: DoCoMo Disallow: User-agent: Nucleus SiteList LinkChecker/1.1 Disallow: User-agent: nuSearch Spider Disallow: User-agent: Nutch Disallow: User-agent: NutchCrawler Disallow: User-agent: explorersearch Disallow: User-agent: obidos-bot Disallow: User-agent: ObjectsSearch Disallow: User-agent: Occam Disallow: User-agent: Octopus Disallow: User-agent: Octora Beta Bot - www.octora.com Disallow: User-agent: Jabot Disallow: User-agent: Offline Explorer 1.* Disallow: User-agent: JetBrains Disallow: User-agent: OmniExplorer_Bot/1.0x Disallow: User-agent: Online24-Bot Disallow: User-agent: Slarp/0.1 Disallow: User-agent: onCHECK Linkchecker Disallow: User-agent: OntoSpider/1.0 Disallow: User-agent: OpenISearch Disallow: User-agent: Openbot/3.0+ Disallow: User-agent: Openbot Disallow: User-agent: DigOut4U Disallow: User-agent: OpenTaggerBot Disallow: User-agent: OpenTextSiteCrawler Disallow: User-agent: OpenWebSpider Disallow: User-agent: OpidooBOT Disallow: User-agent: Oracle iMTCrawler Disallow: User-agent: Oracle Ultra Search Disallow: User-agent: OrangeSpider Disallow: User-agent: Orbsearch/1.0 Disallow: User-agent: Orbiter/T-2.0 Disallow: User-agent: Orca Disallow: User-agent: OutfoxBot Disallow: User-agent: OutfoxMelonBot Disallow: User-agent: Overture-WebCrawler Disallow: User-agent: OWR_Crawler Disallow: User-agent: ozelot Disallow: User-agent: packrat Disallow: User-agent: PageBitesHyperBot Disallow: User-agent: pageboy Disallow: User-agent: Pagebull Disallow: User-agent: PagmIEDownload Disallow: User-agent: puf Disallow: User-agent: Patric/0.01a Disallow: User-agent: Patwebbot Disallow: User-agent: pavuk/0.9pl29b i686-pc-linux-gnu Disallow: User-agent: Pbrowse Disallow: User-agent: PEAR HTTP_Request Disallow: User-agent: PEERbot Disallow: User-agent: pegasus Disallow: User-agent: PeopleChat/Search_Engine Disallow: User-agent: perlcrawler Disallow: User-agent: PEval 1.4b Disallow: User-agent: phantom Disallow: User-agent: phpdig Disallow: User-agent: PicoSearch/1.0 Disallow: User-agent: psbot/0.1 Disallow: User-agent: Piffany_Web_Scraper_v0.x Disallow: User-agent: Marketwave Hit List Disallow: User-agent: HitList Disallow: User-agent: piltdownman Disallow: User-agent: Pimptrain Disallow: User-agent: PingALink Monitoring Services 1.0 Disallow: User-agent: Pingdom GIGRIB Disallow: User-agent: CrawlerBoy Pinpoint.com Disallow: User-agent: Pioneer Disallow: User-agent: pipeLiner Disallow: User-agent: Pita Disallow: User-agent: Pizilla++ ver 2.45 Disallow: User-agent: Plagger Disallow: User-agent: PlagiarBot Disallow: User-agent: PlantyNet_WebRobot_V1.9 dhkang@plantynet.com Disallow: User-agent: plinki Disallow: User-agent: PluckFeedCrawler/2.0 Disallow: User-agent: Pluggd/Nutch-0.9 Disallow: User-agent: Pockey-GetHTML/4.12.0 Disallow: User-agent: POE-Component-Client-HTTP/0.64 Disallow: User-agent: Poirot Disallow: User-agent: polybot Disallow: User-agent: Popdexter/1.0 Disallow: User-agent: Robot/www.pj-search.com Disallow: User-agent: Poppi Disallow: User-agent: gestaltIconoclast/1.0 libwww-FM/2.17 Disallow: User-agent: Port Huron Labs Disallow: User-agent: pjspider Disallow: User-agent: PortalBSpider Disallow: User-agent: Pjspider Disallow: User-agent: portalmmm/2.0 S500i(c20;TB) Disallow: User-agent: pd02_1.0.0 pd02_1.0.0@dzimi@post.sk Disallow: User-agent: potbot 1.0 Disallow: User-agent: PrivacyFinder Disallow: User-agent: Privoxy/3.0 Disallow: User-agent: Privoxy Disallow: User-agent: Production Bot 0116B Disallow: User-agent: PWeBot Disallow: User-agent: ProWebGuide Link Checker Disallow: User-agent: PSurf15a 11 Disallow: User-agent: PSurf15a 51 Disallow: User-agent: psycheclone Disallow: User-agent: PubCrawl Disallow: User-agent: PureSight Disallow: User-agent: PuxaRapido Disallow: User-agent: PycURL Disallow: User-agent: Qango.com Disallow: User-agent: Rotondo/3.1 Disallow: User-agent: WebCompass Disallow: User-agent: QuepasaCreep Disallow: User-agent: QueryN Disallow: User-agent: QuickTime\xaa.7.0.4 Disallow: User-agent: Qweery_robot.txt_CheckBot Disallow: User-agent: QweeryBot Disallow: User-agent: rabaz Disallow: User-agent: Rainbot1.1 Disallow: User-agent: StackRambler Disallow: User-agent: ramBot Disallow: User-agent: Rank Exec Disallow: User-agent: Web Snooper Disallow: User-agent: Raven Disallow: User-agent: rbse Disallow: User-agent: ReadABlog Spider Disallow: User-agent: RMA/1.0 Disallow: User-agent: RealDownload/4.0.0.4x Disallow: User-agent: Reaper Disallow: User-agent: REBOL Disallow: User-agent: RebusnetBot Disallow: User-agent: RedCarpet/1.2 Disallow: User-agent: RedBot/redbot-1.0 Disallow: User-agent: RedKernel WWW-Spider 2/0 Disallow: User-agent: REL Link Checker Lite x.x Disallow: User-agent: Web Link Validator 1.5 Disallow: User-agent: RepoMonkey Bait & Tackle/v1.01 Disallow: User-agent: resumerobot Disallow: User-agent: Rewebber/1.2 Disallow: User-agent: rico/0.1 Disallow: User-agent: RixBot Disallow: User-agent: roadrunner Disallow: User-agent: RHCS Disallow: User-agent: lc/$ROADS::Version Disallow: User-agent: Robbie Disallow: User-agent: RoboCrawl Disallow: User-agent: robofox Disallow: User-agent: Robot du CRIM 1.0a Disallow: User-agent: Robozilla/1.0 Disallow: User-agent: Roverbot Disallow: User-agent: RPT-HTTPClient Disallow: User-agent: RssBandit Disallow: User-agent: RSSMicro.com Disallow: User-agent: RSSOwl Disallow: User-agent: RSurf15a Disallow: User-agent: Rubbot Disallow: User-agent: RufusBot Disallow: User-agent: RuLeS Disallow: User-agent: Rumours-Agent Disallow: User-agent: RX Bar Disallow: User-agent: SafariBookmarkChecker Disallow: User-agent: FANGCrawl Disallow: User-agent: SafetyNet Robot 0.1 Disallow: User-agent: MaSagool/1.0 Disallow: User-agent: SandCrawler Disallow: User-agent: ScanWeb Disallow: User-agent: SquidClamAV_Redirector 1.x.x Disallow: User-agent: ScholarUniverse Disallow: User-agent: schwarzmann.biz-Spider_for_paddel.org Disallow: User-agent: Scope (Mars+) Disallow: User-agent: ScoutAbout Disallow: User-agent: scoutmaster Disallow: User-agent: Scrubby Disallow: User-agent: S&L Spider Disallow: User-agent: Search-AU Disallow: User-agent: search.ch V1.4 Disallow: User-agent: SearchByUsa/2 Disallow: User-agent: SearchdayBot Disallow: User-agent: de.searchengine.comBot Disallow: User-agent: SearchExpress Spider0.99 Disallow: User-agent: SearchGuild/DMOZ/Experiment Disallow: User-agent: Spider-Sleek/2.0 (+http://search-info.com/linktous.html) Disallow: User-agent: Searchit-Now Robot Disallow: User-agent: Searchmee! Disallow: User-agent: searchprocess Disallow: User-agent: SearchSight/2.0 Disallow: User-agent: dev-spider2.searchpsider.com/1.3b Disallow: User-agent: SearchSpider.com/1.1 Disallow: User-agent: page_verifier Disallow: User-agent: GAIS Robot Disallow: User-agent: Seekbot Disallow: User-agent: semaforo.net Disallow: User-agent: Semager/1.1 Disallow: User-agent: semanticdiscovery Disallow: User-agent: Senrigan Disallow: User-agent: Sensis Web Crawler Disallow: User-agent: S.T.A.L.K.E.R. Disallow: User-agent: SeznamBot/1.0 Disallow: User-agent: CloakDetect Disallow: User-agent: SG-Scout Disallow: User-agent: Shagseeker Disallow: User-agent: Shai'Hulud Disallow: User-agent: Shareaza v1.x.x.xx Disallow: User-agent: Agent-SharewarePlazaFileCheckBot/2.0+ Disallow: User-agent: SharewarePlazaFileCheckBot/1.0+ Disallow: User-agent: ShopWiki/1.0 ( +http://www.shopwiki.com/) Disallow: User-agent: Shoula.com Crawler 2.0 Disallow: User-agent: DatenBot Disallow: User-agent: virus_detector Disallow: User-agent: SietsCrawler Disallow: User-agent: sift Disallow: User-agent: Siigle Orumcex Disallow: User-agent: SimBot Disallow: User-agent: SimpleFavPanel/1.2 Disallow: User-agent: Simpy Disallow: User-agent: Argus/1.1 Disallow: User-agent: asterias/2.0 Disallow: User-agent: Sirketcebot Disallow: User-agent: ssearcher100 Disallow: User-agent: SiteSpider Disallow: User-agent: site-valet Disallow: User-agent: SiteBar Disallow: User-agent: SiteRecon+(xx) Disallow: User-agent: SBIder Disallow: User-agent: SiteSucker Disallow: User-agent: SiteTaggerBot Disallow: User-agent: SiteTech-Rover Disallow: User-agent: SiteTruth Disallow: User-agent: +SitiDi.net/SitiDiBot Disallow: User-agent: GhostRouteHunter Disallow: User-agent: Skampy Disallow: User-agent: skymob Disallow: User-agent: SLCrawler/2.0 Disallow: User-agent: Sleek Spider/1.2 Disallow: User-agent: Sleipnir Disallow: User-agent: silk/1.0 Disallow: User-agent: Slider_Search_v1-de Disallow: User-agent: SlimBrowser Disallow: User-agent: SlySearch Disallow: User-agent: SmiffyDCMetaSpider/1.0 Disallow: User-agent: Snapbot/1.0 Disallow: User-agent: Snarfer/0.x.x Disallow: User-agent: snooper Disallow: User-agent: SnoopRob/x.x Disallow: User-agent: Snoopy v1.xx Disallow: User-agent: sna Disallow: User-agent: SnykeBot Disallow: User-agent: nsyght.com/Nutch-0.9 Disallow: User-agent: SocSciBot () Disallow: User-agent: SoftHypermarketFileCheckBot Disallow: User-agent: Softizerbot Disallow: User-agent: sohu-search Disallow: User-agent: solbot Disallow: User-agent: Program Shareware 1.0.2 Disallow: User-agent: MusicWalker2.0 Disallow: User-agent: Deep Link Calculator v1.0 Disallow: User-agent: Space Bison Disallow: User-agent: SpeedDownload Disallow: User-agent: speedfind ramBot xtreme 8.1 Disallow: User-agent: speedy Disallow: User-agent: Sphider Disallow: User-agent: Spida/0.1 Disallow: User-agent: Spinne/2.0 Disallow: User-agent: spider_monkey Disallow: User-agent: SpiderBot/1.0 Disallow: User-agent: SpiderKU/0.x Disallow: User-agent: spiderline Disallow: User-agent: SpiderMan Disallow: User-agent: SpiderMonkey/7.0x Disallow: User-agent: SpiderView Disallow: User-agent: Spock Crawler Disallow: User-agent: sportsuchmaschine.de-Robot Disallow: User-agent: sproose/0.1-alpha Disallow: User-agent: spry Disallow: User-agent: SQ Webscanner Disallow: User-agent: squidclam Disallow: User-agent: Squid-Prefetch Disallow: User-agent: SSurf15a Disallow: User-agent: Stamina/1.4 Disallow: User-agent: Star Downloader Disallow: User-agent: stat statcrawler@gmail.com Disallow: User-agent: Steeler Disallow: User-agent: Strategic Board Bot Disallow: User-agent: Mulder, VCR-1.0 Disallow: User-agent: suchbaer Disallow: User-agent: suchbot Disallow: User-agent: Suchknecht.at-Robot Disallow: User-agent: suchpadbot Disallow: User-agent: suke Disallow: User-agent: Sunrise Disallow: User-agent: suntek Disallow: User-agent: SuperBot Disallow: User-agent: Robot@SuperSnooper.Com Disallow: User-agent: SURF Disallow: User-agent: Submission Spider Disallow: User-agent: SurveyBot Disallow: User-agent: sven Disallow: User-agent: SWB/V1.4 Disallow: User-agent: Swooglebot/2.0. Disallow: User-agent: SWSBot-Images Disallow: User-agent: sygol Disallow: User-agent: SygolBot Disallow: User-agent: Sylera Disallow: User-agent: !Susie Disallow: User-agent: SyncBot Disallow: User-agent: SyncIT/x.x Disallow: User-agent: Syndirella/0.91pre Disallow: User-agent: SynoBot Disallow: User-agent: Syntryx ANT Scout Chassis Pheromone Disallow: User-agent: Szukacz Disallow: User-agent: Onet.pl SA- Disallow: User-agent: tach_bw Disallow: User-agent: LinkProver Disallow: User-agent: tags2dir.com/0.8 Disallow: User-agent: Tagword Disallow: User-agent: Tagyu Agent/1.0 Disallow: User-agent: TAMU_CS_IRL_CRAWLER/1.0 Disallow: User-agent: tarantula Disallow: User-agent: TargetYourNews.com Disallow: User-agent: targetblaster.com/0.9k Disallow: User-agent: tarspider Disallow: User-agent: TCDBOT/Nutch-0.8 Disallow: User-agent: dlw3robot Disallow: User-agent: TechBOT Disallow: User-agent: TECOMAC-Crawler/0.x Disallow: User-agent: Tecomi Bot Disallow: User-agent: Teleport Pro/1.2x(.1xxx) Disallow: User-agent: templeton Disallow: User-agent: Teradex Mapper Disallow: User-agent: TeragramCrawler Disallow: User-agent: terraminds-bot/1.0 Disallow: User-agent: Spider.TerraNautic.net Disallow: User-agent: TerrawizBot Disallow: User-agent: JubiiRobot/version # Disallow: User-agent: NorthStar Disallow: User-agent: w3index Disallow: User-agent: perignator Disallow: User-agent: python Disallow: User-agent: tkwww Disallow: User-agent: webmoose Disallow: User-agent: TheSuBot/0.1 Disallow: User-agent: thumbshots-de-Bot Disallow: User-agent: Sqworm/2.9.89-BETA Disallow: User-agent: TimelyWeb Disallow: User-agent: titan Disallow: User-agent: titin Disallow: User-agent: tivraSpider Disallow: User-agent: TJG/Spider Disallow: User-agent: TJvMultiHttpGrabber Disallow: User-agent: Tkensaku Disallow: User-agent: topiclink Disallow: User-agent: toCrawl/UrlDispatcher Disallow: User-agent: T-Online Browser Disallow: User-agent: Topodia/1.2-dev Disallow: User-agent: TOPOS robot Disallow: User-agent: traazibot/testengine Disallow: User-agent: Trailfire-bot Disallow: User-agent: Trampelpfad-Spider Disallow: User-agent: trendtech Disallow: User-agent: tricosMetaCheck Disallow: User-agent: TSurf15a 11 Disallow: User-agent: TulipChain Disallow: User-agent: TurnitinBot Disallow: User-agent: Turnpike Emporium LinkChecker/0.1 Disallow: User-agent: Tutorial Crawler Disallow: User-agent: Twiceler-0.9 http://www.cuill.com/twiceler/robot.html Disallow: User-agent: Twisted PageGetter Disallow: User-agent: Twotrees Reactive Filter Disallow: User-agent: Willow Internet Crawler by Twotrees Disallow: User-agent: TygoBot Disallow: User-agent: TygoProwler Disallow: User-agent: Superdownloads Spiderman Disallow: User-agent: UCmore Disallow: User-agent: ucsd Disallow: User-agent: UdmSearch Disallow: User-agent: UDM Disallow: User-agent: Mackster Disallow: User-agent: UKWizz/Nutch-0.8.1 Disallow: User-agent: lwp-trivial Disallow: User-agent: UnChaos Disallow: User-agent: Under the Rainbow Disallow: User-agent: Contact Disallow: User-agent: UofTDB_experiment Disallow: User-agent: updated Disallow: User-agent: UptimeBot Disallow: User-agent: URI::Fetch/0.06 Disallow: User-agent: urlck Disallow: User-agent: Search+ Disallow: User-agent: URL Spider Pro Disallow: User-agent: URLBase/6.x Disallow: User-agent: URLBlaze Disallow: User-agent: URLGetFile Disallow: User-agent: USyd-NLP-Spider Disallow: User-agent: Vakes Disallow: User-agent: Valkyrie libwww-perl Disallow: User-agent: VayalaCreep Disallow: User-agent: mylinkcheck/1.02 Disallow: User-agent: vspider Disallow: User-agent: versus Disallow: User-agent: verticrawl Disallow: User-agent: VeryGoodSearch.com.DaddyLongLegs Disallow: User-agent: verzamelgids.nl - Networking4all Bot/x.x Disallow: User-agent: Vespa Crawler Disallow: User-agent: NCSA Beta 1 Disallow: User-agent: Victoria Disallow: User-agent: Matrix Disallow: User-agent: VisBot/2.0 Disallow: User-agent: Visicom Toolbar Disallow: User-agent: Vision Research Lab image spider at vision.ece.ucsb.edu Disallow: User-agent: vision-search Disallow: User-agent: Skywalker Disallow: User-agent: VSE Disallow: User-agent: VLC media player Disallow: User-agent: VMBot Disallow: User-agent: void-bot Disallow: User-agent: KE_1.0/2.0 Disallow: User-agent: Vortex Disallow: User-agent: Voyager Disallow: User-agent: Everest-Vulcan Inc./0.1 Disallow: User-agent: VWbot_K Disallow: User-agent: VWBOT/Nutch-0.9-dev Disallow: User-agent: wapspider Disallow: User-agent: W3SiteSearch Crawler_v1.1 Disallow: User-agent: W3CLineMode/5.4.0 libwww/5.x.x Disallow: User-agent: W3C-checklink Disallow: User-agent: w3m/0.x.xx Disallow: User-agent: W3M2 Disallow: User-agent: w3mir Disallow: User-agent: wadaino.jp-crawler Disallow: User-agent: appie Disallow: User-agent: crawlpaper Disallow: User-agent: SurferF3 1/0 Disallow: User-agent: WannaBe (Macintosh; PPC) Disallow: User-agent: WapOnWindows Disallow: User-agent: Watchfire WebXM 1.0 Disallow: User-agent: WAVcheck Disallow: User-agent: Wavefire Disallow: User-agent: Waypath Disallow: User-agent: Waypath Scout v2.x Disallow: User-agent: WDG_Validator Disallow: User-agent: webbandit Disallow: User-agent: webcore Disallow: User-agent: UtilMind Disallow: User-agent: WebTrends Disallow: User-agent: WebVulnCrawl.blogspot.com/1.0 libwww-perl/5.803 Disallow: User-agent: WebWatcherMonitor/2.01 Disallow: User-agent: wombat Disallow: User-agent: web2express.org/Nutch-0.9-dev Disallow: User-agent: WebAlta Crawler/1.2.1 Disallow: User-agent: pulseBot Disallow: User-agent: WebarooBot Disallow: User-agent: WebBandit/1.0 Disallow: User-agent: web-bekannt Disallow: User-agent: Web-Bot V1.03 Disallow: User-agent: WebCatcher Disallow: User-agent: Webclipping.com Disallow: User-agent: webcollage/1.xx Disallow: User-agent: W3C-WebCon/5.x.x libwww/5.x.x Disallow: User-agent: webcopy Disallow: User-agent: WebCorp/1.0 Disallow: User-agent: webcrawl.net Disallow: User-agent: WebDownloader Disallow: User-agent: Webdup Disallow: User-agent: WebFetcher Disallow: User-agent: WebFilter Robot Disallow: User-agent: WebFindBot Disallow: User-agent: webfoot Disallow: User-agent: Webglimpse Disallow: User-agent: webGobbler Disallow: User-agent: webhack Disallow: User-agent: WebImages Disallow: User-agent: T-H-U-N-D-E-R-S-T-O-N-E Disallow: User-agent: IIITBOT Disallow: User-agent: weblayers Disallow: User-agent: WebLight Disallow: User-agent: weblinker Disallow: User-agent: Weblink's checker/ Disallow: User-agent: Weblog Attitude Diffusion Disallow: User-agent: wlm Disallow: User-agent: webmeasurement-bot Disallow: User-agent: WebMiner Disallow: User-agent: webmirror Disallow: User-agent: WeBoX Disallow: User-agent: WebPix Disallow: User-agent: webquest Disallow: User-agent: WebRACE Disallow: User-agent: WebRankSpider Disallow: User-agent: Web-Robot Disallow: User-agent: webs Disallow: User-agent: savvybot/0.2 Disallow: User-agent: WebSearch.COM.AU/3.0.1 Disallow: User-agent: WebSearchBench WebCrawler Disallow: User-agent: WSB WebCrawler Disallow: User-agent: Website Explorer Disallow: User-agent: WebsiteWorth Disallow: User-agent: websnarf Disallow: User-agent: webspider Disallow: User-agent: Webspinne Disallow: User-agent: Websquash Disallow: User-agent: WebStat/1.0 (Unix; beta; 20040314) Disallow: User-agent: Webster Disallow: User-agent: webster-internet.de pad browser Disallow: User-agent: WOLP Disallow: User-agent: WebTrafficExpress Disallow: User-agent: WebVac Disallow: User-agent: WebVal Disallow: User-agent: Webverzeichnis Disallow: User-agent: webwalk Disallow: User-agent: WebWalker Disallow: User-agent: SiteWinder Disallow: User-agent: Utopia WebWasher Disallow: User-agent: WebWatch Disallow: User-agent: ScollSpider Disallow: User-agent: Wells Search II Disallow: User-agent: WEP Search Disallow: User-agent: WFARC Disallow: User-agent: Whatsup Disallow: User-agent: whatUseek_winona Disallow: User-agent: winona Disallow: User-agent: WhizBang! Lab Disallow: User-agent: PigeonBot1.0 Disallow: User-agent: DaviesBot Disallow: User-agent: whowhere Disallow: User-agent: Hazel's Ferret Web hopper Disallow: User-agent: Wildsoft Surfer Disallow: User-agent: WinampMPEG Disallow: User-agent: WincerSong Disallow: User-agent: msnbot-Products/1.0 Disallow: User-agent: WebFetch Disallow: User-agent: WebGo IS Disallow: User-agent: WebLeacher Disallow: User-agent: WebReaper [info@webreaper.net] Disallow: User-agent: WebReaper [webreaper@otway.com] Disallow: User-agent: WebReaper v9.1 - www.otway.com/webreaper Disallow: User-agent: WebReaper v9.7 - www.webreaper.net Disallow: User-agent: WebReaper v9.8 - www.webreaper.net Disallow: User-agent: WebReaper vWebReaper v7.3 - www,otway.com/webreaper Disallow: User-agent: WebSauger Disallow: User-agent: WebSauger 1.20b Disallow: User-agent: WebSauger 1.20j Disallow: User-agent: WebSauger 1.20k Disallow: User-agent: WebStripper/2.03 Disallow: User-agent: WebStripper/2.10 Disallow: User-agent: WebStripper/2.12 Disallow: User-agent: WebStripper/2.13 Disallow: User-agent: WebStripper/2.15 Disallow: User-agent: WebStripper/2.16 Disallow: User-agent: WebStripper/2.19 Disallow: User-agent: WebWhacker Disallow: User-agent: WebZIP Disallow: User-agent: WebZIP/2.75 (http://www.spidersoft.com) Disallow: User-agent: WebZIP/3.65 (http://www.spidersoft.com) Disallow: User-agent: WebZIP/3.80 (http://www.spidersoft.com) Disallow: User-agent: WebZIP/4.0 (http://www.spidersoft.com) Disallow: User-agent: WebZIP/4.1 (http://www.spidersoft.com) Disallow: User-agent: WebZIP/4.21 Disallow: User-agent: WebZIP/4.21 (http://www.spidersoft.com) Disallow: User-agent: WebZIP/5.0 Disallow: User-agent: WebZIP/5.0 (http://www.spidersoft.com) Disallow: User-agent: WebZIP/5.0 PR1 (http://www.spidersoft.com) Disallow: User-agent: WebZip Disallow: User-agent: WebZip/4.0 Disallow: User-agent: WebmasterWorldForumBot Disallow: User-agent: Website Quester Disallow: User-agent: Website Quester - www.asona.org Disallow: User-agent: Website Quester - www.esalesbiz.com/extra/ Disallow: User-agent: Website eXtractor Disallow: User-agent: Website eXtractor (http://www.asona.org) Disallow: User-agent: Website.Quester Disallow: User-agent: Webster Pro Disallow: User-agent: Webster.Pro Disallow: User-agent: Wget Disallow: User-agent: Wget/1.5.2 Disallow: User-agent: Wget/1.5.3 Disallow: User-agent: Wget/1.6 Disallow: User-agent: Wget/1.7 Disallow: User-agent: Wget/1.8 Disallow: User-agent: Wget/1.8.1 Disallow: User-agent: Wget/1.8.1+cvs Disallow: User-agent: Wget/1.8.2 Disallow: User-agent: Wget/1.9-beta Disallow: User-agent: Widow Disallow: User-agent: Xaldon WebSpider Disallow: User-agent: Xaldon WebSpider 2.5.b3 Disallow: User-agent: Xenu's Disallow: User-agent: Xenu's Link Sleuth 1.1c Disallow: User-agent: Zeus Disallow: User-agent: Zeus 11389 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 11652 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 18018 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 26378 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 30747 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 32297 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 39206 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 41641 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 44238 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 51070 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 51674 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 51837 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 63567 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 6694 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 71129 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 82016 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 82900 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 84842 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 90872 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 94934 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 95245 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 95351 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus 97371 Webster Pro V2.9 Win32 Disallow: User-agent: Zeus Link Scout Disallow: User-agent: b2w/0.1 Disallow: User-agent: cosmos Disallow: User-agent: eCatch/3.0 Disallow: User-agent: hloader Disallow: User-agent: httplib Disallow: User-agent: humanlinks Disallow: User-agent: larbin (samualt9@bigfoot.com) Disallow: User-agent: larbin samualt9@bigfoot.com Disallow: User-agent: larbin_2.6.2 (kabura@sushi.com) Disallow: User-agent: larbin_2.6.2 (larbin2.6.2@unspecified.mail) Disallow: User-agent: larbin_2.6.2 (listonATccDOTgatechDOTedu) Disallow: User-agent: larbin_2.6.2 (vitalbox1@hotmail.com) Disallow: User-agent: larbin_2.6.2 kabura@sushi.com Disallow: User-agent: larbin_2.6.2 larbin2.6.2@unspecified.mail Disallow: User-agent: larbin_2.6.2 larbin@correa.org Disallow: User-agent: larbin_2.6.2 listonATccDOTgatechDOTedu Disallow: User-agent: larbin_2.6.2 vitalbox1@hotmail.com Disallow: User-agent: lwp-trivial/1.34 Disallow: User-agent: moget/2.1 Disallow: User-agent: pavuk Disallow: User-agent: pcBrowser Disallow: User-agent: searchpreview Disallow: User-agent: spanner Disallow: User-agent: suzuran Disallow: User-agent: tAkeOut Disallow: User-agent: turingos Disallow: User-agent: webfetch/2.1.0 Disallow: User-agent: TeamSoft WinInet Component Disallow: User-agent: WinkBot Disallow: User-agent: WinPodder Disallow: User-agent: WinWAP Disallow: User-agent: WIRE Disallow: User-agent: NetSprint Disallow: User-agent: Vagabondo Disallow: User-agent: WISEbot Disallow: User-agent: ZyBorg Disallow: User-agent: WiseWire-Spider2 Disallow: User-agent: wish-project Disallow: User-agent: WordChampBot Disallow: User-agent: woriobot Disallow: User-agent: WWWeasel Robot Disallow: User-agent: wanderer Disallow: User-agent: worm Disallow: User-agent: WorldLight Disallow: User-agent: WorQmada Disallow: User-agent: Wotbox Disallow: User-agent: wume_crawler Disallow: User-agent: Wusage/x.0@boutell.com Disallow: User-agent: West Wind Internet Protocols 4.xx Disallow: User-agent: Wwlib/Linux Disallow: User-agent: WWSBOT Disallow: User-agent: MediaSearch/0.1 Disallow: User-agent: www4mail/2.x libwww-FM/2.14 Disallow: User-agent: WWWC Disallow: User-agent: WWW-Mechanize Disallow: User-agent: WWWOFFLE Disallow: User-agent: wwwster Disallow: User-agent: egothor Disallow: User-agent: Xerka WebBot Disallow: User-agent: XGET Disallow: User-agent: xine Disallow: User-agent: xirq Disallow: User-agent: XMLSlurp Disallow: User-agent: Project XP5 [2.03.07-111203] Disallow: User-agent: MIIxpc/4.2 Disallow: User-agent: Search-Engine-Studio Disallow: User-agent: SiteXpert Disallow: User-agent: Y!TunnelPro Disallow: User-agent: yacy Disallow: User-agent: Yahoo-VerticalCrawler-FormerWebCrawler Disallow: User-agent: Yahoo-Blogs/v3.9 Disallow: User-agent: PostFavorites Disallow: User-agent: Yahoo! Mindset Disallow: User-agent: Yahoo-MMAudVid Disallow: User-agent: Y!OASIS Disallow: User-agent: Yahoo Pipes 1.0 Disallow: User-agent: YahooSeeker Disallow: User-agent: YahooFeedSeeker Disallow: User-agent: YahooSeeker/CafeKelsa-dev Disallow: User-agent: Y!J-BSC Disallow: User-agent: Wir sind die Borg Disallow: User-agent: WebAuto Disallow: User-agent: Yandex/1.01.001 (compatible; Win16; I) Disallow: User-agent: yarienavoir.net Disallow: User-agent: spider.yellopet.com Disallow: User-agent: Findexa Disallow: User-agent: yggdrasil/Nutch-0.9 Disallow: User-agent: RssReader Disallow: User-agent: YodaoBot Disallow: User-agent: yoogliFetchAgent/0.1 Disallow: User-agent: yoono Disallow: User-agent: YottaCars_Bot Disallow: User-agent: YottaShopping_Bot Disallow: User-agent: GulperBot Disallow: User-agent: Z-Add Link Checker Disallow: User-agent: Zao-Crawler Disallow: User-agent: ZeBot_lseek.net Disallow: User-agent: Zearchit Disallow: User-agent: zedzo.digest Disallow: User-agent: Zend_Http_Client Disallow: User-agent: zerxbot Disallow: User-agent: Zbot Disallow: User-agent: Zeus ThemeSite Viewer Webster Pro V2.9 Win32 Disallow: User-agent: Zeusbot Disallow: User-agent: Ziggy Disallow: User-agent: ZipppBot Disallow: User-agent: ZIPPPCVS Disallow: User-agent: Zippy Disallow: User-agent: Zoo Tycoon 2 Client Disallow: User-agent: ZoomSpider Disallow: User-agent: zspider Disallow: User-agent: Pasiphae Disallow: User-agent: Alexibot Disallow: User-agent: Aqua_Products Disallow: User-agent: BackDoorBot Disallow: User-agent: BackDoorBot/1.0 Disallow: User-agent: Black.Hole Disallow: User-agent: BlowFish Disallow: User-agent: BlowFish/1.0 Disallow: User-agent: Bookmark search tool Disallow: User-agent: Bot mailto:craftbot@yahoo.com Disallow: User-agent: BotALot Disallow: User-agent: BotRightHere Disallow: User-agent: BuiltBotTough Disallow: User-agent: Bullseye Disallow: User-agent: Bullseye/1.0 Disallow: User-agent: BunnySlippers Disallow: User-agent: Cegbfeieh Disallow: User-agent: CheeseBot Disallow: User-agent: CherryPicker Disallow: User-agent: CherryPickerElite/1.0 Disallow: User-agent: CherryPickerSE/1.0 Disallow: User-agent: ChinaClaw Disallow: User-agent: Copernic Disallow: User-agent: CopyRightCheck Disallow: User-agent: Crescent Disallow: User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 Disallow: User-agent: DISCo Disallow: User-agent: DISCo Pump 3.0 Disallow: User-agent: DISCo Pump 3.2 Disallow: User-agent: DISCoFinder Disallow: User-agent: DittoSpyder Disallow: User-agent: Download Demon/3.2.0.8 Disallow: User-agent: Download Demon/3.5.0.11 Disallow: User-agent: EirGrabber Disallow: User-agent: EmailCollector Disallow: User-agent: EmailSiphon Disallow: User-agent: EroCrawler Disallow: User-agent: Express WebPictures (www.express-soft.com) Disallow: User-agent: ExtractorPro Disallow: User-agent: EyeNetIE Disallow: User-agent: FairAd Client Disallow: User-agent: Flaming AttackBot Disallow: User-agent: FlashGet Disallow: User-agent: FlashGet WebWasher 3.2 Disallow: User-agent: Foobot Disallow: User-agent: FrontPage Disallow: User-agent: FrontPage [NC,OR] Disallow: User-agent: GetRight/2.11 Disallow: User-agent: GetRight/3.1 Disallow: User-agent: GetRight/3.2 Disallow: User-agent: GetRight/3.3 Disallow: User-agent: GetRight/3.3.3 Disallow: User-agent: GetRight/3.3.4 Disallow: User-agent: GetRight/4.0.0 Disallow: User-agent: GetRight/4.1.0 Disallow: User-agent: GetRight/4.1.1 Disallow: User-agent: GetRight/4.1.2 Disallow: User-agent: GetRight/4.2 Disallow: User-agent: GetRight/4.2b (Portuguxeas) Disallow: User-agent: GetRight/4.2c Disallow: User-agent: GetRight/4.3 Disallow: User-agent: GetRight/4.5 Disallow: User-agent: GetRight/4.5a Disallow: User-agent: GetRight/4.5b Disallow: User-agent: GetRight/4.5b1 Disallow: User-agent: GetRight/4.5b2 Disallow: User-agent: GetRight/4.5b3 Disallow: User-agent: GetRight/4.5b6 Disallow: User-agent: GetRight/4.5b7 Disallow: User-agent: GetRight/4.5c Disallow: User-agent: GetRight/4.5d Disallow: User-agent: GetRight/4.5e Disallow: User-agent: GetRight/5.0beta1 Disallow: User-agent: GetRight/5.0beta2 Disallow: User-agent: GetWeb! Disallow: User-agent: Go!Zilla (www.gozilla.com) Disallow: User-agent: Go!Zilla 3.3 (www.gozilla.com) Disallow: User-agent: Go!Zilla 3.5 (www.gozilla.com) Disallow: User-agent: GrabNet Disallow: User-agent: Grafula Disallow: User-agent: HMView Disallow: User-agent: HTTrack 3.0 Disallow: User-agent: HTTrack [NC,OR] Disallow: User-agent: Harvest Disallow: User-agent: Harvest/1.5 Disallow: User-agent: Image Stripper Disallow: User-agent: Image Sucker Disallow: User-agent: Indy Library Disallow: User-agent: Indy Library [NC,OR] Disallow: User-agent: InterGET Disallow: User-agent: Internet Ninja Disallow: User-agent: Internet Ninja 4.0 Disallow: User-agent: Internet Ninja 5.0 Disallow: User-agent: Internet Ninja 6.0 Disallow: User-agent: Iron33/1.0.2 Disallow: User-agent: JOC Web Spider Disallow: User-agent: JennyBot Disallow: User-agent: Kenjin Spider Disallow: User-agent: Kenjin.Spider Disallow: User-agent: Keyword Density/0.9 Disallow: User-agent: Keyword.Density Disallow: User-agent: LeechFTP Disallow: User-agent: LinkScan/8.1a Unix Disallow: User-agent: LinkScan/8.1a.Unix Disallow: User-agent: LinkextractorPro Disallow: User-agent: MIDown tool Disallow: User-agent: MIIxpc Disallow: User-agent: Mass Downloader Disallow: User-agent: Mass Downloader/2.2 Disallow: User-agent: Mata Hari Disallow: User-agent: Mata.Hari Disallow: User-agent: Microsoft URL Control Disallow: User-agent: Microsoft URL Control - 5.01.4511 Disallow: User-agent: Microsoft URL Control - 6.00.8169 Disallow: User-agent: Microsoft.URL Disallow: User-agent: Mister PiX Disallow: User-agent: Mister PiX version.dll Disallow: User-agent: Mister Pix II 2.01 Disallow: User-agent: Mister Pix II 2.02a Disallow: User-agent: Mister.PiX Disallow: User-agent: NICErsPRO Disallow: User-agent: NPBot Disallow: User-agent: Navroad Disallow: User-agent: NearSite Disallow: User-agent: Net Vampire Disallow: User-agent: Net Vampire/3.0 Disallow: User-agent: NetAnts Disallow: User-agent: NetAnts/1.10 Disallow: User-agent: NetAnts/1.23 Disallow: User-agent: NetAnts/1.24 Disallow: User-agent: NetAnts/1.25 Disallow: User-agent: NetSpider Disallow: User-agent: NetZIP Disallow: User-agent: NetZip Downloader 1.0 Win32(Nov 12 1998) Disallow: User-agent: NetZip-Downloader/1.0.62 (Win32; Dec 7 1998) Disallow: User-agent: NetZippy+(http://www.innerprise.net/usp-spider.asp) Disallow: User-agent: Offline Explorer/1.2 Disallow: User-agent: Offline Explorer/1.4 Disallow: User-agent: Offline Explorer/1.6 Disallow: User-agent: Offline Explorer/1.7 Disallow: User-agent: Offline Explorer/1.9 Disallow: User-agent: Offline Explorer/2.0 Disallow: User-agent: Offline Explorer/2.1 Disallow: User-agent: Offline Explorer/2.3 Disallow: User-agent: Offline Explorer/2.4 Disallow: User-agent: Offline Explorer/2.5 Disallow: User-agent: Offline Navigator Disallow: User-agent: Offline.Explorer Disallow: User-agent: Openfind Disallow: User-agent: Openfind data gatherer Disallow: User-agent: PageGrabber Disallow: User-agent: Papa Foto Disallow: User-agent: PerMan Disallow: User-agent: ProPowerBot/2.14 Disallow: User-agent: ProWebWalker Disallow: User-agent: Python-urllib Disallow: User-agent: QueryN Metasearch Disallow: User-agent: QueryN.Metasearch Disallow: User-agent: RMA Disallow: User-agent: Radiation Retriever 1.1 Disallow: User-agent: ReGet Disallow: User-agent: RealDownload Disallow: User-agent: RealDownload/4.0.0.40 Disallow: User-agent: RealDownload/4.0.0.41 Disallow: User-agent: RealDownload/4.0.0.42 Disallow: User-agent: RepoMonkey Disallow: User-agent: SmartDownload/1.2.76 (Win32; Apr 1 1999) Disallow: User-agent: SmartDownload/1.2.77 (Win32; Aug 17 1999) Disallow: User-agent: SmartDownload/1.2.77 (Win32; Feb 1 2000) Disallow: User-agent: SmartDownload/1.2.77 (Win32; Jun 19 2001) Disallow: User-agent: SpankBot Disallow: User-agent: Sqworm/2.9.85-BETA (beta_release; 20011115-775; i686-pc-linux Disallow: User-agent: SuperBot/3.0 (Win32) Disallow: User-agent: SuperBot/3.1 (Win32) Disallow: User-agent: SuperHTTP Disallow: User-agent: SuperHTTP/1.0 Disallow: User-agent: Surfbot Disallow: User-agent: Szukacz/1.4 Disallow: User-agent: Teleport Pro/1.29 Disallow: User-agent: Teleport Pro/1.29.1590 Disallow: User-agent: Teleport Pro/1.29.1634 Disallow: User-agent: Teleport Pro/1.29.1718 Disallow: User-agent: Teleport Pro/1.29.1820 Disallow: User-agent: Teleport Pro/1.29.1847 Disallow: User-agent: Telesoft Disallow: User-agent: The Intraformant Disallow: User-agent: The.Intraformant Disallow: User-agent: TheNomad Disallow: User-agent: TightTwatBot Disallow: User-agent: Titan Disallow: User-agent: True_Robot Disallow: User-agent: True_Robot/1.0 Disallow: User-agent: TurnitinBot/1.5 Disallow: User-agent: URL Control Disallow: User-agent: URLy Warning Disallow: User-agent: URLy.Warning Disallow: User-agent: VCI Disallow: User-agent: VCI WebViewer VCI WebViewer Win32 Disallow: User-agent: VoidEYE Disallow: User-agent: WWW-Collector-E Disallow: User-agent: Web Image Collector Disallow: User-agent: Web Sucker Disallow: User-agent: Web.Image.Collector Disallow: User-agent: WebAuto/3.40 (Win98; I) Disallow: User-agent: WebBandit Disallow: User-agent: WebBandit/3.50 Disallow: User-agent: WebCapture 2.0 Disallow: User-agent: WebCopier Disallow: User-agent: WebCopier v.2.2 Disallow: User-agent: WebCopier v2.5 Disallow: User-agent: WebCopier v2.6 Disallow: User-agent: WebCopier v2.7a Disallow: User-agent: WebCopier v2.8 Disallow: User-agent: WebCopier v3.0 Disallow: User-agent: WebCopier v3.0.1 Disallow: User-agent: WebCopier v3.2 Disallow: User-agent: WebCopier v3.2a Disallow: User-agent: WebEMailExtrac.* Disallow: User-agent: WebEnhancer Disallow: User-agent: * Disallow: Sitemap: http://www.absolutesystem.com/sitemap.xml Sitemap: http://www.absolutesystem.com/absolutesystem_com.xml Sitemap: http://www.absolutesystem.in/absolutesystem_in.xml