User-agent: NetZIP User-agent: tAkeOut User-agent: Internet Ninja User-agent: Offline Explorer User-agent: larbin User-agent: Go-Ahead-Got-It User-agent: WebReaper User-agent: Web Sucker User-agent: MJ12bot User-agent: WebFetch User-agent: ChinaClaw User-agent: GetRight User-agent: LinkpadBot User-agent: WWWOFFLE User-agent: eCatch User-agent: Dotbot User-agent: WebSauger User-agent: JetCar User-agent: SiteSnagger User-agent: WebCopier User-agent: SuperBot User-agent: Web Image Collector User-agent: FlashGet User-agent: LeechFTP User-agent: Image Stripper User-agent: EyeNetIE User-agent: SmartDownload User-agent: HTTrack User-agent: pavuk User-agent: Grafula User-agent: Gigabot User-agent: RealDownload User-agent: Offline Navigator User-agent: Xaldon WebSpider User-agent: Express WebPictures User-agent: Papa Foto User-agent: Navroad User-agent: Zeus User-agent: ReGet User-agent: NetAnts User-agent: SuperHTTP User-agent: AhrefsBot User-agent: EirGrabber User-agent: Teleport Pro User-agent: WebWhacker User-agent: Mass Downloader User-agent: Surfbot User-agent: Widow User-agent: Website eXtractor User-agent: GetWeb! User-agent: GrabNet User-agent: WebLeacher User-agent: JOC Web Spider User-agent: SemrushBot/1.1~bl User-agent: NearSite User-agent: BlackWidow User-agent: WebZIP User-agent: Go!Zilla User-agent: pcBrowser User-agent: Indy Library User-agent: DISCo User-agent: Bot [EMAIL="craftbot@yahoo.com"]mailto:craftbot@yahoo.com[/EMAIL] User-agent: EmailWolf User-agent: Rogerbot User-agent: SemrushBot User-agent: VoidEYE User-agent: Image Sucker User-agent: ExtractorPro User-agent: Custo User-agent: InterGET User-agent: Exabot User-agent: WebAuto User-agent: PageGrabber User-agent: Octopus User-agent: Download Demon User-agent: Wget User-agent: Mister PiX User-agent: MIDown tool User-agent: Website Quester User-agent: NetSpider User-agent: EmailSiphon User-agent: WebStripper User-agent: WebGo IS User-agent: Net Vampire User-agent: AhrefsBot/5.1 User-agent: HMView Disallow: / User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ User-agent: Googlebot Disallow: /*.png$ User-agent: Screaming Frog SEO Spider Disallow: / User-agent: Baiduspider Disallow: / User-agent: Baiduspider+ Disallow: / User-agent: Baiduspider+(+http://www.baidu.com/search/s User-agent: Baiduspider/2.0;+http://www.baidu.com/search/spider.html Disallow: / User-agent: Baiduspider/2.0 Disallow: / User-agent: +Baiduspider Disallow: / User-agent: +Baiduspider/2.0 Disallow: / User-agent: +Baiduspider/2.0;++http://www.baidu.com/search/spider.html Disallow: / User-agent: Mozilla/5.0(compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) Disallow: / User-agent: spider Disallow: / User-agent: SEOENGWorldBot/1.0+(+http://www.seoengine.com/seoengbot.htm) Disallow: / User-agent: Sosospider+ Disallow: / User-agent: ezooms Disallow: / User-agent: Ezooms/1.0 Disallow: / User-agent: repparser/0.1+(python) Disallow: / User-agent: Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+6.0)+Java/1.6.0_21 Disallow: / User-agent: Mozilla/5.0+(compatible;+MJ12bot/v1.4.3;+http://www.majestic12.co.uk/bot.php?+) Disallow: / User-agent: psbot/0.1+(+http://www.picsearch.com/bot.html) Disallow: / User-agent: Mozilla/5.0+(Windows+NT+5.1;+rv:6.0.2)+Gecko/20100101+Firefox/6.0.2 Disallow: / User-agent: Mozilla/5.0+(compatible;+Ezooms/1.0;+ezooms.bot@gmail.com) Disallow: / User-agent: CybEye.com/2.0+(compatible;+MSIE+9.0;+Windows+NT+5.1;+Trident/4.0;+GTB6.4) Disallow: / User-agent: Mozilla/5.0+(compatible;+Exabot/3.0;++http://www.exabot.com/go/robot) Disallow: / User-agent: CCBot/2.0 Disallow: / User-agent: Mozilla/5.0+(compatible;+008/0.83;+http://www.80legs.com/webcrawler.html;)+Gecko/2008032620 Disallow: / User-agent: AddThis.com+robot+tech.support@clearspring.com Disallow: / User-agent: Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+zh-CN;+rv:1.8.0.11)++Firefox/1.5.0.11;+360Spider Disallow: / User-agent: TosCrawler/Nutch-1.6+(http://www.toshiba.co.jp/rdc/about/crawl_info_en.htm;+'Rdc-crawler+at+ml+dot+toshiba+dot+co+dot+jp') Disallow: / User-agent: Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/28.0.1500.72+Safari/537.36 Disallow: /