Mein Gott, es ist voller Sterne… äääh… Bots… das Netz…
Über meinen Beitrag von vor drei Jahren kann man da eigentlich nur noch lachen… Acht Spider waren es damals in 20 Stunden…
Dieses mal sind es 36 Stunden – also fast doppelt so viel Zeit. Hier die Bots (die mir beim lesen der Log-Files aufgefallen sind):
- BAGL/Nutch-0.9 (BaglBot; http://www.bagl.co.uk; MyEmail)
- Baiduspider+(+http://www.baidu.com/search/spider_jp.html)
- Baiduspider+(+http://www.baidu.com/search/spider.htm)
- boitho.com-dc/0.86 ( http://www.boitho.com/dcbot.html )
- CCBot/1.0 (+http://www.commoncrawl.org/bot.html)
- curl/7.16.4 (i386-redhat-linux-gnu) libcurl/7.16.4 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.8
- Googlebot-Image/1.0
- ia_archiver
- ia_archiver-web.archive.org
- ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com)
- Java/1.4.1_04
- Java/1.6.0_03
- Java/1.6.0_04
- kalooga/KaloogaBot (Kalooga; http://www.kalooga.com/info.html?page=crawler; crawler@kalooga.com)
- Microsoft URL Control – 6.00.8169
- Mozilla/4.0 (compatible; NaverBot/1.0; http://help.naver.com/delete_main.asp)
- Mozilla/5.0 (compatible; askpeter_jeanie_2009_bot/5.0; +http://www.askpeter.info)
- Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)
- Mozilla/5.0 (compatible; Charlotte/1.1; http://www.searchme.com/support/)
- Mozilla/5.0 (compatible; DBLBot/1.0; +http://www.dontbuylists.com/)
- Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html)
- Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org)
- Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
- Mozilla/5.0 (compatible; MJ12bot/v1.2.3; http://www.majestic12.co.uk/bot.php?+)
- Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
- Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)
- Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html)
- Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (support.voilabot@orange-ftgroup.com)
- Mozilla/5.0 (Yahoo-MMCrawler/4.0; mailto:vertical-crawl-support@yahoo-inc.com)
- MSMOBOT/1.1 (+http://search.msn.com/msnbot.htm)
- msnbot/1.1 (+http://search.msn.com/msnbot.htm)
- msnbot-media/1.0 (+http://search.msn.com/msnbot.htm)
- Python-urllib/1.17
- Snoopy v1.2
- Sogou Pic Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
- Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
- sonarv2/1.16
- SonyEricssonK800i/R1ED Browser/NetFront/3.3 Profile/MIDP-2.0 Configuration/CLDC-1.1 (Mediobot/1.0 +http://bot.medio.com)
- Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)
- SurveyBot/2.3 (Whois Source)
- Touche (+http://www.touche.com.ve)
- Yandex/1.01.001 (compatible; Win16; I)
- Yanga WorldSearch Bot v1.1/beta (http://www.yanga.co.uk/)
- Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/)
- Yeti/1.0 (+http://help.naver.com/robots/)
Ausserdem kamen noch von folgenden IP-Adressen Zugriffe, die sich zwar als Browser ausgegeben haben, aber mit Sicherheit auch Bots sind:
- 38.100.41.113 (IE7)
- 38.105.83.12 (IE7)
- 58.107.82.216 (kaputte Browser-Kennung)
- 65.94.162.29 (IE6 – fragt robots.txt ab)
- 65.94.7.199 (IE6 – fragt robots.txt ab)
- 67.68.21.233 (IE6)
- 69.58.178.26 – 41 (Firefox 1.0.7 – fragt robots.txt ab)
- 70.99.174.23 (IE7)
- 76.65.77.73 (IE6 – fragt robots.txt ab)
- 84.166.237.84 (IE6 auf Win98 🙂 )
- 89.149.241.118 (kaputte Browser-Kennung)
- 92.75.218.76 (Firefox 3.0.3)
Eigentlich toll, dass es so viele Sicherheits-Kopien aller Webseiten gibt… 😉