[TYPO3-german] SPAM BOTS
Andreas Becker
ab.becker at web.de
Sun Sep 20 05:21:45 CEST 2009
Hi
Nach einigen Updates von Extensions auf die jeweils neuesten Versionen
mussten wir auf zwei Kunden Servern feststellen, dass nun gigantisch viele
Spam Bots diesen anscheinend aufsuchen. Sie nutzen hierzu immer die eigene
Server IP und rufen die Error Seiten auf mit "
http://www.domain.com/domain.com" ....
188.40.123.123 - - [20/Sep/2009:09:19:46 +0700] "GET /DOMAIN.com
HTTP/1.1" 302 - "http://www.domain.com/domain.com" "Mozilla/5.0
(Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824
Firefox/3.5.3 GTB5"
188.40.123.123 - - [20/Sep/2009:09:19:46 +0700] "GET /page-not-found/
HTTP/1.1" 301 - "http://www.domain.com/domain.com" "Mozilla/5.0
(Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824
Firefox/3.5.3 GTB5"
188.40.123.123 - - [20/Sep/2009:09:19:46 +0700] "GET /page-not-found/
HTTP/1.1" 503 1114 "http://www.domain.com/domain.com" "Mozilla/5.0
(Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824
Firefox/3.5.3 GTB5"
188.40.123.123 - - [20/Sep/2009:09:19:46 +0700] "GET /DOMAIN.com
HTTP/1.1" 302 - "http://www.domain.com/domain.com" "Mozilla/5.0
(Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824
Firefox/3.5.3 GTB5"
188.40.123.123 - - [20/Sep/2009:09:19:46 +0700] "GET /page-not-found/
HTTP/1.1" 301 - "http://www.domain.com/domain.com" "Mozilla/5.0
(Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824
Firefox/3.5.3 GTB5"
188.40.123.123 - - [20/Sep/2009:09:19:46 +0700] "GET /page-not-found/
HTTP/1.1" 503 1114 "http://www.domain.com/domain.com" "Mozilla/5.0
(Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824
Firefox/3.5.3 GTB5"
(IP und Domains wurden ersetzt durch dummies!!! da steht normalerweise die
eigene Domain einmal mit www gefolgt von slash ohne www. und eben die eigene
Server IP)
Diese Spam Bots haben auf dem Kundenservern zur Folge, dass deren MySQL
runterfaehrt, ca. 60 - 90 Minuten nachdem man im Backend aufgehoert hat zu
arbeiten. Ob da ein Zusammenhang besteht versuchen wir noch rauszubekommen,
jedoch ist das bereits jetzt anhand der Logs sehr auffallend. Die Seiten
laufen mit 4.2.8 und eine mit 4.3.alpha3
Meine Fragen an Euch:
*1. Gibt es einen wirksamen Schutz in TYPO3 gegen diese Spam Bots.*
*2. Wenn ja welchen - Extensions??? Kann jemand ewas wirksames empfehlen.*
*3. Gibt es einen weg / Extension - dass diese SpamBot Eintraege dynamisch
mitwachsen? Das Problem duerfte vorallem sein, dass die Bots laufend ihre
Namen aendern wie wir feststellten.*
*4. Hat jemand in letzter Zeit aehnliche Erfahrungen - insbesondere nach dem
Updaten von Extensions vom TER?*
*5. Sind Extensions bekannt die ggf. diese Bots auf die eigene Domain
pointen? Es ist naemlich auffallend, dass auf einem dieser Server nur 3
Domains "betroffen" sind und 2 Domains die wir noch nicht upgedated haben
eben nicht!*
----------------
Mittels .htaccess konnten wir den Amount bereits erhelblich einschraenken.
Die Prozesor Auslastung sank dadurch auf ein viertel von dem was es zuvor
war!
==============
### Blocking Bots and Spams ###
###
http://diveintomark.org/archives/2003/02/26/how_to_block_spambots_ban_spybots_and_tell_unwanted_robots_to_go_to_hell]
###
# Blocking User Agents with no privileges
# User-Agents with no privileges (mostly spambots/spybots/offline
downloaders that ignore robots.txt)
RewriteCond %{REMOTE_ADDR} http://www.DOMAIN.com/DOMAIN.com [OR] # ERROR
SPAM DOMAIN durch die eigene DOMAIN ersetzen
RewriteCond %{REMOTE_ADDR} "^63\.148\.99\.2(2[4-9]|[3-4][0-9]|5[0-5])$" [OR]
# Cyveillance spybot
RewriteCond %{REMOTE_ADDR}
^12\.148\.196\.(12[8-9]|1[3-9][0-9]|2[0-4][0-9]|25[0-5])$ [OR] # NameProtect
spybot
RewriteCond %{REMOTE_ADDR} ^12\.148\.209\.(19[2-9]|2[0-4][0-9]|25[0-5])$
[OR] # NameProtect spybot
RewriteCond %{REMOTE_ADDR} ^64\.140\.49\.6([6-9])$ [OR] # Turnitin spybot
RewriteCond %{HTTP_REFERER} iaea\.org [OR] # spambot
RewriteCond %{HTTP_USER_AGENT} ^[A-Z]+$ [OR] # spambot
RewriteCond %{HTTP_USER_AGENT} anarchie [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} Atomz [OR] # rude bot
RewriteCond %{HTTP_USER_AGENT} cherry.?picker [NC,OR] # spambot
RewriteCond %{HTTP_USER_AGENT} "compatible ; MSIE 6.0" [OR] # spambot (note
extra space before semicolon)
RewriteCond %{HTTP_USER_AGENT} crescent [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} "^DA \d\.\d+" [OR] # OD
RewriteCond %{HTTP_USER_AGENT} "DTS Agent" [OR] # OD
RewriteCond %{HTTP_USER_AGENT} "^Download" [OR] # OD
RewriteCond %{HTTP_USER_AGENT} EasyDL/\d\.\d+ [OR] # OD
RewriteCond %{HTTP_USER_AGENT}
e?mail.?(collector|magnet|reaper|siphon|sweeper|harvest|collect|wolf)
[NC,OR] # spambot
RewriteCond %{HTTP_USER_AGENT} express [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} extractor [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} "Fetch API Request" [OR] # OD
RewriteCond %{HTTP_USER_AGENT} flashget [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} FlickBot [OR] # rude bot
RewriteCond %{HTTP_USER_AGENT} FrontPage [OR] # stupid user trying to edit
my site
RewriteCond %{HTTP_USER_AGENT} getright [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} go.?zilla [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} "efp at gmx\.net" [OR] # rude bot
RewriteCond %{HTTP_USER_AGENT} grabber [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} imagefetch [OR] # rude bot
RewriteCond %{HTTP_USER_AGENT} httrack [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} "Indy Library" [OR] # spambot
RewriteCond %{HTTP_USER_AGENT} "^Internet Explore" [OR] # spambot
RewriteCond %{HTTP_USER_AGENT} ^IE\ \d\.\d\ Compatible.*Browser$ [OR] #
spambot
RewriteCond %{HTTP_USER_AGENT} "LINKS ARoMATIZED" [OR] # rude bot
RewriteCond %{HTTP_USER_AGENT} "Microsoft URL Control" [OR] # spambot
RewriteCond %{HTTP_USER_AGENT} "mister pix" [NC,OR] # rude bot
RewriteCond %{HTTP_USER_AGENT} "^Mozilla/4.0$" [OR] # dumb bot
RewriteCond %{HTTP_USER_AGENT} "^Mozilla/\?\?$" [OR] # formmail attacker
RewriteCond %{HTTP_USER_AGENT} MSIECrawler [OR] # IE's "make available
offline" mode
RewriteCond %{HTTP_USER_AGENT} ^NG [OR] # unknown bot
RewriteCond %{HTTP_USER_AGENT} offline [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} net.?(ants|mechanic|spider|vampire|zip)
[NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} nicerspro [NC,OR] # spambot
RewriteCond %{HTTP_USER_AGENT} ninja [NC,OR] # Download Ninja OD
RewriteCond %{HTTP_USER_AGENT} NPBot [OR] # NameProtect spybot
RewriteCond %{HTTP_USER_AGENT} PersonaPilot [OR] # rude bot
RewriteCond %{HTTP_USER_AGENT} snagger [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} Snapbot [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} Sqworm [OR] # rude bot
RewriteCond %{HTTP_USER_AGENT} SurveyBot [OR] # rude bot
RewriteCond %{HTTP_USER_AGENT} tele(port|soft) [NC,OR] # OD
RewriteCond %{HTTP_USER_AGENT} TurnitinBot [OR] # Turnitin spybot
RewriteCond %{HTTP_USER_AGENT}
web.?(auto|bandit|collector|copier|devil|downloader|fetch|hook|mole|miner|mirror|reaper|sauger|sucker|site|snake|stripper|weasel|zip)
[NC,OR] # ODs
RewriteCond %{HTTP_USER_AGENT} vayala [OR] # dumb bot, doesn't know how to
follow links, generates lots of 404s
RewriteCond %{HTTP_USER_AGENT} zeus [NC]
RewriteRule .* - [F,L]
RewriteCond %{HTTP_USER_AGENT} ^(-?|[A-Z]{10})$ [OR]
# A host which tries to hide itself in reverse DNS lookup
RewriteCond %{REMOTE_HOST} ^private$ [NC,OR]
# Web surveying sites (may require using ipchains)
RewriteCond %{HTTP_REFERER} (traffixer|netfactual|netcraft)\.com [NC,OR]
RewriteCond %{REMOTE_HOST} \.netcraft\.com$ [NC,OR]
# A fake referrer that's often used -- use this unless your pages are
related
# in some way to atomic energy and could really be linked to from
www.iaea.org
RewriteCond %{HTTP_REFERER} ^[^?]*iaea\.org [NC,OR]
# "addresses.com" is a referer used by an email address extractor
RewriteCond %{HTTP_REFERER} ^[^?]*addresses\.com [NC,OR]
# A fake referrer that's used in conjuncting with formmail exploits
RewriteCond %{HTTP_REFERER} ^[^?]*\.ideography\.co\.uk [NC]
# The rule which blocks out further access from the host
RewriteRule .* /cgi-bin/bad.pl [L,T=application/x-httpd-cgi]
# Bad requests which look like attacks (these have all been seen in real
attacks)
RewriteRule
^[^?]*/(owssvr|strmver|orders|Auth_data|redirect\.adp|MSOffice|DCShop|msadc|winnt|system32|script|autoexec|formmail\.pl|_mem_bin|NULL\.)
/cgi-bin/bad.pl [NC,L,T=application/x-httpd-cgi]
# Filter out bad requests (may need to be adjusted to your needs)
RewriteCond %{THE_REQUEST} "^((GET|POST|HEAD) [^/]|CONNECT)" [NC]
RewriteRule .* /cgi-bin/bad.pl [L,T=application/x-httpd-cgi]
RewriteRule
^(/(scripts|msadc|MSADC|./winnt)|.*(default\.ida|[NX]{30}|c\+dir))
/cgi-bin/blocklist.pl [L,T=application/x-httpd-cgi]
==================
Danke Andi
More information about the TYPO3-german
mailing list