No More Suspect Traffic
30 Jan 2005
I’m fed up with the entire cornucopia of suspect tinned meat products that float amongst the web. I leave comments open on all my old posts to encourage further discussion in the future; whether this will remain is in serious jeopardy. While there are many effective ways of lessening comment spam, referrer spam is another beast entirely — an insidious, pointless resource hog that baffles me by its mere existence. Do they really think webmasters
are going to buy SEO tools from spammers?
As a result, I have added some severe ReWrite rules to .htaccess in an attempt to stem the tides. I’ve blocked anything remotely suspicious, including all localhost IP’s (my log files reveal someone is developing locally using one of my CSS files… you idiot); any odd referrers that reside behind 403 Authorization or are ‘vacant’ but return suspicious whois queries.
One tool that I found very useful whilst testing these changes is wannaBrowser, which fakes HTTP user agent strings and referrers.
If I have any reason to believe that I’m blocking legitimate users with these measures I will change the redirection to a page that allows you to contact me and say that you feel hard done by. Otherwise I’ll just leave them in limbo and not waste another precious kilobyte on all this suspect traffic.
Natalie:
It would be great to know the process for blocking spam using .htaccess and modRewrite rules, as most of us suffer from the same problem.
While I’ll be implementing Google’s nofollow also, having someone tell me how to do it using better tools would be very handy indeed.
Many of us feel your pain.
Roger Johansson:
Natalie: How to block referrer spam with mod_rewrite is described in Killing Referral Spam and Killing referrer spam. The articles describe slightly different approaches. Both work, and my referrer logs are usable again
Andrew:
Sure Natalie
Here is a good run down on blocking bad referrers using htaccess.
In my case, each wave of spam correlated to a referral from a domain whose index was a holder page. Once I checked my logs against the timestamps on the comments, I knew that this site was responsible for the attacks.
edit: Good timing Roger!
Geoffrey Sneddon:
Well, after I moved host, OK, my host moved host, I stopped getting a lot of the spam I was receiving, although my email hadn’t changed, which was nice, still getting 9/10 a day though…
As for the .htaccess, I’m still here
Will Chatham:
There seems to be a trend developing here. Just as Google implements nofollow and people start to get a handle on comment spam, referrer spam seems to have stepped up a notch. Everyone seems to be blogging about it now.
What comes next, after we stop referrer spam?
Seth Thomas Rasmussen:
The upcoming ShortStat release has a nice tool for combatting referral spam. A simple click deletes a referal and blocks all future referals from that domain.
Is your spam really that out of hand?
I had problems for a while, but WordPress caught 99% of the spam comments I’ve ever gotten. A quick trip into MySQL Control Center, a few clicks, and the offending records are gone and out of my life.
The mountains of notification emails were really my biggest beef. I wish WordPress had an option to forego the notification if it’s a comment held for moderation. Maybe a batch notification at the end of the day?
I’d suggest this stuff in their forums, but what can I say… I hate their fucking forums.
Andrew:
Seth: Have you seen the new shortstat? I’m very interested in having a look as I have a nice little graphing extension for it on hold until the new version is released…
And yes, my spam was that out of hand. 80% of the problem came from coresat.com — yes, I’m naming names! CORESAT.COM IS A SPAM SERVER!
Seth Thomas Rasmussen:
Yeah, Andrew, Shaun included me in the initial beta testing. He later decided to exclude me for reasons unknown. :shrug:
Geoffrey Sneddon:
I’ve also been getting a lot of spam from coresat.com, but ever since I got the MT-Blacklist list and used it, I’ve almost completely eliminated spam, even though I can’t use the RegEx at the top
Matthew Ross:
I just thought I’d mention that I went to coresat.com (I’m a curious person) and there is a great big “Website suspended.” Nothing else. I’m guessing someone else realized it was a spam server too.
Geoffrey Sneddon:
Nope, it’s their website, they are still sending spam, just deleted one from the moderation queue of my blog…
Andrew:
1? Lucky you – I just deleted 18. Standard morning routine, unfortunately
Geoffrey Sneddon:
I didn’t mean that, I meant as in deleting yet another comment, I had actually just deleted three, but I guess it’s somewhat to do with how long your blog has been online for, and your blog has been online for almost 6 months more than mine, anyhow, just added some extra things to the list of words/URLs/IPs to put comments into the moderation queue… I’ve only got around 2800 entries in the list… :p
Jon:
Seth: It looks to me like my installation of WordPress (1.5) has an option not to notify on comments held for moderation. Take a look in Options > Discussion > E-mail me whenever…
David George:
I have been plagued by referrer and comment spam on a site I run. ModSecurity is probably a better route than ModRewrite – you can tackle both referrer and comment spam plus it is less resource hungry.
Here is a paper I’ve recently written on what I did.
Combatting Comment and Referrer Spam
Jon:
You are asking for trouble if you post links back to the referers. It’s sort posting a note saying “please referer spam me”.
fambizzari:
When i set up my blog, i never expected that spam would be the result.
It’s exciting and rewarding to see the number of people that read your blog, but then the type of things that are being spammed make me sick.
I know another blogger who simply removed comments altogether, and i think i might follow suit.