Blogging Roller: re: The Referer Spam battle rages on

re: The Referer Spam battle rages on

Lance comments on the recent surge in referrer spam. I've been deleting referrers and adding ugly new ignore words to my Roller installation on a daily basis this week. It is really getting bad. Three options I would like to see in Roller:

Referrer white list: a blog only accepts referrers that include words that are listed in the blog's white list.
MT Blacklist: use the MT Blacklist data to filter referrers just as we now (in Roller 0.9.9) filter comments.
Technorati check: a blog only accepts comments that come form other blogs that are in the blog's Technorati cosmos.

Dave Johnson in Roller • 🕒 01:10PM Oct 14, 2004

Tags: Roller

Comments:

Hey don't I get some credit for the white list idea?

Posted by Steven Citron-Pousty on October 14, 2004 at 09:31 PM EDT #

Yes, credit where credit is due. The white list idea was suggested by Steve in #roller on freenode, the MT blacklist support in Roller was added by Lance, and I came up with the Technorati check idea.

Posted by Dave Johnson on October 14, 2004 at 09:45 PM EDT #

=) Thanks

Posted by Steven Citron-Pousty on October 14, 2004 at 09:47 PM EDT #

Just to clarify, the MT Blacklist support is currently only for Comments. The creator of MTBlacklist (sorry, forgot name) didn't think it would be useful against Referer Spam, I'm not convinced. I don't see the other two solutions being of much use: 1) a whitelist would prevent me from seeing most of my referers (I see new addresses every day) and 3) the Technorati check would also fail as I see many "spam referers" from blogspot.com (as well as some legitimate referers). Not that I really understand the Technorati cosmos.

Posted by Lance Lavandowska on October 15, 2004 at 03:09 PM EDT #

Lance, you are right about the whitelist - it would take away one of the primary benefits of referrers, learning about new sites that link to yours.<p /> If we set a threshold on blog authority, then perhaps a Technorati check would weed out the dummy spam blogs that you mention.

Posted by Dave Johnson on October 15, 2004 at 03:43 PM EDT #

[Trackback] I just had a thought: how about Referer throttling? Like implemented elsewhere for Comments (we should do that for Roller ): don't increment Referer count, from the same IP, for 'X' seconds. Not sure exactly how we'd record this efficiently though. ...

Posted by Vanity Foul on October 15, 2004 at 05:33 PM EDT #

What about greylisting
http://projects.puremagic.com/greylisting/whitepaper.html
Some parts of the algorithm would not work because there is no MTA, but we could look at something like number of unique referrers from the same IP within some time period.
Lance, I wonder if your situation is not unique (or at least rare) in terms of new referrers every day. I am not sold on the white list idea, but it would probably work for a good chunk of users and it would be relatively easy to implement.

Posted by Steven Citron-Pousty on October 18, 2004 at 09:30 AM EDT #