How To Stop Forum Spam – Anti Spam Systems

As many of my site visitors know, I own/manage a handful of internet forums including:

• The Printer Forum (on this site)
• BackYardChickens.com
• TheEasyGarden.com
• SufficientSelf.com
• BackYardHerds.com

One of the constant problems with managing a forum is dealing with spammers.   Spammers come in many shapes and forms, but here are the two main types:

1. “Spam Bots” –  These are applications/programs that are automated and are usually targeted to the more popular forum software applications.   They can automatically register and post on thousands of internet forums.
2. Human spammers – These are usually people who are trying to promote their own websites and businesses, or work as “SEO” or “Link Building” professionals that register and spam internet discussion groups, forums, blogs, etc.  Often they will outsource and hire people that are $1 – $2 per hour in India, China, Philippines, etc. to sit in a room and spam all day long… 10 hours a day, 6 days a week.

Originally, I was just working to prevent the “Spam Bots” from registering and posting.   Every day or so we’d see a human spammer come in, but it is usually pretty easy to just delete and ban them manually.

Well, that all changed around the first day of the new year 2011.   All of a sudden we started to see a TON more spam… like a factor of 1,000 times more spam.   On BYC, during the “slow months”, we get about 50 new members a day.   Well, during the first 12 hours of one day last week we saw 300+ new registrations.   They hadn’t posted yet, but we knew they were spammers by the totally irrelevant (and often inappropriate) links in their user profile signatures.

After deleting all the spammers, I embarked on a multi-day and very time/energy tense process of beefing up our anti-spam systems!     Below, I’ll list the anti-spam systems we have in place.  These were not deployed or added in this order, but they are listed in the order a spammer needs to go through them to get access to spamming our community.

(Oh, a side note:  The goals with any anti-spam system are to stop or slow down the spammers so they get frustrated and leave BUT also to not block or restrict legitimate users… not always possible, so compromises have to be made)

Lines of defense against forum spammers:

1. “Bad Behavior” – This is a system that blocks known spammers and bots from accessing your site in the first place.   Within 30 minutes of installing it I had blocked 64 access attempts!  A good number of spammers still get through, but this is a great first roadblock.
2. Human Detectors – Sounds very sci-fi, huh?   Undoubtedly you’ve already seen these.  Often they are in the form of a captcha (the image you have to read and type to prove you’re a person).  One of my favorites is ReCaptcha.  In addition to the image to text test, we also have a question and answer that would be almost impossible for a spam-bot to answer.
   These systems stop almost all bots, but are easy for human spammers to get through.
3. Email Verification – Before someone can join the forum, they must submit an email address, receive an email from our system, then validate their email.   This stops a good amount of spam, but the spam-bots and human spammers can get through it easily, especially with the proliferation of free email account systems.
4. Signature Limits –  About 99% of spammers put a spam link in their signature.  This is how they quickly spam the system even without posting.   We implemented a restriction that doesn’t allow new members to create a signature until they are no longer “new”.
5. URL Posting Limits – This has had the biggest impact in stopping spammers, but also has the most “unintended consequences” to legit members.   How it works:  A new member can NOT post any images or links until they are no longer “new”.    We haven’t seen a single spam post since we implemented this system.  Unfortunately we’ve already seen new members get frustrated that they can’t post links and especially pictures.   Fortunately the limitation isn’t too long, but it is a necessary evil.
6. Reports, Great Members, Excellent Moderators –  This final stop-gap is about 100% effective!   It takes our members about 1.4 seconds to start reporting spammy posts.    Not long thereafter one of our amazing moderators deletes and bans the account.   (Side note:  Our members are so good at reporting that we sometimes get hundreds of reports at once.  Because of this we ended up installing a modification that hides reported posts after multiple people report it.  HUGE time saver!)

There are a few more things we could implement to stop or slow down spammers if they start to get through these systems, but the cost to the members begins to be a significant trade-off.    For now I expect the automated systems to stop 99.701% of spam and the members and moderators to easily pick up the rest.

Footnotes:

Why such a huge spike in spam all of a sudden?   Well, check this out:   Of the thousands of spammer accounts that we blocked and/or deleted, most are porn, affiliate sales, virus, buy my stuff online, etc. spam junk.

BUT I saw a handful of legitimate looking businesses.   They had very good .com domain names and a presence online that looked legitimate with a physical location, telephone number, etc.

Well, guess what I did!?   Today I spent a few hours calling a handful of them, told them I own some internet forums that are being spammed with their company name and URL.   I asked if they were outsourcing their SEO / Link building and if they knew that their brand was getting tainted and that they possibly might get put on some internet blacklists.  The replies from most were:

“Yes, we use an SEO / Link building company and we had no idea they were doing this!”

I’m betting that these legit companies think, “Wow, we don’t want to bother with online marketing, so let’s hire it out!”  They find a company or individual that says, “Sure, we’ll build your presence online for CHEAP!” and then they deploy spam-bots and/or hire cheap labor to spam all day long.

One company finally got me routed through 4 people until I got to their head of internet marketing.   He said they purchase “link packs” off the internet that show “high quality” sites on which they can promote their products.    They then have contractors “participate” on these forums and promote their stuff.

So, it’s a pretty crazy spam war out there!  Unfortunately there are some legitimate companies that are paying “professionals” to market their sites, but those professionals are just turning around and using “black-hat” spamming systems and processes.

Have you had a problem with spam on any of your blogs, sites, etc?   What has, or hasn’t worked for you?   Have you ever been flagged or blocked as a spammer?   As a user, how do you feel about all these anti-spam systems that directly or indirectly affect you?