I love Akismet – it’s a WordPress tool that has literally saved me months of work. It has blocked 4,059,113 comment spams on ProBlogger alone over the last year or two – something that I will be forever grateful for!
However every day or two I get an email like this:
“I have been trying to leave comments on your blog but they never appear! Did I say something wrong?”
Now when a blog gets as many comment spams as I do in a day there are bound to be some false positives – but over the last couple of months these emails are getting more and more frequent. The problem concerns me for two reasons. Firstly I want everyone who wants to comment on my posts to be able to. Secondly many of those whose comments are falsely blocked think that I’m the one behind their comment being blocked. I’ve had angry comments from readers as well as bloggers blogging about me censoring them. The reality is that unless a comment is explicit, spammy or defamatory I don’t delete comments.
I’m not sure what the reason is for these false positives it but it’s becoming apparent that quite a few legitimate blog readers are getting on Akimset’s blacklist falsely. It’s also seems to me that when I mark a comment as ‘not spam’ that Akismet isn’t ‘learning’ of it’s mistake because quite a few people’s comments seem to still be filtered as spam. Akismet says that it can take a couple of days to fix these but I’ve had a few people not be able to comment for weeks now. My approach is generally to ask them to contact Akismet directly. Quite often these people are unable to comment on other blogs also.
Perhaps Akismet could provide us with some way for us as bloggers to add to a whitelist of commenters. I’d love to simply take the emails of those who have problems and add them to such a list that automatically lets people comment. I’m also wondering if there’s some way of notifying commenters that are filtered as spam that this is what has happened and to point them into the direction of some sort of procedure to rectify it if they feel that their comment isn’t spam?
I’ve noticed that with some blogs I’ve owned in the past. I have mass amount of spam and there are some normal comments in there that I totally miss.
Well I suppose we can’t complain with the job it does :)
I’ve found this happens a lot less with Akismet since I uninstalled Span Karma 2.
That’s excelent idea, but it would probably take spammers about 24h to set up some blogs, whitelists themself as legit users.
You can try some other spam-blocking tools like Spam Karma 2 or Bad Behaviour. SK2 has so many options that it’s practicly inposible to know all of them.
On my blog SK2 and BB works great and I turned Akismet off.
I’m in the same position, Darren.
I’ve had to deactivate Akismet for the same reason, but checking each message (99.9 percent is spam) is not the answer, and I’ve not had luck with Bad Behavior either.
What’s working for other readers? I’d like to know through comments.
I noticed recently that more of my regular commenters were having their comments “swallowed up” by Akismet. I put up a post about my needing to “fish out” the missing comment for them. I couldn’t find a pattern to the false positives.
This isn’t a solution – I too would LOVE a whitelist – but since we installed Bad Behaviour, almost all the real spammers are blocked by that plugin. We get a couple of dozen comments a week which get through Bad Behaviour and are caught by Akismet, so it’s very easy to see which are the false positives.
I’ve been using Spam Karma 2 with great success. Do I really need to incorporate Akismet as well?
At the moment, I tend to get only 5 spam comments a day… so it’s pretty easy for me to look through them and double check them. So far no false positives.
However, if/when my comment traffic gets heavier, I can imagine it being much harder to manually check the spam queue. A whitelist would be very handy.
I have the same problem too and find myself combing through the spam from time to time to make sure I haven’t missed a legit comment.
Akismet is letting through an awful lot of spam for me at the moment.
In the past I’ve very rarely monitored comments, everything just seemed to work out fine but in the last I’m receiving hundreds of spam comments a day which I manually have to mark.
The Solution Is Simple
If your comment is being blocked that is and assuming you have more than one blog which most people do.
Write a comment on your ‘other’ blog and when it gets trapped as spam, mark it as non spam, delete it and repeat.
After marking your comments as non spam several times Akismet gets the message and stops filtering you out internet wide. If you spam a LOT however… I suspect you’ll be doing this for days.
I’ve just installed akismet on moveable type in the last few days and am so impressed with how well it works (no accidents so far!).
A whitelist is a really good idea – the tricky part would be designing something that only allowed valid bloggers to submit to the list
Thankfully, I have not had this problem when commenting.
I think the idea of a white list would be very helpful.
I love Akismet, it’s saved me from tens of thousands of spam (which pales in comparison to your numbers, but for a personal blog, five figures of spam is staggering, and annoying!) It’s only very rarely that I find a false positive, but there have been a few over the time I’ve used it.
A whitelist is a great idea, and I sincerely hope the folks at Akismet read this post and act!
I feel the same way. I get so much spam in the Akismet that I can’t check to make sure it’s all spam. I just deleted over 2500 spam comments I and I sure hope there were no legit comments. It’s the life of blogging I guess.
OOOH!! I got a list of them that Akismet couldn’t catch because they were so normal.. yet.. didn’t relate to the post. lol
“To do anything truly worth doing, I must not stand back shivering and thinking of the cold and danger, but jump in with gusto and scramble through as well as I can.”
“Listen. Do not have an opinion while you listen because frankly, your opinion doesn’t hold much water outside of Your Universe. Just listen. Listen until their brain has been twisted like a dripping towel and what they have to say is all over the floor.”
“We are what we repeatedly do.”
Can anyone ever think these were fakes? Some I had to read 2 or 3 times to understand the connection of them with my post.
I’ve yet to have any false positives but Askimet has grabbed all of 15 Spam. That being said, I’ve had about half that get through (which I caught in moderation).
Still, I’d certainly be willing to go in and whitelist people, especially in these early blog development stages.
Luckily I’ve not had any such problems with Akismet, but having said that I don’t get too many comments anyway :(
I’ve also had a few false positives, but when you look at the blocked comment it’s difficult to understand why.
No embedded links, no cursing, no mention of dodgy websites or drugs for sale and yet the filter stops them.
Perhaps Akismet could provide some indication as to why it rejects the comment.
I use Blogger and I don’t have any filter or plugin for my comments and I have only been spammed once. The only thing I use is the “wait for moderation” feature and just approve eveyr comment I get in my email.
My site isn’t nearly as big as ProBlogger, but I’ve run a site on WordPress that barely got 50 people viewing it as its biggest one day total and it gets dozens of blocked spam per day.
Maybe Blogger blocks these spams easily (I dont use captchas) and doesn’t show them in the interface, but I think it’s something inheriently wrong with the Word Press comments that attracts the spam from my very simple experience with the two platforms.
Askimet works very good for me..
I think akismet should provide the ‘scores’ it gives each spam comment. Bloggers should then be able to set their own threshold for review. That way, the bloggers can review the spam that is borderline, but ignore the pure spam.
Our email server spam filter works like this, if the spam ‘score’ is over a certain amount, the server just ditches the email, but if it’s only a little spammy, we get an email saying it caught it. That way if it was wrong, we can correct it.
For what it’s worth, as a comment moderator here on PB, I’ve been using Auntie Akismet (a greasemonkey script for FireFox) to help me with the comments trapped up by spam here.
Seriously, I’ve gone in there to find over 9000 spams at once. This helps me see spam in a nice, one-line format, and scan through those who only left one or two (or 5 or 20) comments. It’s been a great help, but still, there’s no guarantee I won’t miss one here and there.
As Darren does, I too don’t delete comments unless they’re spammy, explicit, or defamatory. And if I’ve got a question about one, I either save it for Darren or email him to let him know.
I’ve used SpamKarma on other blogs, and it seems to take a lot longer to go through and delete the spam. It does work well, I give it credit, but it also hasn’t been updated in a while, so sometimes I wonder. I’ve gone back to using Akismet on all my own blogs at this point. And Bad Behavior never worked right for me… no idea why, but it just never communicated with my server or database properly. *shrug*
Anyway, I just wanted to let everyone know about the AA plugin (don’t know the link at the moment) and reassure you all that I follow Darren’s “rules” when it comes to comment modding here on PB. :)
I tried to comment, and was rejected (I think…), because I included a url for WP SpamFree plugin – I use it now as a standalone instead of Akismet & Bad Behavior, and so far it works like a charm
Many times, it isn’t Akismet. WordPress isn’t perfect at posting comments, and after suggesting that the user try again… it goes through without problem.
A company called Mollom is developing an alternative to Askimet that has an accuracy of 99.77%. They were featured on TechCrunch. Try Googling them, unfortunately currently we’re waiting for a WordPress plugin, but if you run Drupal your in luck.
I think that Akismet should have a faster learning capability, but still takes into account spammers trying to game it.
There needs to be a system in place to value certain blogs over others, so that spammers can’t simply create blogs to de-spam their own comments. Some sort of process that only real human beings would go through.
But if the history of the internet is any indicator, every trick you throw at a spammer will eventually be circumvented. You have to constantly innovate to stay a step ahead. Anti-spam and anti-virus companies definitely have their work cut for them.
However, considering the volume of spam akismet has been able to block (false-positives, notwithstanding), I think they do a pretty good job, especially being a free service.
The question is why is askinet seeing theses comments as spam. I suspect somewhere down the line the bloggers may have left a comment, which was poor quality or negative, this then got marked as span and has hence blacklisted them. I often see new bloggers following the tactic that writing comments in large quantity will gain them traffic, with no regard for quality. This could lead to a comment be marked as spam. On the other hand, must not rule out the accidental marking of comments as spam as well!
I’d like to second (or third) the opinion that SK2 + Bad Behaviour works a treat. I was turned off Akismet from the beginning precisely because of the reasons that you have stated in your post.
I use a combination of Spam Karma 2, Trackback Validator and AntiLeech. I think I’ve only gotten three false positives in the past year and a half.
I LOVE that idea. I had a 2-week NIGHTMARE where ALL my comments went to spam despite all my regular blogs rescuing me over and over again. Took 2 emails to Akismet to fix the problem. :( It was HORRIBLE!
I use WordVerify in combination with Akismet. Reduced my comment spams from 50 to 0 (with 1.500 visits a day). Only some trackback or ping spam left to check in Akismet. A great relief.
I had some ‘false positive’ problems about 6 months ago. While the Akismet team was good about clearing it up, it is especially annoying to know that someone can take away your ability to comment on almost all WordPress blogs with a single malicious or accidental click.
That said, there has been a big increase recently in automated comment spamming so I’m glad there is some help in dealing with that situation.
This is a great idea. I get a whole bunch of spam on my financial blog (no idea where it came from) and I don’t want to be wasting my time sorting through it everyday (especially as my blog grows).
But also I don’t want to block out those that want to comment. A whitelist is a fantastic idea.
Great post Darren, you continue to inspire me to write a better blog
Would be nice to get option to create
yourself white list of commentators!
“I suspect somewhere down the line the bloggers may have left a comment, which was poor quality or negative, this then got marked as spam and has hence blacklisted them” I agree with that. I believe that’s the most provable reason why legitimate comments are swallowed as spam by Akismet.
The most annoying spam comes from spambots which can post thousand of porn and pharma links on your blog. human spammers are usually the kind of “great post” or “I agree” comments.
I use Bad Behavior and Akismet on my blog. While BB takes care of spambots, AK receives those which have passed the first line of defense (not too many indeed).
Having such a short list on AK spam queue it’s not a problem to me to take a look for false positives before mass deleting spam.
Hey! Darren I’m so glad you touched on Akismet, this was a problem for me once. I had to contact Darren and ask him what happened, he advised me to contact Akismet with my problem. I did just that, they contacted me about 4 days later letting me know that problem was fixed.
I must agree it is a problem, and I believe this is the reason. If you see the post on is key-wording on comments spam? Darren wrote, you will notice a lot of webmaster’s will mark a relevant legit comment spam because of the keyword of in the name field.
This also happens for short comments, non-relevant comments, and comments like “great site!,” “love the blog,” etc. When webmaster’s mark these comments spam, Akismet picks up on this and learns and then the commenter will be blocked from every blog that uses Akismet.
Thanks guys. These tips are very helpful. I have the same problem though usually, I am the person who would write:
“I have been trying to leave comments on your blog but they never appear! Did I say something wrong?”
So, I decided to start my own blog instead! Good idea, don’t you think?
http://www.efm.lk
Wouldn’t a solution be for Akismet plugin to have a LOCAL whitelist? Not a big Internet wide one, but a local one on my installation. Then I can add anyone I want to it, and not worry about the rest of the blogosphere. Seems like an easy code for the folks at Automattic.
Well, on the commenter’s side, make sure you add something. Don’t just say “You gave great advice! Loved your post!”
Some people, especially those with popular blogs, may just tag you as Spam out of annoyance. I know I sometimes think it, even when just reading the comments in another blog.
I don’t think a whitelist is a good idea. Generally the commentators were reported for something in the past that got them in the blacklist, and so even if some of them deserved to be moved back, the risk of them actually getting their blacklist turned into a whitelist might end up with even more spammy comments. This way, a spammer could keep a legit profile on big blogs to get whitelisted, then spam a ton of little blogs… just my two cents.
I’m glad that my comments don’t get caught by akismet. Having a blog with much less traffic that Problogger, I don’t have to worry about this quite yet. I’m not sure how to increase the effectiveness of akismet, but perhaps the blame lies in the commenter?
I’ve had the same problem before. Contacting Akismet directly solved the problem within a day.
It’s a sign that Akismet needs some serious revamp or upgrade. With all the WP-powered blogs out there, Akismet may be getting under a lot of strain.
I know there’s a plugin that notifies commenters whenever their comments have been flagged as spam. Just forgot the name though. I’m sure Google knows. :P
I’m also using Akismet on my blog. Sometimes it is considering trackbacks as spam.
And one thing i’m wondering about is that, before using Akismet i was never getting those long spam comments with lots of links.
Regards,
Shaan Haider
@Shaan
Good point. What if the whitelist was blog specific? That way the little blogs would still be protected.
Darren – do what Mike Cherim does. He has a notice on his contact form that says “If your response inexplicably vanishes, it’s Akismet. I will rescue it.”
Perhapsj including such a by-line and a link to your contact page instructing the user to contact you so the problem can be fixed would be a good interim solution.
I also was blocked for leaving a url. Finally got the problem fixed. However I noticed recently my comments are not showing up on one or two blogs for no apparent reason.
I’m tired of dealing with the problem so I just unsubscrided.
The Masked Millionaire
There are a lot more bloggers now, a lot of them using wordpress and akismet. But not all people understand spam: is a couple of generic sentences spam? is using keywords as the name spam? Judging from a previous post about using keywords instead of names, a lot of people seem to be reporting comments (even if they are of good quality) as spam.
Once the commentor is marked, incorrectly, as spam then it’s battle to get that changed.
So, for akisment users, think carefully before you mark someone as a spammer. Maybe a delete would be sufficent? Or, if you have specific requirements from commentors then let them know before they commnent!
Haha! Having said I’ve never had any false positives… I got my first one today :P
I hate to give away my secret, but only for you, Darren. Just because I love you… ha ha.
While Akismet will always have false positives, the trick is to install some kind of friendly CAPTCHA (but not one of those gross image things that make you go blind). On top of Akismet, I use WP-Gatekeeper by Eric Meyer of CSS fame.
It lets you set up your own questions with a free form response (not case-sensitive). I will often put in multiple-choice questions like this: “Identify the weapon in this list: asphalt, bacon, cloud, dagger”. Pretty simple for a visitor to fill out.
You can create a bank of questions and change them every so often if you like (although I’ve not had to as of yet).
Despite Eric’s warning, I’ve been using it since WP 2.1 or so without problems (now using on 2.5.1).
Here’s the key, though. 99.99% of all spam, as you know, is bot-generated. It automatically ceases upon install of Gatekeeper. I haven’t had a single one get through even to Akismet since I installed it (and before that I had 84,000+).
So, the only people who wind up in your Akismet queue are real people with a “spam” flag on either their IP or post-content (depending on your rules).
This happened quite recently to a friend of mine. Unknowingly, he thought he’d be clever and use his break at school (he’s a publik skool teacher) and comment on blogs in an overly promotional way about some of his web projects. His quasi-sploggy comments, combined with a static IP address of a publik skool (used for all sorts of web-badness) caused him to wind up on Akismet’s badboy list.
But when he came to post some comments on my blog, I would see, for the first time in months, an actual comment in my Akismet queue. It was no big deal to go in there and approve his comment since there was nothing else in Akismet.
I’d be curious how well this would work on a larger scale blog. I get a decent amount of traffic but nowhere near the volume of you or T/C – yet I hear you both always worried about the amount of Akismet false positives.
The only way for a spam poster to get into Akismet is to do it manually. And because you have such a highly-ranked blog, you might have more than your fair share of such dorky attempts.
In theory, the known spam commenters who manually fill out the reply box and answer the test should still get caught by Akismet, but I’d think that the majority of your Akismet queue would basically turn into a queue of false positives (rather than the reverse as it is now).
Only other point is that the questions you use can be off-putting to foreign visitors with a limited grasp of English. In my way of thinking, considering that is where the majority of spam comes from, I’m all for it. But I did have one recent lady from England get frustrated by getting the question: “The NFL team in Miami?”. She had to look it up online. She could have refreshed the page to get a different question, but I didn’t have instructions to that effect. There’s probably a way to put a little micro-script on the page like the old CAPTCHA images that says “Can’t read this? Get a new image.”, but I haven’t done it yet myself.