Thankfully Lieutenant Servy was on-duty yesterday to point out a strategic flaw, basically summed up; RRs (robo-reviewers) that currently exist would allow robo-editors to get into the system (and become RRs themselves) and thus allowing the negative feedback loop to continue. Therefore I have come up with several suggestions that may help combat this issue,

Again, I'm not expecting all of these ideas to be implemented (these are only suggestions to fix the mentioned problem). Please bear-in-mind that it is practically impossible to achieve a "robo-less" community, although we can (and hopefully will) reduce their numbers.

Phase 3 (loop regulations)

  • Offer badges for reporting RRs,

    • Robo Scout (bronze): Successfully reported 2 RRs.

    • Robo Spy (silver): Successfully reported 5 RRs.

    • Robo Mole (gold): Successfully reported 10 RRs.

  • Offer X reputation to users that successfully report a RR.

  • In addition to banning a RR from accessing the queue for Y duration; reset the number of reviews the user has made to 0.

  • Deduct Z reputation from users that have been successfully identified as a RR.

  • Increase the physical size of the "Skip" button. And reduce the size of the "Approve" & "Reject" buttons.

  • Increase the waiting period for the "Approve" & "Reject" buttons to become clickable.

I'll now explain the reason(s) behind each idea; offering badges/rep for hunting down RRs will add more incentive for uses to report such people and therefore reduce the number of RRs that have access to the queue (and therefore cause a greater "lean" towards a positive feedback loop).

Resetting the reviews count to 0 will cause reviewers (that are only interesting in getting a badge, which seems to be one of the main reasons why there are RRs) to slow down and check what they're approving/rejecting is actually worthy of the decision they've made. (Allowing RRs to still have access to the queue, but causing them to think more carefully about what they're doing).

Subtracting Z reputation (fear incentive) from such users will generally cause everyone to be more carefully, and to remember it's not a race!

Physically increasing the size of the "Skip" button will make it easier to skip an edit (which is better than falsely approving/rejecting one), and reducing the size of the "Approve" & "Reject" buttons will raise cognitive function and require high levels of awareness (to make sure you're actually going to click the right button).

Increasing the delay period will force reviewers to spend more time on the actual edit they're reviewing (which should hopefully result in better judgments being made).

Of course all ideas/constructive criticism/suggestions are welcome...

80% of the community have agreed with my last plan Which has only been posted for 23 hours, seems a bit quick to say that 80% has agreed with it. –  bluefeet Jun 12 at 13:24
44 users who bothered to vote != the community. –  Bart Jun 12 at 13:26
Servy is only a Lieutenant? I thought he would have been promoted after all this time. –  Frédéric Hamidi Jun 12 at 13:28
We have badges for catching bad posts (Deputy/Marshal). But a badge for catching a person... I don't think that's the way we should go. Besides, what's to keep people from flagging every reviewer as a robo-reviewer, just to get their badges? –  S.L. Barth Jun 12 at 13:50
Well then we just start a war on the robo-flagger-robo-reviewers! –  Jeroen Vannevel Jun 12 at 14:01
Ok so I may have been a bit quick to judge, still these are only suggestions on how to possibly fix the flaw mentioned. –  Sam Jun 12 at 14:05
Resetting the reviewer counter, after someone has a review ban, seems like a good idea to me. I couldn't upvote or downvote your post because I agree with some ideas, disagree with others. –  S.L. Barth Jun 12 at 14:14
Why is this now separate from the original question? –  Joe Jun 12 at 14:24
@Joe Because this could turn out to be another very long winded discussion (the last post has 35~ comments already), so I thought it would be best to start another post regarding the issue. –  Sam Jun 12 at 14:25
Everything that you gamify will need moderation, which will then be gamified when the queue gets large. Break the cycle; remove gamification from moderation tools. –  bd33 Jun 16 at 23:03

1 Answer 1

up vote 15 down vote accepted

Offer badges for reporting RRs

No, this would be terrible. The primary reason we have bad reviewers are the badges. Adding badges for reporting people would quickly cause folks to spam flags on everyone they saw in the hope of getting a badge. Also, it would put tremendous pressure on moderators because you can be sure every single declined flag would lead to a blowup on Meta, as well as a public shaming of the reviewers involved.

Even worse would be

Offer X reputation to users that successfully report a RR.

I'm also not a big fan of


as currently we only impose reputation penalties on spammers or trolls. Poor reviewers are not in that same category, and I don't want moderators to be handed the tools to arbitrarily impose reputation penalties on people. We already have tools for dealing with bad reviewers in the timed review bans. The discipline system on SE is not based on a punitive model, but on first restricting someone's potential to cause harm and then attempting to reform their behavior.

Let's also talk about the term "robo-reviewer" that I see being tossed around. This came about during our brief, horrible period between when badges for review were introduced and when the audits were implemented. During that period, there were a number of people who gamed the system by clicking buttons in review as quickly as they could in order to rack up badges.

The audits have largely done away with these people. New users who haven't hit audits are generally the only ones doing this that I see, and starting someone off with a high dose of audits could deal with that.

When people have been talking about "robo-reviewers" over the last few weeks, what they really mean are people making bad decisions in review. In almost all of the cases that people have flagged for recently, even the really terrible approvals, I'll generally see reasonable rejections, etc. in their history along with those few questionable decisions. What we're arguing about now is generally not someone pushing buttons blindly, but people who aren't careful with what they're reading or who have made mistakes. That's a lot harder to see and to pick out than someone approving every single edit or upvoting every answer in the Late Answers queue.

We're getting into a lot more subjective territory as to what a bad reviewer is. The last part of Shog9's answer to your original feature request talks about this. We all talk about bad reviewers, but everyone on Meta seems to have a different idea of what one looks like, now that we've mostly removed the automated button clickers from the pool.

How bad a review does one need to approve to be considered under your criteria? Is there a certain number of bad reviews they have to make? These are questions I struggle with (and trust me, I've banned a lot of reviewers), so you can see how any badges involving identifying these people will be the source of a lot of heated arguments.

I agree, looking back with hindsight most of those are awful ideas. –  Sam Jun 18 at 8:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .