The Suggested Edits review queue suffers from robo-reviewers. One thing I see regularly is users serially adding one or a few tags to a lot of posts.

The current review audit system does not catch the people who approve of such edit suggestions, most of which should be rejected as "Too minor".

I suggest that, to catch these robo-reviewers, SE creates a new type of review audit in the Suggested Edits queue, that only changes one thing to a question: it adds an obvious meta-tag or a tag that has a clear "DO NOT USE" in its tag wiki excerpt.

I am Andrew Barber, and I approve (of) this feature request! –  Andrew Barber Jun 11 at 13:20
Related:… We need a community consensus of what actually is too minor. –  bjb568 Jun 11 at 23:06
@bjb568 We don't even have a consensus on "too minor" being a valid reason to reject an edit:…. The audits suggested in this post catch a different problem, though: adding invalid tags. –  jpmc26 Jun 11 at 23:28
Any specific tips on avoiding false-positive tag suggestions? –  chrylis Jun 11 at 23:38
@chrylis If by false positive you mean, a tag added that would be appropriate - that's why I suggest using obvious meta-tags, or tags that have "DO NOT USE" in their tag wiki excerpt. –  S.L. Barth Jun 12 at 7:21
Adding a bad tag seems more like vandalism than a too minor edit. Also, related - Add "too minor" audits to review queue –  Dukeling Jun 12 at 7:41
As a side note, please create "First post" audit question with meta/"DO NOT USE" tags as well. E.g. "First post" audit question with "Networking" tag. –  vhu Jun 12 at 8:34
Like @jpmc26 says, you talk about tag-only edits being too minor and then you suggest an audit to catch users who approve incorrect tags. –  Stijn Jun 12 at 9:30
Isn't tagging questions appropriately a good thing ...? –  asteri Jun 12 at 17:29
@JeffGohlke Yes, that's why these audits should only add tags that are provably inappropriate - known meta-tags or "do not use" tags. –  S.L. Barth Jun 12 at 20:58
Fixing a single character bug is too minor. Why? No it's not. It's crucial. See if you can fix something else? Nope..can't see anything. Still want to fix the single character bug. Now what? Is there an answer to this problem (other than the extremely obvious)? –  Poldie Jun 13 at 12:28
@Poldie That seems a little off-topic to me, but: if it is a bug in an answer and you are absolutely sure it is a bug, you can fix it. Usually, though, it is better to point it out in a comment, and leave fixing it to the OP. –  S.L. Barth Jun 13 at 12:36

4 Answers 4

up vote 13 down vote accepted

Adding a bad tag is an "invalid edit", not "too minor"

Bad tag audit cases, i.e. adding an irrelevant tag like C to a Javascript question, should yield "invalid edit". First, it isn't a minor change. It inserts irrelevant material into viewer's feeds. And, it is not like we want the user who submits an irrelevant tag to do more...

a "too minor" audit case would be a quetsion entreily of lwoer case enlgish without punctuition wher da edittor b addin sum corrrect taggs

When the question needs major editing, more than a tag is clearly necessary, and someone adds a minor tag, it is like rearranging the deck chairs on the Titanic. At that point, "too minor" is a proper edit reject reason.


I think this is a good idea, though for such a review audit to work, it must be adding tags that are obviously just nonsense, for any kind of post. A review audit which introduces tags that requires the edit reviewer to have technical knowledge about a specific topic will never work out well (plus I imagine it would be quite complex to implement).

Some examples of tags that are invalid to edit-introduce no matter the topic:

All the "burninate" topics on meta should provide plenty of other such stupid tags that the review audit could use.

I would add any of the company only tags, like microsoft, yahoo, google, etc. –  artless noise Jun 12 at 17:21
Can you explain why you think an audit must be obviously wrong? What use is the audit then? If you're not sure about an edit you're reviewing, skip it. A better idea IMHO would be an audit where a version-specific tag (like is added to a question that is obviously not version-specific. –  CodeCaster Jun 12 at 17:39
@CodeCaster Because the audit system shouldn't contain any trick questions and it should not be designed so that only a person with specific technical knowledge can understand that it is incorrect. We already have some audit systems that can conjure very strange tests, for example the low-quality one is known to come up with audits where there is nothing wrong with the post, then claim that there is. –  Lundin Jun 13 at 6:23
And you have to realize that the audits aren't static, they continuously come up with new tests. If you made such an audit generator that would toss in a completely random tag, chances are it would accidentally add a proper one, then claim that the edit was incorrect. –  Lundin Jun 13 at 6:23

This is a good idea in principle, but in practise it is going to be hard to enforce. If you're not a specialist in a particular tag, you probably won't know at a glance whether the tag edit is correct.

For example, suppose someone asks a question about APN (Apple Push Notifications), and someone adds the "apn" tag. Most people would look at that and think OK. Wrong! The APN tag is specifically not for Apple Push Notifications.

But by the time you've checked, it's already been robo-reviewed, and the system encourages this behaviour, and discourages checking things carefully.

Rather than punishment, maybe what we really need is EITHER more encouragement on the edit review screen to use the "Skip" button when you don't really know what you're looking at (e.g. a written reminder), OR, a guarantee that once you've started an edit review, your effort will count towards the result, which would allow people time to explore the tag descriptions, before the suggested edit has become robo-reviewed.

I think the key point of these audits would be to encourage the use of skip –  OGHaza Jun 12 at 7:52
The carrot is better than the stick, though, and the suggestion in the question does seem like a punishment-based approach. –  S List Jun 12 at 7:55
Aren't review audits meant to be the "stick"? If you have an idea to reward people for not robo-approving, by all means post it as a feature-request! –  S.L. Barth Jun 12 at 7:59
The punishment being a message instructing you to be more careful reviewing. And the punishment for repeat offenders being to ban them from doing a repetitive menial task for 2 days. If there is one thing suggested edit reviewers do not need, it's another carrot to aim for. –  OGHaza Jun 12 at 7:59
The [apn] tag is neither a meta-tag nor a "DO NOT USE" tag, so my feature-request would not catch this particular case. It would, hopefully, catch the people who blindly approve tag edits. With them out of the queue, reviewers who DO read tag descriptions should have a little more time. –  S.L. Barth Jun 12 at 8:02
@SList No problem. If you have a more reward-based solution to the robo-reviewer problem, post it! I for one would be interesting in hearing some. BTW Could you use the dots in my name when @-commenting me? Otherwise I don't get the notification. –  S.L. Barth Jun 12 at 8:30
I have already posted my suggestions in my answer and a comment above. –  S List Jun 12 at 8:34
If you don't know, then don't accept but skip! It would have to be a review audit introducing some nonsense tag like programming, web or whatever. –  Lundin Jun 12 at 12:42
Maybe we should add a minor reward for skipping a suggested edit? Something to tell people "hey, not only is it okay to skip, it is recommended! Skip all the things!" –  Bringer128 Jun 13 at 2:24
@Bringer, that's a good point, skip should absolutely be the recommended action for all reviews, unfortunately I don't think we can incentivise it since it literally requires nothing of the user (except by removing the incentive to act - a man can dream) –  OGHaza Jun 13 at 8:55
@SList If you feel new reviewers could use a bit more guidance, how about you make a feature-request about that? My impression is that SE thinks new reviewers get enough guidance, the perspective from a fresh reviewer could be useful. –  S.L. Barth Jun 13 at 12:46
New reviewers actually get zero guidance, unless they specifically search it out. –  S List Jun 13 at 16:04

I would definitively implement this. And instead of an static list of tags just query the blacklisted tags list. This will assure that the audits are being fairly accurate (there is consensus to blacklist a tag), universal (can be applied to others SE sites) and unmaintained (you don't have to compile a list).


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .