With ~210 add-ons pending full review and more pending flagged preliminary review, just 2 reviewers doing all the work whenever it is feasible for them is quite problematic for both sides.
And without any way of knowing, the flagged add-ons don’t even have any queue criteria; will they be reviewed after all 210 full reviews are done, will they be reviewed after 10 full reviews are done, after each one, etc.? There’s no way of knowing that a flagged add-on will even be reviewed at all, the present picture can even convey the very likely possibility of never being reviewed at all.
When I asked for someone to take a look at my add-on update pending review, no reviewer replied in IRC, same happened when I used the email. Absolutely nothing in return.
And the flagged queue would be everything but confusing, if anything currently it is confusing because the developer has no clue wether he might have his add-on reviewed anytime soon or not. And to make matters worse, developers in this position observe their position in the queue increasing as if someone is bumping ahead or bumping it behind.
The problems that demand an admin review are prefectly understandable, but to be permanently stuck in that position should not happen. The preliminary reviewers should be the ones that set that flag and it should reset once a new version is uploaded because otherwise the preliminary reviewers are completely unable to do anything, especially if the new version has corrected or fixed the problems that led the add-on to be flagged in the first place. Let the preliminary reviewers make their work and leave the admin reviewers (which are only 2) make theirs (which is already plenty). Because, very honestly, the current system that AMO has is absolutely crippled, just 2 admin reviewers for over 200 add-ons? And the preliminary review has only been alright lately because there’s one reviewer blazing through 40+ add-ons per day, otherwise it would also be as choked as the full review queue.
Don’t make the admin flag permanent, reset the flag if the developer uploads a new version. Otherwise, if the developer corrected the problem that led to the admin flag in the first place, the reviewers in the preliminary section cannot do anything, the 2 review admins’ work load is increased without any necessity and the developer grows frustrated of waiting forever to have his add-on reviewed, and in some cases it can even lead to him giving up on AMO or worse, developing for Firefox completely.
And to make matters worse, the prelinimary review queue position is reset everytime a new version is uploaded. This is a problem that has been going unsolved for over 3 years now
and it forces developers to not upload new versions if they are about to be reviewed, and only after they upload a new version which has to be reviewed again. This problem doubles the work when he could have just uploaded the new version and not have its queue position reset.
- Implement a flagged for admin review queue
- Reset the admin flag when a new version is uploaded
- Fix the prelimninary queue position reset when uploading a new version