Angry but constructive criticism

Hello,

I have had a Mozilla Add-on for almost 5 years. Many things have changed in Mozilla Add-ons during this time and that’s why I want to express my concerns.

Recently, I uploaded a new version of my add-on, and since my add-on contains minified code I uploaded the source code with detailed instructions on how my add-on is built. This has been the case for the many previous versions which never had an issue to pass review.

I understand the review process is slow and done by volunteers, and I can deal with that. However, in my case, the reviewer failed to follow the most basic instructions (it can be confusing to build others’ projects so I understand). The problem, is that the 15 days for the review process have elapsed and the Add-on has been delisted.

First, I find it pretty bad to constantly receive emails threatening me of my add-on being delisted when:

  • I provided the source code
  • My add on has been listed for almost 5 years without an issue
  • I responded to the reviewers emails in less than 24h every time
  • I’m helpless when the reviewer fails to follow 4 basic steps to replicate the uploaded version with the sent source code.

Secondly, I think it is pretty poor design to entirely delist a web extension just because the new version has failed to pass review, or in this case, reviewer has failed to build the web extension. In my opinion, if a new version hasn’t passed review, THE OLD VERSION SHOULD STILL BE AVAILABLE. Currently, I’m receiving emails of customers complaining that they cannot use the web extension. Considering the slow review process and the delays we have been experiencing, the least Mozilla could do is keep the OLD versions that have PASSED REVIEW alive in order NOT TO KILL BUSINESSES.

It’s easy, you have a live version, and then a new version isn’t live until it passes review. So if it doesn’t pass review, no one get hurts and the problems can be fixed without pressure.

All in all, this is poor support for developers to work with Mozilla and Firefox.

I understand the goals of the review process and as a Firefox user I appreciate them. We cannot trust third party code. But there’s a difference between thoroughly reviewing addon source codes and delisting addons that have previously passed review and have shown absolutely no issues nor gotten reports. This kills honest businesses.

This is frustrating.

7 Likes

Hi, Mezood -

In the last year, AMO has seen about 155,000 add-ons submitted for review; in terms of size, they almost all fall in the “several thousand lines of code” range. The overwhelming majority of them go through automatic inspection to make sure they meet our existing standards, and if they pass those tests are automatically approved.

Those policies and the tools that enforce them are, necessarily, always evolving; new web technologies and standards emerge, new ways to abuse APIs and other threat vectors are discovered and our tools and processes need to keep up with both.

For all our reviewers’ hard work, we can only give human attention to a small fraction of the add-ons that get submitted to AMO. This isn’t the fault of the team, or how smart or well-intentioned anyone is, or how diligently anyone is working. It’s because Mozilla’s add-on developer community clicks “submit” on roughly a million lines of code every day.

The reason that we require that add-ons be signed - the reason we have the automatic review process at all - is simply that add-ons are made of software. We know that software can be and is routinely abused by the internet’s numerous bad actors, and we believe that we have a moral obligation to protect people who depend on Firefox from that abuse. In the case of good-faith actors, this means that when upcoming changes to our policies are going to break existing add-ons, or those add-ons are causing crashes or performance issues in the wild, we have a point of contact for those developers to let them know. In the other case, it means that when we identify add-ons from bad actors, we can identify patterns of malicious behavior and disable those add-ons in the wild.

But again, as much of this has to be automated as possible. This is the only choice we have that doesn’t end with “we cannot meet our obligations to our users while supporting add-ons at all”. There is no other way; the raw numbers are just too big.

This isn’t a perfect system, and one of the sharpest edge cases is precisely what you’re describing. When our policies change, some developers - well-intentioned developers writing perfectly legitimate, useful add-ons that do no harm whatsoever - get an email saying their long-established add-on has relied on some part of the WebExtensions API that we can no longer support, and the clock starts ticking.

In most cases, add-ons are updated in time, given manual review, and make it back into AMO safely. In other cases, they can be rolled back to an earlier version that passes the new policy restrictions while the developer works on a new version. But in some cases, like yours, when the clock runs out and there isn’t an earlier version available that meets the standards of the new policy, the automation will revoke the entire stack like it’s pulling a zipper.

We believe that this is the right thing. Specifically, we believe that our commitments to user safety, the importance of user trust in Firefox and its surrounding ecosystem, and our obligation to mitigate the reach and impact of malicious software through prompt action outweighs our obligations to avoid all possible false positives or other adverse outcomes for developers.

We do our best to mitigate those adverse outcomes, and make corrections and adapt when we get something wrong, but when we have to make design decisions about which way our processes might fail, we are going to choose “fail-towards-user-safety” every time, and then do our best to do right by developers wrongly caught up in that failure.

You’re welcome to email me directly if there’s any way we can help. Thank you for being a part of AMO.

-Mike Hoye, on behalf of the Mozilla Add-ons team.

1 Like

First of all thanks for taking the time to answer. This felt like a wall and there was no one to contact beyond the reviewers who simply ignore emails.

As I wrote, I understand that you need to make sure of third party code legitimacy.

I just wanted to express how bad my experience was, and why I’ll be dropping off of AMO in the midterm. After all, one thing are the requirements, and a very different one how reviewers follow instructions and communicate.

I still believe delisting all previously accepted versions is unfair, especially without warning and without time to react to send fixes. Not to say that in these cases, extensions should have preference in the review queues.

Having to send the full source code isn’t great either, a risk for the business not to be able to control who has access to it for people to find vulnerabilities. Also having reviewers telling you which files to remove from the extensions without giving a justification, not great. The list could go on.

Simply, the fact that I can’t know which new surprise I’ll get on every review, that the extension can get delisted without warning nor time to fix it and that passing review can take weeks if not months is not the most welcoming scenario for small businesses trying to make a legitimate penny.

Don’t want to bother you any further. It’s a lost battle. As a long term defender of Mozilla and what it means for the open web this has been (and is still being) very frustrating.

EDIT: aaaaand another rejection, big waste of time