User Submitted Data and Privacy?

Add-ons addons.mozilla.org
1

Hi! I make a plugin that blocks NSFW images for users called Wingman Jr.. It uses machine learning, and I’m constantly improving the model. I’d like to provide a way for users to provide feedback about the image blocking so I can make it better for everybody!

Specifically, what I have in mind is something like a “report recent images” feature that would take some number of the last images that the user has viewed and submit that data to a server, where I would review and incorporate it into a dataset for training. Right now that dataset is private, but I would like to keep the option of opening it up someday - I think the lack of a good dataset of both positives and negatives is a limiter of this type of model in research.

Now there’s lots of ways that this could be done well or it could be done poorly. Here’s what I thought would work well, but I’d like some feedback:

  1. Provide a clear privacy policy and disclaimer that make it clear that data is only collected when specifically submitted by the user.
  2. Collect URLs, not the actual image data. This is how e.g. ImageNet etc. approach the creation of such datsets.
  3. Exclude data URLs for the same reason.
  4. Clearly show the user a copy of the data that will be submitted with a “Yes/No” decision before each submission - e.g. the list of URL’s, any metadata. This way the user can have a last chance to review and back out anything they don’t want to submit.
  5. Clearly state once again at the time of submitting the user data what will happen with the collected data.

I think if a plugin provided these bounds, I would feel OK with submitting data if I believed in what they were trying to do.

So what do you think? Good idea, bad idea? What pitfalls haven’t I thought of yet?

Thanks for the feedback!

2

Your checklist is thoughtful — especially #2 (URLs only) and #4 (show exactly what will be sent before each submission).

One gap I’d surface in the UX copy: “URLs only” still doesn’t cover what happens when users save or re-share images elsewhere. The page URL and the image bytes are different privacy surfaces. A JPEG downloaded from a CDN can still carry EXIF/GPS (and increasingly C2PA on some exports) even if you never upload the file to the extension author.

If you ever expand beyond URL lists, I’d avoid collecting raw image bytes without per-image opt-in. A redacted summary (host, MIME type, rough dimensions, maybe whether the response was cross-origin) is usually enough for training signal with less risk than thumbnails.

Also worth a line in the disclaimer that user-submitted URL lists can still contain sensitive query strings or signed tokens — your preview step (#4) is the right place to let people redact those.

Thanks for drafting this in public; more add-on authors should do the same before shipping feedback features.