Archiving of add-on statistics

Add-ons addons.mozilla.org
#1

Currently we get incoming statistics from Mozilla telemetry and parse them into the AMO database. Add-on authors can then choose to make those public. Here’s an example: https://addons.mozilla.org/en-US/firefox/addon/ublock/statistics/?last=30

What most authors might not know is that those stats go back till 2007 for some add-ons. I’d like to propose a minor of change and I just wanted to get some feedback on that:

  • We’ll take the data and archive it on to disk in some format (probably csv).
  • We’ll make that data available through a page and API.
  • That data be available through the exact same permissions as the current statistics are.
  • We’ll limit the data available in graphs to just being in the last year.

The main reason for doing this is that parsing and storing the data for every add-on is becoming an increasing large operational and technical burden for relatively little gain. Most users of the stats pages check out the pages for last few months and leave.

The upshot of this is that the data will still always be available, but for any analysis or graphing beyond a year, you’ll have to download the data.

Bugs for this are being tracked here: https://github.com/mozilla/olympia/issues/1395, if you have any opinions, feel free to reply or email me.

#2

I think most people will be unaffected. Personally, I rarely check old data.

But internally, how come it isn’t already archived? I would think that the graph would pull and crunch the data on demand no? Or does that slow down the stats pages a lot?

#3

In the current architecture data for the graphs still has to exists and is indexed in our database and the views are rendered through elastic search. Both of those are pain points at the moment.

You do make an interesting point though, it should be possible to change the UI so we could render graphs just from the archived on disk. Whether we’ll get to that in the coming months, I don’t know.

1 Like