List of new Mozilla Repos + latest Wiki edits

mkohler · June 30, 2018, 10:34pm

Hi everyone

I’ve created a tool to show the latest new repos across Mozilla github organisations as well as the latest Wiki edits. I’ve started that project 2 years ago, but given the recent interest I got yesterday when I once again tweeted about it, I’d like to post it here as well. Of course the latest wiki edits can be seen on the wiki itself as well, but I think it’s nice to have these two on the same page. The latest repos view definitely allows everyone to find a few new gems from time to time.

Here’s the tool: https://watcher.michaelkohler.info/

You can find the full list of tracked GitHub organisations on the GitHub repo, would be great to hear if I missed any important orgs that should be tracked as well (I’m very happy to include community orgs as well!).

Happy to hear your feedback, do you think there is something that would be as beneficial that could be added to that view?

Cheers
Michael

dreams · July 1, 2018, 1:29am

Hey Michael, great work pulling this all together!

Please feel free to add the mozilla-lockbox organization’s GitHub repos.

More specifically: the iOS and Android ones will be part of a Test Pilot experiment this summer and fall.

hmitsch · July 1, 2018, 5:30pm

Wow, what a nice and handy tool. Now we’d need a way to import the new repos into our Community Analytics repo list …

-Henrik

mkohler · July 1, 2018, 9:47pm

Yes, having new repos automatically be tracked in Community Analytics would be great indeed. Feel free to use the org list to cross check if you’re missing any orgs (even if it currently is not automatic to include new repos).

Little caveat here: even if new repos automatically get added to Community Analytics, it would still be NDA only and I’d like to believe this tool provides a lot of value for non-NDA’d Mozillians as well.

mkohler · July 1, 2018, 9:52pm

On another note, I’m happy to provide an API with the aggregated data if the Community Analytics platform could handle this easier than aggregating the info itself.

bkerensa · July 2, 2018, 9:39am

Hi mkohler,

Very interesting tool! Does this scrape the wiki and how often does it check for updates?

Cheers,
Benjamin
Mozilla Wiki Peer

mkohler · July 2, 2018, 10:31am

It’s not really scraping the website, I’m using the Wiki API. You can find the call here:

github.com

mozillach/mozilla-watcher/blob/master/lib/wiki-helper.js#L6


'use strict';


const debug = require('debug')('Wiki');
const fetch = require('node-fetch');


const BASE_URL = 'https://wiki.mozilla.org/api.php';
const FIELDS = 'title|timestamp|user';
const LIST = 'recentchanges';
const ACTION = 'query';
const LIMIT = 500;
const FORMAT = 'json';
const LAST_EDITS_URL = `${BASE_URL}?action=${ACTION}&list=${LIST}&rcprop=${FIELDS}&rclimit=${LIMIT}&format=${FORMAT}`;


class WikiHelper {
constructor() {
  this.apiUrl = LAST_EDITS_URL;

This is called every hour, so it should not have any negative impact on the wiki

bkerensa · July 11, 2018, 8:57am

Oh I do not think were worried about traffic the wiki is in the cloud and auto-scales to its changing needs.