A toolkit to speed-up WebExtensions add-on development

Add-ons Development
#1

I recently released a little tool to help developing WebExtensions add-ons. It’s called weh (for WebExtensions Helper) and is available at https://github.com/mi-g/weh

To make it short, it’s a set of different things that you can use (or not) independently:

  • a build system that maintains in live a ready-to-load directory distinct from your source code, modifying on the fly manifest.json and .html files:
  • allow using typescript, coffeescript and/or JSX as an alternative to Javascript for both background and content
  • allow using Sass (.scss), Less and/or Stylus as an alternative for CSS
  • provide optional concatenation of scripts and stylesheets, and minification for scripts, styles and html
  • initial pre-processing of any text file through EJS if desired
  • a preference system that produces automatically the settings edition tab from a formal parameter definition including constraints
  • a translation system that allows the user to modify any locale strings from the add-on in her profile
  • a simplified method to communicate between the background and content scripts
  • a separate add-on to debug other weh-based extensions (Chrome/Opera only as some APIs are missing in Firefox):
  • monitor background/content messages (if messaging feature is used)
  • read/write preferences
  • read HTML5 and WebExtensions storage
  • ability to generate a working skeleton add-on with ReactJS UI in one command
  • tested successfully on Firefox, Chrome, Opera and Edge

Just wanted to share in case anyone is interested in using or contributing.

2 Likes
Concern about WebExtensions exclusive strategy from add-on authors and users
#2

Wow ultra cool! Each of my addons requries native messaging. I’ll see if I can give you a PR on this.

Does the translation system allow for plurals?

For Firefox I also had to use hybrid as all my addons support Android. The future about this is extremely confusing though. Scares me.

#3

Hey Noitidart,

No, the weh translation system does not support plural since it is based on a wrapping of the current WebExtensions browser.i18n.getMessage(), which, as far as i know, does not have this feature. The really cool thing would be to have this feature implemented natively into Firefox. I opened bug#1323440 about this.

It was my understanding that having a pure WebExtensions add-on will ensure that it will work on both desktop and Android. Now, you make me doubt :confused:

1 Like
#4

Thanks mig

I also have so much uncertainty with where they are going. All my Android addons are powered by JNI.jsm. They already stated for sure, nothing resembling native messaging will be there for Android. So without JNI I have no idea. Crazy times haha.

Re: Plurals

What I did was add some sugar like this:

for (let i=0; i<100; i++) {
    console.log(getMessage('hotkeys_plurals_' + i));
}

function getMessage(...args) {
    let returnval;
    
    let name = args.shift();
    let count;
    const SUFFIX = '_plurals_';
    if (name.includes(SUFFIX)) {
        name = name.substr(name.indexOf(SUFFIX));
        count = parseInt(name.substr(name.indexOf(SUFFIX) + SUFFIX.length));
        let plurals = eval(browser.i18n.getMessage(name));
        for (let [a_cnt, a_val] in Object.entries(plurals)) {
            if (parseInt(a_cnt) < count) {
                returnval = a_val;
            }
        }
    } else {
        returnval = browser.i18n.getMessage(name, ...args);
    }
    
    return returnval;
}

And the messages.json entries:

{
    "hotkeys_plurals":
        {
            message:
                "
                    {
                        0:'No hotkeys',
                        1:'A hotkey',
                        2:'Couple hotkeys'
                        3: 'Few hotkeys',
                        6: 'Some hotkeys',
                        10: 'Many hotkeys',
                        100: 'Lots and lots of hotkeys'
                    }
                "
        }
}

It doesn’t support replacements though.b

#5

Here’s another rough version of plurals that supports replacements:

for (let i=0; i<100; i++) {
    console.log(getMessage('hotkeys_plurals_' + i));
}

function getMessage(...args) {
    let returnval;
    
    let name = args.shift();
    const SUFFIX = '_plurals_';
    if (name.includes(SUFFIX)) {
        let name_prefix = name.substr(0, name.indexOf(SUFFIX) + SUFFIX.length);
        let count = parseInt(name.substr(name_prefix.length));

        for (let i=count; i>=0; i++) {
            try {
                returnval = browser.i18n.getMessage(name_prefix + i, ...args);
                break;
            } catch(ignore) {}
        }
        throw new Error('No plural exists for quantity/count of:', count, 'name_prefix:', name_prefix);
    } else {
        returnval = browser.i18n.getMessage(name, ...args);
    }
    
    return returnval;
}

And the messages.json entries:

{
    "hotkeys_plurals_0": { message: "No hotkeys" },
    "hotkeys_plurals_1": { message: "A hotkey" },
    "hotkeys_plurals_2": { message: "A couple hotkeys" },
    "hotkeys_plurals_3": { message: "A few hotkeys" },
    "hotkeys_plurals_6": { message: "Some hotkeys" },
    "hotkeys_plurals_10": { message: "Many hotkeys" },
    "hotkeys_plurals_100": { message: "Lots and lots of hotkeys" },
}
#6

I must say i prefer the 2nd version :slight_smile:

In the first version:

  • doesn’t it break the messages.json legacy structure ?
  • object.entries() is experimental technology, not supported on Opera, possibly Edge
  • doing an eval(browser.i18n.getMessage(name)) seems to me highly dangerous !

In the second version:

  • name.includes() might not be supported on all browsers (Opera, maybe Edge). name.indexOf(name)>=0 is probably safer
  • i’d prefer not throwing an exception in case a plural is not available as this kind of thing may only trigger in some production situations (add-ons code is often not tested thoroughly). It may be better to have a console warning message but still returns something that makes sense, even if it’s grammatically incorrect.
  • i’m not a great fan of the loop for (let i=count; i>=0; i++). What if count is greater than the maximum defined plural ? And even then, what if the maximum defined plural is one billion ? we’d bump into performance issues. Note that if the weh build system is used, all the locale keys are known (array weh.i18nKeys from auto-generated file background/weh-i18n-keys.js) which might be useful here
  • from your code, it looks to me like the exception is always thrown when plural is used
  • if substitutions are to be used, they will have to be defined in every single plural form which might quickly become heavy

If you can sort that out, a PR would be very welcome. You can insert your function into file src/common/weh-i18n.js as function GetMessagePlural(), call GetMessage() instead of browser.i18n.getMessage() to get the benefit of user-defined translations, and map a weh.xxx() property to GetMessagePlural() just like weh._() is mapped to GetMessage(). weh._s() would look like a good name no ?

1 Like
#7

I actually use babel to turn my es6 to es5 so I forget what all is main stream haha.

I strongly agree with you on the for loop, like if its a billion.

A better solution would be to assume there is to XHR the xhr(’/_locales/message.json’) and then get all entries that have the name_prefix.

#8

You are right about babel, it is used in the weh build system on every .js file, so it would have worked anyway. But the thing about weh is to not compel the developer to use all the tools. We could imagine someone willing to have the translation system without the build system. So it’s better to use widely supported functions.

Oh, would xhr('/_locales/message.json') work on a webextension add-on static resource ? I did not realize that. That might be useful. An issue i see is that the actual locales files are located at /_locales/XX/messages.json, with XX being the locale and i don’t think xhr can list the available entries.

Anyway, if the build system is to be used and we need the whole locale meta definitions (including substitutions), it can be done the same way weh.i18nKeys is generated. Search for i18nKeys in gulpfile.js.

#9

Ah yeah correct /XX/ - to get XX I do thi:

// REQUIREMENTS:
// _locales/[LOCALE_TAG]/ directories
async function getExtLocales() {
	let { xhr:{response} } = await xhrPromise('/_locales/');

	let locales = [];
	let match, patt = /^.*? ([a-z\-]+)\//img;
	while (match = patt.exec(response))
		locales.push(match[1]);

	return locales;
}

// REQUIREMENTS
// messages.json in _locales/** directories
async function getSelectedLocale(testkey) {
	// returns the locale in my extension, that is being used by the browser, to display my extension stuff
	// testkey - string of key common to all messages.json files - will collect this message from each of the extlocales, then see what browser.i18n.getMessage(testkey) is equal to
	// REQUIRED: pick a `testkey` that has a unique value in each message.json file
	let extlocales = await getExtLocales();

	let errors = [];
	let msgs = {}; // localized_messages `[messages.json[testkey]]: extlocale`
	for (let extlocale of extlocales) {
		let msg = (await xhrPromise('/_locales/' + extlocale + '/messages.json', { restype:'json' })).xhr.response[testkey].message;

		if (msg in msgs) errors.push(`* messages.json for locale "${extlocale}" has the same "message" as locale ${msgs[msg]} for \`testkey\`("${testkey}")`);
		else msgs[msg] = extlocale;
	}

	if (errors.length) throw 'ERROR(getSelectedLocale):\n' + errors.join('\n');

	return msgs[browser.i18n.getMessage(testkey)];
}

Usage getSelectedLocale(SOME_KEY)

#10

Good.

In our case, since we only need to read the meta of the locale keys (we are not interested by the translated string, just the available substitutions), we can use field the default_locale property in manifest.json to directly pick the right messages.json file.

Also, i’m a bit concerned with using xhr to read locales since because it is a promise, the translation is not available when the script first runs. I mean some add-on code doing let title = weh._s("title"); at the very top level of a script would fail.

#11

One great addition to weh would be to use your dynamic locale file reading method to provide an export function through the weh “inspect” interface (in the same way preferences and storage are already exported). Then from the weh-inspector, we could provide a user interface that would query the add-on in development and have an audit feature: show the missing/extra keys, what strings are empty or untranslated, …

#12

This is very cool! I have taken a similar approach in building my own extension (Social Fixer for Facebook), extracting many general extension tools out to a library I call “X”. It addresses many of the common needs of web extensions and provides out-of-the-box solutions for them. Things like persistence, pub/sub messaging, mutation observers, html sanitizing, abstracting ajax, integrating with Vue.js, etc.

I have a gulp build script that can target multiple outputs, of which web extension is just one. I also build a safari extension and a greasemonkey script right now. All are built from the same source. I’ve not tackled i18n at all, though my users are asking for it. It seems too much to do at the moment. :slight_smile:

My end goal is to release my X framework as a general extension-building framework, so that others can benefit from it and also so I can benefit from community development in solving the more mundane tasks of cross-browser extension development.

I will definitely take a look at your toolkit and see what we might have in common. It could be that my framework fits on top of yours. We will see. Do you want to keep this a solo project or are you open to other collaborators?

#13

This sounds great ! I’d love to see your framework.

I am not seeing weh as a solo project and encourage anyone to participate. That’s why it is available on github for anyone to fork and submit PRs.

#14

I just submitted to amo the first weh-based extension, as update 2.0.0.0 of CouponsHelper.

Since by weh nature, the add-on files and the source code are different, i will add to weh the ability to generate automatically the source tarball that goes along with the submission.

#15

The latest version of weh includes support for Angular (1.6+). There is now an Angular-based skeleton template to init project from and both settings and translation UI have been rewritten so that add-ons using Angular do not need to include React libraries.