Webextension: Variable match pattern for content script

Add-ons Development
#1

I am considering how to convert my addon to a webextension: https://addons.mozilla.org/en-US/firefox/addon/jolla-askbot-unseen-posts/

It includes two page scripts which were originally written to work with this site:
https://together.jolla.com/questions/
but I coded it so that it would work with any site that used the ‘askbot’ software, such as:
https://ask.fedoraproject.org/en/questions/
It seemed like a good idea at the time!

The user can specify what askbot sites to target, in simple preferences, by providing the match pattern in a parameter called ‘includes’. For example in my case ‘includes’ is set to:
[“https://together.jolla.com/questions/","http://askbot.org/en/questions/”,“https://ask.fedoraproject.org/en/questions/","http://ask.sagemath.org/questions/”]

The main script calls the content script like this:
pageMod.PageMod({
include: includes,
…

How can I handle this variable match string in a webextension?

I think I cannot do it with a “content_scripts” key in the manifest: that is a fixed string. So it seems I must use tabs.executeScript() and that requires host permission from the target site (or activeTab permission, but that requires user action). And because the target site(s) are variable it must apply to all_urls. I must code the content scripts to check if the URL is a target and if not do nothing.

I don’t like this solution, it’s inefficient. Is there a cleverer way of calling the content script only on the sites specified in the ‘includes’ parameter?

#2

One way I know of is to just filter execution of the content script in the content script itself. So you would attach it to all the pages you listed and then check if the current page the script is running in matches the subset the user selected.

EDIT: see reply

#3

I think your way is the only feasible way. Setting up the fine control for executeScript is the other way.

I thought to getManifest and modify it but there is nothing to update manifest. I tried searching.

#4

Yepp. It’s still the best you can do.

content_scripts from the manifest.json are completely static. You can’t change what they apply to.

executeScript is the only other way to run content scripts (exept for evaluating code from within them, but to do that you need one in the first place).
And to be allowed to use executeScript on sites not known when you pack your extension, you need to specify <all_urls> as permission.
Also executeScript can’t really run at document_start, and, unless you wait for every async call to return before you issue the next one, I don’t think there is a guarantee they the scripts are in fact executed in the order you specified.

1 Like
#5

Thanks all. My two content scripts are not huge - about 100 lines each - but it seems daft to inject both of them into every page when only one of them will do anything - perhaps in one page in 1000.

I went for a walk in the woods and had an idea.

I could write a very small ‘detector’ content script which I inject into every page from the manifest. It will check whether the page URL matches my ‘includes’ variable, and if so which of the two main content scripts is required, and send a message to the persistent script. On receipt of that it will inject the appropriate script using tabs.executeScript(). The detector script can use ‘run at: document_start’ since it doesn’t need the DOM loaded.

A couple of questions:

1 The detector script will need access to the ‘includes’ variable, from the webextension’s options page, which will be in chrome.storage.local. Will it be able to read that?

2 Is there a way to ensure that the content script gets injected into the page that sent the message? Might delays in message handling lead to it injecting into the wrong page - the user having opened a new tab, say?

#6

You could do that, neither of you two points would be a problem.
But, you say you don’t need the DOM to decide if you want to run your script. Can you make the decision in the background script instead of the “very small ‘detector’ content script”? that way Firefox won’t have to create the Sandbox environment in the content processes for every page, which I _think_s more costly than including a large script in it. After all that script isn’t executed, and one would hope that Firefox doesn’t re-compile it every time anyway.

#7

[quote=“NilkasG, post:6, topic:13144”]
But, you say you don’t need the DOM to decide if you want to run your script. Can you make the decision in the background script instead of the “very small ‘detector’ content script”?[/quote]The only information I need to know whether to inject my content script (and which of the two to inject) is the URL of the page. I can get that in a content script (I assume before the DOM is loaded). Can I get it in the background script? If I can, that solves the original problem.

#8

Yes, you can. Depending on when exactly you need to inject, browser.tabs.onUpdated or browser.webNavigation.onCommitted/Compleated will inform you about navigations and the tabs new URL.

#9

Niklas - thank you very much.

I should be able to use:

  • webNavigation.onCompleted along with UrlFilter to match the user’s target sites.
  • tabs.executeScript() to insert the appropriate content script.
    In the manifest I must give the addon host permission to all URLs.

I should have realised that there must be some way of triggering events based on the page loaded.

#10

Replying to myself to amend here:
What you’d want optimally is the declarativeContent API, which sadly isn’t available in Firefox yet, but it would solve exactly this issue.

#11

Interesting, but if declarativeContent does what I think it does I don’t think it is what I want. These two content scripts started life as Greasemonkey user scripts, and all they do is change the colour of elements. No action is required of the user. The only oddity is that the user can define what site(s) to run them on.

I wanted to define a transition route to webextensions now, well in advance of needing to, because this addon must run on Android. I now know that two of the APIs I need are not available on Android yet - tabs.executeScript() (bug 1260548 I think) and options (bug 1302504).

I’d also like to find a way of transferring the existing simple prefs to webextension’s options. Maybe I can get the existing SDK addon to write the prefs into chrome.storage.

#12

What declarativeContent lets you do is attach a content script based on conditions you define at run-time, which is what you are asking for, essentially. Sadly content scripts are only an experimental feature even in chrome.

To migrate your data you will have to use an embedded WebExtensions.

#13

[quote=“freaktechnik, post:12, topic:13144”]
What declarativeContent lets you do is attach a content script based on conditions you define at run-time, which is what you are asking for, essentially.[/quote]OK. It wasn’t clear from the link that you could use it to inject a content script - indeed it says it allows you do things ‘without needing to … inject a page script’. My misunderstanding. Never mind - I have a method that should work.

Thanks for the link to Embedded webextensions There is no mention there of Android compatibility but I assume it’s not available for Android yet.

My sdk addon currently runs on Android and I would like to make a transitional version of my addon (that stores the simple prefs, in preparation for conversion to a webextension) but I’d like it to work on Android too. Is it possible to code it so that it will not fail on Android while embedded webextensions are not available, but will use the feature once it’s released?

I want to avoid a plethora of versions:

  • android without transition features
  • desktop with transition features
  • desktop and android both with transition features
  • desktop webextension (by Fx 57)
  • both as webextension (by Fx ??)

Do you see what I mean?

#14

According to the dev who implemented it embedded WebExtensions also work on Android, however it will limit you to only use features Firefox for Android supports in your WebExtension.

#15

That’s good (and worth documenting). I think I only need storage.local and runtime.sendMessage() and both are available for Android.

Thanks for your help.

#16

Testing this in Fx 53.0a2 (2017-01-29) (64-bit), and following the documentation here , in my background script I have:

browser.webNavigation.onCompleted.addListener(indexPageListener, UrlFilter1 ) ;

But I get:

TypeError: browser.webNavigation is undefined

(The browser object is defined.)

#17

You need to add the "webNavigation" permission!

#18

Thanks, yet again!
…

#20

I’ll continue this thread for another issue from converting my addon from SDK to webextensions. I’m testing with Fx 53.0a2 (2017-01-29) (64-bit)

In the SDK addon one of my content scripts sends a message to the background script, which then reads the site’s RSS feed and sends a response. (I couldn’t do that in the content script because the RSS feed has a different origin.) It uses port.emit in both content and background script.

I have converted that to use runtime.sendMessage. The content script how has:

message = [... an array of strings ...] ;       // from SDK version code  
// 'message' now has to be sent in an object
var messageObj = new Object() ;
messageObj['theMessage'] = message ;
var sending = browser.runtime.sendMessage( messageObj ) ;
sending.then(handleResponse, handleError);
 ...
  
/* Message error */  
function handleError(error) {
  console.log(`SendMessage error: ${error}`); 
}    
  
/* Process response from background script to RSS check */
function handleResponse(RSSresponseObj) {
...

This generates an error:
SendMessage error: Error: Constructor Request requires ‘new’

In case it’s relevant, the background script still has the SDK’s request API so that might be implicated, though the error comes from the sendMessage error handler.

Subsidiary question. It’s not clear whether the SDK request API will still work in a webextension. The existence of the old code doesn’t generate an error, but I don’t think it’s been executed yet. The documentation says I should use webextensions instead, and I set out today to convert it … but I’m not clear what API is the equivalent of request.get().

#21

It’s not clear whether the SDK request API will still work in a webextension.

You can’t use anything you needed to get through require() in the SDK.

You can use this code. Besides being much more readable, it should give you a stack trace to your error:

var sending = browser.runtime.sendMessage({
    messageObj: [ /* ... an array of strings ... */ ], // from SDK version code
})
.then(RSSresponseObj => {
    // ...
})
.catch(error => console.error('SendMessage error', error));

And since you are using Firefox 52+ anyway, you could even use async/await.