My add-on needs to examine many of the images that the page itself loads. In previous versions at the add-on level there was chrome privilege to read any image. With the web-ext version there is a security error.
Is a content script permitted to read any image that the page itself reads?
How do I go about this?
I’m unsure what special privilege you would need to access images from a content script. Assuming those images come from <img/> tags, you can access the DOM from your content script (using for instance document.querySelectorAll("img")) to get the Image objects. From there, you have access to their width and height. If you want to access the image content, you can create (from your content script) a Canvas element, get a 2D context from this canvas (canvas.getContext('2d')) and copy the image data to this canvas context using drawImage(). You have now access to any single image pixel with getImageData().
How are you exactly accessing the images on webpages?
Previously, add-on scripts used to have system principle, now, in WebExtensions, they have content principle of that webpage they’re running in with some exceptions: they have different fetch and XMLHTTPRequest injected which have no CORS restrictions.
That’s the same issue [1] I’ve stumbled upon with my add-on. I need to access document.styleSheets[…] and my workaround is: remove affected stylesheets, fetch() their code, inject into the page as inline stylesheet, fix all relative links inside. You probably can make something like this: fetch() image data, construct a data: url from it’s data and do whatever you want with them. Obviously, that’s not the prettiest way but that’s all you can do now.
In the bug I’ve created [2] firefox developers decided not to extend WebExtensions content script principle to WebExtensions CORS policy. Instead, they’re going to “digg a holes” for each such usecase as already done for fetch() and XMLHTTPRequest. So, you should create a bug for your case.
At the moment the images I am interested in are the background images. These are specified by URLs in the background-image entry in CSS. A security restriction was introduced in an earlier version of FF that prevented the add-on from reading the content of an IMG object into a canvas. So I resorted to having the chrome-privileged part of the add-on load the image with an IMG element and copy its data to a canvas.
Now I find that the background script cannot access any image URL. I would like to have a security policy like “img-src https://* http://*” but the documentation says that that is not allowed. It seems strange that the add-on cannot load things that the page has already loaded.
I haven’t yet tried loading the URL within the content script but I doubt that it would work.
… A security restriction was introduced in an earlier version of FF that prevented the add-on from reading the content of an IMG object into a canvas …
Ouch…
Another possible solution: if you know the URL of the images, you may be able to use the new feature webRequest.filterResponseData(). Basically, you install a listener on webRequest.onBeforeRequest, then when you see a URL request matching one of your images, you setup a filter with webRequest.filterResponseData() and grab the image data on the fly. Note that this feature is available only on Firefox >= 56.
Your suggestion led me to reread the host permissions stuff, so, silly me. Adding a host permision of <all_urls> solves the problem.
Very good !
I must say i was surprised about what you wrote about a content script not being able to access image data. I gave it a try and it seems to work for me, on both Firefox 58.0a1 and Chrome 61.0.3163.100.
Here is my test code:
var imgs = document.querySelectorAll("img");
if(imgs.length>0) {
try {
var img = imgs[0];
var canvas = document.createElement("canvas");
canvas.setAttribute("width",img.width);
canvas.setAttribute("height",img.height);
var context = canvas.getContext("2d");
context.drawImage(img,0,0);
var imgData = context.getImageData(0,0,img.width,img.height);
var data = [];
for(var i=0;i<10;i++)
data.push(imgData.data[i]);
alert("Size "+imgData.data.length+" "+JSON.stringify(data));
} catch(e) {
alert(e.message);
}
}
The script was injected using browser.tabs.executeScript().
I haven’t tried former Firefox versions but apparently that security restriction is no longer there.
That might simplify my code then.
It was the getImageData() that was restricted, if there was some cross-site access.
Thanks
… if there was some cross-site access
Oh, that might be the thing. I only checked on the first image which is probably hosted on the same domain.