Taking screenshots from the "chrome" context

Add-ons Development
1

Does anybody know how to take screenshots from the Firefox “chrome” context like so:

const { GeckoDriver } = ChromeUtils.importESModule(
    "chrome://remote/content/marionette/driver.sys.mjs"
);
const image = await GeckoDriver.prototype.takeScreenshot();

The error I get is: “Browsing context has been discarded”.

I’m rewriting the text-based browser https://brow.sh and have already been successfully using the screenshotter from the Web Extensions API. And of course I know that I can take screenshots directly from Webdriver (namely GeckoDriver in the case of Firefox), but they’re just too slow, having to send the screenshot to Webdriver and then back to the web extension. So there are 2 reasons I’d like screenshots from the chrome context:

  1. It’d save me having to manage the Browsh web extension. Having to have both a separate build process and the dedicated signed publishing process isn’t fun.
  2. The current version of Browsh only renders the “content” context of the browser, which is arguably the most important part I know, but it’d be nice if Browsh could also render things like the permissions dialog, URL history suggestions, etc.

So any other ideas that could address one or both of those points would be much appreciated.

2

Hey @Thomas_Buckley-Houston, welcome to the Add-ons section of Mozilla’s Discourse server!

Sorry, I don’t follow the question. Where are you executing the code snippet you shared? There isn’t a ChromeUtils object in extension JS contexts, so attempting to execute the snippet you shared in an extension will throw ReferenceError: ChromeUtils is not defined.

I’m also not clear on what you mean by the “screenshotter from the Web Extensions API.” There’s isn’t “screnshotter” namespace; do you mean browser.runtime.captureVisibleTab()?

Are you perhaps you’re referring to Firefox’s support for writing experimental WebExtensions APIs? If so, I’m not too familiar with them, but I might be able to dig up some info once I get a better picture of what you’re trying to do.

3

Thanks for the welcome!

And sorry for not being clear. I execute that snippet by sending a JS payload via Geckodriver (so nothing to do with the extensions) using the so-called “chrome” context which has access to Firefox’s GUI.

And indeed yes, my Browsh project currently and successfully uses the browser.tabs.captureVisibleTab method in the Web Extensions API (the only limitation being that it can’t access the browser’s UI). I hadn’t heard of the experimental Web Extensions API, it looks promising, though I can’t quite see if it enables screenshotting the browser UI.

BTW I’ve been having fun playing with SVG’s <foreignObject> support which has surprisingly comprehensive HTML rendering support. I’ve managed to get some basic screenshots of the browser UI with it! But it requires quite a bit of massaging the brower’s CSS.

So I really am open to any ideas.

4

Experimental WebExtensions APIs may not be the right solution. It’s essentially a system that gives Firefox contributors a relatively low friction way of implementing and testing a new WebExtensions API. I haven’t worked with it myself, but as I understand it an experimental API’s JS is executed in a privileged context, so I think you may be able to screenshot the browser chrome. The catch is that you can’t distribute an experimental API like you can an add-on: it’s a development-time capability, not a separate extension system.

Firefox doesn’t provide a way for extensions to capture a screenshot of the entire browser window, but I see a way you could accomplish this using a coordinating native application and the Native Messaging API. Here’s the hand-wavy version:

  1. Your extension opens a native messaging channel to your native app.
  2. The extension post a message to that channel to request a screenshot of the current browser window.
  3. The native application receives the message on stdin, recognizes the request, and calls out to native system APIs to capture a screenshot of the browser window.
  4. Once the native app has the screenshot, it serializes that image into a Base64 string and pass it back to the extension.
  5. The extension de-serialize the message into a buffer array and uses that to instantiate a new Image.

Bada bing bada boom, the extension has a screenshot of the full browser window and page content.

Of course, this approach requires writing bespoke apps/scripts for each platform you want to support, a decent bit of trial and error to wrap your head around native messaging, and figuring out how to distribute the native component. But at least it doesn’t require rebuilding Firefox :sweat_smile:

1 Like
5

Oh that privileged context in the Experimental WebExtensions API sounds veeery interesting. You see my Browsh project actually runs Firefox headlessly so I think the fact that the addon would be running in development mode wouldn’t be notable to the user.

And also that Native Messaging API could well be useful for other reasons. Currently I have Browsh the application communicate with Browsh the WebExtension via both Marionette and a custom websocket server. Being able to have Browsh and the extension talk directly over STDIN/OUT could make things a lot simpler.

Bada bing bada boom indeed. Thanks for your insights!

1 Like