[RFC] Third party services support

(Fabrice) #1


One of the scenarii we want to support with Link is to interact not only with your physical home devices, but also with online services. A simple example is to use a voice command to book a taxi.

The Adapter approach

In Link’s current world, that would require either an adapter for each taxi service or a generic taxi adapter that can itself be configured to support different providers (eg. Lyft, Uber, you name it). However both of these options have quite severe limitations:

  • one adapter per taxi service is hard to scale, especially since adapters are trusted code: we have no plans for an “adapter store”. That would also be quite inflexible when the service itself evolves.
  • a taxi meta-adapter would have to implement its own way of describing various services and interacting with them, and to let users manage their preferences/accounts/etc. That would be a lot to implement for a single domain (the taxis) and would have to be redone for other use cases (eg. restaurant bookings). We would obviously try to share the underlying mechanism, but that looks once again like an attempt to describe the world, which is… a hard problem.


Let’s consider that the Link user has an account with the brand new GreatTaxis service. Instead of expecting Link to have ad-hoc support for GreatTaxis, Link will offer a way for GreatTaxis to deploy on their website an application that interacts with the Foxbox. This is already possible since all Link apps are “remote” apps that access Link apis on behalf of the authenticated user.

What is missing though, is that this application needs to be able to run (eg. waiting for the user to say “call me a great taxi”) even when the web app is not currently opened in a user’s browser. That means that the GreatTaxis app needs to run in some kind of distributed fashion, with some part of its code having permanent access to the Foxbox apis.

Web pages happen to have a way to spawn off the main thread processing : web workers. Since all the communication with the worker happens through message passing, there is no reason we can not “detach” the worker from the web browser and run it in the Foxbox instead. By not shutting down the worker when the user navigates away from the app, and reviving the message channel on demand if the user goes back to the app we can achieve what we need.

The benefits of this approach are multiple:

  • no adapter bloat in the codebase.
  • full control of the experience by 3rd parties.
  • leverages a well known and understood security model: the web.

The capabilities available to Foxbox workers should be similar to what a “real” web worker gets in terms of web APIs. In the Foxbox, they will run on behalf of the user that created them (with their own data jar), and so will get access to all the apis this user has access to. For instance, push notifications can be sent to notify users, or voice recognition triggers can be set up.


This plan relies on being able to run Javascript workers, with a sizeable chunk of the web stack. The big picture here is to re-use a trimmed down version of an existing web runtime tailored for our needs. Obvious candidates are Gecko and Servo. There are pros and cons to both, that I will not detail in this post. Overall the main challenge here is to use just enough resources while providing the needed set of web apis.

One interesting issue is how we setup the message passing channel between the web browser and the Foxbox worker. The first prototype will use a websocket connection, but we will likely move to a webrtc datachannel in a more finalized version. The main advantage of using a webrtc datachannel is that the traffic won’t have to go through the Link bridge when you’re outside your home network.

We will have to keep the workers “always green” by checking for potential updates and replacing old instances by newer ones when needed.

On the client side, we will expose a RemoteWorker object with capabilities as close as possible as a normal web worker. One limitation is that we can’t support transferable arguments in postMessage().

Future work

Once we are done with this first feature set, there are even more interesting things we can achieve with remote workers. One idea I want to investigate further is to use them to provide content from a publicly accessible url. Much like Service Workers can intercept network requests and provide content, our workers could register themselves to provide content for a given uri space.

For instance, let’s imagine that I build a “lifestream” webapp, that let me post pictures and text. It will use a remote worker to actually store the content on my Foxbox, and register itself to serve public content at /fabrice/public/lifestream/feed.json so that anyone interested in my public posts could access them.

Another example would be a chat application, where each participant would connect to its buddies through their public worker endpoint. That would be some kind of distributed, always up-to-date messaging application.

These are basic ideas and there are lots of details to figure out, but I’m convinced that it’s a powerful way of leveraging the web technologies!

Feedback welcome!


(David Teller) #2

Side-note: we should consider one.


(David Teller) #3

I believe that these workers are a good idea. Lots of work, I suspect, but a good idea.

How do you envision their security? I assume that they can send messages to the Foxbox REST API and to their origin, right? Anywhere else?


(Fabrice) #4

They are following the same rules as normal web workers. So same origin only by default, unless CORS is used (which is the case in the foxbox).


(Michiel de Jong) #5

So this would basically replace the current Thinkerbell rule syntax with full JavaScript powers for scripts, correct?

My main concern is if the user will understand that there’s semi-trusted code running on their Box which they may expect to have shut down by closing the browser tab which it came from.

The Box should then have some sort of interface to see “Installed apps”, displaying resource usage and with “pause”, “close”, and “revoke access” buttons (but that last button is needed anyway).


(David Teller) #6

I believe that we would want to keep both. Thinkerbell is pretty useful by itself. It doesn’t scale up to complex scripts, and that’s just how it should be.



(David Teller) #7

We probably also want at least IndexedDB.


(Francisco) #8

Excellent proposal Fabrice!

Really like the idea of moving to ‘workers’, seems with that we will have a real expandable and customisable system.

Let me add some comments to the proposal:

As an alternative, could we use SpiderMonkey directly https://github.com/servo/rust-mozjs ?

I know it won’t provide the sandboxing that web workers provide by default and the access to the web apis, and also I’m not sure if we could have webrtc for communication but seems to me (still would need to dive on it to check it) those problems could fixed.

I guess we will have problems with some web apis, but avoiding that, and despite of using pure web workers or a javascript runner, what do you think about introducing a JS api in those workers that could provide access to the box:

  • Access to taxonomy.
  • Access to services (I know they are available through taxonomy, but a js api will be easier to use).
  • Access to a sandboxed filesystem.

I didn’t think about the complete API, but something that could help devs sounds good to me.

An example of this is YQL, a system built in java that allowed people to define little tiny pieces of JS to be executed (via Rhino), with some of this extra sugar to make it helpful: https://developer.yahoo.com/yql/guide/yql_execute.html#yglobal

All of this sounds incredible! \o/

Whatever is our final implementation, it sounds good to me the direction, by leveraging web technologies, making easier for devs to interact and for users to expand their capabilities. My only concern (yes I have one, but just one), is the amount of work that we need to have this ready, but as well, I know there are a bunch of smart engineers working here :slightly_smiling: .

Again great work!



(Francisco) #9

I would like to avoid the adapter stores.

Specially trying to avoid having tons of different stores for different iot products -> https://apps.athom.com/


(Fabrice) #10

Yes you could implement a rule engine this way, and it could be more featureful than Thinkerbell. Do you volunteer for ThinkerWistles.js ? :slightly_smiling: It’s much too early to discard Thinkerbell, but time will tell whether we need to keep it or not.

I agree we’ll need some management interface, but I don’t think users need to be especially aware of anything. This is very similar to what happens with Service Workers: the page registers something that will be triggered to run in the background, and you can manage them through about:serviceworkers

You trust your browser to run them, and you should also trust your Foxbox.

IndexedDB is usable from workers in Gecko. Servo currently lacks any indexedDB support - I’ve been talking to the team about that; it’s a large project to tackle though.

Yes I should have added rust-mozjs to the list of options. I looked at https://github.com/Ms2ger/runtime/ and indeed running JS scripts is easy. However implementing ourselves the subset of dom apis we need for workers is already a lot of work (look at the full dependencies of xhr for instance…).

That looks a lot like “privileged apis” to me, so I’d rather not go this route. We can instead write a JS api on top of the http api (probably the same we use client side).


(Fabrice) #11

There was some discussion on irc in #foxlink about the choice of the web runtime so I’d like to elaborate a bit on that.

My starting point was to figure out what we want to offer to developers. I’d like the experience to be as similar as possible compared to what they are used to when building front-ends: no completely new APIs and tools, no need to submit to a store, same security model. No surprises, and KISS. Testing and deploying also needs to be a breeze. A very simple way to test your code should be to run it as a “normal” web worker, no foxbox needed at this point, and all the browser debugging tools work.

One could argue that it’s possible to run a lot of node.js modules in a browser, and so that we should just use node.js or spidernode on the foxbox. I’m not favorable to that because the execution environment would be very different on browser and in the foxbox, and we would have to come up with a security model (cors, cookies, etc.). I don’t think this is the best use of our resources.

Implementation-wise, there are different options for the runtime: Gecko, Servo, and roll-your-own. Let’s try to assess the strengths and weaknesses of each solution.


  • Pros: complete API coverage, battle tested implementation.
  • Cons: hard to really trim down, no headless mode.


  • Pros: still relatively lightweight, and likely easier to adapt to our need. There is a headless mode.
  • Cons: missing apis, and not mature.

Roll-your-own (ie. SpiderMonkey + our own implementation of the apis we need):

  • Pros: Would provide just what we need. Sounds nice and clean, and would still leverage SpiderMonkey so no big surprises in perfs, etc.
  • Cons: Huge pile of work - just look at all the features an api like fetch() needs to be implemented properly.

If we target to ship a product by the end of the year, I can’t see how we can go the “roll-your-own” path. I also wonder how close from that we can be with a Servo build that doesn’t include features we don’t need (that sill looks like significant work, but not so bad).

Right now I’m still on the fence between Gecko and Servo… I know it’s doable with Gecko, but I would rather like to use Servo. However there are missing features in Servo that may be blockers. More discussion around these is needed with the Servo team.

What is the bare minimum feature set of features needed? Here’s my list (feel free to add to it!):

  • P1: per-user data jars (cookies, indexedDB). Gecko: yes, Servo: no.
  • P1: fetch(), xhr. Gecko: yes, Servo: yes.
  • P1: indexedDB api. Gecko: yes, Servo: no.
  • P1: web sockets. Gecko: yes, Servo: yes.
  • P2: WebCrypto: Gecko: yes, Servo: ???
  • P3: OffscreenCanvas: Gecko: yes, Servo: ???

The implementation has 3 main components: the runtime, the foxbox, and the in-browser part.

The client library needs a channel to communicate with the foxbox to proxy the onMessage / postMessage worker api. We can either set up a websocket connection, or a webrtc datachannel to not go through the bridge after the ICE negotiation.

The runtime also needs a channel to communicate with the foxbox. This will be used to multiplex worker management commands/events, and the worker messages. The target here is to use an efficient IPC mechanism, like the one provided by the ipc-channel crate. In the mean time, we could also use a websocket if that simplifies implementation (in the case of Gecko, we would not have to implement the Rust <-> geckoism integration if we use a web socket).

My current implementation plan (for a v0) is to implement :

  1. in the foxbox: either a taxonomy based adapter or a custom router to communicate with the client library.
  2. In the foxbox: the interface with the runtime.
  3. the runtime itself. It needs to create user-specific workers, run them, relay onmessage/postMessage. The simplest way in Gecko is likely to have a chrome window create content subframes with a userContextId and to load the worker in these iframes.
  4. a client side library, exposing a RemoteWorker constructor to web pages.

I started on 1. and 2. Digging into 3. is also something that could start right away, and 4. once we have something usable.


(Fernando Jiménez Moreno) #12

Thanks for the detailed explanation, Fabrice!

In case of using Gecko, would we be using something like https://wiki.mozilla.org/Embedding/IPCLiteAPI ?

It seems that there are not even cookie jars yet

XHR is available in Servo, Fetch isn’t.

It looks like someone recently signed up to do this work, but there does not seem to be much activity yet.

Servo only supports crypto.getRandomValues()

What do we need this for?

It seems that 3 is blocked on deciding which web runtime we can use.

I would also love to see this done with Servo, but reading your description and digging a little bit on Servo’s wiki page and roadmap, it seems that it’s not going to be ready for what we need at the end of this year :\ So it looks like Gecko is the winner.


(Fabrice) #13

That never landed and I don’t know if that’s on the radar in the new “easier embedding” effort.

Wrong link? But yes, they have a single profile that stores everything, and no cookie jars.

Looks like fetch is in: https://github.com/servo/servo/tree/96a86bd952c4c0e41d6164fbd16244ea4420106a/components/net/fetch

Right, I think several people said “Sure, it should not be hard”. And then they look at the spec…

I’m not sure, that’s for creative people :wink: Clearly lower priority anyway, and not a choice blocker.

I also have the feeling that our safe choice right now is Gecko if there’s no commited help from the Servo team.


(Fernando Jiménez Moreno) #14

If we don’t use this, what’s your idea for this work? Maybe something like Graphene?

Yeah, wrong link, sorry, I just edited it.

Hmm, it doesn’t seem to be exposed to web content.


(Fabrice) #15

Yes, a very basic Gecko app. I posted a patch at https://bugzilla.mozilla.org/show_bug.cgi?id=1274093 for that. Build with --enable-application=jsworkers and run with a --ws=<ws_url> parameter. It doesn’t do anything useful yet!