How to send blobs or streams in the WoT?

WebThings
#1

For now the Web of Things API seems to focus on simple types when sending properties, actions or events. I have seen booleans, numbers, integers, strings and colours (which are RGB strings).

When looking at current devices in the IoT realm, I see home automation that also sends pictures, audio, videos and streams of such. The data can either be produced in the network (e.g. a camera at the entrance door) or on the Internet (e.g. an online radio).

Would a gateway proxy these blobs/streams, as it does with simple values?
How about sending URLs to the content? What if the target device can’t reach the source or the source requires some authorization?

Use cases:

  • I want to get a notification when the house alarm goes off with a photo of the front door.
  • When shopping, I want to check the content of the fridge via the fridge cam.
  • I want to tell the TV / Media Centre (e.g. Kodi) to play a video from the NAS. (Can we control DLNA via WoT?)
  • I want the internet radio to play a radio station in the morning. The WoT gateway could have rules for news on weekdays and calming tunes on the weekend.

Is sending these data types within the scope of the WoT standard? I think it should be.
How much is WoT focusing on home automation for end-customers vs. automation of :smirk:every-Thing?
Is there a place to bring in my ideas around these forms of data?

#2

We’ve had some people send still images and other binary content as base-64 encoded strings. While this is not super efficient, it does work with our JSON-based descriptions. Another option, as you said, is to include a URL to the image/stream/etc. I think this is totally valid, as you’re still sticking with web-based protocols.

Some of your use cases don’t really need this functionality at all. For instance, playing an internet radio station could be as easy as setting a string property (station) and sending a play action. The same goes for the media center.

However, there certainly are cases where streaming functionality is required, not the least of which is cameras.

We’re still trying to work through security standardization for the spec. The W3C has done a fair amount of work, but that’s far from finalized. For right now, you could bridge existing hardware and services by writing a standalone web thing (or gateway add-on).

It would be worthwhile to file issues to our repos, here and/or here, especially if you have some property descriptions/schemas that you think would be capable of handling these use cases.

#3

I think this is an argument for changing the href of a property into a links array (like the current forms array in the W3C spec but links), so that you can provide a link for a property with an alternative MIME type to JSON, for example:

...
  "properties": {
    "image": {
      "links" [{
        "href": "/things/lamp/properties/image",
        "mediaType": "image/jpeg"
      }]
    },
  }
...

Note that type can be omitted in this case.

This solution was also recently proposed for the W3C spec.

This would also allow for other binary encoded data formats.

I have filed an issue on the Web Thing API specification regarding binary properties.

#4

Is there a concept how updates for links’/forms’ content would be propagated via HTTP and Websocket?

The current W3C draft does not touch the topic as part of WoT.
I guess for streaming content there is no real time when the content updates, but for a webcam that operates on an interval this would be interesting.

#5

For a gateway like Mozilla Things I see it as a useful feature to proxy local link/form URLs to authenticated clients.

For that purpose I suggest for the gateway to point link/form URLs at itself.