Developing an extension - getting access to the core

I’m developing an extension to add facilities to create groups of things and to add pages (or views) with different combinations of Things and, as a stretch goal, add user permissions. I will publish the code when I have something a bit more demonstrable - I estimate 1-2 weeks.
To achieve this, I will first need to get a list of Things out of core (their internal id and the name) so I can create the structures that will later become the groups and pages.
Later, I’d like to be able to unbundle the individual Things displays (from the current Things and Floorplan views) and to be able to display them in different combinations on different end-user created views.
What are the less intrusive and more-structured and clean approaches ways to get access to the core to do this? As opposed to, for example, just opening another connection to the core database and directly pulling data out?
Ultimately and if it turns out well, I’d like to have these facilities in the Gateway, so I’d like to attempt to do it right.

You’ll want to create an Adapter class and use the handleDeviceSaved callback to get a list of all saved devices.

You can create your views with an Extension and an APIHandler.

1 Like

Some screenshots to show the progress so far
The basic extension


some Groups - eventually will be able to add Things to groups
image
Adding a new Group
image
Deleting - need to add the name of what’s about to be deleted
image
Showing an empty group
image
Adding something to a group (theres something wrong, I’m adding a group to itself)
image
And here it complains
image

I’ve created all the screens so far starting from the example-extension.
Would the Adapter class be in the same package or a separate package? If separate, how would they communicate?

The Adapter runs on the server. You’ll want to create an Adapter that is owned by the APIHandler, or vice versa, so that the Adapter can share the device info with the APIHandler.

1 Like

The example-adapter actually uses this pattern, although it uses the adapter for much more than you’ll need to. You won’t need to create any devices of your own.

I’ve done this and it’s working, thank-you for your quick response.

I’ve completed the basics of a platform to setup groups of things and the structure behind pages. I’ll spend some time to go through to ensure all the components are working and that all the code is fairly clean and documented. Then I’ll publish on GitHub.

Before anyone gets excited: This doesn’t do anything for anyone yet.

I have not even created any display of the new pages. The hard graft will be to see how to to chop up the basic Things page and FloorPlan page, then see how to get the devices displaying and updating on the new pages that will eventually appear.

I’ve published the extension so far at https://github.com/chas-iot/pages-extension

It is not ready to be added to the of published addons. So far, it is only some internal infrastructure and the UI to manage that infrastructure - it does not create pages that display updating things.

If you want to test, please note that the path to the database is hard-coded to /home/pi/.mozilla-iot/pages/pages.sqlite3 - make sure that path exists or edit src/db.js

Hi,

can anyone advise why this code leaves me with config === undefined ?

class PagesAPIHandler extends APIHandler {
    constructor(addonManager) {
        super(addonManager, manifest.id);
        addonManager.addAPIHandler(this);

        let pages_db_location = '/home/pi/.mozilla-iot/pages';    // hard-coded fallback

        const db = new Database(manifest.id);
        db.open().then(() => {
            db.loadConfig();
        }).then((config) => {
            if (config && config.dblocation) {
                pages_db_location = config.dblocation;
            } else {
                console.error(`pages-api-handler (B): "dblocation" is not in extension configuration ${JSON.stringify(config)}`);
            }
            PagesDB.open(pages_db_location);
        }).then(() => {

I’ve lightly edited from the actual code to remove some not relevant initialisation.

I just added the code to load the config and it is failing. With the fallback hard-coded location, the extension otherwise runs OK.

config will only be defined there after the user has configured the add-on via the UI. Defaults can be accessed from the manifest object.

How does the Floorplan page work?

I started to look at the Gateway code but got lost. From what I can work out from watching the Gateway HTML, the Floorplan redirects from the Things page. I would like to replicate that.

I assume there is some MVC or similar pattern used to do this. What should I research so I can do the same thing?

Thanks in advance :slight_smile:

Generally speaking, the UI is a single page app. The navigation just hides/unhides appropriate sections.

There is a loose MVC framework that manages the WebSocket connection to /things, e.g. property updates and such. The thing bubbles are all custom web components.

With that said, I’m not sure how much of it is directly accessible from an extension, so you’ll probably just have to play with it. The main logic you’ll want to look at (and probably attempt to replicate) is in thing.js.

1 Like

If possible, I’d like to achieve this at the browser level rather than reproduce a large chunk of the gateway.

Alternative approaches being researched:

  • Ideally/fantasy: find a web-api that replicates and dynamically mirrors sections of the DOM, so all I need to know is the ids on the Things page and I can magically display each one separately as I please on as many different pages as I please.
  • Build a some mirroring api from the MutationObserver api. Looks like a lot of work :rofl:
  • Very hacky, possibly also a fantasy: manipulate the DOM to ‘steal’ the components of the Things page and put them on the pages that the extension creates. Need to switch them all around as different pages are selected and to be able to put them back on the Things page when that is selected.

My question regarding the Floorplan page vs the Things page. Are both being separately being updated in the DOM, even when one (or both) is hidden? Or do they do some variant of one of the approaches I mentioned. I did start looking but this will take me many hours over several days to familiarise myself enough with the code to work it out for myself.

If you can give me an idea of exactly which things you’d need access to, I can potentially extend the Extension class (or elsewhere) to give developers a bit more access. I wonder if, instead, it would be simpler to just have a basic link for each device that takes you to the individual thing pages, though?

What I am trying to do is create additional pages that look like the Things page or the Floorplan page but with different sets of Things visible. Use case: I have a lot of non-physical Things (timers, intervals,email etc) that clutter up those two pages, and I’m not usually interested in seeing.

Where possible, I don’t want to rebuild existing functionality. Which is why I’d like to re-use the existing mechanisms that display Things.

I have further steps in mind.

  • There will be groups of things that will appear as a single ‘thing’ that, when opened, show the individual Things. Use case: I have a light with two different bulbs that I’d like to show as one until I need to look at the detail.
  • I’d like to give different users different levels of access to pages. Some pages they could see the items, other pages, control the items. Use case: when bored, my girlfriend can be a bit of a trickster…

So, as I said before, it’s probably easiest to just use text links at this point, given the current set of constraints.

Another option is to build this into the gateway itself. I’d love to have that feature and would gladly review a pull request…

1 Like

I’d love to build it into the gateway. I’ve been thinking about an approach. Here are my notes. Criticism is needed

Roadmap

  • Start behind an experimental flag
  • Start with the basic definition of additional Displays, which are similar to the current Things page
  • Later additions might include
    • Floorplans (I’d personally want this quite quickly)
    • Groups (a container of Things) that displays as a unit
      • Display a closed container
      • Open the container to display the contents inline in the context of the Display, still graphically showing they are part of a container (eg. Boxed with a name in the border of the box)
      • Be able to assign properties from the contents to the container
      • If, for example, there are two coloured lights in a container, could set up to be able to turn both on, to change both to the same colour
      • Stretch goal – target a group with rules
    • Permissions
      • Users have the permissions to view a Display, with no changing the contents or properties;
      • Permission to Remove Things from Displays;
      • Permission to Add Things to the Display (from Things page). Actually adding Things into the Gateway should still be performed on the Things page;
      • Permission to Modify Things properties on the Display;
      • Eventually, remove general permissions to access Things Page – it should be considered privileged
      • Some way to enforce that there always remains a most privileged user and some way to regain access if the password is lost

Design decisions

  • Have a new menu entry below Logs, named Displays (in English) (previously I called them pages)
  • Question: sub-menu within the sidebar or links in the main body?
    • I prefer a submenu.
    • This submenu would dynamically grow/shrink as users add or remove displays
    • The submenu to disappear when another main menu is chosen
  • New table: container
    • Where a Display (and later a Group) may be defined
    • Column: name
    • Column: rowid (primary key)
    • Column: rowtype – D for Display, G for Group
    • Column: settings – a JSON string where a link to a floorplan image, etc may be placed
  • New table: link
    • Where a relationship between a container and a thing may be set up
    • Column: rowid (primary key)
    • Column: container -> container.rowid
    • Column: contained -> things.id
    • Column: rowtype – T for Thing, U for user
    • Column: settings – contains a JSON string of settings, such as the x,y co-ordinates of the placement on a Floorplan
    • Column: order – the sequence to display on a page (JSON handling is not yet built into sqlite3, so this is too painful to put into the settings column)
    • Later, contained could point to users.id as part of permissions (shame that the id has different types in the two tables)
  • Question: in the browser, does it make sense to
    • Have multiple copies of a Thing view, one on each Display where it is placed, plus one on the Things page and another on the Floorplan?
    • Or, to re-parent the Thing view to the Display being displayed/unhidden?
    • In this case a Thing view is contained on the current Thing page in, for example, <div class="thing connected" draggable="true" id="thing-zb-14b457fffe4f65ce" data-layout-index="4">
    • My gut feel is that it is cheaper to reparent elements rather than have multiple copies. Multiple copies of a Thing view would have to be addressed, say, by sharing a class and then iterating through the class as each update arrives in the model. And each might have to have a separate id, with the issues around namespacing to avoid clashes
  • In the Displays sub-menu, the last entry will be Add Display
  • The Things page remains key, as a system level management page
    • On each Display, the ‘big + in a circle’ (the same icon as on the Things page) would allow the user to add a Thing that exists on the Things page.
    • On each Display, removing a Thing removes it only from that Display. To fully remove a Thing from all Displays requires removing it from the Things page.
    • Eventually, the Floorplan page might become redundant
    • Maybe, the Things page should migrate to Settings

Wow, this is thorough! Some thoughts:

  1. I’d leave the permissions bit out for now, as that is a much larger issue.
  2. Why are you wanting to reimplement the floorplan?
  3. Submenu: I like the idea, but that will require a fair amount of re-architecting things. Instead, I’d go with something similar to the Settings page.
  4. DB tables: You should be able to do this with one additional containers table containing id, name, etc., then the description in things could be amended to add a containers array. This is essentially what we do for the floorplan and positioning on the main Things page.
  5. Browser question: it depends. The Thing view is different between the main things page and the floorplan view, so for this new use case, it will depend on the desired functionality. It does seem simpler to reparent, but you may decide otherwise once you start digging in.

Thanks for the response @mrstegeman

No problem, that’s far and distant.

I’d like to have a Floorplan with some Things left off (e.g. virtual things like timers, email and weather). I’d like to have a larger scale partial Floorplan showing some Things in more detail.