Learning WebThings - Professional Data Engineer/Scientist

WebThings
#1

Hi All,
I’m a azure data engineer/Scientist and have been in the profession about 5 years and absolutely love the command that I have over data.
I’m embarking on an extensive home renovation project in the coming months and am setting aside a small portion of the budget for home automation. Naturally being a data engineer I’m reluctant to pay someone to install an IoT system with off the shelf parts (and a little bit jealous of the fun he’ll be having), so will be taking it upon myself to design and setup my own IoT home automation system.

Im starting from the true zero here, so hopefully you all can assist me in my journey.
My skillset is SQL (Expert), Python (intermediate), c# (basic).
The end goal of this project is to ingest the data from my devices to my own Azure Data Platform. Within the platform I can create machine learning algorithms to fully automate the control of the subsystems within the homes.

But I need to get at the data. Which is where (im hoping) you all can help me.

The project in question is a 3 bedroom semi-detached property, approx 110 sqm (1200sqft), if there’s some traction on this forum I’ll post a floor plan. The house is being rewired so I’ll have access to a qualified electrician.

The solution design initially will address 2 use cases but will be scalable to address other use cases that may appear.

  1. Lumen-ary Automation - Motion detected light activation. Time-sensitive dimming. etc
  2. Heating Control - Automation of pnematic valves to heat rooms based on occupancy. (sign into room to trigger heating activation).
  3. Air Quality monitoring.

Everything else is a blank page.
Im attracted to WebThings because of its ability to run python scripts. I am familiar with the process of securely connecting to Azure via python so this is a massive upside for me.

I have ordered a sonoff zigbee switch and raspberry pi and am preparing to butcher my desk lamp to create a basic test rig.

So over to you all, I’ll need assistance in hardware design and connectivitiy. But happy to read any documentation to increase my knowledge would also be well received.

1 Like
#2

I would also research Home Assistant which has a much larger deployment, lots of active support for way more devices, and supports Python. I have started transitioning to Home Assistant as a learning excercise.

WebThingsIO is my primary HA system and like it. It still works great in a docker on an rpi using a zwave usb.

Development/support for the core software and addons has slowed over time after the transition from Mozilla to WebThingsIO. I hope new users (developers) can join the project to help maintain and improve the system moving into the future.

#3

Hi @63BL,

Welcome! Sounds like a fun project.

It’s true that development on WebThings was a bit slow for a while but it’s picking up again now and we’re just about to release WebThings Gateway 1.1, which involved a significant refactoring of the back end into TypeScript for better stability amongst other enhancements.

Your Python skills will come in handy in case you need to develop any new gateway add-ons for your project, but there are already over 100 to choose from. If you want to get really hands-on you can also use Python to develop your own web things using the WebThings Framework.

The best way to access the data from the gateway (and monitor and control the devices) is to use the Web Thing API, which includes both a REST API and WebSockets API, using OAuth2 for authentication of third party services.

Having your house re-wired opens up additional hardware opportunities such as wired rather than wireless devices (no need to change the batteries), but also smarter lighting. For example, it can be hard to retrofit smart light switches in houses which don’t have a neutral connection to the light switch.

For lighting I have used Phillips and IKEA smart bulbs and push buttons (all Zigbee). I haven’t yet done much with heating controls, but I know some people have experimented with connected radiator valves. You can see a list of hardware which is known to work with WebThings on the wiki.

Most of our documentation is at webthings.io/docs and the wiki. The documentation is due an overhaul so let us know if you find anything that doesn’t make sense, and also to contribute to improving the documentation yourself.

Looking forward to sharing your WebThings journey!

Ben