Where to start debugging when things don't update

WebThings
#1

Hi all,
I’m pretty new to the Mozilla IoT/WebThings and started to create a bunch of own sensors (ESP32/ESP8266 based, so I’m using the Arduino WebThing lib and version 0.9.2 of the gateway).

I noticed very strange behaviour of the Temperature und MultiLevel Sensors (e.g. only the latest LevelProperty that was added gets updated - I worked around this problem by simply having multiple devices on a single adapter). However, I noticed that at some point all sensors that I created stop updating (reproducible). I didn’t find a pattern yet when this happens, but the experimental logs feature clearly show that at some point they simply stop updating and so do the downloadable logs- although the devices are still reachable (http://IP/things/DeviceName/properties).

I’m a bit lost on how to debug this without any logs showing a device failure or the devices being unreachable, they are still accessible and provide valid data. Any hints on how I can figure out why the gateway stops updating the things? In some cases the devices start to update again after hours, in other cases they don’t or just some of them (even though they are served by the same adapter on the same device and still providing valid data). Is there any way to find out why a thing wasn’t updated anymore? Frankly speaking, I didn’t figure out yet if a call to adapter->update() pushes the values to the gateway or if the gateway pulls the data regularly and the call just updates the internal data structure that is being published.

I’ll file issues for it once I’m able to confirm it’s really a problem with the gateway and not with my code.

I’m very grateful for any hints. Thanks for all the effort you put into the project so far, great work, I love it!

Thanks,
Chris

MultiLevelSensor Only Updates Second Thing
#2

Is there anything relevant in your log files? It would probably be prefixed by “thing-url”.

You can find the log files in ~/.mozilla-iot/log or in the UI via Settings -> Developer -> View Internal Logs.

#3

I think I found the problem by accident… it seems to be related to mDNS and caused by my router.
The logs didn’t give anything useful other than no more logs for the devices - at some point it simply stopped. Is there anyway to see when the gateway tried to reach the devices the last time? What happens exactly when you fire adapter->update()?

#4

No, I don’t believe that adapter logs anything when it’s polling, but you could add a log if you want.

  • mkdir ~/.mozilla-iot/addons/thing-url-adapter/.git
  • Add a console.log() in poll() in ~/.mozilla-iot/addons/thing-url-adapter/thing-url-adapter.js
  • Disable the “Web Thing” add-on in your UI, wait a few seconds, then re-enable it.
MultiLevelSensor Only Updates Second Thing
#5

Excellent, this is where the actual adapter code is… I’ll try to debug from here and will update the thread once I found the problem.

Cheers! Dank Dir!

#6

As promised, a - hopefully - final reply on the topic. The issues I faced seem to have multiple root causes:

  1. Bad USB cable to power the RaspPi
    No kidding, a shitty, cheap USB cable (in my case it was very likely just too thin in diameter) caused the RaspPi to struggle. A very well known problem… dmesg complained about under voltage: Under-voltage detected! (0x00050005). This caused the RaspPi to reboot in most of the times when you put load on it, e.g. viewing the page of the experimental log feature

  2. ESP8266s behave slightly different to ESP32s:
    In my case, whenever the device rebooted (e.g. wtd kicked in) or reconnects to WIFI the mDNS didn’t work reliable afterwards. With ESP32 that’s no problem, but for the ESP8266 an additional MDNS.update() in the main loop worked miracles… Don’t forget to check for connectivity in the main loop of your thing and re-establish if required - this check is missing in every sample code I’ve seen so far.

  3. Deep-sleep doesn’t work with the gateway, the devices lose connectivity and will only occasionally recover. Bear in mind that the ThingAdapter polls rather than getting the data pushed from the thing.

I’ll polish my code a bit and commit it to my repo for reference. So far I have samples for BME280, soil sensors and I’m working on IR and RF examples to control e.g. sockets or LED strips.

Thanks,
Chris

#7

Hi,

I am using an ESP8266 and have trouble with the updating of the dashboard. Where do I put MDNS.update() in my code? If I put that code in the main loop I get the following error: expected unqualified-id before ‘.’ token

What am I missing here? Thanks in advance for your time and help. Danielle.

#8

MDNS.update() is done for you every time you call adapter->update().

#9

Thank you very much!
I have tried your suggestion but it doesn’t seem to make a difference. I am beginning to suspect the dashboard not updating has to do with the Arduino Nano 33 IoT.
I have a Nodemcu ESP8266 device running with a basic code (reading an analogue sensor and sending that to the server) which works like a charm and updates real time.
Today I replicated this code on my arduino nano and the dashboard updates every 3 seconds or stops updating completely.
This is the code I am using:

#include <Arduino.h>
#include "Thing.h"
#include "WebThingAdapter.h"
#include "secrets.h"

// initialize pins

const int ldrPin = A1;        // LDR
const int ledPin = 13;        // Debug LED

WebThingAdapter *adapter;

const char* ldrTypes[] = {"MultiLevelSensor", nullptr};
ThingDevice ldr("tafelLDR", "Tafel LDR A1", ldrTypes);
ThingProperty ldrRead("sensor", "Sensor value", NUMBER, "LevelProperty");
ThingPropertyValue ldrValue;

int ldrVal;
int LastLdrValue;
const int thresholdLDR = 5;

void setup() {
  pinMode(ledPin, OUTPUT);
  // connect to wifi
   Serial.begin(115200);
  Serial.println("");
  Serial.print("Connecting to \"");
  Serial.print(ssid);
  Serial.println("\"");
#if defined(ESP8266) || defined(ESP32)
  WiFi.mode(WIFI_STA);
#endif
  WiFi.begin(ssid, password);
  Serial.println("");

  // Wait for connection
  bool blink = true;
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
    digitalWrite(ledPin, blink ? HIGH : LOW); // active low led
    blink = !blink;
  }
  digitalWrite(ledPin, HIGH); // active HIGH led

  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
  adapter = new WebThingAdapter("w25", WiFi.localIP());

  ldr.addProperty(&ldrRead);
  adapter->addDevice(&ldr);

  adapter->begin();
  Serial.println("HTTP server started");
  Serial.print("http://");
  Serial.print(WiFi.localIP());
  Serial.print("/things/");
  Serial.println(ldr.id);

}

void loop() {
  adapter->update();
  // lezen licht sensor
  ldrVal = analogRead(ldrPin);
  checkAndSetLdr();
  delay(50);
}

void checkAndSetLdr(){ 
  if (ldrVal - LastLdrValue > thresholdLDR || LastLdrValue - ldrVal > thresholdLDR) {
    Serial.print("ldrValue: ");
    Serial.println(ldrVal);
    ldrValue.number = ldrVal;
    ldrRead.setValue(ldrValue);
    adapter->update();
    LastLdrValue = ldrVal;
  }
}

Maybe I am missing something?
Hope someone can help because I can’t really use the webthings platform this way… Thanks very much in advance.

#10

The difference here is that the ESP8266 and ESP32 have WebSocket support, so you get instant property updates. With other boards, the gateway has to poll the device. By device, the poll interval is 5 seconds.

#11

Thanks!
The documentation says: The communication on WiFi and Bluetooth is managed by a NINA W102 ESP32 based module (https://www.u-blox.com/sites/default/files/NINA-W10_DataSheet_(UBX-17065507).pdf). So that should be the same, right?
Also very often the devices just stay greyed out in the dashboard and don’t appear to be connected at all even though the json file shows the transmitted values.
Any suggestions to make this smoother are very welcome. TIA.

#12

The documentation says: The communication on WiFi and Bluetooth is managed by a NINA W102 ESP32 based module (https://www.u-blox.com/sites/default/files/NINA-W10_DataSheet_(UBX-17065507).pdf). So that should be the same, right?

We use this library for WebSocket support, which only supports the ESP8266/ESP32. The Arduino Nano board uses the Arduino SAMD platform.

Also very often the devices just stay greyed out in the dashboard and don’t appear to be connected at all even though the json file shows the transmitted values.

Try adding your devices by IP address, rather than using the auto-discovered ones. That seems to work better.

#13

Thanks very much for the info!
I have already added the devices by URL.

Does this mean that there is no way to use the Arduino Nano 33 IoT and get a more stable connection with the dashboard? This would be problematic for as I already build 3 devices using the Nano to work with your platform… Do you have any tips on how to proceed?
Thanks very much for your help.

#14

If by “stable” you mean instant updates, you can either decrease the poll interval in the add-on settings, or yes, switch to ESP8266/ESP32 – unless you can find us a WebSocket library we can use for the Nano board!

#15

Hi,
I will try to decrease the poll interval to see if that helps the performance.
By stable I mean that the Things show up when they are online and that they stay online.
Now I have to use add new devices (I don’t add any, I just visit the page and let it search) before the Things are activated/connected. They often get greyed out after a restart of the device or when I upload new code. Or the values freeze and I can’t get them to update any more. Can you shed any light on this?

I am not a developer and I don’t know much about websockets. I did a search and all the libraries I could find were also for ESP8266/32. But I suppose you did this search too. :slight_smile:

Again many thanks.

#16

There are 2 reasons that a device would be marked as disconnected:

  1. The gateway receives a “service down” mDNS event from the device.
  2. The gateway failed to poll the device. In this case, there should be something about it in the gateway’s log.

Either of the 2 cases above could be caused by the device crashing and restarting. Is that happening?

#17

Yes, I restart after uploading new code. I also have to press the reset button often because otherwise the device doesn’t connect to internet. I will try and check the log.