Broke the Logs with long retention period

WebThings
#1

Hi,

I’m trying to use the gateway with BME280 as a weather station, since it’s a sensor with temp/hum/pressure/dew measurements. When running it with Logs retention for 7 weeks it worked fine, but I want to have a 1-year archive to extrapolate data and it broke. The whole gateway became irresponsive when trying to calculate such a period of time. I cannot even delete the logs from the UI, and since I don’t know how to do it without the front end I’m stuck.

Can anyone help?

Thanks,
Preslav

#2

I’m not all that surprised. The logging is built on top of SQLite, rather than a proper time-series database, so all of the math and such is much heavier.

You can do the following (on the Raspberry Pi) to delete logs from the gateway manually:

# stop the gateway
sudo systemctl stop mozilla-iot-gateway

# find log IDs
sqlite3 ~/.mozilla-iot/log/logs.sqlite3 "SELECT id, descr FROM metricIds"

## now, find the ID (the first field) in the output above,
## to use in the next steps in place of <id>

# delete logs with that ID
sqlite3 ~/.mozilla-iot/log/logs.sqlite3 "DELETE FROM metricsNumber WHERE id = <id>"
sqlite3 ~/.mozilla-iot/log/logs.sqlite3 "DELETE FROM metricIds WHERE id = <id>"

# start the gateway
sudo systemctl start mozilla-iot-gateway
#3

Thank you!

Really it is not the proper DB for me. Tried the InfluxDB add-on to log the measurements from the sensor and then visualize with Grafana. Works smooth. The only problem is that I cannot access Grafana server through the gateway from outside . May be some Grafana integration would be usefull.

Thanks again and cheers!
Preslav