[IN PROGRESS] Integration Influxdb

Development
1

I’ve made considerable progress on the InfluxDB topic. Those with many features will be happy to be able to store their data over multiple years.

The data types pushed into InfluxDB are:

  • decimal
  • integer
  • binary

I’ll soon push this to my production environment to see how it performs. I’ll get back to you quickly to test.

In the meantime, some screenshots:

Gladys

Not much here, we only configure the information for its InfluxDB server

image
image668×402 32.6 KB

image
image776×552 21.1 KB

InfluxDB

In the metadata section we find:

  • Feature name
  • Room
  • Type

image
image1287×849 81.3 KB

6 Likes
2

Great, thanks for the screenshots!

How does it work — are the data stored twice?

I would really appreciate being able to create charts in InfluxDB if needed, over several months and multiple sensors. In Gladys, aggregated data alone is sufficient for my dashboards (if necessary) :blush:

3

Yes, it’s duplicated — each state change is reflected in InfluxDB.

You can therefore

1 Like
4

Do you think we could add an option (or make it the default) to retain aggregated values in Gladys for the periods 24h, 7d, 1m and 1y?

That way we won’t have duplicates but will keep visibility in Gladys.

5

I didn’t get it @lmilcent :sweat_smile:

6

I’m referring to the improvements we discussed with @pierre-gilles.
I suggested the following to reduce database size, and therefore storage needs and improve speed:

  • keep 24 hours of raw data
  • only keep aggregated values for the last 7 days (roughly 100 values/day)
  • only keep aggregated values for the last month
  • keep aggregated values for the last year.

That way, Gladys ends up storing relatively little data and the dashboard graphs remain functional.

And finally, for visualizing and storing all raw data, we could offer the InfluxDB integration you’re working on.

7

Cool @VonOx! :slight_smile:

I still agree with what we had discussed but it’s another topic for me, it doesn’t have much to do with In

8

Haha wait, you haven’t seen the code yet :eyes::grin:

3 Likes
9

It’s in production on my instance

image
image1853×479 111 KB

I’ll leave it running like this for a while; I have the tests and the documentation to write…

5 Likes
10

Almost a month, no worries :slight_smile:

image
image1842×896 166 KB

image
image1849×385 73.9 KB

I’ll let you guess when I went on vacation :stuck_out_tongue:

4 Likes
11

On the database side, how is it evolving? Really x2?

12

Db gladys?

13

You said a second DB was needed for InfluxDB. Hence my question: by using it you’ve doubled the disk space used, is that right?

What happens when you delete a device from Glaydys? Do you keep the data in InfluxDB or is it deleted as well?

14

It’s stored, it’s not the same technology.

With InfluxDB you create buckets with a retention. You push time-series data into those buckets

My Influx DB is 300 MB for 2 months of data

2 Likes
15

What are you pushing into your InfluxDB that makes it only 300 MB?

Otherwise it’s exactly what I was hoping for: viewing sensor data independently of Gladys when it comes to removing devices. It’s more complicated if you really want to delete all of a sensor’s data, but it allows addressing different needs and situations.

Can’t wait for

16

All the states :slight_smile:

17

I have to admit I don’t understand how that’s possible.

2 months at 300Mb for InfluxDB and 4.5G for 1 month in Gladys, what’s the trick?

18

Well, I only have the states on the Influx side; Gladys I have everything (aggregation, localization)
And the InfluxDB technology is made for that!

1 Like
19

I can’t wait :slight_smile:

20

I’m already using a build on my prod vonox/gladys:influxdb (it’s alpha)