Gladys Plus problem

We’re just trying to see if something gets saturated at some point, or if everything seems fine :slight_smile:

So look at CPU/RAM/network/disk write/disk read over time. Once you’ve installed it, take screenshots for us and post them here!

1 Like

Okay, here we go again today… loss of the Gladys Plus connection. This apparently happened between 11:30 and 12:30 since I had been trying to test regularly.

Here’s the log that caught my attention during the relevant period (12:06) :

2024-07-03T12:06:35+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = EFATAL, message = EFATAL: Error: read ENETUNREACH
2024-07-03T12:06:55+0200 <warn> index.js:914 (Socket.<anonymous>) Socket disconnected client side. Trying to reconnect...
2024-07-03T12:06:57+0200 <info> index.js:884 (Socket.<anonymous>) Gladys Gateway: connected in websockets

The netdata screenshots are so large that I can’t post them here, nor upload them to Imgur or elsewhere ^^

Let me know if any specific data would interest you!
I’ll paste a few things quickly for you:

Ah, this log is very interesting!

We can see that it’s not just Gladys Plus losing the connection, it’s also Telegram, and the Telegram error is very clear ENETUNREACH = network unreachable

So this is clearly a network issue

By any chance aren’t you using OpenDNS somewhere in your home network stack?

From memory, DNS-wise I’m on Cloudflare’s… But I need to check that!

I’m connected via Ethernet on my mini PC.

I’ve never noticed any problems with my router; at the same time, if it’s a brief outage it’s quite possible I wouldn’t even notice.
It’s a Freebox Pop, I’ll check if there’s a connection history I can consult.

EDIT: I confirm that I’m on Cloudflare with 1.1.1.1 and 1.0.0.1

1 Like

AH !!!

However, I’m having trouble understanding why, when my connection drops and then comes back, the Gladys Plus connection doesn’t come back.

Ah indeed!

Normally Gladys Plus reconnects if the connection comes back; maybe in your case the connection never returns to Gladys?

Well, actually I realize that something’s wrong because when I log in to plus.gladysassistant.com, my session appears fine, but today’s charts are all missing: and for good reason, the last displayed data dates from June 2.

What I have locally (up to date):

What I have on Gladys Plus:

@guim31 did you refresh the page? Because indeed, if your instance disconnects, the front-end will miss all the events that pass by, and there’s no workaround; only a refresh (or switching tabs/returning to the tab) can bring you back up to date

I just did a refresh and a Ctrl+F5 and it doesn’t change anything.
What’s strange is that it happens to me from my phone but also on my computer.

I had also disconnected my GLadys Plus from my local instance, then reconnected, and then it worked.

Restarting the Gladys container also works.

I just deleted the cookies on my computer, which ended my

Ok, and on the browser side, if you open the browser console on your computer, do you see a specific error?

Indeed:

Ok, this error means that the frontend decrypts the message but can’t authenticate that it was encrypted with the Gladys instance’s private key :smiley:

But you told me you restarted your container and that afterwards it worked again — how did you end up in this situation? Another outage?

Each time it happened, I restarted the container.
But I can’t tell you more about any network outages in the last 10 days; the Freebox doesn’t show me anything earlier :confused:

Ah sorry, I think I misunderstood your question: right now I left it as-is to run some tests. It was the previous times when I had restarted the container :wink:

1 Like

Ok, that’s what I wanted to know, indeed

Actually, it’s hard to know to what extent the errors we see are due to a failing network, or whether the failing network puts Gladys in a situation where she has trouble recovering, and if that’s the case, why ^^

Knowing that in my opinion something like this must be happening here: your instance gets disconnected for several minutes, the messages to send via Gladys Plus « pile up » and that probably creates one hell of a queue

Ok I see…

I’ll try to see if my Free network is « stable » but I admit I don’t really know what to look for.

What’s also extremely suspicious is that on your Gladys Plus frontend, only the charts don’t work, everything else works perfectly from what I can see (or I didn’t understand)

Do the charts usually work?

No, no, you’ll notice that the data are from June (lower temperatures) but also the latest changes I made (like creating the pool fill button) are no longer there.

Ok, if you want we can have a call sometime so you can explain it to me because I don’t think I understand right now ^^

Unfortunately, with my 30th birthday I won’t be available over the next few days, and after that I’ll be very busy with work — I can’t give you any date before July 15 :sweat_smile:

1 Like

It’s no big deal, we’ll find something then, don’t worry :wink: thanks already for this first help!!

1 Like