A CDN for the forum's static assets!

Hello everyone :slight_smile:

Just a quick message to let you know, I’ve deployed a CDN to manage the forum’s assets (CSS/JS/icons/images)

It’s pretty simple, but I realized that the forum was making a lot of requests to load the CSS/JS/icons/images and it was a shame that these requests were made on the origin and not on a CDN close to the user.

Some pages load a lot of static resources, and it’s better to offload the forum’s backend so it can focus on dynamic content :slight_smile:

The goal is also to improve the forum’s Web Vitals to rank better on Google.

(Also, for users like me in Asia, this greatly improves loading times :innocent: )

To set this up, Discourse offers a turnkey configuration, which I deployed very simply. I used Bunny CDN as the CDN.

For now, everything is working, but if you notice any loading bugs at any time, let me know :slight_smile:

Great idea!
So far so good…

First minor issue (oops), as the CDN contacts the forum with the same IP (it’s a « pull Â» type CDN, so the cache needs to « warm up Â»), the forum’s internal rate limit (Nginx) sometimes blocks the CDN with 429 Too Many Requests.

For now, it hasn’t happened too much, but I’ve seen a few go through:

I’m restarting the forum with the fix!

Not yet, 6 PM… :sweat_smile:

The cache is gradually filling up all over the world :slight_smile:

Why am I doing this?

In short, for those who don’t know, Google has started measuring « Core Web Vitals Â» (Comment les seuils des mĂ©triques Core Web Vitals ont Ă©tĂ© dĂ©finis  |  Articles  |  web.dev), which are values that help estimate whether a site is truly fast for the end user and provides a good experience (not a crappy site with content that constantly shifts due to loading images, ads, etc.).

Google uses these measurements to influence the site’s ranking in Google, so it is crucial to have a fast site.

The Web Vitals of the Gladys site (gladysassistant.com) are excellent (normal, it’s a fully static site and deployed on a CDN - Cloudflare), but the forum’s Web Vitals are not always great, as the forum is not static and hosted on a central server at Hetzner.

Here’s what I see on my side in Cloudflare (I use their tool to measure web vitals for end users):

As you can see, in 18% of cases it’s bad! This mainly affects mobile browsers when things don’t go well (people browsing the forum on their phones in the subway, for example).

So, we need to improve these numbers for these users!

One approach was to add a CDN to the forum. I used BunnyCDN, which is a « Pull Â» CDN. This means the forum replaces all URLs of static content with the CDN domain, and the CDN:

  • On the first request, goes to the origin
  • Then caches the file for all subsequent requests

This helps offload the forum from asset requests and also allows the user to load all static content from a server close to them :slight_smile:

Cloudflare also provides me with a list of file calls that are sometimes slow, and unsurprisingly: it’s images :slight_smile: The CDN will do all its work here for these files.

Now, there’s one thing I haven’t done yet. I need to « regenerate Â» all the forum posts to replace the image URLs in the posts with the CDN URL (I didn’t realize I had to do this myself :p).

I’ll keep you updated here on the evolution of the forum’s web vitals following these changes!

Quick question then, as users, does the forum seem fast or not on a daily basis?

@VonOx @AlexTrovato @lmilcent @terdious you who are quite active, I’m open to any feedback on your experience with the forum :smiley:

It’s up to me

I had some latency yesterday on the first page refresh, but after that, I didn’t notice anything.

For my part, it’s the same, it’s fluid enough to have a good experience, whether on mobile or PC, knowing that I often only have poor 4G connection.
If you need the *.har files, I can send them to you by email, I have done the home page and the Latest French page.
Like @AlexTrovato, it happens that the home page takes a little longer to load after clearing the cache (between 5 and 10 seconds) but I really blame that on my connection!!

Okay, thanks for your feedback!

@Terdious no need, I already have a lot of metrics via Cloudflare analytics, I’ll monitor the progress :slight_smile:

I’ll post the progress here in the coming weeks.

Knowing that on the Google search console side, I also have a pretty average rating for the forum, which I hope will improve with these changes :slight_smile:

On my end, I’ve never had any latency issues, so I didn’t notice any particular change. :sweat_smile:
But it’s rather good news in the end!

For information, I will be launching the re-generation of all posts with images via the CDN, which will take a little while (about fifteen minutes I think), and the forum may be a bit slow during this time :slight_smile:

Yesterday I just moved all static assets (JS/CSS/icons) + images from new posts, and now it will also be the images from previously written posts!

Actually, I think the problems are rather for some users who find the forum via Google and land on a very long post with tons of photos.

Example: the tutorial « Flasher un Sonoff Â» on the forum is very well referenced on Google and drives a lot of traffic to the forum every day :smiley: Except that users arrive out of nowhere, have nothing in local cache, and have to load all the images (including icons, css, js).

So doing it via a CDN close to them allows to speed up the loading times for these people.

It’s working!!

haha it’s not me who works!! :stuck_out_tongue:

25% of the re-generation!

Screenshot 2021-10-09 at 15.06.58

Good, but it actually creates a task per post on the forum in Sidekiq (the background task manager used by Discourse). Given that there are 49k posts on the forum, we will now need to process the queue :joy:

It will take some time, but that’s just how it is :slight_smile: (and the queue is in « ultra low priority Â» so it doesn’t block normal forum execution)

OK, I understand system notifications on Discourse better

Hello everyone,
A long time ago, I made this request:

A link to the forum would also be interesting because indeed, we always go through a search engine to get there, Qwant for me.

Good point, sorry I must have missed this request!

But there is one!