We were discussing this with @AlexTrovato and @VonOx, given that the release is soon we will need to change our development flow to introduce more testing before each release.
Before debating this topic, just one constraint we have:
The release builds of Docker images for ARM platforms are extremely slow (about 1 hour), and we can’t necessarily do them unlimitedly. I don’t know the limits of CircleCI, but a few builds per week/month are okay, several builds per day is not. We can’t therefore say that we systematically build a branch (master? dev?), it must be a manual action to say « we want to build a DEV build so that everyone can test this pre-release ».
@AlexTrovato I’ll let you speak if you have any suggestions
Oh hello, let me introduce myself, my name is… no we’ve already done the introductions
I do indeed think it would be interesting to have a « testing » or « pre-release » branch that in some ways continues the current mode, a candidate version that must be tested as much as possible (we know it’s difficult to test everything, especially when connecting to hardware…).
In short, we should stay in a mode of generating an image per new service, with as much testing as possible, to correct feedback without getting confused.
I say « per new service » but if we find enough testers, we could go up to 3 services; but I also think it’s important to limit the scope.
This means we would be at a maximum of one image per day (because time is also needed to process feedback).
So:
wait for the integration of N new features (services or core)
generate an image with a scope limited to these N features (+ other bugs if bugs)
We can get closer to an Agile + Git Flow process:
provide an amount of work for a limited « duration/scope » (with a task list)
integrate these new features into a dev branch from the PRs
validate (depending on availability) the new features (listed) one by one
when everything is okay, we merge everything into master
a new version has just been born
Success comes with good organization.
I’m talking, talking, but your opinions interest me.
Yes, I like the idea of the equipment list.
I start with the idea that if you enter your equipment, it means you want it to be integrated into Gladys and for that it needs to be tested.
I’m for it.
For the dialogue between Alex and Pierre-Gilles, I don’t understand everything either. But it’s interesting to read this topic because most of the time these conversations are private and non-developers cannot participate by giving their point of view.
It would indeed be great to have a test branch to validate new features/evolutions before integrating them into the main branch.
But should we detail them by service? I don’t think so, otherwise, we won’t test the potential side effects on other services under development.
It would also be great to have a branch to fix bugs detected on the Release → Git flow
I am also in favor of defining task lists linked to PRs, to be integrated by milestone, but I realize it is difficult to be agile when working on it only in the evening…
If we had a way to flexibly adjust which PRs will be integrated into the next Release, depending on the evolution of our schedule, that would be great.
By the way, I found a nice bug collection tool that handles node. There is a great article about it in the Linux Magazine special issue that just came out:
In my opinion, I’m rather in favor. After what scares me with this approach is that this branch becomes an « everything » branch, and we reach a stage where, for example, we have 2 features that are ready to be merged, and they are blocked by a « rotten » feature that prevents the merge.
For example: imagine you’re working on the Sonoff HTTP PR, and someone is working on the Zigbee2mqtt PR.
If we put both in « pre-release », but one of them introduces a critical bug that takes time to fix, then the Sonoff PR is blocked (and the entire development flow with it). Whereas in our current approach, we work « per individual branch per feature ».
I would be interested in seeing how large open-source projects close to our use case do it (I think of Home Assistant for example, they have a lot of contributors).
Yes, I use it on many projects. However, it’s adapted for cloud projects, not for self-hosted open-source projects.
1/ It sends all your data to Sentry, which is the opposite of the project’s philosophy
2/ Not to mention the pricing/limitations. It’s not at all made for our use case
If you can, we can talk about the error tracking/telemetry part in another topic. It’s a subject I’m not opposed to, but it must be done well and with great precautions to respect the privacy of Gladys Assistant users.
To bounce back on the processes of large open-source projects equivalent, here’s how Home Assistant proceeds:
A « dev » branch which is the equivalent of our master branch. All PRs are merged into this branch, and once a week, a « beta » image is built. I imagine some power users run Home-Assistant beta instead of the « prod » version.
A master branch, which is the production version. Every two weeks, they rebase the « master » branch on the dev branch. (master becomes the dev content which is normally in the testing phase for 1 week via the beta tag)
I find that pretty good, but my question is: what happens when there is a major issue on their dev branch?
Also, in the end, I see little difference with our current flow.
We merge PRs into master (which is the equivalent of their dev). Then, we just need a manual CI task to build master when needed (to a Docker dev tag), and when we are ready, we make a versioned tag for it to go to production!
Exactly, a PR should only be merged if it really works perfectly.
By the way, that reminds me it would be great if each PR at least did an amd64 build in the automated tests, to ensure that the Docker image always works perfectly. (I say amd64 because these are fast builds, if we have to test ARM builds it takes a lot). Currently, we are a bit in the dark when we merge on the Docker side. @VonOx what do you think?
CircleCI has partnered with Docker to ensure that our users can continue to access Docker Hub without rate limits. On November 1st, with few exceptions, you should not be impacted by any rate limits when pulling images from Docker Hub through CircleCI.
However, these rate limits may go into effect for CircleCI users in the future. That’s why we’re encouraging you and your team to add Docker Hub authentication to your CircleCI configuration and consider upgrading your Docker Hub plan, as appropriate, to prevent any impact from rate limits in the future.
It’s sure it’s not a long-term solution but for now it does the job
After that, for me it’s obvious that they will implement a solution for open-source projects for this rate-limit issue because for this we will really not be the only ones to have this problem.
For the story of builds + push in the PR I am 100% agree that we still have the problem of secrets, but build amd64 only I think it’s okay!
