Docker and you :)

Random
1

Hello everyone :partying_face:

I know many of you use Docker personally and/or professionally.
I soon have a test infrastructure to set up on Docker and therefore have a lot of questions about your usage. :slight_smile:

While reading the recommendations from ANSSI, I saw a few things on the security side:

  • remap uid
  • unprivileged

Remap

For remap uid, if we take lxc, from the moment it is unprivileged:

  • lxc user = 1000, host user = 101000

So in case of escape, no problem. Is this something you use a lot? Systematically?

Unprivileged

Do you use it a lot? In my view, it should never run with full privileges. If, for example, you need 4 USB ports, I prefer to set them specifically.

Storage

From what I remember, a Docker is volatile, application data must be bound inside. So they must be stored elsewhere. How do you manage this part? Do you export your logs, for example?

Cloud

Do you use public cloud? Private cloud?

I will start with self-hosted infrastructure / image, etc., limiting external dependencies to the strict minimum. So I came across OKD recently, it seems pretty good.

If anyone has recommendations, I’m all ears :slight_smile:

The final idea is to offer a platform to a few developers so they can be autonomous with image deployment.

In short… if you can share your Docker experiences, your uses, your needs/constraints, it will allow me to get an overview.

The people for whom I will do this want to use Docker, but I don’t think they have features like HA in mind, etc. :slight_smile:

Thanks to you!

2

In a private or professional context?

3

Both are interesting.

My solution will be put in the professional framework, hence the security aspect above all :slight_smile:

4

Hi

I had never seen your post, sorry!

On my side, personally, I use Docker with several options to remove root rights by default.

For this:

  1. User namespace option (UID and GID remapping)
    This is great because a process in root in the container is no longer root on your OS. This is what public CI/CD platforms use, I think.
    Only downside: some applications need to be adapted not to use privileged ports like port 80 or 443 or some system actions will not work (like creating a network card)

  2. Remove all CAPABILITIES by default with drop_cap=all then add only those that are necessary.

  3. Limit the maximum performance of the container in CPU, RAM and number of processes launched.
    This prevents a container that is buggy or compromised from using all the host’s resources.

  4. Install docker in its userland version, without the daemon that runs as root. It’s radical but better in security.

That’s what I was thinking for the moment :sweat_smile:

An example of Docker-compose with these elements without the PID limitation part:

5

No problem :slight_smile:

That’s what I also use with LXC

Thanks, this will probably be useful to me

I might do a lab before the end of the year on OKD by the end of the year to see how it turns out.

Since we will be providing custom Docker images to the developers, with custom Apache, PHP, PHP-FPM, etc., I see a potential issue that may arise.

For example, if there is an apache2/php7.4 image with certain PHP modules. If for a certain project a developer needs an extension that is not in it, another developer needs another extension… it may become unmanageable. This would mean creating an image per project. I’m having a bit of trouble visualizing this part.

6

Exactly, in general, this paradigm is used.

One project = one Docker image.

It is the project developer who designs their Docker image and updates their Dockerfile as the project evolves.

The Dockerfile is treated like any other code file, just as you add dependencies to the package.json (or equivalent for PHP projects), you add system dependencies to the Dockerfile as the project evolves.

7

In our case, no, we are more in the role of the host and therefore the web server, mail server, or other configurations, if there is a flaw in them, it’s up to us. It’s more a matter of scope and responsibility.

Today, we deliver VMs to host the code according to their needs. I am starting to work with Ansible to automate the VM build process. Tomorrow, they want to start using Docker on a lab platform to see what they can do. Therefore, this principle where the image is made available by the host will remain. This is likely to be complicated :slight_smile:

8

Ah, I see okay! :slight_smile: I thought you were talking about internal development. In this case, yes, it’s up to you to create a fairly comprehensive image to suit everyone, or to propose several versions with different levels of comprehensiveness depending on the projects.

9

This would rather be the second case. There will be a lot of discussion to find an organization.

Even with several images, it’s complicated. In the sense that you can have a project that needs exec, which is not recommended by default.
So you put an image with exec and one without and the dev can make their choice.

For exec, we work blocked by default. If there is really a need, we warn about the risk and activate it. If the dev can make their choice, I’m afraid it will become image consumption without necessarily being aware of the risks of these images, etc.

Without falling into caricature, I have already seen a dev do a chmod 777 because they were stuck … Euhhhhhhhh :nauseated_face: :crazy_face:

10

Hardening

On that note, you did well to read the recommendations from ANSSI. I can also recommend those from CIS BENCHMARK (docker and all kinds of systems).

Remap

This is done at the Docker configuration level directly. Example of my config:

/etc/docker/daemon.json

{
     "dns": ["9.9.9.9", "1.0.0.1"],
     "live-restore": true,
     "log-level": "debug",
     "max-concurrent-downloads": 30,
     "selinux-enabled": true,
     "no-new-privileges": true,
     "userland-proxy": false,
     "userns-remap" : "default"
}

Unprivileged

I would like to add that there is also a suite of very practical micro executables to address this issue: GitHub - just-containers/s6-overlay: s6 overlay for containers (includes execline, s6-linux-utils & a custom init) · GitHub

In summary, this project allows you to create containers that will execute a set of actions as root if necessary (change permissions, create certain specific files, create a simple user in the container, etc.) then execute the program that should be launched in the container with the lowest privileges.
It’s very well thought out and very practical. This is what I use in my image here: Dockerfiles/seedbox/flood at master · LM1LC3N7/Dockerfiles · GitHub

Storage

I confirm that by default, storage in a container is not persistent. This is the idea of docker volumes, to keep storage even after the deletion of a container.
The advantage is that you can do rm -rf /, it will do nothing on the host server (but your container will be down).
Most of my applications need to use persistent storage, for databases for example.
I manage this a bit in the spirit of Docker: one container = one process.

So for a blog that has a web server part and a database part, I have two containers:

  1. Web server, with html, JS, image files, etc.
    Nothing is persistent (as embedded in the image), except the logs, uploads and images folders, which are likely to change.
  2. The database.
    Same idea, nothing is persistent except the folder that stores the database files (for example /var/lib/mysql.
    My server uses a free distribution based on RedHat (Oracle Linux), which by default integrates SELinux. This allows you to protect volumes so that a container cannot escape or read restricted files on the system. It works very well!

Platform

Since it’s personal, I use a dedicated server at Online.net (now Scaleway.fr). But it is possible to use Kubernetes cloud instances for example, or directly container creation with AWS, Azure, Google Cloud, etc.

11

I hadn’t seen this thread about Docker.
It’s very interesting, thank you. :slight_smile:
I use it a bit at work.
So, do you have any tools for managing instances and creating high availability, please?

12

The best-known is the Kubernetes project; I have no experience with it, though. If you adapt an application to it, you can indeed manage high availability, automatic load balancing, etc. But from what I understand, it’s far from simple to do.

More basic, there are web interfaces like https://www.rancher.com/ (which has an open-source version) that could help you.

13

@lmilcent thanks for your feedback. I’ll take a look during the holidays. :slight_smile: