aboutsummaryrefslogtreecommitdiffstats
path: root/mds/securedocker.txt
diff options
context:
space:
mode:
authorterminaldweller <devi@terminaldweller.com>2024-05-20 02:23:40 +0000
committerterminaldweller <devi@terminaldweller.com>2024-05-20 02:23:40 +0000
commitfe8f6fc5246abf4d99db034d8848c0297f9c9c1a (patch)
tree15e7f63528f8e62c91034f146027081a3407f56a /mds/securedocker.txt
parentWIP (diff)
downloadblog-fe8f6fc5246abf4d99db034d8848c0297f9c9c1a.tar.gz
blog-fe8f6fc5246abf4d99db034d8848c0297f9c9c1a.zip
new article
Diffstat (limited to '')
-rw-r--r--mds/securedocker.txt611
1 files changed, 611 insertions, 0 deletions
diff --git a/mds/securedocker.txt b/mds/securedocker.txt
new file mode 100644
index 0000000..62a4796
--- /dev/null
+++ b/mds/securedocker.txt
@@ -0,0 +1,611 @@
+== Docker, Linux, Security. Kinda.
+
+We will be exploring some Linux features in the context of a docker
+application container. Another way of explaining it would be to say we
+will talk about how to make more secure application containers. We will
+not talk about firewall and apparmor because they are tools that enhance
+security on the host in general and not specific to a docker application
+container. A secure host means a more secure application container but
+that is discussion for another post. We will focus on Linux containers
+since FreeBSD containers are still experimental(see
+https://wiki.freebsd.org/Docker[here] and
+https://github.com/samuelkarp/runj[here]). Yes, windows containers
+exist. We will not discuss performance. Here be performance penalties,
+but again that is not the focus of this post.
+
+Before we begin, Linux docker containers are Linux. They are using most
+of the functionality that existed before application containers in the
+form of docker were a thing. Knowing Linux better means you know Linux
+Docker containers(application containers is a more correct term) better.
+We will see this point throughout this post.
+
+=== Base Image
+
+We start with the first building block of a new docker image, The base
+image. By far the most used base images are the Alpine docker base
+image, followed by Debian and Ubuntu docker base images. These distros
+have two major differences that we want to focus on:
+
+* C standard library implementation
+* the userspace utility implementation
+
+Debian and Ubuntu(we are not forgetting that Ubuntu itself is a Debian
+derivative) both use glibc, as in gnu’s
+https://www.gnu.org/software/libc/[libc] implementation. Alpine uses
+https://www.musl-libc.org/[musl-libc] as its C standard library
+implementation. The major difference here which will come into play
+later on again is glibc has been around for much longer, so it has to
+keep backwards compatibility for a much longer period of time and for
+far more many things. Also the general attitude with the glibc team is
+that they have to support everything since if they don’t then who will?
+Libmusl on the other hand, does not try to support everything under the
+sun, a relatively newer project, comparatively, and, they keep their
+codebase lean. As a result not all applications are supported by libmusl
+but a good number of them are. In simpler terms, libmusl has a far
+smaller attack surface compared to glibc.
+
+On to our second point, which is the cli utilities’ implementation.
+Debian and Ubuntu use gnu’s
+https://www.gnu.org/software/coreutils/[Coreutils] while Alpine uses
+https://busybox.net/[Busybox](remember, we are talking about the most
+used application container bases. You can install a desktop version of
+Alpine with GNU coreutils). Here we have the same situation as before,
+The GNU coreutils are bigger, do more and have a larger attack surface.
+Busybox is smaller, does not support as many features as GNU Coreutils
+but does support enough of them to make them useful. Needless to say,
+busybox is small and hence, it has a smaller attack surface.
+
+To get a feel for how this plays out in the real world, you can look at
+some of the popular images that come in both Debian and Alpine flavours
+on dockerhub. Take a look at the number of reported vulnerabilities for
+both bases. The theme we observe is simple. The bigger the attack
+surface the bigger the number of vulnerabilities.
+
+Alpine images are small, lean and functional, just like libmusl and
+busybox but there are still quite a few things on an alpine image that
+are extraneous. We can take them out and have a perfectly functioning
+application container.
+
+That’s how we get
+https://github.com/GoogleContainerTools/distroless[distroless].
+Distroless base images follow the same pattern as alpine base docker
+images, as in, less functionality while still keeping enough
+functionality to be able to do the job and minimize the attack surface.
+Minimizing a base image like this means that the base images are very
+specialized so we have base images for golang, python, java and the
+like.
+
+=== Dokcer Runtimes
+
+By default docker uses containerd which in turn uses runc for the
+runtime. There are two additional runtimes that we want to focus on who
+try to provide a more secure runtime environment for docker.
+
+* gvisor
+* kata
+
+==== gvisor
+
+gVisor creates a sandbox environment. Containers interact with the host
+through this sandboxed environment. gvisor has two components. Gofer and
+Sentry. Sentry is a kernel that runs the containers and intercepts and
+responds to system calls made by the application so as not to have an
+application directly control the syscalls that it makes. Gofer handles
+filesystem access(not /proc) for the application. The application is a
+regular application. gVisor aims to provide an environment equivalent to
+Linux 4.4. gvisor presently does not implement every system call,
+`/proc` file or `/sys` file. Every sandbox environment gets its own
+instance of Sentry. Every container in the sandbox gets its own instance
+of Gofer. gVisor currently does not support all system calls. You can
+find the list of supported system calls for amd64
+https://gvisor.dev/docs/user_guide/compatibility/linux/amd64/[here].
+
+....
+ -------------
+ |Application|
+ -------------
+ |system calls
+ |
+ -------- 9p -------
+ |Sentry|<------->|Gofer|
+ -------- -------
+ | limited |system
+ | syscalls |calls
+ ---------------
+ | Host Kernel |
+ ---------------
+ |
+ |hardware
+....
+
+==== kata
+
+Kata creates a sandbox environment for containers to interact with as
+proxy, not too dissimilar to gvisor but the main point of difference is
+that kata uses a VM to achieve this.
+
+gVisor and katacontainers allow us to implement defense in depth when it
+comes to application containers and host system security.
+
+=== Capabilites and Syscalls
+
+Let’s talk about capabilities for a bit.
+
+From
+https://manpages.debian.org/bookworm/manpages/capabilities.7.en.html[man
+7 capabilities]:
+
+[source,txt]
+----
+For the purpose of performing permission checks, traditional UNIX implementations distinguish two
+categories of processes: privileged processes (whose effective user ID is 0, referred to as
+superuser or root), and unprivileged processes (whose effective UID is nonzero). Privileged
+processes bypass all kernel permission checks, while unprivileged processes are subject to full
+permission checking based on the process's credentials (usually: effective UID, effective GID, and
+supplementary group list).
+
+Starting with Linux 2.2, Linux divides the privileges traditionally associated with superuser into
+distinct units, known as capabilities, which can be independently enabled and disabled.
+Capabilities are a per-thread attribute.
+----
+
+Capabilities give you a more granular control over which privileges to
+give instead of just root and non-root. Docker let’s us choose which
+capabilities to give to a container. So we can for example allow a
+non-privileged process to bind to privileged ports using capabilities.
+As an example, a simple application making calls to API endpoints and
+writing results back to a database does not require any capabilities. It
+can run under a non-privileged user with no capabilities and do all the
+tasks that it needs to do. That being said, determining which
+capabilities are required can be a bit challenging when it comes to
+certain applications since there is no straightforward way of achieving
+this. In certain cases we can get away with dropping all capabilities,
+running our application and then trying to figure out, based on the
+received error messages, which capability is missing and needs to be
+given to the application. But in certain cases this may not be feasible
+or practical.
+
+From
+https://manpages.debian.org/bookworm/manpages-dev/syscalls.2.en.html[man
+2 sycalls]:
+
+[source,txt]
+----
+The system call is the fundamental interface between an application and the Linux kernel.
+----
+
+The Linux kernel lets us choose which ones of these interface calls can
+be allowed to be made by an application. We can essentially filter which
+syscalls are allowed and which ones are not on a per application basis.
+Docker enables this functionality with an arguably more friendly
+approach. Capabilities and syscall filtering are tools to implement
+principle of least privilege. Ideally, we would like to allow a
+container to only have access to what it needs and just that. Not more,
+and obviously not less.
+
+==== capabilities in the wild
+
+Capabilities are a Linux feature, docker allows us to use that with
+application containers. We’ll look at a very simple example of how one
+can set capabilities for a regular executable on Linux.
+https://manpages.debian.org/bookworm/libcap2-bin/setcap.8.en.html[man 8
+setcap] lets us set capabilities for a file.
+
+==== syscall Filtering in the wild
+
+As an example we will look at
+https://manpages.debian.org/bookworm/bubblewrap/bwrap.1.en.html[man 1
+bwrap]. https://github.com/containers/bubblewrap[Bubblewrap] allows us
+to sandbox an application, not too dissimilar to docker. Flatpaks use
+bubblewrap as part of their sandbox. Bubblewrap can optionally take in a
+list of syscalls to
+https://www.kernel.org/doc/html/v4.19/userspace-api/seccomp_filter.html[filter].
+The filter is expressed as a BPF(Berkley Packet Filter program -
+remember when I said docker gives you a
+https://docs.docker.com/engine/security/seccomp/[friendlier] interface
+to seccomp?) program. Below is a short program that defines a BPF
+program that can be passed to an application using bwrap that lets us
+log all the sycalls the application makes to syslog.
+
+[source,c]
+----
+#include <fcntl.h>
+#include <seccomp.h>
+#include <stdbool.h>
+#include <unistd.h>
+
+void log_all_syscalls(void) {
+ scmp_filter_ctx ctx = seccomp_init(SCMP_ACT_LOG);
+ seccomp_arch_add(ctx, SCMP_ARCH_X86_64);
+ seccomp_export_bpf(ctx, 1);
+ seccomp_export_pfc(ctx, 2);
+ seccomp_release(ctx);
+}
+
+int main(int argc, char **argv) {
+ log_all_syscalls();
+}
+----
+
+Building is straightforward. Just remember to link against `libseccomp`
+with `-lseccomp`.
+
+[source,bash]
+----
+gcc main.c -lseccomp
+----
+
+Running the above code we get this:
+
+[source,txt]
+----
+ > 5@#
+# pseudo filter code start
+#
+# filter for arch x86_64 (3221225534)
+if ($arch == 3221225534)
+ # default action
+ action LOG;
+# invalid architecture action
+action KILL;
+#
+# pseudo filter code end
+#
+----
+
+[source,bash]
+----
+#!/usr/bin/dash
+
+TEMP_LOG=/tmp/seccomp_logging_filter.bpf
+
+./a.out > ${TEMP_LOG}
+
+bwrap --seccomp 9 9<${TEMP_LOG} bash
+----
+
+Then we can go and see where the logs end up. On my host, they are
+logged under `/var/log/audit/audit.log` and they look like this:
+
+....
+type=SECCOMP msg=audit(1716144132.339:4036728): auid=1000 uid=1000 gid=1000 ses=1 subj=unconfined pid=19633 comm="bash" exe="/usr/bin/bash" sig=0 arch=c000003e syscall=13 compat=0 ip=0x7fa58591298f code=0x7ffc0000AUID="devi" UID="devi" GID="devi" ARCH=x86_64 SYSCALL=rt_sigaction
+type=SECCOMP msg=audit(1716144132.339:4036729): auid=1000 uid=1000 gid=1000 ses=1 subj=unconfined pid=19633 comm="bash" exe="/usr/bin/bash" sig=0 arch=c000003e syscall=13 compat=0 ip=0x7fa58591298f code=0x7ffc0000AUID="devi" UID="devi" GID="devi" ARCH=x86_64 SYSCALL=rt_sigaction
+type=SECCOMP msg=audit(1716144132.339:4036730): auid=1000 uid=1000 gid=1000 ses=1 subj=unconfined pid=19633 comm="bash" exe="/usr/bin/bash" sig=0 arch=c000003e syscall=13 compat=0 ip=0x7fa58591298f code=0x7ffc0000AUID="devi" UID="devi" GID="devi" ARCH=x86_64 SYSCALL=rt_sigaction
+type=SECCOMP msg=audit(1716144132.339:4036731): auid=1000 uid=1000 gid=1000 ses=1 subj=unconfined pid=19633 comm="bash" exe="/usr/bin/bash" sig=0 arch=c000003e syscall=13 compat=0 ip=0x7fa58591298f code=0x7ffc0000AUID="devi" UID="devi" GID="devi" ARCH=x86_64 SYSCALL=rt_sigaction
+type=SECCOMP msg=audit(1716144132.339:4036732): auid=1000 uid=1000 gid=1000 ses=1 subj=unconfined pid=19633 comm="bash" exe="/usr/bin/bash" sig=0 arch=c000003e syscall=13 compat=0 ip=0x7fa58591298f code=0x7ffc0000AUID="devi" UID="devi" GID="devi" ARCH=x86_64 SYSCALL=rt_sigaction
+type=SECCOMP msg=audit(1716144132.339:4036733): auid=1000 uid=1000 gid=1000 ses=1 subj=unconfined pid=19633 comm="bash" exe="/usr/bin/bash" sig=0 arch=c000003e syscall=14 compat=0 ip=0x7fa5859664f4 code=0x7ffc0000AUID="devi" UID="devi" GID="devi" ARCH=x86_64 SYSCALL=rt_sigprocmask
+type=SECCOMP msg=audit(1716144132.339:4036734): auid=1000 uid=1000 gid=1000 ses=1 subj=unconfined pid=19633 comm="bash" exe="/usr/bin/bash" sig=0 arch=c000003e syscall=13 compat=0 ip=0x7fa58591298f code=0x7ffc0000AUID="devi" UID="devi" GID="devi" ARCH=x86_64 SYSCALL=rt_sigaction
+type=SECCOMP msg=audit(1716144132.339:4036735): auid=1000 uid=1000 gid=1000 ses=1 subj=unconfined pid=19633 comm="bash" exe="/usr/bin/bash" sig=0 arch=c000003e syscall=1 compat=0 ip=0x7fa5859ce5d0 code=0x7ffc0000AUID="devi" UID="devi" GID="devi" ARCH=x86_64 SYSCALL=write
+type=SECCOMP msg=audit(1716144132.339:4036736): auid=1000 uid=1000 gid=1000 ses=1 subj=unconfined pid=19633 comm="bash" exe="/usr/bin/bash" sig=0 arch=c000003e syscall=1 compat=0 ip=0x7fa5859ce5d0 code=0x7ffc0000AUID="devi" UID="devi" GID="devi" ARCH=x86_64 SYSCALL=write
+type=SECCOMP msg=audit(1716144132.339:4036737): auid=1000 uid=1000 gid=1000 ses=1 subj=unconfined pid=19633 comm="bash" exe="/usr/bin/bash" sig=0 arch=c000003e syscall=270 compat=0 ip=0x7fa5859d77bc code=0x7ffc0000AUID="devi" UID="devi" GID="devi" ARCH=x86_64 SYSCALL=pselect6
+....
+
+Docker allows us to do the
+https://docs.docker.com/engine/security/seccomp/[same]. We can give
+docker a seccomp profile to filter out the syscalls that are not
+required for a specific container. You can find the default docker
+seccomp profile
+https://github.com/moby/moby/blob/master/profiles/seccomp/default.json[here].
+
+=== Namespaces
+
+....
+A namespace wraps a global system resource in an abstraction that makes it appear to the processes
+within the namespace that they have their own isolated instance of the global resource. Changes
+to the global resource are visible to other processes that are members of the namespace, but are
+invisible to other processes. One use of namespaces is to implement containers.
+....
+
+From
+https://manpages.debian.org/bookworm/manpages/namespaces.7.en.html[man 7
+namespaces]. You can think of namespaces as almost the same thing as a
+namespace does in some programming languages. Docker uses its own
+namespaces for the containers so as to further isolate the application
+containers from the host system.
+
+==== Namespaces in the Wild
+
+As an example let’s look at the script provided below. Here we are
+creating a new network namespace. The new interface is provided by
+simply connecting an android phone for USB tethering. Depending on the
+situation you have going on and the `udev` naming rules the interface
+name will differ but the concept is the same. We are creating a new
+network namespace for a second internet provider, which in this case, is
+our android phone. We then use this network namespace to execute
+commands in the context of this specific network namespace. Essentially,
+we can choose which applications get to use our phone internet and which
+ones use whatever it is we were previously connected to.
+
+[source,sh]
+----
+#!/usr/bin/env sh
+PHONE_NS=phone_ns
+IF=enp0s20f0u6
+
+sudo ip netns add ${PHONE_NS}
+sudo ip link set ${IF} netns ${PHONE_NS}
+sudo ip netns exec ${PHONE_NS} ip link set ${IF} up
+sudo ip netns exec ${PHONE_NS} ip link set dev lo up
+sudo ip netns exec ${PHONE_NS} dhclient ${IF}
+----
+
+[source,sh]
+----
+$ sudo ip netns exec home_ns curl -4 icanhaveip.com
+113.158.237.102
+$ curl -4 icanhasip.com
+114.201.132.98
+----
+
+*_HINT_*: The IP addresses are made up. The only thing that matters is
+that they are different.
+
+Since we have the android phone’s interface on another namespace the two
+cannot interfere with each other. This is pretty much how docker uses
+namespaces. Without a network namespace we would have to make a small
+VM, run a VPN on the VM and then make a socks5 proxy to the VM from the
+host and then have applications pass their traffic through a socks5
+proxy with varying degrees of success. *_NOTE_*: since we are not
+running the script on a hook, you might blow out your net having two
+upstreams at the same time. In which case, run the script, then restart
+NetworkManager or whatever you have.
+
+=== SBOM and Provenance Attestation
+
+What is SBOM? NIST defines SBOM as a ``formal record containing the
+details and supply chain relationships of various components used in
+building software.''. It contains details about the components used to
+create a certain piece of software. SBOM is meant to help mitigate the
+threat of supply chain attacks(remember xz?).
+
+What is provenance?
+
+....
+The provenance attestations include facts about the build process, including details such as:
+
+ Build timestamps
+ Build parameters and environment
+ Version control metadata
+ Source code details
+ Materials (files, scripts) consumed during the build
+....
+
+https://docs.docker.com/build/attestations/sbom/[source]
+
+==== Example
+
+Let’s review all that we learned about in the form of a light exercise.
+
+For the first build, we use a non-vendored version. Vendoring means that
+you store your dependencies locally. This means you are in control of
+your dependencies. You don’t need to pull them from a remote. Even if
+one or more of your dependencies One of the more famous examples is Lua.
+The Lua foundation actually recommend vendoring your Lua dependency.
+Vendoring helps with build reproducability.
+
+We will use https://github.com/terminaldweller/milla[milla] as an
+exmaple. It’s a simple go codebase.
+
+[source,dockerfile]
+----
+FROM alpine:3.19 as builder
+RUN apk update && \
+ apk upgrade && \
+ apk add go git
+WORKDIR /milla
+COPY go.sum go.mod /milla/
+RUN go mod download
+COPY *.go /milla/
+RUN go build
+
+FROM alpine:3.19
+ENV HOME /home/user
+RUN set -eux; \
+ adduser -u 1001 -D -h "$HOME" user; \
+ mkdir "$HOME/.irssi"; \
+ chown -R user:user "$HOME"
+COPY --from=builder /milla/milla "$HOME/milla"
+RUN chown user:user "$HOME/milla"
+ENTRYPOINT ["home/user/milla"]
+----
+
+The first docker image build is fairly simple. We copy the source code
+in, get our dependencies and build a static executable. As for the
+second stage of the build, we simply put the executable into a new base
+image and we are done.
+
+The second build which is a vendored build with a golang distroless
+base. We copy over the source code for the project and all its
+dependencies and then do the same as before.
+
+[source,dockerfile]
+----
+FROM golang:1.21 as builder
+WORKDIR /milla
+COPY go.sum go.mod /milla/
+RUN go mod download
+COPY *.go /milla/
+RUN CGO_ENABLED=0 go build
+
+FROM gcr.io/distroless/static-debian12
+COPY --from=builder /milla/milla "/usr/bin/milla"
+ENTRYPOINT ["milla"]
+----
+
+Below You can see an example docker compose file. Milla can optionally
+use a postgres database to store messages. We also include a pgadmin
+instance. Now let’s talk about the docker compose file.
+
+[source,yaml]
+----
+services:
+ terra:
+ image: milla_distroless_vendored
+ build:
+ context: .
+ dockerfile: ./Dockerfile_distroless_vendored
+ deploy:
+ resources:
+ limits:
+ memory: 128M
+ logging:
+ driver: "json-file"
+ options:
+ max-size: "100m"
+ networks:
+ - terranet
+ user: 1000:1000
+ restart: unless-stopped
+ entrypoint: ["/usr/bin/milla"]
+ command: ["--config", "/config.toml"]
+ volumes:
+ - ./config.toml:/config.toml
+ - /etc/localtime:/etc/localtime:ro
+ cap_drop:
+ - ALL
+ runtime: runsc
+ postgres:
+ image: postgres:16-alpine3.19
+ deploy:
+ resources:
+ limits:
+ memory: 4096M
+ logging:
+ driver: "json-file"
+ options:
+ max-size: "200m"
+ restart: unless-stopped
+ ports:
+ - "127.0.0.1:5455:5432/tcp"
+ volumes:
+ - terra_postgres_vault:/var/lib/postgresql/data
+ - ./scripts/:/docker-entrypoint-initdb.d/:ro
+ environment:
+ - POSTGRES_PASSWORD_FILE=/run/secrets/pg_pass_secret
+ - POSTGRES_USER_FILE=/run/secrets/pg_user_secret
+ - POSTGRES_INITDB_ARGS_FILE=/run/secrets/pg_initdb_args_secret
+ - POSTGRES_DB_FILE=/run/secrets/pg_db_secret
+ networks:
+ - terranet
+ - dbnet
+ secrets:
+ - pg_pass_secret
+ - pg_user_secret
+ - pg_initdb_args_secret
+ - pg_db_secret
+ runtime: runsc
+ pgadmin:
+ image: dpage/pgadmin4:8.6
+ deploy:
+ resources:
+ limits:
+ memory: 1024M
+ logging:
+ driver: "json-file"
+ options:
+ max-size: "100m"
+ environment:
+ - PGADMIN_LISTEN_PORT=${PGADMIN_LISTEN_PORT:-5050}
+ - PGADMIN_DEFAULT_EMAIL=${PGADMIN_DEFAULT_EMAIL:-devi@terminaldweller.com}
+ - PGADMIN_DEFAULT_PASSWORD_FILE=/run/secrets/pgadmin_pass
+ - PGADMIN_DISABLE_POSTFIX=${PGADMIN_DISABLE_POSTFIX:-YES}
+ ports:
+ - "127.0.0.1:5050:5050/tcp"
+ restart: unless-stopped
+ volumes:
+ - terra_pgadmin_vault:/var/lib/pgadmin
+ networks:
+ - dbnet
+ secrets:
+ - pgadmin_pass
+networks:
+ terranet:
+ driver: bridge
+ dbnet:
+volumes:
+ terra_postgres_vault:
+ terra_pgadmin_vault:
+secrets:
+ pg_pass_secret:
+ file: ./pg/pg_pass_secret
+ pg_user_secret:
+ file: ./pg/pg_user_secret
+ pg_initdb_args_secret:
+ file: ./pg/pg_initdb_args_secret
+ pg_db_secret:
+ file: ./pg/pg_db_secret
+ pgadmin_pass:
+ file: ./pgadmin/pgadmin_pass
+----
+
+We are assigning memory usage limits for the containers. We are also
+limiting the size of the logs we are keeping on disk. One thing that we
+did not talk about before is the networking side of compose. As can be
+seen, the postgres and pgadmin container share one network while the
+postgres container and milla share another network. This makes it so
+that milla and pgadmin do not have access to each other. This is inline
+with principle of least privilege. Milla and pgadmin don’t need to talk
+to each other so they can’t do that. Also we refrain from using host
+networking. We are also binding the open ports to the host’s localhost
+interface. This does not let us connect to the endpoints directly. In
+our example we don’t need the ports to be exposed to the internet but we
+will need access to them. What we can do is bind the open ports to the
+host’s localhost and then use ssh to forward the ports onto our own
+machine, assuming the docker host is a remote.
+
+[source,sh]
+----
+ssh -L 127.0.0.1:5460:127.0.0.1:5455 user@remotehost
+----
+
+While building milla, for the second stage of the build, we made a
+non-privileged user and our now mapping a non-privileged user on the
+host to that user. We are removing all capabilities from milla since
+milla will be making requests and has no server functionality. Milla
+will only need to bind to high-numbered ports which does not require a
+special privileges. We run both postgres and milla with gvisor’s runsc
+runtime since it’s possible to do so. Finally we use docker secrets to
+put the secrets into the container’s runtime environment.
+
+Now onto the attestations. In order to view the SBOM for the image we
+will use docker https://docs.docker.com/scout/install/[scout].
+
+[source,sh]
+----
+docker scout sbom milla
+docker scout sbom milla_distroless_vendored
+----
+
+The SBOMs can be viewed
+https://gist.github.com/terminaldweller/8e8ecdcb68d4052aecb6804823648b4d[here]
+and
+https://gist.github.com/terminaldweller/f4ede7122f159506f8e6e6be2bfd6a8b[here]
+respectively.
+
+Now lets look at the provenance attestations.
+
+[source,sh]
+----
+docker buildx imagetools inspect terminaldweller/milla:main --format "{{ json .Provenance.SLSA }}"
+----
+
+And
+https://gist.github.com/terminaldweller/033ae07a9e685db85b18eb822dea4be3[here]
+you can look at the result.
+
+=== Further Reading
+
+* https://manpages.debian.org/bookworm/manpages/cgroups.7.en.html[man 7
+cgroups]
+* system containers using https://github.com/lxc/incus[lxc/incus]
+* https://katacontainers.io/[katacontainers]
++
++
+timestamp:1716163133
++
+version:1.0.0
++
+https://blog.terminaldweller.com/rss/feed
++
+https://raw.githubusercontent.com/terminaldweller/blog/main/mds/securedocker.md
++