It is sometimes useful to use docker-run, but with a different image
than the default one. This commit allows to override the image being
used by only defining IMAGE if not already passed in the environment.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: s/\t/ /g]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
After switching to a fresh Fedora 38 installation with SELinux disabled,
we noticed that utils/docker-run doesn't work as the applications
running inside the container are not allowed to accept the data mounted
through the bind mount.
Since we do not really need to isolate and confine the build, but rather
to provide a known environment, we don;t really need to enforce any
SELinux confinment in the container.
So, we tell docker to turn off label confinement for the container:
https://manpages.org/docker-run
--security-opt=[]
Security Options
[...]
"label=disable" : Turn off label confinement for the container
Suggested-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: use Antoine's proposal]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, using utils/docker-run expects that the current working
directory is the working copy. This means that it is not possible
to use docker-run with an out-of-tree build (one using O=).
Add the current working directory to the list of mountpoints, and
use that as working directory in the container.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
If buildroot is checked out as part of a 'repo' manifest, docker-run
doesn't fully bind mount the .git directory, leading to commands such
as `utils/docker-run make check-package` to fail.
Signed-off-by: Nicolas Boichat <drinkcat@google.com>
[yann.morin.1998@free.fr: use newly introduced mountpoints list]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
If the user has defined $BR2_DL_DIR in the environment, it would be
nice to have it accessible inside the Docker container, and the
BR2_DL_DIR environment variable set to access it.
This commit does exactly this: it mounts the host $BR2_DL_DIR as /dl
in the container, and sets BR2_DL_DIR=/dl in the container.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: use the new mountpoints list]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
For now, we only ever mount two mountpoints, the main directory (i.e.
the working copy), and the git directory.
To pave the way for adding new mountpoints, we introduce a list of them,
that we sort to ensure that we never mount a shallower mounpoint after a
deeper one (that would shadow the deeper mountpoint).
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 9079079092 (utils/docker-run: fix support for git-worktrees)
got last-minute changes when it was applied, and the case when the
current working directory is not the top of the current working copy
got broken.
Fix that by duplicating (and thus reinstating) the 'cd MAIN_DIR' to
match what is done when retrieving the git-common-dir.
Fixes: 9079079092
Reported-by: Brandon Maier <Brandon.Maier@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The docker-run script attempts to support git-new-workdirs and
git-worktrees by resolving the symlink at '$GIT_DIR/config' to get the
true $GIT_DIR. However this does not work for git-worktrees as they do
not use symlinks, instead they change the $GIT_DIR into a regular file
that contains the path to the real $GIT_DIR. To complicate things
further, we actually want the $GIT_COMMON_DIR which is the superset of a
worktree's $GIT_DIR.
git-rev-parse supports the '--git-common-dir' which will resolve the
$GIT_COMMON_DIR for us. However it does not work for git-new-workdirs,
so we still need to detect and handle them.
'--git-common-dir' also appeared only with git 2.10.0, released in 2016,
so it will not be available in older "enterprise-grade" distributions.
In that case, 'git rev-parse --git-common-dir' would return the option
flag '--git-common-dir' as-is, which is incorrect. So, we instruct it to
never return flags.
'--git-common-dir' also returns just '.git' for the main working copy,
but 'docker run' want an absolute path, so we canonicalise it.
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
[yann.morin.1998@free.fr:
- support git versions before --git-common-dir was introduced
- don't mount GIT_DIR if unknown (i.e. not needed)
- fix expanding MAIN_DIR
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
It is quite customary to use a single repository with multiple workdirs,
one for each active branch, with either the aging 'git new-workdir' or
the more recent 'git worktree'.
However, in a workdir/worktree, most entries in .git/ are only symlinks
to the actual files in the main repository.
Currently, utils/docker-run only bind-mounts the current working copy.
If that is a workdir/worktree, then it is going to be missing the actual
git data, resulting in errors like:
$ ./utils/docker-run make check-package
fatal: not a git repository (or any parent up to mount point [....]/buildroot)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
No files to check style
make: *** [Makefile:1257: check-package] Error 1
So, we also bind-mount the actual git directory. If that is a subdir
of the current working copy, then it is already mounted and thus the
bind-mount is superfluous but harmless; for simplicity, we mount it
unconditionally.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@datacom.com.br>
Currently, utils/docker-run spawns a container with a tty, so that he
user can interact properly in the container.
However, that requires a tty when calling docker-run, which is not
always guaranteed, e.g. if called from a git hook.
Since the script is a bash script already, we can use an array to store
options passed to docker, and only add the -t option when there is
actually a tty available.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
In utils/docker-run line 10:
--user $(id -u):$(id -g) \
^------^ SC2046: Quote this to prevent word splitting.
^------^ SC2046: Quote this to prevent word splitting.
The suggestions from shellcheck can be applied.
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Since this is a helper script there is not much reason to show the
command that's been issued. Furthermore, the incantation has been
slightly extended since the script was introduced.
The only interesting reason to print the command is to know what image
it is being spawned into. However, this is prominently displayed by
docker the first time the script is run, as it can't find the image
locally and has to fetch it first. Afterwards, users can still use
'docker image ls' to see what images they have locally.
So let's remove 'set -x' before running docker.
Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
[yann.morin.1998@free.fr: reword and expand commit log]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, the docker-run script starts a container based on the
reference image, in which the user has UID:GID set to 1000:1000,
which may or may not be the same as local user uses, which may
prevent the commands in the container from creating files, which
foils the plan of using the container to run a build...
Additionally, the paths in the container differ from the paths
the user expects, as the current Buildroot tree is mounted over
the in-container user home directory. This is a bit confusing...
Finally, the container is left hanging around after the command
finishes, and thus there are as many lingering containers as the
user runs docker-run. This is not very nice...
We fix all of that (yeah, should be different patches, but meh):
- we use --mount instead of -v, which allows us to bind-mount
the Buildroot tree at the same place in the container, as
Docker will create the destination if it does not exist, while
-v seems to expect it to exist [0].
- as a consequence, we can also set the working directory as the
Buildroot top-directory;
- use --user to force the same UID:GID in the container as the
local user, so that files created in the container belong to
the local user, and so that files from the local user are
accessible from the container;
- use --rm to remove the container once it terminates; starting
a new container is very quick anyway, so it is as good as
re-using a previous container.
[0] the documentation is not clear about that. It clearly states
that the host directory (i.e. the origin, the source) is created
if missing, but it says nothing of the destination:
https://docs.docker.com/engine/reference/commandline/run/#mount-volume--v---read-only
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Giulio Benetti <giulio.benetti@benettiengineering.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
Tested-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Without -i, the container cannot be used interactively. Allow using it as
in:
$ ./utils/docker-run
+ exec docker run -v /home/ceresoli/devel/buildroot:/home/br-user -it registry.gitlab.com/buildroot.org/buildroot/base:20220206.1756
br-user@1e9508a4ccbb:~$ ls
arch board boot CHANGES Config.in Config.in.legacy configs COPYING DEVELOPERS docs fs linux Makefile Makefile.legacy output package README support system toolchain utils
br-user@1e9508a4ccbb:~$
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
/usr/bin/bash does not exist on Ubuntu 2018.04. Use the more portable
alternative "#!/usr/bin/env bash" which we are already using in quite
a few scripts.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Fixes: 242e9d72e7 ("utils/docker-run: new script")
[yann.morin.1998@free.fr: use "#!/usr/bin/env bash", not "@!/bin/bash"]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Add a small script to run commands in the same docker image used in the
GitLab CI.
For instance, one can run check-package unit tests without installing
pytest directly in the host:
$ ./utils/docker-run python3 -m pytest -v utils/checkpackagelib/
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: fix shellcheck errors; add exec]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
