daniels changed the topic of #freedesktop to: https://www.freedesktop.org infrastructure and online services || for questions about freedesktop.org projects, please see each project's contact || for discussions about specifications, please use https://gitlab.freedesktop.org/xdg or xdg@lists.freedesktop.org
ykaelig has quit []
Haaninjo has quit [Quit: Ex-Chat]
shashanks_ has joined #freedesktop
shashanks has quit [Ping timeout: 480 seconds]
i509vcb has quit [Quit: Connection closed for inactivity]
phire has quit [Remote host closed the connection]
phire has joined #freedesktop
ximion has quit [Quit: Detached from the Matrix]
bmodem has quit [Ping timeout: 480 seconds]
bmodem has joined #freedesktop
privacy has joined #freedesktop
sima has joined #freedesktop
AbleBacon has quit [Read error: Connection reset by peer]
bmodem has quit [Ping timeout: 480 seconds]
bmodem has joined #freedesktop
sentriz has quit [Ping timeout: 480 seconds]
sentriz has joined #freedesktop
dcunit3d has joined #freedesktop
alatiera has quit [Quit: Connection closed for inactivity]
<bentiss>
PSA, gitlab security update incoming (should be relatively short)
<bentiss>
that should be it now
damian has joined #freedesktop
mvlad has joined #freedesktop
i509vcb has quit [Quit: Connection closed for inactivity]
damian has quit []
thaller has joined #freedesktop
cadubentzen has quit [Ping timeout: 480 seconds]
Haaninjo has joined #freedesktop
nnm has quit [Remote host closed the connection]
nnm has joined #freedesktop
cadubentzen has joined #freedesktop
<pinchartl>
bentiss: thank you
<pinchartl>
does that mean that gitlab-runner runs on bare metal on the runner machines, not in a container ?
bmodem has quit [Ping timeout: 480 seconds]
<bentiss>
pinchartl: yes, gitlab-runner on bare metal, not in a container, but it then spins up containers for each job
<bentiss>
and because some jobs are special, we have to keep running as priviledged containers, which is less than ideal
<bentiss>
thus the runner restrictions because we had script kiddies escaping the containers and mining bitcoins on them
<pinchartl>
ouch, escaping containers...
<pinchartl>
I'm trying to run gitlab-runner in a container (mostly because the machine I want to run it on won't give me bare metal access to run gitlab-runner)
<pinchartl>
next step is making sure that the container-in-container setup will work fine
<bentiss>
I had that setup with the coreos deployment
<pinchartl>
the "inner" container image I want gitlab-runner to run is 231GB in size, it's available on the host already, I need to make sure the gitlab-runner container won't try to download it
<bentiss>
you are running a container within a gitlab job or is it the job itself using that big container?
<pinchartl>
the 231GB container is the one that my gitlab job references in "image:"
<pinchartl>
so it's the latter, the big container is used by the job itself
<pinchartl>
thanks
<bentiss>
OK, so it's easy enough to cache it locally
<bentiss>
basically, you tell gitlab-runner to use the docker or podman socket from the host, and then magic happens
<pinchartl>
yes, that's what I'm trying to do
<pinchartl>
hopefully it will work fine
<pinchartl>
it's docker in this case. I'd use podman, but I don't have full control over this machine
<pinchartl>
I'll transition to podman next month
<bentiss>
It's in the "podman run" command that the magic happens: "/bin/podman run -v /etc/gitlab-runner:/etc/gitlab-runner:z -v /run/podman/podman.sock:/run/podman/podman.sock --name gitlab-runner registry.gitlab.com/gitlab-org/gitlab-runner:latest --log-level info"
<bentiss>
and you can replace podman with docker
<pinchartl>
I'll have a look at that, thanks
<pinchartl>
and as you're here, I'll ask the same question I asked daniels
<bentiss>
sure
<pinchartl>
"podman container run --rm" vs "podman container create ; podman container start"
<pinchartl>
any preference ?
<bentiss>
it depends I think. If you just want to run the command, podamn run is simpler, but if you want to attach another container to the same network of this pod, then create + do something + start
<bentiss>
also with run --rm, you can ensure to be running on the latest image, while create+start can allow you to be more in control
vkareh has joined #freedesktop
thaytan has joined #freedesktop
<pinchartl>
I was thinking that create + start is nice in that it preserves the state of containers across runs, even using volumes. that would probably backfire though, as a "podman prune" will then happily delete all that "permanent" state
<pinchartl>
so using "run --rm" may be best, as it will ensure there won't be bad surprises
AbleBacon has joined #freedesktop
alatiera has joined #freedesktop
dcunit3d_ has joined #freedesktop
dcunit3d has quit [Ping timeout: 480 seconds]
ximion has joined #freedesktop
mripard has quit [Quit: mripard]
mripard has joined #freedesktop
dcunit3d has joined #freedesktop
dcunit3d_ has quit [Ping timeout: 480 seconds]
todi has quit []
todi has joined #freedesktop
dcunit3d has quit [Ping timeout: 480 seconds]
dcunit3d has joined #freedesktop
dcunit3d_ has joined #freedesktop
dcunit3d__ has joined #freedesktop
dcunit3d has quit [Ping timeout: 480 seconds]
dcunit3d_ has quit [Ping timeout: 480 seconds]
ximion has quit [Quit: Detached from the Matrix]
dcunit3d__ has quit [Ping timeout: 480 seconds]
mripard has quit [Quit: mripard]
mripard has joined #freedesktop
dcunit3d has joined #freedesktop
dcunit3d_ has joined #freedesktop
dcunit3d has quit [Ping timeout: 480 seconds]
MrCooper has quit [Remote host closed the connection]
* pinchartl
wonders if he shouldn't use the simpler gitlab-runner shell executor
aswar002_ has joined #freedesktop
MrCooper has joined #freedesktop
pzanoni` has joined #freedesktop
<bentiss>
pinchartl: I strongly advice against using the shell executor
aswar002 has quit [Ping timeout: 480 seconds]
pzanoni has quit [Ping timeout: 480 seconds]
<bentiss>
unless you are in a VM that is created for each job
<pinchartl>
the goal here is to run very specialized jobs. but I understand it's a shortcut that comes at the price of security
<pinchartl>
so I'll likely not do that
<pinchartl>
I'm reaching 3 levels of inception: gitlab-runner in a container, executing the job in another container, and the job itself creating yet another container
<pinchartl>
(not using docker/podman for the latter, it's the Chrome OS SDK, they isolate the SDK from the host manually)
vkareh has quit [Quit: WeeChat 4.1.2]
i509vcb has joined #freedesktop
<bentiss>
pinchartl: not too bad :)
<bentiss>
pinchartl: you should probably add a qemu in the middle, just to be sure ;)
<MrCooper>
the good news is that the jobs should finish in no time, due to how time works differently in lower levels of inception
<bentiss>
MrCooper: but maybe it can finish before it started, which would be even better!
<MrCooper>
hehe
<karolherbst>
bentiss: typo in the commit message
<MrCooper>
(this makes me want to watch the movie again)
<bentiss>
karolherbst: oh, yes, well... who cares ;)
damian has joined #freedesktop
<daniels>
pinchartl: wait, are you running gitlab-runner on the actual end device itself?
<pinchartl>
bentiss: you mentioned script kiddies escaping containers. is that something I should be worried about, and consider that, in the case the CI jobs can be specified by users who are not 150% trusted, I should consider the runner machine as being compromised by design and have an infrastructure that makes it easy to redeploy on a regular basis ?
todi has quit []
pzanoni` is now known as pzanoni
<pinchartl>
if I register my runners only for projects I control, do I need an equivalent of the runner-gating.sh pre-script ?
<pinchartl>
daniels: no, not on the end device
<pinchartl>
it could be tempting though, that would remove a few layers :-)
<daniels>
noooonnnnn :)
<pinchartl>
this runner is meant for the sole purpose of compiling libcamera for chrome os. this currently requires a 231GB image
<pinchartl>
so I want to make sure this compilation job runs on a machine that has the image available locally
<pinchartl>
in the future we'll investigate how to reduce the size of that image, and if we can get it down to a much more reasonable size, we'll move that compilation step back to shared runners
todi has joined #freedesktop
alanc has quit [Remote host closed the connection]
alanc has joined #freedesktop
todi has quit []
<daniels>
pinchartl: ahhhh yeah, gotcha
<pinchartl>
so, given that this is supposed to be temporary, and that the only job running on that runner is well-known, I could use the shell executor. but I suppose it's "interesting" to learn about the docker executor
todi has joined #freedesktop
todi has quit []
<pinchartl>
what's the reason why the containers for the docker executor run in priviledged mode ?
Kayden has quit [Quit: -> JF]
<alatiera>
there are a lot of thins you need cap_sys_admin for
<alatiera>
it's going to be a big effort to make our existing CI setups run unprivileged
Kayden has joined #freedesktop
<mupuf>
pinchartl: mostly building new containers with buildah
<alatiera>
that works unprivileged
<mupuf>
alatiera: not rootfull
<alatiera>
it's things like kvm that are trickier
* alatiera
has words about rootful containers
<mupuf>
Well, -v /dev/kvm works
<alatiera>
also I think rootfull too should work
<mupuf>
alatiera: i tried, it didn't. Still need to move to a different user... But we may end up reimplementing gitlab-runner anyway for unrelated reasons... So the MR just sits in ci-tron, un-merged
<alatiera>
mupuf: it should work if you give it the right seccomp profile
<alatiera>
buildah has one in the repo
<alatiera>
(i think)
<mupuf>
Oh, interesting!
<alatiera>
need to pass that to docker/podman through the runner config
<alatiera>
with docker it was like very annoying, you needed to serialize the json into the toml
<mupuf>
We ended up restricting who can use these runners anyway
<alatiera>
but with podman as the daemon, you can pass it a file in the seccomp config in gitlab-runenr
<alatiera>
good news is that the shared runner with proccess isolation works again now!
<jenatali>
\o/
<alatiera>
I will make an MR in mesa to enable the runners again
<jenatali>
Thank you!
* alatiera
wants the two weeks of his life back
<pinchartl>
how does gitlab pick the tag for the gitlab-runner-helper image ? I'm trying to run gitlab-runner with a pull policy of "never" as my runner is meant to run a specific container, which should be available locally
<pinchartl>
I get
<pinchartl>
WARNING: Failed to pull image with policy "never": Error response from daemon: failed to find image registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-f5da3c5a: registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-f5da3c5a: No such image (manager.go:163:0s)
<pinchartl>
if I pull that manually the next run finds the image fine
<pinchartl>
but how can I know what tag will be used, to provision the machine accordingly ?
<alatiera>
you can either define helper_image in the toml to specifically configure it
<alatiera>
or you can figure out what scheme gitlab-runner is using
<pinchartl>
ah good point, I'll pin the helper image in the config, and pick the version that matches the runner
<pinchartl>
thanks
sima is now known as Guest10372
sima has joined #freedesktop
Guest10372 has quit [Ping timeout: 480 seconds]
sima has quit [Ping timeout: 480 seconds]
privacy has quit [Quit: Leaving]
mvlad has quit [Remote host closed the connection]
damian has quit []
lsd|2 has joined #freedesktop
lsd|2 has quit []
lsd|2 has joined #freedesktop
todi has joined #freedesktop
<daniels>
alatiera: thankyou so so much
<bwidawsk>
whot, bentiss: I deleted old pipelines, but if you're interested in a link to the error, I can get one