NoGuest17 has quit [Remote host closed the connection]
NoGuest17 has joined #freedesktop
yk has joined #freedesktop
chomwitt has joined #freedesktop
i-garrison has quit []
<mupuf>
bentiss: nice job with the ci-template for b2c. I was wondering, should I move boot2container to /gfx-ci/boot2container before we start referencing it more?
ximion has joined #freedesktop
<mupuf>
oh, and BTW, encrypted local volumes incoming in b2c. Will allow storing traces locally without making them accessible to all jobs in the farm
<bentiss>
mupuf: whatever you want (regarding moving b2c)
<mupuf>
ack
<bentiss>
and nice regarding encrypted local volumes
* bentiss
is fighting against regressions introduced by cbuild
<mupuf>
good luck :)
<bentiss>
mupuf: BTW, thanks for the heavy groundwork on b2c. It is now much simpler I think to use qemu than before thanksto that (once I figure out all the ci-templates regressions and teweaks to do)
<bentiss>
the nice thing is we can build the image regularly, and then use the same image either with or without qemu :)
Haaninjo has joined #freedesktop
<mupuf>
bentiss: good to hear! Using the same containers everywhere is indeed the goal of the project
<mupuf>
along with making developement and reproduction of issues much much easier, as things can be run locally :)
<bentiss>
mupuf: I was wondering if you don't want to mount by default /dev in podman from the initrd
<mupuf>
isn't it what --privileged does?
<bentiss>
nope, privileged does a copy of the state, but doesn't update it after
<bentiss>
while I was surprised to see that using "-v /dev:/dev" was doing the expected thing
<bentiss>
(expected for me)
<mupuf>
oh, that is indeed a funny difference!
<bentiss>
actually.... maybe I should re-test, but I thing that's the case
<bentiss>
I think
<mupuf>
actually, I am running containers that do detect hotplugs of devices... so it seems like privileged is doing the right thing
<mupuf>
but... I wouldn't bet my life on it D:
<bentiss>
hmm, OK, then maybe I checked only *without* ystemd-udevd running inside the pod
<mupuf>
maybe... I opted out of using udevd in boot2container to save space
<bentiss>
itś best not to use it
<bentiss>
but my test suite needs it *in the container*
<bentiss>
and I realized last week I can just start this process in the background in the container, and it works
<bentiss>
which goes along the lines of "initrd does nothing but starts podman"
<mupuf>
initrd does the bare minimum to set up the hardware and abstract the different machines
<bentiss>
yep, and that's good enough :)
<bentiss>
mupuf: also, FTR, I fought a little bit with how to use and store local data
<mupuf>
yeah, I am reworking that part right now
<bentiss>
I think I have a good solution now: I dump everything in the current workdir in the cache disk, then start the container, then dump the other way around
<mupuf>
yeah, I saw that...
<mupuf>
not sure what to do better in b2c, but with the overall valve infra, I would like to provide two storages: one for the job, one persistent across jobs
<bentiss>
are you trying to make that experience smoother?
<bentiss>
oh, ok
<bentiss>
what's the point of having 2 disks?
<bentiss>
'across jobs' like different CI pipeline runs?
<mupuf>
2 disks? There should be only one disk, with one partition
<mupuf>
and yes, I mean across CI pipeline runs
<bentiss>
not sure the across CI pipeline runs should be handled by b2c
<bentiss>
seems like it's the caller of b2c that needs to set this up
<bentiss>
but having am official way of adding data to the qcow2 image would be a plus
<mupuf>
if you are thinking from a qemu PoV, then yeah, it is a little weird to do it like that
<mupuf>
but for real machines, it makes sense, as you definitely can have big big files you don't want to re-download every time you run
<mupuf>
What I am working on right now would work for you... but not sure you want to depend on minio :s
<bentiss>
mupuf: FWIW we already have a secure minio in place for fdo
* mupuf
is working on having his executor set up a minio bucket, extract a user-provided archive there, then tell boot2container to populate the job volume with the data
<mupuf>
and then, having b2c upload the result back to minio when the container is done executing
<bentiss>
hmm, can we opt out???? :)
<mupuf>
of course :D
<mupuf>
everything is opt-in
<bentiss>
the thing I am not comfortbale with here is the 'having his executor set up a minio bucket'
<bentiss>
you need way too big privileges for that
<bentiss>
but I would need to see the code I think
<mupuf>
true. it can be a folder just as well :)
<mupuf>
as long as we can write a policy just for the folder
<bentiss>
mupuf: have you looked at our minio-opa deployment/usage?
i-garrison has joined #freedesktop
<bentiss>
just in case you haven't, we got 2 pieces of interest there:
<bentiss>
first is the use of the JWT token provided by gitlab: every job gets one which ensures the user of minio is from our CI
<mupuf>
but anyway, from b2c's PoV, the point is that there would be something like b2c.job_data="http://LOGIN:PASSWORD:10.42.0.1:9000/PATH", and b2c would download everything from minio before running the container, then run it with the cache mounted somewhere, then upload the new state after
<mupuf>
tanty described to me the process, and I looked more into it after he left, but I had not found the OPA project
<bentiss>
second we rely on OPA to fine tune the permissions and accesses to the various folders/buckets/objects
<mupuf>
I looked at the minio doc, and the policy seemed good-enough for me
<mupuf>
I could create users per job, make them expire based on the job timeout, and add a source IP address requirement
<bentiss>
the nice thing is you don't have to provide passwords or token, gitlab does the auth for you
<bentiss>
nah, you want STS for that (short token service)
<mupuf>
Hmm... I need something that works for github and custom jobs too...
<bentiss>
OPA is actually orthogonal to jwt
<bentiss>
the policy could be used without jwt
<mupuf>
the STS didn't feel as good though: minimum lifetime of 15 minutes. No limits on IP
<bentiss>
*however* the minio devs decided to deprecate this and all the doc is gone, just FYI
<bentiss>
anyway, sorry, got to go have lunch, bbl
<mupuf>
have a good one!
<mupuf>
I'll dig more into STS and OPA before getting into b2c changes
blue__penquin has quit [Quit: Connection closed for inactivity]
blue__penquin has joined #freedesktop
ximion has quit []
<bentiss>
mupuf: finally managed to get the regressions fixed, and I just tested without `-v /dev:/dev`: I confirm that this is required for me at least, without it, new devices created by the container are not exposed
<ndufresne>
gitlab returning 502, any info ?
<ndufresne>
ah, seems back
<bentiss>
yeah, 502 are common and usually transient
<bentiss>
means that one webservice pod is not available, but we should be running 8 in parallel, so it just needs time to realize one is stuck
jstein has joined #freedesktop
blue__penquin has quit []
veganaiZe has joined #freedesktop
mixfix41_ has joined #freedesktop
mixfix41 has quit [Ping timeout: 480 seconds]
Kayden has quit [Quit: Leaving]
ximion has joined #freedesktop
ngcortes has joined #freedesktop
tobtoht has quit [Remote host closed the connection]
tobtoht has joined #freedesktop
Kayden has joined #freedesktop
chomwitt has quit [Remote host closed the connection]
chomwitt has joined #freedesktop
laguneucl has joined #freedesktop
chomwitt has quit [Read error: Connection reset by peer]
danvet has quit [Ping timeout: 480 seconds]
laguneucl has quit [Remote host closed the connection]