daniels changed the topic of #freedesktop to: https://www.freedesktop.org infrastructure and online services || for questions about freedesktop.org projects, please see each project's contact || for discussions about specifications, please use https://gitlab.freedesktop.org/xdg or xdg@lists.freedesktop.org
DodoGTA has quit [Remote host closed the connection]
whot1 has quit []
whot has joined #freedesktop
bbhtt has joined #freedesktop
AbleBacon has joined #freedesktop
mvlad has quit [Remote host closed the connection]
i509vcb has joined #freedesktop
Kayden has quit [Remote host closed the connection]
<hakzsam>
it looks like Marge isn't doing anything right now?
damian has joined #freedesktop
<dj-death>
yeah I think not much code is going to land today ;)
<bentiss>
I just kicked marge back once again
<MrCooper>
bentiss: Marge can't do anything about no runners with "mesa-ci-x86-64-lava-asus-C523NA-A20057-coral" tag being online
<bentiss>
MrCooper: yeah, but eric_engestrom was complaining that marge's cache was not good
<bentiss>
FWIW, reworking marge deployment at the moment to rely on the git main branch of fdo/marge-bot, and a push would update it automatically
GNUmoon has quit [Remote host closed the connection]
GNUmoon has joined #freedesktop
luna has joined #freedesktop
luna has left #freedesktop [#freedesktop]
blatant has joined #freedesktop
AbleBacon has quit [Remote host closed the connection]
<eric_engestrom>
nice
<eric_engestrom>
did you do other changes to it recently?
<eric_engestrom>
I'm not sure why it got broken yesterday and hasn't been able to merge anything since
<eric_engestrom>
bentiss: ^
<kode54>
cool
<kode54>
git did automatic garbage collection on my local copy of the mesa repo, breaking it
<kode54>
and now recloning from scratch is getting reset by peer at random completion points
<mupuf>
kode54: I see you are having a great day!
<daniels>
weird, gitlab-runner on our side just ... stopped doing anything
<daniels>
it's back now
<daniels>
eric_engestrom: we haven't made any changes to marge, no - I looked at it last night and it was certainly fetching fdo/main and rebasing on to it, so I have nfi why it would be confused about it
<daniels>
the only thing I can think of is to be adding a bunch more logging to it to try to figure out why it's doing what it's doing
<daniels>
but it was certainly fetching both repos and rebasing, with no reported errors, soooo ... \_o_/
<eric_engestrom>
let's see if marge manages to merge it at the end of the pipeline
florida has joined #freedesktop
<mupuf>
daniels: I mean, here, everything is white :p Heck, even the inside of the car windows are frozen
florida has quit []
<dj-death>
mupuf: river is starting to freeze here
florida has joined #freedesktop
<mupuf>
dj-death: cool, you'll be able to skate on them soon!
<mupuf>
No idea how the baltic is doing right now. I assume it will take a little longer to freeze over
<mupuf>
all I know is that my heat pump's condensation exhaust was all frozen and overflowed despite the use of heating wires during the defrost cycle. I guess I have more tuning to do :p
suporte has joined #freedesktop
<daniels>
mupuf: :D
<eric_engestrom>
bentiss, daniels: marge merged an MR \o/
<eric_engestrom>
I'm telling #dri-devel they can start marging again, we'll see if it holds
<daniels>
eric_engestrom: let’s see if we can manage two
<daniels>
if not I think we’d have to look at making it nuke the repo between merges for now
<pinchartl>
but that doesn't seem to exist anymore
<daniels>
pinchartl: yeah so I think it’s mainly Weston you want to look at for that, but probably use virtme-ng rather than our form of the old dead one
<pinchartl>
OK
<daniels>
whot is the one who’d know about the templates
<pinchartl>
so it's "just" a matter of docs being outdated ?
<pinchartl>
I would send a Reviewed-by tag if it was on a mailing list
<eric_engestrom>
You can post it as a comment, or just "looks good to me" or equivalent
<eric_engestrom>
I haven't interacted with the ci-templates repo much so I'm not sure what the custom is there
<eric_engestrom>
bentiss, daniels: second MR merged by Marge
<eric_engestrom>
I think we can consider the issue resolved :)
<eric_engestrom>
I've let #dri-devel know
bmodem has quit [Ping timeout: 480 seconds]
jmondi has joined #freedesktop
lsd|2 has joined #freedesktop
<bentiss>
eric_engestrom: \o/
<bentiss>
(sorry I was out for a long lunch break)
<bentiss>
pinchartl: re multi-arch: yes, the container that is capable of building arch specific images is multi-arch capable, so all you need is to run the same image on the arch target and you are good
<pinchartl>
bentiss: ok... no idea how that works, but I'll see what happens :-)
<bentiss>
pinchartl: we basically build the image twice, one for each arch, then merge them together in a single tag, mentioning which arch which image is from. And then docker/podman deals with it :)
<pinchartl>
I need to run tests on a custom kernel, so I'll neet to use virtme-ng anyway
<pinchartl>
I saw b2c and I was told it was for something else
<bentiss>
pinchartl: it works really well ;)
<pinchartl>
should I do that instead of manually running virtme-ng ?
<bentiss>
even for qemu
<bentiss>
they are slightly diferent
<bentiss>
virtme-ng IIRC uses the curent dir as the "root" of your booted system, when boot2container (through vm2c.py or /app/boot2container in that image) uses a generic container as a base root image
<pinchartl>
virtme-ng seems better for my use case then
<bentiss>
I find boot2container easier to use, but that's becasue I have been converted to containers for a lot of things
<pinchartl>
as I want to build in the container, then run a VM inside the container to run tests, using the build artifacts
<bentiss>
both would work just fine
* bentiss
does just that for HID ci tests
<pinchartl>
is there a good example of how to use b2c for this kind of use case, where tests have to run on a custom kernel, but with the build artifacts available ?
<bentiss>
the link I posted just above :)
<bentiss>
b2c would mount your current working dir in the VM, so you can exchange files easily
<pinchartl>
I'm looking at your link, how do I specify the kernel image ?
<bentiss>
I must confess, for hid, I rely more or less on my vmtest.sh script at tools/testing/selftests/hid/vmtest.sh
<bentiss>
or /app/boot2container --kernel bzImage_that_I_just_built
<pinchartl>
another stupid question, if I want to build the kernel image once and store it (as I'm testing userspace code that needs particular kernel features, not testing the kernel itself), where should I store it ?
<pinchartl>
I could upload it to random.server.I.own.com
<bentiss>
in an artifact?
<pinchartl>
but I suppose it's best to store it on fd.o
<bentiss>
or at s3.freedesktop.org if you want
<bentiss>
how long do you need the artifact to be stored?
<pinchartl>
what's the best practice, building the kernel manually and uploading it somewhere somehow ? or having a job to build it ?
<bentiss>
s3.fd.o now has a retention policy for these artifacts of 30 days
<pinchartl>
I expect the kernel image to remain the same forever
<bentiss>
use a package then
<pinchartl>
(forever probably being a couple of years)
<pinchartl>
what's a package ? :-)
<bentiss>
there is a package registry you can use to send/store files
<bentiss>
that's used to push releases, these kind of stuff
<bentiss>
let me find a pipeline that's close enough to your needs
<pinchartl>
my brain is saturating with lots of new information
<bentiss>
basically I have 2 containers: one for building, one for testing, in the "build" stage I build everything, and one of the jobs is "rebuild kernel" that looks for a generic package, and if it's not there rebuilds it, then the "pytest" job runs b2c on the test image with the kernel I've built
<bentiss>
so close enough to your use case
<pinchartl>
building the kernel as a job sounds very good
<pinchartl>
better than doing it manually
<bentiss>
yeah, I just have to bump the KERNEL_VERSION variable, push and it builds a new one ;)
<pinchartl>
jmondi: once I get this working I'll let you try the Chrome OS build. that one will be fun too
<pinchartl>
a naive try to create a container with the CrOS SDK gives a ~80GB image
<bentiss>
heh
<pinchartl>
which I doubt the runners would be happy to pull for every run
<bentiss>
pinchartl: they are now in the same datacenter than the rest of gitlab, so... meh?
<pinchartl>
so we should try and run when the admins start chasing us ? :-)
<bentiss>
unless they are using runners from Google, which in case would require a proxy
<pinchartl>
no, standard runners
<pinchartl>
except for running the test job
<bentiss>
well then...
<pinchartl>
that will be a custom runner we will self host
<pinchartl>
but the artifacts will be about ~10MB in size so that should be OK
<bentiss>
well, that is going to be an issue (though we don't actively track egress now)
<bentiss>
for the 80GB I mean, 10 MB is peanuts
<pinchartl>
the 80GB image is for building only
<jmondi>
do I read this right or maybe I'm overconcerned with the ChromeOS SDK container size ?
<jmondi>
seems like I haven't read it right :)
<jmondi>
bentiss: hello, I'm working on libcamera CI as well
<pinchartl>
the DUTs will be preflashed, only one small artifact will need to be pushed
<jmondi>
so much fun -.-
<bentiss>
jmondi: \o
<jmondi>
hello there
<bentiss>
so you folks are working on the linux-media group that has been created this summer?
<bentiss>
or is libcamera in another group?
<pinchartl>
it's a separate group
<pinchartl>
we have a camera group
DodoGTA has quit [Quit: DodoGTA]
<pinchartl>
and libcamera is one project in that group
<bentiss>
I see fancy :)
DodoGTA has joined #freedesktop
<pinchartl>
during these experiments I'm currently storing .gitlab-ci.yml in a separate tree to avoid touching the main libcamera tree for now, and I reference that in the libcamera gitlab configuration. will the contents of .gitlab-ci/ from the tree that provides .gitlab-ci.yml be available in the runner too ?
<bentiss>
pinchartl: yes, there is a setting for that
<bentiss>
you can use a relative file to the project, a file in another project on the same instance, or an external URL
<pinchartl>
but it only mentiones the configuration file
<bentiss>
the pipeline is parsed/created by gitlab itself, and then the runners pick the instructions from gitlab
<pinchartl>
if, in my .gitlab-ci.yml, I have a script that runs a shell script from .gitlab-ci/ (that's very common, from what I've seen), will the .gitlab-ci/ directory of the tree that hosts .gitlab-ci.yml be available ?
<bentiss>
nope
<pinchartl>
OK
<bentiss>
you only have the clone of the target repo if you set the GIT_STRATEGY
<bentiss>
then you can clone the external repo and have the script
blatant has quit [Ping timeout: 480 seconds]
<pinchartl>
so if I want to use CI scripts inside the container, they have to be in the tree that the pipeline is running on, or I have to pull them manually in somehow
<pinchartl>
makes sense
<bentiss>
yeah I'd advise to git pull the scripts first
<pinchartl>
(this is the point where I expect someone to shout "store your CI config in the same tree" :-))
<bentiss>
well, it depends :)
<bentiss>
I'm rebuilding my upstream CI, and having the scripts located in a separate repo allows for retrying the job without rebuilding/testing everything
<bentiss>
but having the tests in the tree allows to keep them in sync with the various fixes
<pinchartl>
yes, that I agree with
<pinchartl>
it's easier to keep something in sync with itself
<bentiss>
definitely :)
<bentiss>
right now I have the tooling in a separate repo, and the tests in the tree
<bentiss>
though I'm also trying to merge as much as I can in the tree for that sync reason
<bentiss>
daniels: it's in case it's not, if I hit merge and redeploy it, I'm afraid something might break
<bentiss>
(though it would probably be safer to do on a morning, not a late evening)
rgallaispou has left #freedesktop [#freedesktop]
<daniels>
bentiss: yep, exactly the same
ximion has joined #freedesktop
<pinchartl>
continuing a bit with random newbie questions (sorry), I see that lots of projects use explicit dependencies in their .gitlab-ci.yml (i.e. "needs:") to depend on jobs that are listed in a previous stage. as far as I understand, the dependency will already be fullfilled due to ordering of stages. what's the reason for the explicit dependency ?
anholt has joined #freedesktop
<bentiss>
daniels: thanks! I'll change the deployment tomorrow then
<bentiss>
pinchartl: 2 reasons: ensure you get the artifacts of the "needs" jobs IIRC and quicker pipelines as if you describe explicitely the dependencies, your stages can be run in parallel, i.e. as soon as a job has all its dependencies ready, it can start even if all of the previous stage jobs haven't finished
<pinchartl>
ah, I didn't know that "needs" overrides the stage ordering
<pinchartl>
thanks
<MrCooper>
and as a bonus you get a nice dependency graph on the pipeline page
<pinchartl>
if all jobs have explicit dependencies, can the stages be omitted, or are they still mandatory ?
<pinchartl>
looks like stages are still mandatory
<daniels>
bentiss: thankyou so much <3
<MrCooper>
pinchartl: in principle you can put all jobs in the same stage then, might look a bit odd on the pipeline page though :)
blatant has quit [Ping timeout: 480 seconds]
privacy has joined #freedesktop
alanc has quit [Remote host closed the connection]
alanc has joined #freedesktop
suporte has quit []
florida has joined #freedesktop
florida has quit []
scrumplex has quit [Quit: Quassel - Signing Off]
scrumplex has joined #freedesktop
blatant has joined #freedesktop
mvlad has quit [Remote host closed the connection]
lsd|2 has quit [Read error: No route to host]
lsd|2 has joined #freedesktop
lsd|2 has quit []
blatant has quit [Ping timeout: 480 seconds]
lsd|2 has joined #freedesktop
suporte has joined #freedesktop
AbleBacon has joined #freedesktop
sima has quit [Ping timeout: 480 seconds]
appusony__ has quit [Read error: Connection reset by peer]
appusony__ has joined #freedesktop
tarun_ has quit [Read error: Connection reset by peer]
tchar has quit [Read error: Connection reset by peer]
Ford_Prefect has quit [Read error: Connection reset by peer]
suporte has quit []
tchar has joined #freedesktop
Ford_Prefect has joined #freedesktop
SanchayanMaity_ has quit [Read error: Network is unreachable]
jstultz has quit [Read error: Network is unreachable]
ogabbay has quit [Read error: Connection reset by peer]
pendingchaos has quit [Read error: Connection reset by peer]
narmstrong has quit [Write error: connection closed]
Leftmost____ has quit [Read error: Connection reset by peer]
seanpaul_ has quit [Read error: Network is unreachable]
Leftmost____ has joined #freedesktop
pendingchaos has joined #freedesktop
jstultz has joined #freedesktop
narmstrong has joined #freedesktop
SanchayanMaity has joined #freedesktop
seanpaul has joined #freedesktop
ogabbay has joined #freedesktop
tarun_ has joined #freedesktop
elibrokeit has quit [Remote host closed the connection]
raghavgururajan has quit [Write error: connection closed]
rpigott has quit [Write error: connection closed]
cat has quit [Write error: connection closed]
ifreund has quit [Write error: connection closed]
ajhalili2006 has quit [Write error: connection closed]
bnieuwenhuizen has quit [Read error: Connection reset by peer]
bnieuwenhuizen has joined #freedesktop
blatant has joined #freedesktop
blatant has quit [Quit: WeeChat 4.1.1]
vkareh has joined #freedesktop
<whot>
pinchartl: you still neeed stages for sequential ordering of any dependency chain, but the difference with needs: is that each chain runs individually instead of waiting for every other chain to finish the current stage