daniels changed the topic of #freedesktop to: https://www.freedesktop.org infrastructure and online services || for questions about freedesktop.org projects, please see each project's contact || for discussions about specifications, please use https://gitlab.freedesktop.org/xdg or xdg@lists.freedesktop.org
itaipu has joined #freedesktop
DodoGTA has quit [Remote host closed the connection]
Haaninjo has quit [Quit: Ex-Chat]
alatiera has quit [Quit: Connection closed for inactivity]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold_ has quit [Remote host closed the connection]
jarthur has joined #freedesktop
mripard has quit [Remote host closed the connection]
mripard has joined #freedesktop
<daniels>
ndufresne: file an issue please and I’ll set up the ACLs
jarthur has quit [Ping timeout: 480 seconds]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold has joined #freedesktop
Leopold has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold has joined #freedesktop
Leopold has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
<ndufresne>
Ok
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold has joined #freedesktop
Leopold has quit [Remote host closed the connection]
krushia has quit [Ping timeout: 480 seconds]
ximion has quit [Quit: Detached from the Matrix]
Leopold has joined #freedesktop
Leopold has quit [Remote host closed the connection]
AbleBacon has quit [Read error: Connection reset by peer]
Leopold_ has joined #freedesktop
bmodem has joined #freedesktop
bmodem has quit []
Leopold_ has quit [Remote host closed the connection]
sima has joined #freedesktop
<mupuf>
ndufresne: If you must have to use rootfses, I would suggest creating them at run time by pulling the container you already have, then extracting it.
<mupuf>
Add your kernel/initrd there and boot
<mupuf>
This way, no need to maintain two build scripts
<mupuf>
And reproduction of the test environment by devs will be easier
<mupuf>
If you must use*
Leopold has joined #freedesktop
bmodem has joined #freedesktop
Leopold has quit [Remote host closed the connection]
mrpops2ko has quit [Ping timeout: 480 seconds]
Leopold_ has joined #freedesktop
mrpops2ko has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
mrpops2ko has quit [Ping timeout: 480 seconds]
Leopold_ has joined #freedesktop
Haaninjo has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
peelz has quit [Read error: Connection reset by peer]
Leopold_ has quit [Remote host closed the connection]
Leopold has joined #freedesktop
Leopold has quit [Remote host closed the connection]
Haaninjo has quit [Quit: Ex-Chat]
<bentiss>
ndufresne, pinchartl: just in case you need a little bit of background: s3.freedesktop.org is self hosted (provided by ceph) and allows jobs to put data in there that can be either publicly accessible, or only accessible to a restricted set of persons. It has nothing related to AWS, except for the API. It was also developped to prevent traffic from GCS when we had stuff there,
<bentiss>
but now using an artifact or s3.fd.o is roughly the same.
<pinchartl>
bentiss: ah, interesting
<pinchartl>
thanks
<bentiss>
so any job store publicly available data on s3.fd.o with the CI_JOB_JWT* tokens. We need to update to the id_token before gitlab 17.0 IIRC
<pinchartl>
is there a standard open-source S3 API implementation ? or was it home-grown ?
<bentiss>
pinchartl: it's using ceph rados GW
<bentiss>
so opensource
<pinchartl>
ok, so aws is still evil, but not s3.fd.o. good :-)
<bentiss>
but any data put in the "artifacts" bucket of s3.fd.o has a hard ACL cleanup policy of a couple of weeks (maybe more, can't remember)
<bentiss>
so it's useful to put there data that are used outside of the gitlab world.
<pinchartl>
is it better to upload, e.g. a rootfs/container image used in a lava farm, to s3.fd.o, or to the gitlab.fd.o package/container registry ?
<bentiss>
we have other buckets that have longer life cycle (i.e. never cleans up), but they are specific (mesa-rootfs, mesa-tracie-*, and tarball of git sources)
<bentiss>
nowadays pick whatever is simpler to use for you
rgallaispou has joined #freedesktop
<pinchartl>
ideally with a cache in front of the board farm, right ?
<bentiss>
though we might say that the container has the problem of having the manifest replicated on the various clones, and so we might never be able to clear the blobs
<bentiss>
pinchartl: yeah, if the farm is not located at equinix :)
<pinchartl>
:-)
<pinchartl>
jmondi: ^^
<pinchartl>
something for your todo list I'm afraid
<pinchartl>
(adding a pass-through registry cache)
<jmondi>
for what sorry ?
<pinchartl>
to pull container images for the DUTs
<pinchartl>
I think we should build those images on gitlab.fd.o
<pinchartl>
and so they should be cached in the board farm
<pinchartl>
instead of being pulled for every boot
<bentiss>
pinchartl: basically, s3.fd.o allows to have stable rootfs URLs, and you can even request them to be ACL guarded to a few dedicated people starting the CI, but you'll probably need a proxy cache in front of your farm, and registry.fd.o requires you to have a registry proxy cache, but you can not have fine grain ACL
<bentiss>
pinchartl: and if you plan on using b2c, then no brainer, use the registry :)
<pinchartl>
that sounds like the easiest option, yes
<pinchartl>
hmmmm...
<pinchartl>
I'll have to think about ACLs too
<bentiss>
also, if you need to put private test data (mesa has traces of proprietary games that can not be reditributed), we can set up a bucket where only the CI bot and a handful of people can access
<bentiss>
not sure if you'll need it, but this is possible to do
<pinchartl>
no, that should be fine
<pinchartl>
speaking of CI
<pinchartl>
any best practice to implement regression tracking ? we have CI jobs that are supposed to pass fully, those are easy. we have other jobs that run a set of unit tests, and some of them are known to fail. we would like to fail the job only when there's a regression (as in a test that previously passed and now fails)
<bentiss>
no best practice, but you might simply drop a json file of the expected results of your test in tree directly, and you compare with it. The nice thing is that when you fix a test, you can checkout the file as well in the commit
<pinchartl>
that's one option indeed
<pinchartl>
although at the moment we store the CI configuration in a separate tree
<bentiss>
in my CI, for sparse, I go back to the previous commit before the branch is branched out, run the tests, store the sparse result, then compare with the new commit run. This works because sparse doesn't rebuild everything, but you need to clear up a bit the files/diffs to only get what's interesting
<pinchartl>
I'm thinking of doing the same when integrating an ABI checker in the CI
<pinchartl>
as it also needs to operate on a diff
<pinchartl>
the tricky part is to find the right base commit
<pinchartl>
it's easy for merge requests, less so when pushing to a branch
<bentiss>
basically, the problem you'll have is that you need to know against *what* to compare to. Having the file in-tree solves a lot of problems. But storing a "result" files outside will always be problematic, and so manually walking up the tree might be an option
<daniels>
yeah, external expectations are definitely pain
<daniels>
pinchartl: if I were you I'd look at deqp-runner which now supports fluster in an MR
* pinchartl
wonders what deqp is
<daniels>
it has a few nice things: firstly you can specify multiple test suites to run with their own options etc, secondly it has xfail/flake/skip files which are really useful, thirdly it has flake detection via re-running, and lastly it has a stable subtest sort order so it always does the same runs
<mupuf>
In ci-tron, we use minio as a way to pass data around between the DUTs and the client
<daniels>
krastevm: see #dri-devel, tomeu is looking at that
<mupuf>
we chose to use S3 because it is a standard based on HTTP, it is file-oriented, and unlike NFS it asynchronous
* mupuf
would dread to see his jobs stuck or influenced by what the other DUTs or what the gateway is doing
<mupuf>
oh, and S3 is also spoken by other services than AWS: backblaze or digital ocean have (partial) support for it too
<mupuf>
in ci-tron, we use minio because it has good access control support (by IP address, and through throwaway credentials we create on the fly for every principal related to the job)
Leopold_ has quit [Remote host closed the connection]
<krastevm>
tomeu, daniels: thank you, it's not really pressing to me as I have all artifacts needed for my purposes locally, just meant to stir general awareness ; ]
ofourdan has quit [Ping timeout: 480 seconds]
bmodem has quit [Ping timeout: 480 seconds]
<krastevm>
lava-ci Q: in a device dictionary extending qemu I try to augment self.job.device["actions"]["test"]["methods"]["docker"] with some extra args, namely "global_options" and "options" arrays, but lava-docker-test-shell completely ignores those, as if https://github.com/Linaro/lava/blob/master/lava_dispatcher/actions/test/docker.py#L188 never executes. I'm running lava 2023.10. Has somebody actually managed to augment docker-method test options
<krastevm>
successfully through their device dictionary?
ximion has joined #freedesktop
ximion has quit [Quit: Detached from the Matrix]
lsd|2 has joined #freedesktop
Reiver has joined #freedesktop
DodoGTA has quit [Quit: DodoGTA]
DodoGTA has joined #freedesktop
<Reiver>
The mesa-commit mailing list appears to have stopped sending two days ago. No new emails received or in list archive. I'd presume gitlab is the problem.
vkareh has joined #freedesktop
tsa has quit [Quit: leaving]
mripard has quit [Remote host closed the connection]
privacy has quit [Quit: Leaving]
mripard has joined #freedesktop
Leopold_ has joined #freedesktop
i-garrison has quit []
i-garrison has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
itaipu has quit [Remote host closed the connection]
itaipu has joined #freedesktop
todi1 has quit []
i-garrison has quit []
jarthur has joined #freedesktop
i-garrison has joined #freedesktop
AbleBacon has joined #freedesktop
tzimmermann has quit [Quit: Leaving]
psykose has joined #freedesktop
<hakzsam>
I'm the only with gmail+fdo who has issues with emails? like a lot of them aren't delivered ?
<pendingchaos>
I also use gmail and am missing a lot
<hakzsam>
ah
<hakzsam>
did the issue start recently? like couple of weeks ago?
<pendingchaos>
I think there might have been a discussion about it several days ago? I don't really remember the contents of it though
<pendingchaos>
a couple weeks ago sounds roughly right, but I could be completely wrong about that
<hakzsam>
I think it started like mid december for me
<hakzsam>
good to know that I'm not the only one, at least :)
<pendingchaos>
it definitively started after mid-december for me
<hakzsam>
okay
<pendingchaos>
it's pretty annoying, I keep on finding MRs that I missed completely
Leopold_ has joined #freedesktop
<hakzsam>
it's definitely annoying...
<Reiver>
And the mesa-commit list not having received/sent any emails for two days is maybe coincidental to the apache setup change...and still a maybe a gitlab issue.
<MrCooper>
there have been issues with gmail list subscriptions for a long time, it merely got worse recently
<hakzsam>
I never had any issues with ML and gmail in the past, only recently FWIW
<MrCooper>
you were lucky for a long time then :)
alarumbe has quit [Remote host closed the connection]
alarumbe has joined #freedesktop
sentriz has quit [Read error: Connection reset by peer]
alpernebbi has quit [Ping timeout: 480 seconds]
alpernebbi has joined #freedesktop
sentriz has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
<daniels>
Reiver: yeah sorry, I didn't get around to sending a mail about these changes yet
<daniels>
it was deliberate - I disabled the commit mailouts since they're giant identical mails which tend to mostly not get read, which tends to anger spam filters
<daniels>
so just trying to improve our sender reputation
<daniels>
you can get Atom/etc feeds of the commits instead
<Reiver>
Ok, thanks for the update. I find emails easier to keep track of commits and bookmark a list of significant commits before doing a rebuild of mesa.
<daniels>
that's reasonable. sorry to break your workflow :(
<daniels>
there are some atom -> email gateways I believe
ximion has joined #freedesktop
<hakzsam>
ah, mesa-commits will be dead forever? I also use that workflow :)
<hakzsam>
the ML seems much better because it has code but not the atom :/
<pinchartl>
daniels: how about storing the notifications in a public-inbox archive ? people could then query it with lei
___nick___ has joined #freedesktop
Haaninjo has joined #freedesktop
Leopold_ has joined #freedesktop
alpernebbi has quit [Ping timeout: 480 seconds]
___nick___ has quit []
alpernebbi has joined #freedesktop
alanc has quit [Remote host closed the connection]
alanc has joined #freedesktop
___nick___ has joined #freedesktop
___nick___ has quit []
Reiver has left #freedesktop [#freedesktop]
Leopold_ has quit [Remote host closed the connection]
vkareh has quit [Quit: WeeChat 4.1.2]
sima has quit [Ping timeout: 480 seconds]
keithp is now known as Guest14572
keithp has joined #freedesktop
Guest14572 has quit [Ping timeout: 480 seconds]
alpernebbi has quit [Ping timeout: 480 seconds]
alpernebbi has joined #freedesktop
Haaninjo has quit [Quit: Ex-Chat]
Leopold has joined #freedesktop
Leopold has quit [Remote host closed the connection]