ChanServ changed the topic of #freedesktop to: https://www.freedesktop.org infrastructure and online services || for questions about freedesktop.org projects, please see each project's contact || for discussions about specifications, please use https://gitlab.freedesktop.org/xdg or xdg@lists.freedesktop.org
haaninjo has quit [Quit: Ex-Chat]
kode54 has joined #freedesktop
airlied has quit [Remote host closed the connection]
airlied has joined #freedesktop
todi1 has joined #freedesktop
todi has quit [Ping timeout: 480 seconds]
JanC is now known as Guest12171
JanC has joined #freedesktop
zerozero1 has joined #freedesktop
Guest12171 has quit [Ping timeout: 480 seconds]
zerozerozero has quit [Ping timeout: 480 seconds]
kasper93 has quit [Remote host closed the connection]
ximion has quit [Remote host closed the connection]
airlied has quit [Ping timeout: 480 seconds]
airlied has joined #freedesktop
jturney has quit [Ping timeout: 480 seconds]
noodlez1232 has quit [Remote host closed the connection]
noodlez1232 has joined #freedesktop
swatish2 has joined #freedesktop
jturney has joined #freedesktop
sima has joined #freedesktop
MrCooper_ has joined #freedesktop
sghuge has quit [Remote host closed the connection]
sghuge has joined #freedesktop
ximion has joined #freedesktop
MrCooper has quit [Ping timeout: 480 seconds]
scrumplex has joined #freedesktop
scrumplex has quit []
AbleBacon has quit [Read error: Connection reset by peer]
scrumplex has joined #freedesktop
ximion has quit [Remote host closed the connection]
jsa1 has joined #freedesktop
swatish21 has joined #freedesktop
mvlad has joined #freedesktop
<slomo> fdo-gitlab-runner-coordinator-x86-2 (at least) has no space left
swatish2 has quit [Ping timeout: 480 seconds]
MrCooper__ has joined #freedesktop
MrCooper__ is now known as MrCooper
<hakzsam> bentiss: daniels: would you be able to fix this? Thanks!
<bentiss> hakzsam, slomo: restarting them, this should prune the VMs and restart fresh
<hakzsam> thanks for the quick fix. Can I reassign to Marge now?
MrCooper_ has quit [Ping timeout: 480 seconds]
<bentiss> yes, it's restarting now
swatish21 is now known as swatish2
<hakzsam> great
tzimmermann has joined #freedesktop
MrCooper_ has joined #freedesktop
MrCooper has quit [Ping timeout: 480 seconds]
MrCooper_ is now known as MrCooper
<hakzsam> token issue it seems?
<daniels> hakzsam: no, expand the ‘artifacts: prepare’ section and you’ll see you’ve introduced a duplicate file
<hakzsam> oh
<hakzsam> my bad, sorry for the ping..;
<daniels> np
swatish2 has quit [Ping timeout: 480 seconds]
MrCooper_ has joined #freedesktop
<hakzsam> are the runners up? looks like https://gitlab.freedesktop.org/mesa/mesa/-/pipelines/1389362 has been queued for like 30 minutesd
MrCooper has quit [Ping timeout: 480 seconds]
MrCooper_ is now known as MrCooper
swatish2 has joined #freedesktop
<bentiss> hakzsam: sorry, was afk a bit. The runners are still up, yeah
<hakzsam> yeah, they do work now
haaninjo has joined #freedesktop
<valentine> bentiss: getting no space left on device again https://gitlab.freedesktop.org/mesa/mesa/-/jobs/73343602
swatish21 has joined #freedesktop
<hakzsam> oh valentine reported it already :)
<valentine> :)
swatish2 has quit [Ping timeout: 480 seconds]
<bentiss> yay, so gst-plugins-rs is actually eating all of the volumes once it's finished (just purged 118GB on that runner)
<bentiss> not pointing fingers at anyone, but that's the issue with rust: it just rebuilds everything :(
<bentiss> I'll try to find a workaround soon (tm)
guludo has joined #freedesktop
MrCooper_ has joined #freedesktop
<bilboed> slomo: ^^
MrCooper has quit [Ping timeout: 480 seconds]
jsa1 has quit [Ping timeout: 480 seconds]
<slomo> bentiss: you mean after jobs finished it doesn't clean up properly? or that while the jobs are running a lot of space is needed?
<slomo> or said differently, is there anything I can do to improve on that situation? :)
<nirbheek_> Is it that the containers balloon while the build is occurring and don't get cleaned up after the job by gitlab?
<slomo> bentiss: also are we talking about the gst-plugins-rs CI pipelines, or the cerbero pipeline (triggered from gst-plugins-rs and gstreamer)?
larunbe has joined #freedesktop
<__tim> apache on annarchy needs a kick I think, freedesktop.org seems down
<bentiss> slomo: the job was in #<Project id:1422 slomo/gst-plugins-rs>>
<slomo> ok so not cerbero
<bentiss> but again, this is normal, it's just that gitlab is too conservative and keep *all* container states after a run, which is not what we need in those cases
<bentiss> slomo: don't worry, next objective is for me to see if there is a gitlab runner config to *not* keep the volumes of the old jobs
<bentiss> I'll do that after lunch
<eric_engestrom> bentiss: would an after_script: be able to clean up what shouldn't be kept?
<bentiss> eric_engestrom: maybe, but for the fleeting runners, being more aggressive should be fine (and reliable)
<eric_engestrom> feels like the right place to make that decision
<slomo> bentiss: ok, let me know if the CI jobs are doing something silly that should be improved :)
<eric_engestrom> ack, maybe we should hav eboth
<bentiss> but yeah, for kvm, this could be interesting as a rule of thumb for rust and go projects
MrCooper_ is now known as MrCooper
alarumbe has quit [Ping timeout: 480 seconds]
<__tim> anyone who can kick apache? :)
<MrCooper> bentiss: are you thinking of completely destroying the container after each job, or just pruning its filesystem?
erdem has joined #freedesktop
<daniels> __tim: done
<__tim> ta
haaninjo has quit [Quit: Ex-Chat]
mvlad is now known as Guest12189
mvlad has joined #freedesktop
MrCooper_ has joined #freedesktop
Guest12189 has quit [Ping timeout: 480 seconds]
MrCooper has quit [Ping timeout: 480 seconds]
<bentiss> (prevent volumes creation)
swatish21 has quit [Ping timeout: 480 seconds]
<zmike> are all the gitlab runners up now?
imre has quit [Quit: leaving]
<daniels> not really, no
<daniels> bentiss: do you want me to push the disable_cache bit?
kasper93 has joined #freedesktop
<bentiss> sigh... again, not enough runners at this location
<mupuf> bentiss: :s And I assume there is no free peering between two Hetzner DCs, right?
<bentiss> there is, but I've been trying differetn instance types and locations :(
<mupuf> Is it cheaper to get one beefy instance that we sub-divide, or smaller instances?
<bentiss> right now I don't have any ccx43, 33, 23, in any of the 3 european locations
swatish2 has joined #freedesktop
<bentiss> it managed to start 2 in hel1
<bentiss> well, we managed to get non dedicated servers now
<bentiss> definitely not ideal :(
<bentiss> luckily I already have the 4 dedicated servers I ordered yesterday, but I'll need a little bit of time to configure them :(
<erdem> mupuf: what is a `beefy` server?
<mupuf> erdem: more than 32 cores? 😅
DodoGTA has left #freedesktop [#freedesktop]
swatish2 has quit [Ping timeout: 480 seconds]
<erdem> mupuf: make sense :)
MrCooper__ has joined #freedesktop
DodoGTA has joined #freedesktop
MrCooper_ has quit [Ping timeout: 480 seconds]
jsa1 has joined #freedesktop
haaninjo has joined #freedesktop
MrCooper__ is now known as MrCooper
DodoGTA has quit [Quit: DodoGTA]
DodoGTA has joined #freedesktop
dri-logger has joined #freedesktop
marcheu_ has joined #freedesktop
dri-logg1r has quit [Ping timeout: 480 seconds]
marcheu has quit [Ping timeout: 480 seconds]
marcheu has joined #freedesktop
dri-logg1r has joined #freedesktop
slomo has quit [Quit: The Lounge - https://thelounge.chat]
dri-logger has quit [Ping timeout: 480 seconds]
marcheu_ has quit [Ping timeout: 480 seconds]
slomo has joined #freedesktop
MrCooper_ has joined #freedesktop
* bentiss is uploading the registry dataa to fastly at 300 MiB/s, that's much better than towards hetzner :)
MrCooper has quit [Ping timeout: 480 seconds]
MrCooper__ has joined #freedesktop
MrCooper_ has quit [Ping timeout: 480 seconds]
<hakzsam> https://gitlab.freedesktop.org/mesa/mesa/-/jobs/73367744 queued for 30 minutes too :/
jenatali has joined #freedesktop
<jenatali> Hm the Windows runners haven't changed, that's surprising
<__tim> There seems to be a bit of a backlog because we're basically catching up on 2 weeks of GStreamer MRs now :)
<eric_engestrom> should windows be disabled in mesa ci for now, until gst is done catching up? we don't want merging gst MRs to prevent merging mesa MRs ^^
<eric_engestrom> daniels, sergi: reminder that you haven't re-enabled the collabora farm, in case you forgot 🙈
<jenatali> If you have to disable it, that's probably fine, but I'm hesitant because nearly every time it gets disabled there's some kind of breakage and then it's on me to fix the breakage before they can be re-enabled
<hakzsam> the windows job just started...
<daniels> eric_engestrom: we haven’t forgot, but the caching snippet is broken - it doesn’t cache anything - so we’re just testing the fix and making sure it also works for traces requiring auth as well
<hakzsam> but I think it will just timeout the pipeline because there is only 5 minutes left
<daniels> also -1 to disabling windows, tbh the linux-x86 runners are just as backed up atm, and the breakage will be pain
<daniels> I don’t mind pulling the marge timeout up temporarily until everything has settled down though
<eric_engestrom> yeah, I don't like disabling farms either :/
<eric_engestrom> so ack, keeping windows in mesa ci
jsa1 has quit [Ping timeout: 480 seconds]
<eric_engestrom> daniels: is that a bug in the url fix, or another bug?
<hakzsam> damn, it passed one minute too late :D
MrCooper__ is now known as MrCooper
<daniels> eric_engestrom: a bug in the url fix
<daniels> eric_engestrom: s3.fd.o returns a 301 redirect to xxx.your-objectstorage.net/fdo-opa/$path/?foo, where 'foo' is a huge set of temporary short-lived tokens and timestamps
<hakzsam> jenatali: same thing is happening again https://gitlab.freedesktop.org/mesa/mesa/-/jobs/73374485 is there contention somewhere?
<daniels> so nothing will ever get cached since it never hits the same URL twice
<jenatali> hakzsam: See __tim's comment: "There seems to be a bit of a backlog because we're basically catching up on 2 weeks of GStreamer MRs now :)"
<daniels> we have a fix, I've just been ploughing through ci-fairy/traces-db/piglit/etc to make sure that it also works with auth'ed requests
<hakzsam> jenatali: okay, I see
<eric_engestrom> daniels: oh, I see, that's a problem indeed
<eric_engestrom> I'm cancelling merging my fix MR since it's actually broken :(
<eric_engestrom> daniels: btw since you're mentioning testing with traces & piglit, did you fix the piglit bug?
<daniels> yeah
<eric_engestrom> nice! looking forward to re-enabling all the traces jobs 😄
tzimmermann has quit [Quit: Leaving]
<hakzsam> given the very high contention of windows-msvc, I would suggest to disable it or to make it optional. Otherwise, all pipelines will just timeout
alanc has quit [Remote host closed the connection]
alanc has joined #freedesktop
AbleBacon has joined #freedesktop
ninelegturnip has joined #freedesktop
ninelegturnip has quit [Quit: ninelegturnip]
haaninjo has quit [Remote host closed the connection]
haaninjo has joined #freedesktop
jsa1 has joined #freedesktop
mvlad has quit [Remote host closed the connection]
haaninjo has quit [Quit: Ex-Chat]
jsa1 has quit [Ping timeout: 480 seconds]
guludo has quit [Ping timeout: 480 seconds]