ChanServ changed the topic of #freedesktop to:
ngcortes has quit [Remote host closed the connection]
wirez has joined #freedesktop
bengal has joined #freedesktop
danvet has joined #freedesktop
wirez has quit [Ping timeout: 480 seconds]
Yeldham has joined #freedesktop
bcarvalho has quit [Remote host closed the connection]
alanc has quit [Remote host closed the connection]
alanc has joined #freedesktop
sumits has quit [Quit: ZNC - http://znc.in]
bengal has quit [Remote host closed the connection]
bengal has joined #freedesktop
bengal has quit []
xexaxo has joined #freedesktop
sumits has joined #freedesktop
bengal has joined #freedesktop
bengal has quit []
sumits has quit [Quit: ZNC - http://znc.in]
bcarvalho has joined #freedesktop
xexaxo has quit [Ping timeout: 480 seconds]
sumits has joined #freedesktop
bengal has joined #freedesktop
bengal has quit []
K`den has joined #freedesktop
Kayden has quit [Read error: Connection reset by peer]
bengal has joined #freedesktop
xexaxo has joined #freedesktop
xexaxo has quit [Remote host closed the connection]
xexaxo has joined #freedesktop
reillybrogan has joined #freedesktop
reillybrogan_ has quit [Ping timeout: 480 seconds]
Adrinael has joined #freedesktop
<Adrinael> Getting test timeouts on gitlab runners, at least fdo-packet-m1xl-1 fdo-packet-m1xl-3
<Adrinael> Also some earlier on fdo-packet-m1xl-2
<Adrinael> daniels, ^
<daniels> Adrinael: please provide job links
<daniels> thanks!
<shadeslayer> Hi! I'm getting AccessDenied errors when marge tries to run pipelines that upload artifacts to minio https://gitlab.freedesktop.org/mesa/mesa/-/jobs/12304901, has anyone seen these?
<tomeu> daniels ^
<tomeu> those are quite weird
<tomeu> shadeslayer: did you retry any jobs in that pipeline?
<shadeslayer> yes, I've assigned the MR to marge twice, and it failed with the same error afaict https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12075
<tomeu> maybe those are related with the timeouts reported just above
<tomeu> eg. the runner took so long to pick up a job, that the token had already expired by then
<tomeu> maybe we should make the TTL of the tokens a bit higher
<daniels> that was transient whilst I was dealing with some breakage
<daniels> retry and it'll be fine
<daniels> I was hoping no-one would catch the 10-15min window where nothing worked :P
<shadeslayer> oops :P
<bentiss> daniels: out of curiosity, what was broken?
<bentiss> sigh large-5 is now down, ceph is timeing out
<daniels> bentiss: k3s-minio-1 was logging a ton of errors about how the master's cert was expired so the node wasn't polling for updates
<bentiss> daniels: oh, ok
<daniels> so the opa pod was still running, but when I deleted it (or scaled the rs/deploy) it wasn't being recreated
<bentiss> daniels: IIRC you can solve that error by restarting k3s
<daniels> bentiss: exactly!
<bentiss> daniels: m1xl-3 is having quite some load
<bentiss> Load average: 3222.30
<bentiss> lots of virgl_test_server --use-egl-surfaceless
<daniels> sweet jesus
<daniels> how many of them?
<bentiss> a lot :)
<tomeu> ouch, that already happened once
<tomeu> do we know if it's from mesa or virglrenderer's CI?
<bentiss> also we have quite some docker pods still running for the past 43h
<daniels> https://gitlab.freedesktop.org/tintou/mesa/-/jobs/12307660 and https://gitlab.freedesktop.org/tintou/mesa/-/jobs/12307122 are both running virgl tests, but they claim to only be running 8 threads ...
<bentiss> daniels: I wonder if that is not jobs that have not been killed in the background
<bentiss> I count 1818 [virgl_test_serv] on m1xl-3
<bentiss> I paused m1xl-3, once its jobs are pruned, I'll killall virgl_test_serv
<daniels> ouch yeah, m1xl-3 is utterly slammed with 3492 load avg and all CPUs 100% utilised and a lot in D state, otoh m1xl-2 has 1400 load avg but CPUs (whilst virgl-test was running anyway, it's finished now) mostly idle ... ?
<bentiss> (side note, why can't I lock m1xl-3?)
ezequielg has quit []
ezequielg has joined #freedesktop
* bentiss is tempted to simply cancel https://gitlab.freedesktop.org/mesa/mesa/-/jobs/12306110 -> doesn't seem to be in a healthy state
<bentiss> Unable to get a valid fd
<daniels> yeah I’d just kill it
<daniels> maybe that’s the issue then - we’re stuck in infinite coredump hell?
<bentiss> coredump hell or fork bomb, don't know
<bentiss> using docker kill -s 9 CONTAINER_ID is not working
wyre_ has left #freedesktop [#freedesktop]
wyre has joined #freedesktop
<bentiss> I guess a reboot of m1xl-3 is in order
<bentiss> hard reboot seems like my last option :/
<daniels> bentiss: I think my first attempt at disabling core dumps was broken, so I've tried again on all of the shared runners
<daniels> I don't think it's forkbomb
<daniels> and every single dEQP coredumping would definitely account for coredump hell
<daniels> so fingers crossed that does the trick ... ?
<daniels> Adrinael: ^ that should solve your problem I hope
<Adrinael> native x86 run passed now, but https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/jobs/12311248 and https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/jobs/12311249 (arm64 and mips, via qemu) are still feeling ill
<daniels> there are 1675 igt tasks on packet-m1xl-1 in D state ...
<daniels> can you please make sure you're using meson test with the --parallel arg as ${FDO_CI_CONCURRENT:-8} so it doesn't try to execute 48 igts in parallel?
<daniels> (and similarly with -j for your build jobs)
<daniels> yeah ok, igt is DoSing itself
<Adrinael> gah patch coming up
<daniels> I've already sent one to the list
<daniels> oh no I haven't
<daniels> ffs
MrCooper has quit [Quit: Leaving]
<daniels> there we are, fixed git-send-email
MrCooper has joined #freedesktop
<daniels> (think it's stuck in igt-dev@ moderation ... having actual MRs for igt sure would be nice tbh)
<Adrinael> Hopefully coming in the future, but not in the near future
<MrCooper> sure would be
<daniels> oops, patch is broken, v2 incoming
<Adrinael> My first one was also broken =(
<daniels> hahaha
<Adrinael> daniels, qemu ones worked with your patch but then there's https://gitlab.freedesktop.org/adrinael/igt-gpu-tools/-/jobs/12312790
<Adrinael> hmm that's the only test job in this pipeline that went to fdo-packet-m1xl-2
<Adrinael> https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/jobs/12313619 one job from post-merge went to fdo-packet-m1xl-1 and is timeouting
<Lyude> daniels, bentiss - did either of you two get a chance to read the email I sent?
ezequielg has quit []
<bentiss> Lyude: which mail?
ezequielg has joined #freedesktop
<Lyude> bentiss: [Input needed ASAP] Things to cover in Gitlab Commit presentation for X.org?
<bentiss> Lyude: I don have that email
<Lyude> bentiss:??? I sent it to site wranglers
* bentiss avoided that list :)
<bentiss> more like dodged, not avoided
<Lyude> bentiss: need me to forward it then?
<bentiss> yes please :)
<Lyude> bentiss: sent to your rh email
<bentiss> k, got it now
<Yeldham> Howdy! I would like to ask if the 'shared-mime-info' repo is open for a PR to add MIME types of the Godot engine.
jarthur has joined #freedesktop
K`den is now known as Kayden
rektide has quit [Remote host closed the connection]
<Lyude> Yeldham: this channel is for fdo infra issues, might need to ask somewhere else
<Lyude> (…I thought this used to be in the topic, not sure what happened to it)
ngcortes has joined #freedesktop
bengal has quit [Ping timeout: 482 seconds]
xexaxo has quit [Ping timeout: 480 seconds]
<bl4ckb0ne> freenode to libera migration?
Kayden has quit [Quit: to lunch and the office]
<Yeldham> Lyude: Any idea as to where I could ask?
<imirkin_> bl4ckb0ne: we all went to oftc, but the topic went to libera? :)
<bl4ckb0ne> yes oftc, sorry
<alanc> Yeldham: https://gitlab.freedesktop.org/xdg/shared-mime-info shows merge requests from other people, so you could just try submitting one and see if they accept it
<Yeldham> Yeah, that seems like the best shot.
<daniels> Lyude: saw it but won’t have much of a chance to do anything with it until Sunday I’m afraid
Kayden has joined #freedesktop
Seirdy_ has joined #freedesktop
Seirdy_ has quit []
Seirdy has quit [Ping timeout: 480 seconds]
Seirdy has joined #freedesktop
danvet has quit [Ping timeout: 480 seconds]
bengal has joined #freedesktop
Haaninjo has joined #freedesktop
bengal has quit [Ping timeout: 480 seconds]
Kayden has quit [Quit: go home]