ChanServ changed the topic of #freedesktop to: https://www.freedesktop.org infrastructure and online services || for questions about freedesktop.org projects, please see each project's contact || for discussions about specifications, please use https://gitlab.freedesktop.org/xdg or xdg@lists.freedesktop.org
danvet has quit [Ping timeout: 480 seconds]
alanc has quit [Remote host closed the connection]
Leopold_ has quit []
genpaku has quit [Remote host closed the connection]
genpaku has joined #freedesktop
kattupoochi has joined #freedesktop
<nirbheek>
Is there some outage going on? I get this when I try to do a git fetch:
<nirbheek>
> rpc error: code = Unavailable desc = The git server, Gitaly, is not available at this time. Please contact your administrator.
<nirbheek>
Also job pages load and then disappear with an error
<nirbheek>
> An error occurred while fetching the job.
<SanchayanMaity>
I see the same here when using SSH urls. HTTPs urls seem to work.
kattupoochi has left #freedesktop [#freedesktop]
ximion has quit []
itoral has joined #freedesktop
<bilboed>
and getting 500 when trying to post an issue also
<bentiss>
we have one server down, not sure what happened, rebooting it now
<bentiss>
(and on that server was one gitaly pod, which explains the git issues)
<bentiss>
should be back-ish
SanchayanMaity has quit []
SanchayanMaity has joined #freedesktop
pjakobsson has joined #freedesktop
Satan3 has quit [Ping timeout: 480 seconds]
miracolix has quit []
Satan3 has joined #freedesktop
ofourdan has joined #freedesktop
danvet has joined #freedesktop
rgallaispou has joined #freedesktop
<bentiss>
FWIW, I still have a few services not properly binding to their IPs (not in gitlab AFAICT), so I'm going to upgrade and reboot all servers, it won't hurt to do this
scrumplex_ has quit []
scrumplex has joined #freedesktop
<nirbheek>
Just FYI getting this now, please ignore if this is expected :)
<nirbheek>
> Unable to load the merge request widget. Try reloading the page.
<bentiss>
nirbheek: yeah, I am currently upgrading/rebooting all servers one by one, so kind of expected
<nirbheek>
👍️
<Venemo>
good morning, gitlab gives me a HTTP 502
<bentiss>
Venemo: yep, known, see above, I am upgrading/rebooting all servers and it takes a little bit of time
<Venemo>
it works now
<Venemo>
bentiss: thanks. I wasn't sure if this was the same issue you had a few hours ago or a new problem
<bentiss>
Venemo: technically, it's the continuation, because I have at least one service on ceph not happy, so I figured I should flush all servers
<bentiss>
(this was way overdue FWIW)
<Venemo>
okay
<Venemo>
thanks for taking care of it :)
<bentiss>
no worries
ximion has joined #freedesktop
<eric_engestrom>
bentiss: are you doing something on gitlab? it's not loading anymore :'(
<bentiss>
eric_engestrom: yes, rebooting the last server of the flock
<eric_engestrom>
ok, thanks :)
<bentiss>
and gitaly-3 is not happy, as it was on that server and the mount poiont had issues migrating to other servers
<eric_engestrom>
ah, so possibly not just a couple of minutes of down time?
<bentiss>
roughly
<bentiss>
roughly a couple of minutes I would say
<eric_engestrom>
ok
<bentiss>
the machine is still in the BIOS
<bentiss>
gitaly-3 is starting up
<bentiss>
eric_engestrom: should be OK for you now
<eric_engestrom>
yeah, it loaded but it also logged me out, not sure if that's a normal side effect of what you did
<bentiss>
eric_engestrom: probably redis being wiped out
<eric_engestrom>
logged back in all fine, everything works
<eric_engestrom>
thanks!
<bentiss>
great :)
<eric_engestrom>
oh btw for the CI runner tags I mentioned a couple of days ago, we decided we don't need to filter like that anymore, so we don't need those tags anymore
<daniels>
eric_engestrom: oh that's easy, I can close that tab then
Haaninjo has joined #freedesktop
<eric_engestrom>
daniels: not sure what you mean ^^
<daniels>
eric_engestrom: I meant that I was about to go through and add the tags to all the different devices, but now I don't have to
Haaninjo has quit [Quit: Ex-Chat]
Haaninjo has joined #freedesktop
<eric_engestrom>
ah, thanks for that!
<eric_engestrom>
and indeed it's not needed anymore
itoral has quit []
MajorBiscuit has joined #freedesktop
mvlad has joined #freedesktop
ybogdano has joined #freedesktop
Leopold_ has joined #freedesktop
pjakobsson has quit [Ping timeout: 480 seconds]
pjakobsson has joined #freedesktop
<tomeu>
anholt: something seems to be wrong with the swrast runners other than #2
<tomeu>
venus-lavapipe jobs time out
<tomeu>
I think daniels had to blacklist some vmware-related kernel modules to prevent vsock (or vhost?) from stopping working properly after some time
<daniels>
tomeu: iirc that was terminal to vsock working at all, not just introducing failures - the vmw stuff greedily occupied vsock and the normal one didn't load
<eric_engestrom>
tomeu: thanks!
<tomeu>
hrm
<tomeu>
wonder if something similar about vmware stuff interfering with normal work of upstream vhost stuff
<tomeu>
as that is a host resource that all VMs have in common
<daniels>
looking at https://gitlab.freedesktop.org/sergi/virglrenderer/-/jobs/31623002 I'm seeing a ton of venus tests failing with a crash, and only towards the end do we start getting prints about guessing the syslog fd failing - I wonder if there's something like an FD leak or space exhaustion after enough tests have failed
MrCooper has quit [Remote host closed the connection]
<bentiss>
robclark: it should be enough yeah, wondering if you have another installed ci-fairy in the current image
<robclark>
it's possible.. keep in mind I don't know what I'm doing ;-)
<bentiss>
robclark: also s/minio:/https:/
<bentiss>
(in your s3cp command)
<robclark>
oh, I missed that in trying to port over the mesa change
jarthur has joined #freedesktop
miracolix has quit []
ybogdano has quit [Ping timeout: 480 seconds]
ybogdano has joined #freedesktop
<eric_engestrom>
tomeu: mesa_ci_status.py is eating almost 4.5 GB of ram and counting, and it printed "Processing pipelines pages 1..4" at the start but hasn't printed anything in over half an hour
miracolix has joined #freedesktop
<eric_engestrom>
(I ran it with no arguments, default everything)
<eric_engestrom>
I'm assuming this isn't normal, right? ^^'