daniels changed the topic of #freedesktop to: GitLab is currently down for upgrade; will be a while before it's back || https://www.freedesktop.org infrastructure and online services || for questions about freedesktop.org projects, please see each project's contact || for discussions about specifications, please use https://gitlab.freedesktop.org/xdg or xdg@lists.freedesktop.org
co1umbarius has joined #freedesktop
columbarius has quit [Ping timeout: 480 seconds]
_DOOM_ has joined #freedesktop
<_DOOM_>
I have a question about DBus, how is a client supposed to handle the NameOwnerChanged signal? I guess based on the args what should the client assume? I apologize if there's an obvious answer but I'm just unsure.
AbleBacon has quit [Read error: Connection reset by peer]
aninternettroll has quit [Ping timeout: 480 seconds]
aninternettroll has joined #freedesktop
bmodem has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
sima has joined #freedesktop
alpernebbi_ has joined #freedesktop
lsd|2 has joined #freedesktop
An0num0us has joined #freedesktop
Leopold_ has quit [Remote host closed the connection]
Leopold_ has joined #freedesktop
tzimmermann has joined #freedesktop
<bentiss>
I guess it's now time to attempt the migration to gitlab 16.0.8 (then 16.1, 16.2, and 16.3)
<bentiss>
(not now-now, in a few, there are a few preparatory steps first)
<mupuf>
bentiss: crossing fingers!
<mupuf>
bentiss: will you speedrun through them all, or just space them out by a day or so?
<bentiss>
mupuf: don't know. It'll depend on how it goes
<mupuf>
ack
<mupuf>
better be safe, indeed!
AbleBacon has joined #freedesktop
<MrCooper>
bentiss: FWIW, I noticed in the 16.3 announcement an explicit mention of it being a "required stop", which conversely implies it might be possible to skip some of the earlier ones?
<bentiss>
mupuf: and not sure it's interesting to let users adapt to one release at a time, because weĺ ll have to go to 16.3. So if there are breaking changes, it's pointless to give users 1 day to adapt
<mupuf>
good point
jarthur has quit [Ping timeout: 480 seconds]
<bentiss>
MrCooper: yes, I know. But from my experience, going from one major to one other usually requires a big db migration, which takes time. And if we add all the other migrations at the same time, the chances that the migration times out are high
<emusia>
hi folks, gitlab seems to be down for us, how long outage can take?
<bentiss>
emusia: between few seconds to few hours. We are migrating to the latest gitlab version, and no ideas on how long the db migration job will take
<emusia>
:(
pkira has joined #freedesktop
<bentiss>
looks like the migration of the db for 16.2 is almost done now
<bentiss>
the new webservice pods are almost ready, we need to kick the old ones
<emusia>
<3
bmodem has joined #freedesktop
jdr112 has joined #freedesktop
<bentiss>
and sorry, we are only at 16.1, 16.2 will be there shortly after
<bentiss>
mupuf: I suspect that we probably need to change the runners and turns them back to debian
<pq>
Awesome! Thank you for the hard work!
<bentiss>
because it might be the same network issue I'm having on the new cluster
<mupuf>
so, it would be an issue with coreos?
<bentiss>
maybe?
<bentiss>
or it's a BGP issue with the datacenter, but it feels not right
<bentiss>
BTW, we got ml-24 completly offline
<mupuf>
are both the runners and gitlab in the same datacenter?
<bentiss>
mupuf: no, the runners are in DC, and the gitlab servers in NY
<mupuf>
right. And we are hoping to move gitlab to DC at some point, right?
<bentiss>
but now that the registry is severed from GCS, we don't require podman anymore, so maybe we can switch back to debian+docker, it was more stable
<bentiss>
that was my point with that migration
<bentiss>
but it went wrong
An0num0us has quit [Ping timeout: 480 seconds]
pulsar12 has joined #freedesktop
<kxkamil>
swatish2: mupuf: should we wait a little longer and re-run ci pipe? Failed to connect to gitlab.freedesktop.org port 443 after 129963 ms: Couldn't connect to server
<mupuf>
kxkamil: trying one more time
<kxkamil>
btw is merging new commits safe? if we do not care about ci pipeline
<MrCooper>
huh, looks like GitLab comment notification e-mails create proper sub-threads if one quotes a previous comment in the same GitLab thread; is that a new feature, or do I just happen to notice it for the first time?
<bentiss>
MrCooper: probably a new feature if you noticved it just now ;)
<MrCooper>
well, there could be bias due to the knowledge of GitLab having been updated :)
<bentiss>
MrCooper: it wasn't updated for 3 months, except for a few security updates. So my bets are that it's a new feature
<MrCooper>
the bias I mean is: notice X for the first time + knowledge that the system exposing X was just changed ⇒ X must be a result of the change
pulsar12_ has quit [Ping timeout: 480 seconds]
<MrCooper>
it could just as well be coincidence though
<bentiss>
MrCooper: I know, I know. But in that case, that wouldn't surprise me because we were so behind in terms of features
<bentiss>
like I still need to get used to that new interface
pkira_ has quit [Ping timeout: 480 seconds]
<MrCooper>
heh, GNOME + gitlab.com got it a while ago (some others are still holding out though)
<mupuf>
koike: for the daemon, it would be good to make it clear that this is effectively a soft-disable
<koike>
mupuf: in the sense that is the docker gitlab-runner that is stopped and not the hardware runner? Or do you mean a different thing?
<mupuf>
I mean that we would indeed not be running on the hardware we said we would, and what we would run is effectively /bin/true
<mupuf>
And we should complain loudly somewhere when this happens (IRC?)
<mupuf>
And probably in the MR too
<koike>
mupuf: ah, right, it is a way to disable a job
<koike>
let me rephrase that on the ticket
pulsar12 has quit [Ping timeout: 480 seconds]
<koike>
mupuf: I edited the ticket, please check if I got right this time
<mupuf>
koike: thanks, it is clearer now :)
<koike>
awesome
<mupuf>
The first solution is a bit hand-wavy though. What policy would you propose for gitlab?
<mupuf>
Users having different priorities?
<koike>
I guess? I'm not sure if there is another way, or maybe some sort of mechanism that people could write their own policy, but this starts to become complicated
jsto has quit [Quit: jsto]
<mupuf>
Yeah... It would have to run server-side
<mupuf>
Good luck getting that accepted
<mupuf>
The endpoints would allow every runner to make their decision
jsto has joined #freedesktop
<koike>
yeah, I like the endpoint proposal
<mupuf>
And I think, in the end, we probably want that + the daemon (because farms can go offline)
<dj-death>
have people noticed that you can't post backtraces in gitlab comments?
<dj-death>
"Your comment could not be updated because your comment has been recognized as spam. please, change the content to proceed.."
<koike>
mupuf: for the deamon, we would also need to understand how many pending jobs in a given tag we need to wait to take a decision to soft-cancel the job
<koike>
dj-death: I didn't know about it, but you can get around it by posting on paste.ee and pasting the link to it
<mupuf>
koike: I would say: one job that waited over 40 minutes
<mupuf>
(Marge job*)
<mupuf>
The lava jobs will be annoying since sometimes the gitlab runner is free but no DUTs in lava are... But that's a bug to fix there
<mupuf>
(The gitlab runner should poll for jobs only if a DUT is free to execute it, and if it does poll the DUT needs to be reserved for it
<mupuf>
AKA, pretend the DUT is taken until the poll is complete
tzimmermann has quit [Quit: Leaving]
swatish2 has quit [Ping timeout: 480 seconds]
<daniels>
mupuf: we did that, but we kept getting DoSed by people running stuff in personal branches. so we accept a larger number of jobs and use LAVA’s priority mechanism to make sure Marge jobs get in before any personal jobs
mripard has quit [Remote host closed the connection]
<mupuf>
daniels: makes sense to do that until prioritisation is implemented in the gitlab runner side
<mupuf>
What's your oversubscription? 2x?
<mupuf>
AKA, twice as many gitlab runners as physical DUTs?
<daniels>
mupuf: 2x iirc but DavidHeidelberg[m] can confirm
<DavidHeidelberg[m]>
I recall 2x too
<mupuf>
Ack, thanks!
<mupuf>
dj-death: can you provide a link to the back trace?
i509vcb has joined #freedesktop
An0num0us has joined #freedesktop
<daniels>
(fwiw I’m not ignoring the CIstrophe, but on holiday and back Wed)
<daniels>
(which means that I am ignoring it, but for arguably quite good reasons; too distracted by Basque hills)
<mupuf>
daniels: of course, enjoy your holidays! Sorry for disturbing
ximion has joined #freedesktop
<daniels>
not at all :)
AbleBacon has joined #freedesktop
alanc has quit [Remote host closed the connection]
Haaninjo has joined #freedesktop
alanc has joined #freedesktop
<dj-death>
mupuf: yeah I can attach a file I guess
<mupuf>
dj-death: I was not proposing to do that. I wanted to see the content of the backtrace. We definitely want users to be able to upload logs!
pkira__ has quit [Ping timeout: 480 seconds]
iNKa has joined #freedesktop
vyivel_ has joined #freedesktop
wyre_ has joined #freedesktop
SirDeiu_ has joined #freedesktop
samuelig_ has joined #freedesktop
enunes- has joined #freedesktop
ifreund has quit [Remote host closed the connection]
raghavgururajan has quit [Remote host closed the connection]
elibrokeit has quit [Remote host closed the connection]
moses has quit [Write error: connection closed]
rpigott_ has quit [Write error: connection closed]
nuclearcat2 has quit [Read error: Connection reset by peer]
ndufresne has quit [Read error: Connection reset by peer]
vyivel has quit [Write error: connection closed]
jvaclav has joined #freedesktop
Brocker has quit [Read error: Connection reset by peer]
jani has quit [Remote host closed the connection]
samuelig has quit [Read error: Connection reset by peer]
infernix has quit [Read error: No route to host]
damian has quit [Remote host closed the connection]
thecollaboran14 has quit [Read error: Connection reset by peer]
jvaclav has quit [Write error: connection closed]
philn has quit [Remote host closed the connection]
SirDeiu has quit [Read error: Connection reset by peer]
wyre has quit [Write error: connection closed]
raghavgururajan has joined #freedesktop
elibrokeit has joined #freedesktop
moses has joined #freedesktop
rpigott has joined #freedesktop
ifreund has joined #freedesktop
thecollaboran14 has joined #freedesktop
jani has joined #freedesktop
philn has joined #freedesktop
damian has joined #freedesktop
vyivel_ is now known as vyivel
nuclearcat2 has joined #freedesktop
enunes has quit [Remote host closed the connection]