12:28 <sergi> daniels, it seems there is a question I forgot to answer from you. When I was creating issues programmatically (using python-gitlab and an access token) the API was throwing an exception with code 409 (I sent the error message). After change the issue description and increase the time between issue creations, it didn't ban me again

12:29 <daniels> right, so 'it is now banning me' was producing the 409 to ask you to solve the CAPTCHA

12:31 <sergi> I didn't solve any captcha

12:31 <sergi> but I've changed the text of the issue description

12:31 <daniels> yeah, changing the text - especially if it contains links - is more likely to trigger CAPTCHA

12:32 <daniels> we don't currently have 'trusted user' opt-outs

12:32 <sergi> Links where internal within freedesktop's gitlab

12:39 <daniels> nod

12:40 <daniels> I don't think the filter is really that discriminating

13:21 <tintou> Is the CI under heavy load? Marge triggered pipeline hasn't started after 10minutes

13:28 <hakzsam> overall gitlab fdo seems a bit slow right now

13:36 <bentiss> FWIW, disks are all up, but the webservices pods are under heavy use, I really wonder what is happening

14:59 <zmike> hm now getting some unhappy messages from git

16:05 <anholt_> alatiera: are you htz windows admin? been seeing some connectivity issues this week from CI: https://gitlab.freedesktop.org/mesa/mesa/-/jobs/24977648 https://gitlab.freedesktop.org/mesa/mesa/-/jobs/23973779 (doesn't seem correlated with any core fd.o outages)

16:07 <alatiera> anholt_ I will check, thanks

16:08 <alatiera> anholt_ I think those are issues with docker itself not being able to resolve dns properly

16:09 <alatiera> I've seen it happen all the time, for example when there's an ADD in a dockerfile

16:09 <alatiera> but not sure what to do about it, windows itself has been able to resolve things properly every time I tried to debug this

16:11 <alatiera> also I've only seen this happen when docker itself is trying to resolve domains, not inside containers

16:11 <alatiera> for extra fun

16:16 <anholt_> wish gitlab had a nice ui for getting from failed jobs to list of jobs run by that runner at that time (was this a transient thing, or is it just intermittent and a retry would help?)

16:16 <anholt_> though, that would also require understanding gitlab ci's retry mechanism more than I do.

16:18 <alatiera> anholt_ it sometimes works on retry

16:18 <MrCooper> even just filtering https://gitlab.freedesktop.org/mesa/mesa/-/jobs for failed jobs would be nice

16:18 <alatiera> I would retry: 3 the docker building job personally

16:19 <anholt_> MrCooper: yeah, I would enjoy an interface for that that wasn't my inbox :)

16:19 <alatiera> the good thing is that once the image is built it doesn't hit it again

16:23 <jenatali> alatiera: That's not true. This job wasn't trying to build the image, just download it

16:23 <alatiera> oh no then

16:24 <alatiera> I guess we haven't seen this issue in gst for a while cause the image has been cached

16:25 <alatiera> nslookup resolved fine, sigh

16:25 <jenatali> And the first one anholt_ sent was a network timeout, not a DNS failure. Probably the same root cause though

16:28 <alatiera> we are on the latest build, sigh 20.10.9

20:13 <Kayden> gitlab seems down?

20:14 <Kayden> hm, no

20:14 <Kayden> git fetch was denying my keys now it's just being very slow

20:15 <anholt_> similar trouble here

20:49 <karolherbst> noooooo "remote: GitLab: Internal API unreachable" :(

20:50 <zmike> F5 intensifies

20:55 <daniels> yeah, something is weird specifically with the mesa repo

21:13 <daniels> looks like stuff is bottlenecking badly on postgres, so I'm going to try a vacuum

21:46 <daniels> hopefully more performant now