ChanServ changed the topic of #freedesktop to: https://www.freedesktop.org infrastructure and online services || for questions about freedesktop.org projects, please see each project's contact || for discussions about specifications, please use https://gitlab.freedesktop.org/xdg or xdg@lists.freedesktop.org
karolherbst has quit [Read error: Connection reset by peer]
<bentiss>
emersion: IIRC it's a manual renewal that daniels does every 3 months
<bentiss>
one more thing we should automate
<daniels>
heh, I see you started running certbot already
<daniels>
I do it with ~danielsR/le-mail-renew.sh
<daniels>
emersion: we use dns-01 challenges for smtp
<emersion>
oh, and it's a manual process :/
<bentiss>
daniels: please ping me when the cert is renewed, I'm in the console waiting to retry the dead job
<bentiss>
FWIW (so I get it archived): Sidekiq::DeadSet.new.each { |job| if job.queue == "mailers" then p job; job.retry end}
<daniels>
bentiss: try that
<bentiss>
daniels:will do
<bentiss>
seems good now :)
<bentiss>
it needs to flush the queu now ;)
<bentiss>
daniels: thanks
* bentiss
is hammering too much the smtp server: "ActiveRecord::ConnectionNotEstablished: connection to server at "10.41.x.x", port 5432 failed: FATAL: remaining connection slots are reserved for non-replication superuser connections"
<pq>
were any email notifications permanently lost?
rgallaispou has joined #freedesktop
<bentiss>
pq: I just finished to empty the queue
<pq>
I don't understand what that means.
<bentiss>
so all missed emails should now have found their recipients
<pq>
cool, thanks!
<bentiss>
pq: no emails loss in other words
<hakzsam>
it's back, thanks for the quick fix!
<daniels>
bentiss: that’s psql, not smtp
<bentiss>
pq: whenever an email is sent, it is using a sidekiq job, which can be parked when it fails 3 times
<bentiss>
pq: so we can resurect all the items in the dead queue to resend them
<bentiss>
daniels: not sure what you are talking about, the link I gave above clearly points at the smtp configuration
<pq>
that's nice, but I'll likely have forgotten about that by the next time :-)
<bentiss>
pq: the only thing to remember is that we should not lose emails if we catch this soon enough
<pq>
"should not" and "if"... ;-)
<bentiss>
yes, very much conditional :)
<pq>
I have no idea when those conditions might trigger, so it's just easier for both of us to simply ask "were any lost this time?"
<bentiss>
if you want :)
eroux_ has joined #freedesktop
<bentiss>
I think that so far we haven't lost any emails, if that matters
<pq>
that's really good
eroux has quit [Ping timeout: 480 seconds]
<daniels>
bentiss: 5432 is postgres
<daniels>
also, if the SMTP fails too hard then the job just becomes ‘dead’ which you can rescue for absolutely ages
<bentiss>
daniels: oh... OK. but it doesn't change the fact that too many emails were trying to be sent at once
<daniels>
I usually just use that through the web UI
<daniels>
sure :)
<bentiss>
daniels: not sure if you cleaned up the queue recently but the amount of old stuff in the queue is rather small
<bentiss>
so I wonder if there is not a new rule in place that clears it after a certain amount of time
<daniels>
bentiss: in the dead section?
<daniels>
bentiss: one thing I do now realise is that I forgot to restore the sidekiq runner to do the batch repo checks, so those have been piling up
<bentiss>
daniels: yes, the oldest one is 4 days ago (I also cleared all of the reactive_caching queue)
<bentiss>
daniels: BTW, I won't be at XDC this year
<daniels>
bentiss: ah that's a shame :(
<bentiss>
daniels: I'm trying to go to LPC though
<daniels>
bentiss: in Dublin? nice, I should be htere
<bentiss>
daniels: cool :)
<bentiss>
well, I don't have a pass yet, trying to get a talk accepted first
<pohly>
udflare-certificates/), but support for it doesn't seem to be enabled for freedesktop.org GitLab. When I go to the pages/New Domain dialog, it says "Support for custom certificates is disabled. Ask your system's administrator to enable it. " Would that be possible?
scrumplex has quit [Ping timeout: 480 seconds]
thaller is now known as Guest4163
thaller has joined #freedesktop
Guest4163 has quit [Ping timeout: 480 seconds]
<bentiss>
pohly: should be, yes. It is disabled because we don't use the gitlab certs and setup for pages, we've got something slightly different that works better, because we are in control
<hakzsam>
looks like gitlab-mirror emails for commits are broken too?
thaller is now known as Guest4167
thaller has joined #freedesktop
Guest4167 has quit [Ping timeout: 480 seconds]
Rainer_Bielefeld_away has quit [Remote host closed the connection]
<bentiss>
pohly: basicaly you need to make syncevolution.org point at our gitlab IP (147.75.198.156) and I can automate the generation of the certificate for both pages.syncevolution.org and syncevolution.org
<pohly>
pages.syncevolution.org is just temporary, I am still experimenting with CloudFlare.
<bentiss>
pohly: that would be a good first test then :)
<pohly>
Should I really use a fixed IP (147.75.198.156)?
<bentiss>
pohly: for pages.xx you can use a CNAME, but for the root of the domain, you can only use a fixed IP
<bentiss>
unless cloudfare has some magic
<pohly>
I can enter syncevolution.pages.freedesktop.org as "Content" for a CNAME entry. I was assuming that it then does the DNS lookup.
<bentiss>
pohly: gitlab.freedesktop.org is better for the CNAME
<pohly>
I can change that.
<bentiss>
but AFAIK, you can not use a CNAME for the top most DNS entry (syncevolution.org in your case)
<pohly>
The "A " entry for syncevolution.org still points to 198.145.21.19, the Drupal server.
<bentiss>
pohly: we'll need to point it at our gitlab server
<bentiss>
but we can do it in a second time, first ensures pages.syncevolution.org works, then the other one
<pohly>
Right.
<pohly>
So I have pages.syncevolution.org pointing to gitlab.freedesktop.org in CloudFlare, and in GitLab I have the pages.syncevolution.org as additional domain.
<bentiss>
pohly: do you really need cloudfare?
<pohly>
But trying to access anything there gives me a 404 error from Cloudflare.
<bentiss>
I am not sure it'll work properly with our let's encrypt
<pohly>
Cloudflare seemed like a nice way to manage the site and get some caching.
pseigo has quit [Quit: left]
<bentiss>
pohly: that's not the way the other part is handled, so kind of expected it doesn't work
<pohly>
The free plan seemed sufficient for my needs.
pseigo has joined #freedesktop
<pohly>
I also need to support downloads.syncevolution.org.
<pohly>
That is currently 30GB of partly historic data that I need to host somewhere.
<bentiss>
pohly: right now we only support when external domains points directly at us
<bentiss>
we can also easily add custom routes (though not with a nice and shiny interface like cloudflare, but by talking to us over IRC)
<bentiss>
30GB of data is not a lot to handle too
<pohly>
So you are saying I could rsync it to some freedesktop machine and have it appear under downloads.syncevolution.org?
<bentiss>
I'd need to check with the other admins (daniels emersion and Mithrandir), but yes that's what I am saying
<pohly>
That might be simpler. I am struggling a bit to put all of this together.
<bentiss>
pohly: I just checked, and we can not automatically generate certificates if you are using cloudflare. Because we would need an API token and what not
<bentiss>
pohly:so it'll be much easier if you just make pages.syncevolution.org a CNAME to gitlab.freedesktop.org, and ditch clouflare
<bentiss>
thaller: are you the one responsible for modemmanager.org too?
<bentiss>
pohly: sorry, not sure we'll be able to have teh certificate up and running soon, modemmanager.org is having too many issued certificates that it is blocking us for getting a new one
<pohly>
No hurry.
<bentiss>
daniels: FWIW, I had to disable teh ingress validating webhooks because it was complaining for unknown variables that were declared in the http-snippet part :(