12:41 <emersion> > But given that our db is 90% CI, this split should hopefully speed up the instance quite a bit by having normal operations handled on a much smaller db.

12:41 <emersion> unless we have the DB currently overloaded i'm not sure it would change anything

17:00 <bentiss> emersion: that would still give us twice the db connections available, hopefully :-)

17:00 <emersion> but is that a bottleneck atm?

17:28 <bentiss> when we had the spam users created as a wave, the UI felt very sluggish

17:29 <vyivel> i thought sluggish ui was a gitlab feature

17:30 <bentiss> but TBH, I think what makes things hard for the cluster are the backups, as we are compressing and sending a lot of Giga bytes everyday (the backups of gitaly takes between 13 to 15 hours to complete)

18:23 <alyssa> bentiss: Is there a non-neglible chance of downtime on June 5?

18:24 <alyssa> No worries if so - just need to know if I should setup a non-fd.o mirror in case the downtime is a bit longer than expected

18:25 <alyssa> ...and thank you in advance for the upgrade ^^

18:39 <bentiss> alyssa: technically it shoudn't be with much risks, but we never know. Thus the 48h delay, as I expect this to be more like 12-24h. But given that the split of teh db consists in taking another snapshot of the db, and re-replicating it in a second postgres instance, and then clearing up the 2 dbs, I suspect this will take at least twice the usual time, so 24h is not unexpected

18:40 <bentiss> 48h should be in case something goes wrong, and we had to redeploy the backup. So I think we can make that decision on Monday evening here on Tuesday morning and we'll let folks know

18:50 <alyssa> sounds great, thanks! and good luck :)

20:32 <daniels> bentiss: should we downgrade gitaly backups to 2x/wk?

20:47 <bentiss> daniels: that's a good idea actually

20:48 <bentiss> not sure we need a daily backup given the history

20:49 <bentiss> maybe we could rotate them as well, gitaly-0, then -1 then -2 and then -3, which roughly gives 2x/week, but one everyday :)

21:01 <daniels> oh yeah! can we do that … ?