05:18 <DavidHeidelberg[m]> bentiss: we download big traces binaries; there is risk that the binary gets corrupted due multiple factors (while on transit, rest...) so idea was to be able do sha256sum with value gathered from x-amz-content-sha256 after the download

06:17 <mupuf> DavidHeidelberg[m]: define big, and are you going to be able to fit them in memory, for runners with limited amount of RAM? Or do you download them and store them on the local NFS server for the duration of the job?

06:19 <bentiss> DavidHeidelberg[m]: the headers of the object contains the "etag" which is the md5sum used when uploaded. And with https://teppen.io/2018/10/23/aws_s3_verify_etags/ you can compare the local tag with the remote

06:24 <bentiss> I just did `URL=https://s3.freedesktop.org/git-cache/bentiss/mesa/mesa.tar.gz ; curl -LO $URL && python verify_object.py mesa.tar.gz $(curl -I $URL | grep etag | cut -d ' ' -f 2 | jq -r)` (the last jq is to strip the \r from the string)

06:24 <bentiss> and it says "Local file matches"

08:40 <bentiss> daniels: mind if I assign https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19076 to marge? (I am never fully certain about the process of when and who assigns marge to a MR)

09:27 <aleksander> hey! I have a bunch of pipelines failing with errors failing to pull images from registry, https://gitlab.freedesktop.org/cmeerw/libmbim/-/pipelines/727457

09:27 <aleksander> "manifest unknown"

09:27 <aleksander> any idea how to solve this?

09:29 <aleksander> daniels bentiss broken registry cache like last time?

09:30 <bentiss> looks like it

09:30 <jadahl> or make a naive assumption that we only check symbol conflicts if its a statically linked executable

09:30 <jadahl> oops, sorry

09:32 <bentiss> aleksander: https://gitlab.freedesktop.org/cmeerw/libmbim is "internal", is that correct?

09:32 <bentiss> the registry cache is probably not able to retrieve the tag because you need to be authenticated for that

09:52 <daniels> bentiss: done now, thanks!

09:52 <bentiss> daniels: great thanks!

10:11 <aleksander> bentiss, internal? you mean that user cloned the repo privately?

10:11 <bentiss> aleksander: almost privately, it's "internal", not public

10:11 <bentiss> only authenticated users can have a look

10:11 <aleksander> ah I see

10:12 <aleksander> is there an easy way to force merge without pipelines passing by using the web UI?

10:12 <bentiss> it's in your project settings

10:13 <bentiss> if you need I can turn the public setting on on that particular repo, but I don't know if this was intentional or not

10:13 <aleksander> nah nah, don't worry

10:13 <aleksander> I'll merge manually

10:13 <bentiss> worse case, you pull the MR locally, merge locally, and push to main, it will mark the MR as merged

10:14 <aleksander> yep yep

10:14 <aleksander> thanks

10:14 <bentiss> no worries

11:25 <DavidHeidelberg[m]> bentiss: You helped me a lot! Thank you

11:28 <DavidHeidelberg[m]> mupuf: we could eventually keep them in RAM _but_ the issue is running in parallel ~ four ~ 300M traces which are brotli compressed (so decompressed in memory it takes huge amount) so we would meet OOM. For some targets we already have to decrease parallelism because memory constraints

11:29 <mupuf> yeah, I am not advocating for that, I was just curious :)

11:29 <mupuf> And I wish our traces were 300M big :D

11:30 <mupuf> Well, *some* are, but most are measured in GB

11:33 <DavidHeidelberg[m]> We have usually one frame traces :D for restricted some are > 1G

11:40 <DavidHeidelberg[m]> mupuf: oh, you reminded me zram ;-)

11:41 <mupuf> DavidHeidelberg[m]: this may help, indeed :)

23:25 <DavidHeidelberg[m]> bentiss: so good tip, seems to solve the issue, now we can check for the etag and size even when resource is cached to ensure it didn't got damaged or something