17:30 <austriancoder> bl4ckb0ne: I might have some time today. I will be on holiday for the next 5 weeks and had to prep some stuff.

17:31 <bl4ckb0ne> ive sent a mail to the ML with all of the infos, i'd happily provide any file

17:32 <bl4ckb0ne> we've also noticed that the DMA address changes if it's dumped earlier rather than later

17:37 <austriancoder> DMA address will change as FE is not idel

17:38 <austriancoder> idle

17:38 <austriancoder> bl4ckb0ne: I will try my luck with your dump

17:39 <bl4ckb0ne> ty

17:40 <austriancoder> bl4ckb0ne: can you do a re-upload - https://litter.catbox.moe/agmpcv.gz is dead

17:40 <bl4ckb0ne> yup give me a minute i'll boot 6.1

17:40 <bl4ckb0ne> darn i removed the archive locally

17:44 <ManMower> I have several from the same test, I can tar one up quickly if that's useful

17:45 <bl4ckb0ne> im packing one clean from 5.15

17:45 <ManMower> I'll just sit over here and eat popcorn then. :)

17:47 <bl4ckb0ne> wanna get both desktop and terrain for the mmu fault

17:49 <bl4ckb0ne> ManMower: pack them, I was on the stock partition without the mesa cmdstream dump :(

17:53 <ManMower> here's one: https://litter.catbox.moe/fdmyvr.gz

18:00 <bl4ckb0ne> https://litter.catbox.moe/eck44w.gz and here's one on 5.15 for both terrain and desktop glmark

18:00 <bl4ckb0ne> former causes a mmu fault, latter doesnt but both hang

18:01 <austriancoder> thx

18:02 <bl4ckb0ne> ManMower and I are both investigating the same issue on the same platform

18:24 <bl4ckb0ne> hang with terrain confirmed on 6.6

18:25 <bl4ckb0ne> but the mmu fault is not there

18:26 <bl4ckb0ne> wait no its there, nevermind

18:26 <bl4ckb0ne> > [ 78.849647] etnaviv-gpu 53100000.gpu: MMU 0 fault (page not present) addr 0x00000000

18:26 <austriancoder> k

18:28 <ManMower> is that address really what the gpu is trying to read from?

18:28 <austriancoder> yes .. and there is no entry in the iommu

18:32 <ManMower> is 0 a special address that we avoid, or is there usually something mapped there?

18:38 <austriancoder> 0 is not special - the etnaviv stack is used on gpus with mmuv2 (even mesa ci has them) maybe I should fire up glmark2 on my gc7000 sitting in front of me :)

18:40 <ManMower> we're working with gc7000 6009. unsure if this problem is specific to that variant or not. :)

18:40 <ManMower> oh, I guess that 6009 is in the register dump already.

20:12 <austriancoder> bl4ckb0ne: ManMower: the iommu v2 looks okay - I have a started on a very hacky validation code for viv-unpack but this does not help for the no-mmu fault but gpu hang case. so lets focus on the gpu hang. I looked at the cmd stream you provided for the terrain and it looks sane. Can you try to run you gpu hanger with ETNA_MESA_DEBUG=draw_stall?

20:13 <austriancoder> glmark2-drm -b terrain --run-forever - is running for 15 minutes without a hang

20:14 <austriancoder> mesa 24.2.0-devel with kernel 6.7.12

20:14 <bl4ckb0ne> hangs/fault immediately with 6.6

20:17 <bl4ckb0ne> anything to share yet for the viv-unpack?

20:18 <austriancoder> bl4ckb0ne: the iommu v2 code I am working on is not needed to find the bad part in the dump - it just verifies that every bo is actually mapped in the iommu

20:18 <austriancoder> a devcoredump with ETNA_MESA_DEBUG=draw_stall would be nice

20:19 <bl4ckb0ne> ill send you that in a few minutes

20:19 <austriancoder> great

20:19 <bl4ckb0ne> with or without mesa cmd stream?

20:19 <austriancoder> without

20:20 <austriancoder> the devcoredump includes the cmd stream of the job that hangs

20:20 <bl4ckb0ne> a preference on terrain or desktop?

20:21 <bl4ckb0ne> or both

20:21 <ManMower> does it always? I've seen devcoredumps where it looked like the dma address pointed at a previous command buffer. (or even in the ring buffer) is the dma address not particularly useful?

20:21 <austriancoder> it does not matter which one

20:22 <austriancoder> ManMower: the FE can either be in the kernel ring buffer or in a user provided command buffer

20:23 <bl4ckb0ne> https://litter.catbox.moe/assl57.gz there you go

20:24 <austriancoder> ETNA_MESA_DEBUG=draw_stall adds a sync between FE and PE (start and end of the pipeline) so we can easier pinpoint which draw command is the bad one

20:30 <bl4ckb0ne> how are you able to find the 905/903 address in all of this?

20:37 <ManMower> haven't been able to light up display on 6.1 yet, testing with glmark2 --off-screen there