07:23 <mwalle> marex: well you see, I don't have a clue ;) But from what I see from my traces, the VIVS_MMUv2_CONTROL_ENABLE is still set when the device is opened a second time and thus has a wrong mtlb address

11:29 <marex> mwalle: well you see, I don't have a clue either ;) I've been hunting this bug for like half a year now

11:30 <marex> mwalle: oh ... hmmm ?

11:30 <marex> mwalle: and the GPU was closed inbetween ?

11:34 <mwalle> marex: yes, glmark2-es-drm, strg+c, glmark2-es-drm

11:35 <marex> mwalle: do you have runtime pm enabled and does it power off the GPU after a bit ?

11:35 <mwalle> I've removed that "if (already_enabled) return 0;" check, and then my smmu errors are at least gone. but then I still get

11:35 <mwalle> [ 34.288043] etnaviv-gpu f0c0000.gpu: MMU fault status 0x00000001

11:35 <mwalle> [ 34.294117] etnaviv-gpu f0c0000.gpu: MMU 0 fault addr 0xfff80000

11:35 <mwalle> but not fault status 1 (previously it was 2 IIRC)

11:36 <marex> mwalle: if so, that could tear down the MMU for you I think, so the test would be, turn off the glmark, wait a bit for runtime pm to kick in, _then_ try the glmark again with disabled MMU and see what happens

11:36 <marex> mwalle: hmmmmm, the vivante kernel driver has the same check for already_enabled though

11:36 <mwalle> marex: I don't think ls1028a is powering off anything, but I may be wrong. the layerscale are notoriously bad at power saving

11:38 <mwalle> marex: could you point me to the code? I've just briefly looked at https://github.com/Freescale/kernel-module-imx-gpu-viv/tree/master/ and yep is looks awful *g

11:39 <mwalle> and that test will be easy, just a moment ;) I'll have to revert my changes

11:43 <mwalle> marex: btw do you know the difference between secure mode and normal one?

11:46 <marex> mwalle: nope, sorry

12:34 <mwalle> lol

12:34 <mwalle> it works, or at least I don't trigger the error anymore

12:35 <marex> mwalle: what did you change ?

12:35 <mwalle> I've noticed that etnaviv_gpu_hw_init() is called repeatedly during the glmark2 run

12:36 <mwalle> tracing that, it turns out it comes from the PM.. no idea why, but commenting out the pm ops, it will do just fine

12:37 <mwalle> looks like now I have to dig into the PM subsystem, argl

12:38 <mwalle> marex: btw, did you try to disable the AXI caches and see if your fault will go away?

12:39 <marex> mwalle: I didn't , hmmmm , why do you ask ?

12:40 <marex> mwalle: as for etnaviv_gpu_hw_init(), could it be that's called every time a test changes in the glmark ?

12:41 <mwalle> strange enough, the ls1028a register map for the GPU doesn't really match that one of the driver, regarding the MMU, very strange

12:41 <mwalle> marex: no, its called multple times during the first test

12:44 <marex> uh :)

12:44 <mwalle> marex: (cache) well you could exclude that cache as the source of your problem

12:47 <mwalle> marex: btw what is your glmark2 score on which soc?

12:48 <mwalle> I can't really tell what is some okish value

12:48 <marex> mwalle: I would have to check, on the GC400, damn low :-)

12:48 <marex> the GC400 is the lowend of the lowend

12:49 <marex> mwalle: you might want to ask mntmn , I think the mx8mq has comparable GPU

12:49 <marex> the LS is some gc7000l, right ?

12:49 <mwalle> marex: yes

12:49 <mntmn> gc7000ul i think

12:49 <marex> so that is more like mx8mp

12:51 <mwalle> currently running "glmark2-es2-drm --off-screen" which works without any ETNA_MESA_DEBUG flags

12:52 <mwalle> oh.. not really.. "desktop" seems to break things

12:52 <marex> desktop ?

12:53 <marex> is that the flying windows with penguin demo ?

12:53 <marex> that one worked for me

12:53 <marex> the fault 00000002 happened during random tests for me, not one specific test

12:55 <mwalle> marex: no that particular GC7000 still need some patches, the workaround for now is to use ETNA_MESA_DEBUG=no_supertile,no_ts (and some MESA patches) but that is totally unrelated to the MMU fault

12:55 <mwalle> but that really degrade the performance, although I really don't have a clue what no_supertile and no_ts does ;)

13:23 <marex> mwalle: no_ts disables fast_clear , no_supertile disables I think 64x64 detiled sampling (it uses 16x16 like the older gpus I think)

14:50 <cphealy> mwalle: do we know what the capabilities of the display controller in the LS1028A are in comparison to the Vivante GPU? I'm curious if the LS1028A is a step back to i.MX6q land where the display controller does not support Vivante tiled formats so the GPU had to do a tile->linear blit after rendering.

14:51 <cphealy> i.MX6qp and i.MX8M solved this by adding detiling capabilities with the display controller but in the case of the LS1028A it's an ARM display controller, IIRC.

15:00 <mwalle> cphealy: I really don't have any clue, in fact, I'm having a hard time parsing your sentence ;)

15:01 <mwalle> cphealy: the display controller is a mali dp500

15:01 <marex> mwalle: if your scanout engine cannot sample the tiled goo buffer that comes out of the GPU directly, you have to do extra mem2mem detiling pass to convert that goo buffer to whatever linear buffer the scanout engine can sample

15:02 <mwalle> cphealy: I could have a look at the older reference manuals where they didn't redact the chapter about the display controller

15:03 <mwalle> if I'd know what I should look for

15:08 <cphealy> ohh, I wasn't aware there was an older reference manual with the Mali DP500 present. Is it anywhere on the web that you can point to?

15:09 <mwalle> cphealy: its not really available on the web

15:09 <cphealy> One thing you could do to learn more is run "modetest" on the target with the vendor BSP to see what buffer modifiers the display controller advertises. With the latest version of libdrm (2.4.107) the buffer modifier list is even human readable.... ;-)

15:13 <mwalle> btw, it seems that at least the LS1028A has only 32bit address bus connected to the GPU

15:14 <mwalle> if we can trust the RM

15:14 <mwalle> that would explain why it doesn't really work with DMA_BIT_MASK(40)

15:15 <mwalle> i wonder if the imx has a 40/48bit bus connected or if the dma addresses happens to be <32bit

15:22 <cphealy> Doesn't the LS1028A have an SMMU or something like that?

15:43 <mwalle> cphealy: yeah its got an SMMU with 48bit address translation

15:45 <mwalle> but NXP disables the iommu in their bsps.. at least for the ls1028a, so there were enough bugs to chase in the beginning

16:53 <mwalle> mntmn: how do you boot debian?

16:53 <mwalle> EFI or flash-kernel (and booti)

16:54 <mntmn> mwalle: u-boot

16:55 <mntmn> setenv bootcmd 'sf probe;sf read $load_addr 0x900000 0x20000;hdp load $load_addr 0x2000;ext4load mmc 0:1 $fdt_addr fsl-ls1028a-rdb.dtb;fdt addr $fdt_addr;ext4load mmc 0:1 $load_addr Image;booti $load_addr - $fdt_addr;'

16:56 <mntmn> mwalle: i load my dtb and kernel from sd card.

16:56 <mwalle> mntmn: so booti, I just saw you're using debian and EFI boot works with the debian installer out-of-the-box but the main os is still missing the device trees

16:57 <mwalle> so i was curious what you're using

16:57 <mntmn> mwalle: ah ok, i have never tried EFI

16:58 <mwalle> thats still a missing piece of the puzzle to have debian work out of the box, just by using te installer, like on any other x86 board ;)