ChanServ changed the topic of #asahi-dev to: Asahi Linux: porting Linux to Apple Silicon macs | Non-development talk: #asahi | General development | GitHub: https://alx.sh/g | Wiki: https://alx.sh/w | Logs: https://alx.sh/l/asahi-dev
deteg1337 has joined #asahi-dev
zzywysm has joined #asahi-dev
nsklaus has quit [Quit: ZZZzzz…]
deteg1337 has quit [Ping timeout: 480 seconds]
hightower3 has joined #asahi-dev
hightower2 has quit [Ping timeout: 480 seconds]
A_L_I_C_E has joined #asahi-dev
amw has joined #asahi-dev
maria28547 has joined #asahi-dev
maria2854 has quit [Ping timeout: 480 seconds]
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
i509vcb has quit [Quit: Connection closed for inactivity]
Graypup__ has quit [Quit: meow]
Graypup_ has joined #asahi-dev
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
A_L_I_C_E has quit [Ping timeout: 480 seconds]
chipxxx has quit [Remote host closed the connection]
chipxxx has joined #asahi-dev
A_L_I_C_E has joined #asahi-dev
chipxxx has quit [Remote host closed the connection]
chipxxx has joined #asahi-dev
A_L_I_C_E has quit [Remote host closed the connection]
chipxxx has quit [Remote host closed the connection]
chipxxx has joined #asahi-dev
Graypup_ has quit [Quit: meow]
Graypup_ has joined #asahi-dev
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
A_L_I_C_E has joined #asahi-dev
rpirea has quit []
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
___nick___ has joined #asahi-dev
deteg1337 has joined #asahi-dev
hightower3 has quit [Read error: Connection reset by peer]
deteg1337 has quit [Ping timeout: 480 seconds]
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
chadmed has quit [Remote host closed the connection]
deteg1337 has joined #asahi-dev
nsklaus has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
<marcan>
why can't MTP allocate DMA memory... is something wrong with 42-bit t8110? :/
<marcan>
Failed to alloc io_pgtable_ops
<marcan>
really...
<marcan>
ohh wait. IAS is 42. I don't think we've ever seen that.
<marcan>
does that mean these DARTs have one more level?
<marcan>
I mean I guess they have to...
<jannau>
is that the first dart you're looking at?
<marcan>
it might be the first one that has to work, since dwc3 as far as I know knows how to fall back to bypass?
<marcan>
all the DARTs are 42->42
<marcan>
and the only other thing using a DART is PCIe which is broken
<marcan>
FOUR_LEVELS=1
<jannau>
the dcp/disp0 handling in kboot.c isn't used yet?
<marcan>
no, I haven't enabled that yet
<marcan>
I see three levels though, so that bit sure feels like a lie
<jannau>
ok, that should fail then too and probably explains the SError OI saw yesterday on the m2 pro mini
<jannau>
I suspect the first level is the ttbr which is flat for dart-t8110
<marcan>
yeah
chadmed has joined #asahi-dev
<jannau>
dart-dcp0 has FOUR_LEVELS=0
chipxxx has quit [Remote host closed the connection]
<marcan>
it's probably optional if you don't need that much address space
chipxxx has joined #asahi-dev
<marcan>
but the first problem is the dart code just gives up if ias is too large right now
<jannau>
does it work if you limit it to 36 instead of reading DART_T8110_PARAMS3_VA_WIDTH?
chipxxx has quit [Remote host closed the connection]
chipxxx has joined #asahi-dev
chipxxx has quit [Remote host closed the connection]
<jannau>
we should take's the device's dma mask into account in apple_dart_finalize_domain(), I doubt we need more than 36-bit for most devices
chipxxx has joined #asahi-dev
<marcan>
probably, but MTP is actually using funny addresses in macOS at least
<marcan>
the HID descriptors look fine in the Python dump so I wonder what happened here, maybe just a bug
Dementor1 has joined #asahi-dev
Dementor3 has joined #asahi-dev
ChaosPri1 has joined #asahi-dev
Dementor2 has joined #asahi-dev
Dementor has quit [Ping timeout: 480 seconds]
Dementor2 is now known as Dementor
Dementor1 has quit [Ping timeout: 480 seconds]
<jannau>
sigh, no idea what "vm-base" is. in the m2 pro adt it looks more like ASC DRAM offset. I guess that still holds for the m2 except that it is so small that it already affects the iova in the dart
Dementor3 has quit [Ping timeout: 480 seconds]
<marcan>
jannau: for MTP, macOS is using an offset but that doesn't work for me (it seems to put the full address through the DART). I think the DAPF config may be wrong in that case.
<ChaosPri1>
marcan: you told me to poke you about fpwm dts, so, if you have time, here is the poke :P
<marcan>
ack, I'll just pull them directly into my branch
<jannau>
I wonder where iboot gets the memory for framebuffer page tables
<jannau>
laso what's up with the single page before the framebuffer mapped directly behind it
MajorBiscuit has joined #asahi-dev
<kettenis_>
marcan: the performance_domain in arch/arm64/boot/dts/apple/t602x-common.dtsi is a typo isn't it?
<kettenis_>
(should be performance-domain)
<marcan>
yes, that's from chadmed :p
<marcan>
fixing
<marcan>
this is interesting, with the DART ias limit patch using that high bit in MTP addresses works,but not without it, but macOS does use ias=42 for this
<marcan>
something else is going on
<jannau>
0x100_0000_0000 fits into 42-bit ias, so you have map the memory at that offset. that seems to be what happens with dcp on m2
<marcan>
ah, I'm dumb, macOS does map it there
<marcan>
I read the dart dump wrong
<marcan>
yeah ok
<marcan>
jannau: how do we deal with that? is there a way to specify the DMA address range?
<marcan>
I know there's a thing in the dart, but is there one for the device? or should this be in the dart?
<marcan>
jannau: I get the feeling vm-base and vm-size are that (the VM range you're supposed to use for iovas/dvas), except when the dart IAS doesn't fit it the top bits just get truncated off
<marcan>
(both at its input and in the page tables of course)
<marcan>
looks like this is only settable at the iommu, so just like macOS, it has to go in the DART...
<jannau>
I think I deal only explicitly in m1n1 with it
<jannau>
only change in linux was to use the dart's ias and rely on linux to allocate iova from the top
<marcan>
yeah that's... not ideal :/
<jannau>
if we want to deal with it properly in linux we have to set domain->geometry.aperture_{start,end} based on vm-base and vm-size
<marcan>
yeah, we could also use dma-ranges but that only works on buses and it feels like very much the wrong abstraction
<marcan>
let's do that
<jannau>
at least vm-size seems to be only a suggestion, we have so far not encoutered a device which fails with top of address space iovas
<jannau>
but most devices are limited to 32-bit. we have to update drivers to not use dma_set_mask...(32);
<marcan>
yeah
<jannau>
finally, dcp works to the point where it detects the missing display
<jannau>
marcan: if you want to give dcp a spin ^^^
<jannau>
works here up to "apple-dcp 389c00000.dcp: dcp_hotplug: connected: 0"
deteg1337 has joined #asahi-dev
deteg1337 has quit [Remote host closed the connection]
deteg1337 has joined #asahi-dev
siglazin has joined #asahi-dev
i509vcb has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
siglazin has quit [Remote host closed the connection]
deteg1337 has joined #asahi-dev
deteg1337 has quit [Remote host closed the connection]
deteg1337 has joined #asahi-dev
deteg1337 has quit [Remote host closed the connection]
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
deteg1337 has joined #asahi-dev
<marcan>
jannau: pushed and added an apple,dma-range thing, hopefully it does what you want for DCP and we can get rid of the DRAM mask story.
<marcan>
I really need to get some dinner and sleep now :)
<marcan>
MTP works as intended now with the high DVAs
<marcan>
will test DCP tomorrow
<marcan>
robher: what's the best practice for DMA addresses in the DT? use the device cell sizes, or hardcode it to 64-bit always, or something else? (so kettenis_ doesn't hate me when we change the bindings when upstreaming)
<marcan>
this is a property for IOMMU device nodes to specify the allowable DMA address range, I have it as two u64s <base size> right now.
<marcan>
not sure if it makes sense to make it generic, in principle other IOMMU drivers could easily support this but I'm not sure if they care
<marcan>
it's somewhat questionable whether this is a property of the IOMMU or the device, since it's really a property of how the IOMMU interacts with the device's view of memory (and we don't even know exactly how that is implemented)
<marcan>
Apple puts it in the IOMMU and on Linux the IOMMU is the only easy place for this to be set, since that driver controls the aperture fully (devices can only set a DMA mask)
<marcan>
there's the dma-ranges stuff that can also limit this, but that's for buses and this isn't a bus...
<marcan>
I think it also makes sense to put it in the IOMMU because we have a special case where that range can exceed the IOMMU's own input range, and in that case it is assumed the high bits are dropped
<marcan>
that wouldn't work properly if we fully decoupled it from the IOMMU driver
deteg1337 has quit [Ping timeout: 480 seconds]
<marcan>
wait no I think I messed this up
<jannau>
looks like it should work in principl, i.e. review of the intention
deteg1337 has joined #asahi-dev
<marcan>
jannau: the problem is the device DMA mask is not set yet, so we get 32bit. the iommu stuff happens too early.
<marcan>
I don't think there's a solution for this, let's just drop that idea... setting the DMA range at the DART will already do the right thing wrt clamping IAS anyway.
<jannau>
ah, same problem for my patch to limit ias then. drop that as well
<marcan>
yeah, I just dropped it and fixed the conflict
<marcan>
[ 0.630506] specified DMA range outside IOMMU capability
<marcan>
lovely...
<marcan>
same check, happening in the IOMMU core, similarly way too early
Dementor has quit [Remote host closed the connection]
deteg1337 has quit [Ping timeout: 480 seconds]
Dementor has joined #asahi-dev
<kettenis_>
btw:
<kettenis_>
./t600x-j375.dtsi:13.10-24.4: Warning (alias_paths): /aliases: aliases property name must include only lowercase and '-'
<kettenis_>
that'll need to be fixed before it goes upstream
<jannau>
that suggests we should use "dma-ranges" see of_dma_configure_id()
<jannau>
kettenis_: disp0_piodma. I'll fix it
<kettenis_>
that needs a fix in m1n1 as well isn't it?
<jannau>
yes
Dementor has quit [Remote host closed the connection]
Dementor has joined #asahi-dev
<marcan>
yeah, though now I'm wondering if this really makes sense
<kettenis_>
there are a few more warnings, but those have less impact
<marcan>
robher: this codepath is getting called before the driver probes, so the DMA mask is unset, which means we end up limiting everything to 32bit. I see that if we set dma-ranges to a dummy single covering the entire 64bit AS, we'll probably get what we want... but is this what we're supposed to do?
<kettenis_>
I thought the default was that there are no dma restrictions unless there is a dma-ranges property
<marcan>
yeah, that'd make sense logically, which is why I think this is a workaround and this kernel codepath is just broken
<marcan>
going to do this for now pending robher's response
<marcan>
robher: the call chain here is really_probe -> platform_dma_configure -> of_dma_configure_id. that ends up pulling a default 32-bit DMA mask since the device driver probe() hasn't run yet. that seems wrong.
<marcan>
also I think the reason nobody noticed this yet is that the subsequent arch_setup_dma_ops -> iommu_setup_dma_ops codepath only uses the size for a sanity check, it doesn't keep it anywhere. so it will pass as long as IOMMUs will at least *support* the 32-bit DMA range, and we're probably the first one that doesn't when we specify a restricted high aperture.
<robher>
marcan: a default greater than 32-bit mask would break lots of things.
<marcan>
robher: ^^
<marcan>
the problem is not the default, it's that the OF code is running before that default can be overridden and then eventually that codepath ends up in a sanity check that we fail
<marcan>
(only, that's all the max is used for, which is why this hasn't actually broken everyone by *forcing* 32bit DMA for everything)
<marcan>
put another way: of_dma_configure_id() is doing something fairly silly, taking the device DMA mask as a default size (or 32bit if not set), then clamping the DMA mask to that again
A_L_I_C_E has quit [Read error: Connection reset by peer]
<marcan>
IMO the default size in that function should be 64-bit on 64-bit platforms period; the actual dma_mask can remain 32-bit by default of course
A_L_I_C_E has joined #asahi-dev
<robher>
marcan: generally, if you want anything other than all 32bit or all 64bit, then you need to use dma-ranges.
<marcan>
how do you get "all 64bit"?
<robher>
As for the specifics in of_dma_configure_id(), that's a conversation to have with Robin who is not here.
<marcan>
but also isn't that kind of backwards? what I want is drivers to be able to set their own DMA mask and OF not get in their way by clamping it further
<robher>
A device that supports >32bit should set their mask to what they support (at the device, ignoring bus level restrictions)
deteg1337 has joined #asahi-dev
<robher>
With an iommu in the middle, I don't know if you can get all 64-bits.
<marcan>
yes, but that isn't working because that OF codepath is calling arch_setup_dma_ops/iommu_setup_dma_ops with a 32bit mask *before* that can happen
<robher>
by default
<marcan>
and that breaks because those ops see that the device is asking for 32 bits and the IOMMU *cannot support that* (because its range is restricted to only a higher aperture than the low 32 bits)
<jannau>
I think we ought to set dma-ranges although it's counterintuitive since the iommu dt bindings say it will be ignored if we use the iommu
<marcan>
heh it's actually impossible to declare all 64-bit AS as dma-ranges because 1<<64 does not fit in a u64
<marcan>
and dma-ranges = <0 0 0 0 0xffffffff 0xffffffff>; does not work because it overflows then fails the zero check
<marcan>
(since it gets adjusted)
<marcan>
dma-ranges = <0 0 0 0 0xffffffff 0xfffffffe>; I guess? This feels silly.
<marcan>
we shouldn't be working Linux silliness in the device tree
<marcan>
*working around
<robher>
marcan: I think for 32-bit sizes we handle that case (DT with ~0 size).
<marcan>
you do (with a warning), but for 64bit it overflows the size and breaks
<robher>
marcan: Again, talk to Robin as he understands the IOMMU interaction much better than I do.
<marcan>
ack
deteg1337 has quit [Ping timeout: 480 seconds]
<marcan>
*finally* it works with that
<marcan>
jannau: pushed
<marcan>
I hope I didn't break anything
<marcan>
now I really need to sleep :p
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
deteg1337 has joined #asahi-dev
linuxgemini13 has quit []
linuxgemini13 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
c10l has quit [Quit: Bye o/]
A_L_I_C_E has quit [Ping timeout: 480 seconds]
c10l has joined #asahi-dev
bcrumb has joined #asahi-dev
linuxgemini13 has quit []
linuxgemini13 has joined #asahi-dev
bcrumb has quit [Ping timeout: 480 seconds]
deteg1337 has joined #asahi-dev
A_L_I_C_E has joined #asahi-dev
Theincognitoman has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
A_L_I_C_E has quit [Ping timeout: 480 seconds]
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
deteg1337 has joined #asahi-dev
<kettenis_>
pci0 at aplpcie0
<kettenis_>
ppb0 at pci0 dev 0 function 0 "Apple M1 PCIe" rev 0x01
<kettenis_>
pci1 at ppb0 bus 1
<kettenis_>
vendor "Broadcom", unknown product 0x4434 (class network subclass miscellaneous, rev 0x04) at pci1 dev 0 function 0 not configured
deteg1337 has quit [Ping timeout: 480 seconds]
<kettenis_>
vendor "Broadcom", unknown product 0x5f72 (class network subclass miscellaneous, rev 0x04) at pci1 dev 0 function 1 not configured
<kettenis_>
ppb1 at pci0 dev 2 function 0 "Apple M1 PCIe" rev 0x01
<kettenis_>
pci2 at ppb1 bus 3
<kettenis_>
vendor "Aquantia", unknown product 0x04c0 (class network subclass ethernet, rev 0x03) at pci2 dev 0 function 0 not configured
<kettenis_>
ppb2 at pci0 dev 3 function 0 "Apple M1 PCIe" rev 0x01
<kettenis_>
pci3 at ppb2 bus 4
<kettenis_>
xhci0 at pci3 dev 0 function 0 "ASMedia ASM2142 xHCI" rev 0x00: msi, xHCI 1.10
<kettenis_>
usb0 at xhci0: USB revision 3.0
<kettenis_>
so it seems they just moved the registers around a bit
<kettenis_>
in particular, the CORE_LANE_* registers
<kettenis_>
those used to start at offset 0x84000 in the "rc" register region
<kettenis_>
but they know start at offset 0xc000 in the region that startx at 0x59e000000
<kettenis_>
s/know/now/ s/startx/starts/
___nick___ has quit [Ping timeout: 480 seconds]
Theincognitoman has quit [Quit: Leaving]
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
c10l has quit [Quit: Bye o/]
MajorBiscuit has quit [Quit: WeeChat 3.6]
c10l has joined #asahi-dev
<jannau>
kettenis_: is that the only change? seems not be enough for the linux driver and dt
<jannau>
for backward compatibility: append the region and access it via reg-names. if it is absent compute a base from "rc" so that the offsets are equal
<kettenis_>
that is the only change I made to the OpenBSD driver
<kettenis_>
I don't have the darts yet though, so devices don't work
deteg1337 has joined #asahi-dev
<kettenis_>
but then I don't have drivers for most of the PCIe devices in OpenBSD
<kettenis_>
and presumably the asmedia xhci needs firmware to work
cylm_ has quit [Ping timeout: 480 seconds]
<jannau>
it timeouts on getting CORE_RC_PHYIF_STAT_REFCLK back from CORE_RC_PHYIF_STAT
deteg1337 has quit [Ping timeout: 480 seconds]
<jannau>
which isn't use in in the openbsd driver. links do not come up with that removed
<kettenis_>
I can access config and mmio space on the devices, so the link must be up in my case
A_L_I_C_E has joined #asahi-dev
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
siglazin has joined #asahi-dev
<richyliu2>
what's the status on external display support on m1 mbp? anything i can help with?
<jannau>
richyliu2: work in progress. works as proof of concept. the main issue appears to be the integration with the typec framework and wonkyness of the dwc3 xhci part
<jannau>
vm-base for dcp/disp0 is not used by iboot/macos and the reserved regions / locked dart do not work with 4 levels so it's using an aperture of 0, vm-size (2**36)
<jannau>
the aperture from dt change misses to limit ias based on dma_max, fixup included
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
deteg1337 has joined #asahi-dev
deteg1337 has quit [Ping timeout: 480 seconds]
chadmed_ has joined #asahi-dev
<chadmed_>
marcan, kettenis_: oof sorry about that, i guess nothing complained about it when i booted because it wasnt wired up :p
deteg1337 has joined #asahi-dev
siglazin has quit [Remote host closed the connection]