ChanServ changed the topic of #asahi-gpu to: Asahi Linux: porting Linux to Apple Silicon macs | GPU / 3D graphics stack black-box RE and development (NO binary reversing) | Keep things on topic | GitHub: https://alx.sh/g | Wiki: https://alx.sh/w | Logs: https://alx.sh/l/asahi-gpu
chengsun has joined #asahi-gpu
chengsun_ has quit [Ping timeout: 480 seconds]
<chadmed>
the russians arent content with just invading ukraine, they want our IRC channels too
<Tramtrist>
hhaah
PhilippvK has quit [Ping timeout: 480 seconds]
phiologe has joined #asahi-gpu
user982492 has joined #asahi-gpu
chadmed has quit [Remote host closed the connection]
user982492 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<phire>
but there is nothing actually mapped at 0x18000 in the gpu VA
<phire>
however, userspace tracing shows a region is mapped with the the VA of 0x18000
<phire>
so there is this weird, extra address space? that apparently only pipeline defintions live in?
mrkajetanp has joined #asahi-gpu
<phire>
it's only the pipeline defintion. That references straight back to regular gpu VAs
<marcan>
phire: interesting. maybe it's some kind of control block that is mapped separately?
<marcan>
also, are we sure AGX firmware can alter the page tables? I'm going to have to include that info in my tech doc since I've been saying proprietary blobs don't have full RAM access and that seems to be an exception
<marcan>
but I also wouldn't be surprised if Apple did *something* and in fact it's more restricted
<marcan>
e.g. it could be that the reason for that special VA range is because that's the one thing AGX firmware *can* access
<marcan>
(just a random guess)
<phire>
Well, it populates it's own pagetables for the 0xf8000000000 range when you boot it
<marcan>
and those just point to regular RAM?
<marcan>
or to the GFX special regions?
<marcan>
e.g. it could be that the UAT/GART filters the high range/TTBR to only allow translation to stuff within a specific carveout region
<phire>
They cover the agx-asc firmware blob that iboot put in main ram, and the IO range for agx-asc's registers
<marcan>
where are the page tables themselves?
<phire>
main ram. The L0 page is at gpu-region-base, kernel points L0[1] at gfx-shared-region-base
<phire>
the gpu uses gfx-shared-region-base for it's page tables
<phire>
I haven't fully investiated the permission bits. It's possible the pagetables are setup in such a way that the firmware can't modify them once booted
<marcan>
what region is the agx init struct in, do you know?
<marcan>
(the one passed to the first/whatever command)
<phire>
it's a virtual address, after the pagetables are already initilized
<marcan>
yes, but what vaddr range is it in, and what paddr does it point to?
<phire>
it's in the 0xfa0_00000000 vaddr range (everything the macos kernel passes into it is in that range)
<phire>
and I think it's just a regular physical memory address
<marcan>
hm, yeah, I don't see how that would work based on VA ranges, but it could still be what you say about the PTs getting locked
<marcan>
or rather, the PTs themselves are mapped with flags that might mean "RO from ASC"
<marcan>
so only the initial mechanism to populate the PTs would have to be locked off
<phire>
some of the log messages I've seen talk about a "microPPL"
<marcan>
there's also the existing VMSA lockdown stuff
<marcan>
their cores already have the ability to lock translation on
<phire>
interesting. I get the feeling we are going to have to RE this blob just so we know it's security model
<phire>
Ah... found the Pipeline data I was missing
<phire>
it lives at VA 0x1100018000
<marcan>
possibly a per-context VA prefix is involved?
<marcan>
that would make sense
<phire>
yeah, there is an L0 table for every context
<phire>
they all share the same L0[1] ptr
<marcan>
but yeah, if the AGX firmware maps its own PTs read-only, and then VMSA locks itself, early in init, that's all the code we need to audit to say "no backdoor here"
<marcan>
and doing it that way is quite Apple-like
<phire>
each L0 is 16 bytes, so they sit next to eachother at gpu-region-base
<marcan>
ah, right
<phire>
at this point I'm going to have to start on a m1n1 agx driver
<phire>
I understand close to enough to launch a compute kernel
<marcan>
in python right?
<phire>
yes
<marcan>
that would be awesome
<marcan>
btw, stringsing the kexts, they seem to call the PTs UAT and GART somewhat interchangeably
<marcan>
I'm guessing UAT is the current generation of GART or so
<phire>
yes. UAT seems to mean Unified Address Translater, So I'm guessing there used to be a seperate GART and DART
<phire>
but they got combined
<marcan>
yeah
<marcan>
makes sense
<marcan>
also I wonder how this works together with the ARM's actual page tables
<marcan>
i.e. does this sit entirely outside of the ARM's own MMU?
<phire>
I think it might be a modified ARM MMU
<marcan>
maybe they run in parallel?
<phire>
ponting at the same data?
<phire>
well, same pagetables
<marcan>
yeah, like maybe the ARM's actual sysreg TTBR1 gets pointed to the same L1[0] and that all uses the standard VMSA lockdown stuff
<marcan>
and then it's possible the ARM never actually has to R/W data in the lower half of the address space directly
<marcan>
we have rtkit-private-vm-region-base which points to what very much looks like a typical ARM top half VM address space base
<phire>
Everything I've seen points to a unified address space
<phire>
yeah, rtkit-private-vm-region-base is 0xf80_00000000 to 0xf8f_ffffffff
<marcan>
more Fs
<marcan>
0xffffff8000000000
<phire>
it gets truncated to 40 bits
<marcan>
that could be PAC?
<phire>
it just seems to be sign extention of bit 39
<marcan>
also I see it going to f8..fa
<marcan>
and yes, that's how the PAC truncation works
<marcan>
you sign extend at some bit
<povik>
11:38 < phire> I understand close to enough to launch a compute kernel
<povik>
exciting
<marcan>
phire: yeah, the ARM's TTBR gets pointed straight at gfx-shared-region-base
<marcan>
looking at the firmware init a bit
<marcan>
so these should be ARM format page tables
<phire>
it doesn't seem to be far off, from what I can tell. But I don't know the ARM MMU very well
<phire>
It gets pointed at gfx-shared-region-base, and not gpu-region-base?
<marcan>
gpu-region-base is the TTBRs, right?
<marcan>
the ARM does not have that PT level, TTBRs are sysregs
<marcan>
the firmware calls gpu-region-base "UAT GPTBAT" fwiw
<phire>
so it just copies the two entires from gpu-region-base into into TTBR0 and TTBR1?
<marcan>
not from gpu-region-base; those are patch args to the firmware
<phire>
I guess if the ARM core never touches the lower address range, it doesn't need to care about the gpu contexts and handle updates to gpu-region-base
<marcan>
yeah
<marcan>
phire: it does have a function to switch TTBR0 around to different contexts
<marcan>
the cool thing is the GXF stuff somehow has symbols here
<marcan>
and by the nature of it, if it can guarantee those PTs never point to crazyland, then all we need to do is audit the GXF code (which is one page) to convince ourselves that there is no backdoor
<marcan>
I think that's good enough for the "no, really, Apple don't have a backdoor here" argument
<phire>
From what I can see, context switching is controlled by second field in work submission commands.
<phire>
sometimes the 3rd field, for the compute work submission
<sven>
iirc GXF also has (had?) some symbols in some XNU kernel
<sven>
but then they were removed the next released again
<sven>
*release
<sven>
and then there was ofc that one kernel with llvm bitcode released under the open source license. that should contain full symbols and then some for GXF as well :D
<phire>
if GXF is so small, could we rewrite it, and then graft the rest of apple's firmware onto it?
<sven>
i'm not sure i understand that question
<sven>
GXF is what apple called "guarded exception level". it's a lateral level to EL2 called GL2 with a different interpretation of pagetable permissions and usually the only code that gets to modify pagetables
<phire>
What I'm wondering is could we take the bulk of apple's gfx-asc firmware, but replace the GFX level code with our own code?
<sven>
ah
<phire>
as an alternative to auditing
<sven>
so we can't easily replaced the firmware
<sven>
*replace
<sven>
it's in a region of RAM that's write protected
<Jamie[m]1>
and it's loaded early in boot, right?
<phire>
oh, that region is write protected?
<sven>
i think apple calls that CTRR. iBoot loads the firmware of all co-procs into a memory region and setups CTRR to write protect it
<sven>
they do the same for XNU but that protection can be disabled in 1TR
<phire>
didn't we have to disable CTRR to boot m1n1, or am I thinking of something else?
<sven>
almost!
<sven>
they have two CTRR regions
<sven>
one for the kernel and one for the co-processor firmware
<sven>
we can only disable the first one
<phire>
Does that mean since we have control over the pagetables, we could hijack all of the agx-asc firmware except for the GFX level?
<sven>
i didn't follow the pagetable discussion, but I think that if you control whatever TTBR0/1 points to you can even hijack the GL2 code
<sven>
yeah. if we can modify the full pagetable we can just map the firmware (or parts of it) somewhere else
<sven>
(including the code running in GL2)
<sven>
hrm... unless they also limit the physical addresses that are allowed to contain code like they do for the XNU kernel
<phire>
though, that might not enough to satisfy the people conserned about backdoors. It would be a racecondition to replace the pagetables before too much of the agx-asc firmware runs
<marcan>
hmm... I don't think this MiniPPL is very well done
<marcan>
as far as I can tell it reads the GPTBAT base from a random global that is not in a PPL-only write protected page?
<marcan>
and then reads the TTBR0 from that for context switches
<marcan>
so you can just redirect TTBR0 to any page table you want by changing the GPTBAT base
<marcan>
unless I'm confused about the MMU maps
<marcan>
the page tables are indeed readonly though, AFAICT
<marcan>
but yeah, without protecting PPL data, this is a bit pointless isn't it
<phire>
awkward
<sven>
and i guess that GPTBAT pointer is not inside the CTRR firmware region either?
balrog has quit [Quit: Bye]
balrog has joined #asahi-gpu
yuyichao has quit [Ping timeout: 480 seconds]
yuyichao has joined #asahi-gpu
<phire>
well, I just found what might be the gptbat pointer being passed in with the initdata
<phire>
this initdata is a mess
<phire>
I zeroed out the region I had labled as "heap", and I got an "Instruction Abort" crashlog
<marcan>
sven: it's data
<marcan>
phire: probably jumping to a 0 function pointer
<marcan>
we know the firmwares are quite pwnable, the bigger question is what is the backdoor surface and whether Apple are going to harden the PPL stuff to the point where we can say we don't need to audit the whole firmware
<sven>
:/
user982492 has joined #asahi-gpu
nafod has quit [Quit: Ping timeout (120 seconds)]
nafod has joined #asahi-gpu
balrog has quit [Remote host closed the connection]
balrog has joined #asahi-gpu
MajorBiscuit has quit [Ping timeout: 480 seconds]
user982492 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
user982492 has joined #asahi-gpu
Gaspare has joined #asahi-gpu
boardwalk has quit [Quit: Ping timeout (120 seconds)]
boardwalk has joined #asahi-gpu
Gaspare has quit [Read error: Connection reset by peer]
user982492 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
VinDuv has quit [Remote host closed the connection]