<rqou_>
works well enough to encode a single frame with hardcoded settings
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
<kameks>
Hey there
<kameks>
I bought a mac to develop an iOS app, but at the same time, as a graphics programmer I'm very much interested to tinker with GPU RE
<kameks>
one thing I'm wondering, is whether I risk being banned from access to iOS sdk by doing so
<kameks>
I mean, the iOS SDK license forbid RE, but does it also apply to mac as well ? Or do you think Apple would be more relax about it ?
alcazar has joined #asahi-re
<kameks>
I know you guys are not Apple's lawyers, but I'm asking for your opinion
alcazar has quit [Ping timeout: 480 seconds]
<dottedmag>
kameks: This is so country-specific that any opinion will be at best worthless and at worst harmful
alcazar has joined #asahi-re
<kameks>
I live in Japan, and from what I understand, so does the lead of the Asahi Linux project. My concern is not so much about being sued, but more about Apple's openness to this, provided we do follow the RE best practices
<kameks>
as even if I do not break any law, Apple still has the right to terminate my access to the iOS SDK for whatever reason.
<kameks>
It's more a question about historically, how Apple has handled mac tinkerers
<kameks>
For instance, they haven't locked down boot on M1 macs. So my understanding is that they expect people will try to run other software, and eventually need to RE to get proper drivers
<kameks>
even if they won't help, because they don't want to spend resources, and want macos to be the os of choice
alcazar has quit [Ping timeout: 480 seconds]
<kameks>
is there stance something like "we don't want to help linux run, we don't want to publish our internal documentations because it also contains trade secrets and we don't want to spend time for a shippable documentation. but if people are going to do the work for us for free, sure why not?" or do you think I really risk being kicked out of the access to iOS SDK and should refrain from participating?
alcazar has joined #asahi-re
amarioguy has joined #asahi-re
<chadmed>
it really is country specific but RE is not illegal and generally clauses in EULAs saying that they will ban/sue/kill you or remotely brick your device for doing it are wholly unenforceable
<sven>
they can pretty much decide to stop doing business with you for whatever reason
<chadmed>
if youre really worred about your apple developer account, use a pseudonym when contributing RE
alcazar has quit [Ping timeout: 480 seconds]
amarioguy has quit [Ping timeout: 480 seconds]
amarioguy has joined #asahi-re
<chadmed>
sven: if you really, really, really cared about access to your apple id, you could take them to court in most common law countries over this. if you can prove they nixed your account because of RE you have a case
<sven>
not convinced about that, but i'm also not a lawyer
<chadmed>
the US is very permissive about this actually, the EU and Australia not so much but if you can demonstrate that you were REing the for the purposes of "interoperability" (in this instance, interfacing one piece of software/hardware with another) then youre clear
<sven>
there's a difference between criminal with civil law
<sven>
s/with/and/
<chadmed>
yeah this applies to the interpretations of RE clauses in EULAs
alcazar has joined #asahi-re
<kameks>
to be clear, I don't want to get involved with a lawsuit with Apple. Even if I have a case, I don't have time, money and courage to fight one of the biggest corporate
<chadmed>
i mean idk why anyone would take apple to court over losing access to an apple id, but hey you could do it and (assuming you have unlimited money) may even win
<kameks>
it's really about do you think apple cares or not so much as long as we don't do anything harmful to their IP
<chadmed>
yeah ok my point is merely that its not really "illegal" to do it, and the risk of them actually taking any action against you is probably extremely low
<chadmed>
unless youre outright copying code and implementations and publishing it all in your capacity as an apple-associated developer
<chadmed>
which, of course, doing the former will just get you banned from here and also probably sent to jail anyway
<chadmed>
and doing the latter is just silly
<kameks>
"i mean idk why anyone would take apple to court over losing access to an apple id" -> Epic did, but that's a whole other story, and Apple will fight tooth and nails for iOS
<chadmed>
i believe our current apple psychoanalys on this goes like: a) we want to sell apple silicon macs. b) we do not want to spend money porting linux to the macs. c) a non-trivial minority of people find these machines highly desirable, IFF they can run linux. d) there are people willing to port linux to our machines for free. e) this will sell more macs. f) let these people have their fun so we sell more macs
<kameks>
"unless youre outright copying code and implementations" -> that's copyright infringement, so yeah, definitely not okay
<chadmed>
well yeah we dont want anything to do with that regardless (its also not fun and not rewarding so why do it :P)
alcazar has quit [Ping timeout: 480 seconds]
<kameks>
I read Alyssa's blog posts about GPU RE, I'm not sure what's the exact status as of today. What I'd like to do is write a raw-level API for M1 GPU, similar to what we have on certain game consoles
alcazar has joined #asahi-re
<chadmed>
alyssa's mesa driver is only the userspace half, so the part that takes the opengl shaders and compiles them into AGX instructions. we still rely on the macos kernel to submit those to the gpu via the firmware. Asahi Lina is working on reverse engineering the interface for that so we can do it in Linux
<kameks>
On these consoles, they also have a higher level API, built on top of the low level API. That higher level API is fully open source (to licensed developers), and it is a bliss to work with something with no hidden pieces
<chadmed>
do consoles no longer just submit precompiled shaders to the GPU directly?
<kameks>
basically, we allocate memory, map it for GPU access, and memcopy precompiled shader code
<kameks>
and give the pointer to the "driver"
<chadmed>
right
<kameks>
but the driver is just going to format it into a command packet
<kameks>
no black magic, that's very low level
<chadmed>
AIUI we have to submit work via the RTKit firmware the GPU runs anyway, so your raw command submission API will probably look pretty similar to the linux kernel driver
<chadmed>
since they will basically do the same thing
<kameks>
do you have to talk to the kernel for every single command you record ?
<chadmed>
phire had some scripts to talk to the GPU's firmware directly from m1n1
<kameks>
on console API, a command buffer is a void*. we create a command encoder, give it a range of preallocated memory, and a callback for when it runs out of memory. The encode writes command to the memory we control as we go. We are then responsible to submit all the buffers at once.
alcazar has quit [Ping timeout: 480 seconds]
<kameks>
only the submission is talking to the kernel
<kameks>
the recording of commands is fully user-space
<kameks>
a texture descriptor is just a raw struct where we put pointers and properties, we don't have to register and destroy them with the driver
<chadmed>
i915 works similar to that i think
<kameks>
the only things which are going into kernel is initializing the GPU, opening the video port, and command submission. The rest is just memory we allocate on our own heap, and we fill with the help of low level functions
<kameks>
the only drawback is that it's not portable at all
<chadmed>
doesnt really need to be on a console heh
<chadmed>
but yeah fwiw thats basically exactly how i915 works in linux
<chadmed>
intel's mesa stuff creates the buffer batches and then flicks them off to the kernel driver for submission
<kameks>
but even if the platform were to not support a high level API such as Vulkan, we could implement it ourselves if we had the time for it. we got all the tools
<kameks>
on M1 GPU, the command buffer still reside in kernel space?
alcazar has joined #asahi-re
<marcan>
kameks: on consoles there are different security considerations than on Linux, and very different multitasking requirements. how this works or can be made to work varies from platform to platform.
<marcan>
usually the draw command buffer is filled by userspace but submission will go through the kernel, and the kernel maintains the main command ring buffer
<marcan>
also some consoles may share page tables between apps and the GPU, which may or may not be possible on Linux
<marcan>
either way the kernel interface is intended to be as low level as possible while maintaining security and multitasking guarantees, so the chances you can improve upon whatever we come up with there aren't terribly high; memory management is another question but it's also not something you can just rip out, and if you want to propose something radically different from what DRM supports today, ...
<marcan>
... that'd have to be discussed with upstream
alcazar has quit [Ping timeout: 480 seconds]
<marcan>
so basically it sounds like what you actually want is to just not use mesa or talk directly to lower layers of mesa, and keep the same kernel interface
<marcan>
I don't know whether Linux supports unified GPU/CPU page tables, but if it doesn't and the M1 can do it, that'd be a good thing to work on a general solution for, since I think some other architectures can too
<kameks>
What'd like to have/do is a user-space driver, that ideally also run on mac. So I wouldn't be responsible for page allocation and protection or the ring buffer. But mostly raw command encoding, helpers to tile textures, find memory footprints (where to put mips) and so on. If that's more or less already what the kernel is doing, that's all the better
<kameks>
ideally, my stuff would also run on a stock macos
<kameks>
about RE, I've read it and I fully agree to follow the practices
alcazar has quit [Ping timeout: 480 seconds]
<marcan>
that's pretty much already how it works in every sane graphics driver, yes
alcazar has joined #asahi-re
<marcan>
AFAIK the only notable difference is the memory management as I said, i.e. with Linux you do need to ask the kernel to GPU-map buffers since the page tables are maintained separately for GPU and CPU
<kameks>
yup, I mean, have it in a user accessible API, not driver internal
<kameks>
on console as well, we ask the kernel for GPU map. some of them have unified memory space, some of them don't. but the kernel still keeps proper permission to avoid privilege escalation using the GPU
<marcan>
AIUI on some consoles the GPU and CPU share page tables, so no explicit GPU map operations are necessary
<kameks>
right, but the mapping still go through the OS, you can't just randomly change permission for kernel owned pages
<marcan>
yes, but the point is your userspace pages are automatically GPU pages in the context of your process
amarioguy has quit [Remote host closed the connection]
<kameks>
well I think the 360 was exploited because the GPU could write any address, including kernel
<marcan>
with the same VA
amarioguy has joined #asahi-re
<kameks>
yes, that's the case
<marcan>
right, and I'm saying that's not on Linux
<kameks>
ok, got it
<marcan>
by the way, you're probably going to have a bad time trying to make this work and maintain it on macOS, because Apple makes no kernel<->userspace ABI guarantees
<marcan>
so any random macOS upgrade can and likely will break your code
<kameks>
;-(
<kameks>
you have a point
<marcan>
Go tried it with boring old UNIX syscalls and a macOS upgrade broke all Go apps; and obviously the GPU API is much, much less stable than that
<marcan>
so yeah
alcazar has quit [Ping timeout: 480 seconds]
<kameks>
how did go solve it? did they just stopped using those syscalls?
yuyichao has quit [Ping timeout: 480 seconds]
<marcan>
they gave up on that idea and now just call libc like everyone else
<marcan>
the only stable graphics APIs on macOS are Metal and OpenGL, so basically anything else is impossible to make forward-compatible
<kameks>
I see. so the only way on mac is to completely replace the kernel driver and reimplement metal as well (assuming it is possible)
<marcan>
it isn't even clear if the *firmware* ABI is stable, and Apple internal kernel magic also definitely isn't stable, so no
<marcan>
this isn't going to work stably, ever
<kameks>
by firmware, you mean the GPU ioctls ?
<marcan>
I mean the GPU firmware
<marcan>
nevermind that messing with kernel drivers like this one means putting the system into permissive security mode and manually loading your own kernels, which is terrible UX for normal macOS users and can't even be done without the KDK and you can't redistribute that
<marcan>
basically, forget about it
<marcan>
it's not happening
<kameks>
so a macos could also update the GPU side firmware and break the ABI ?
<marcan>
indeed macos updates all come with their own GPU firmware
<kameks>
is that a problem for linux too?
<kameks>
eg, you update macos, and now you can't dual boot linux anymore
<marcan>
not the same way, because we can choose what firmware versions we support
<marcan>
no, firmware is *per OS*
<kameks>
ah, I see
<marcan>
we currently support only one firmware set for linux, 12.3
<marcan>
and we will stick with that until we have a good reason to support another one (e.g. new hardware or a critical bugfix)
<kameks>
so during boot, you upload the firmware? or somehow you have the guarantee that it will not be impacted by macos
<marcan>
iBoot loads the firmware for us
<kameks>
that makes sense
<marcan>
the firmware is packaged with the OS and our installer installs the full 12.3 set
<kameks>
well, you have a perfect point about it being impossible to do on macos
<kameks>
how much work is left to do on the linux driver?
<kameks>
is there still stuff I can help with?
<marcan>
on the userspace side, GLES2 compliance is at 95% or something
<marcan>
on the kernel side we have nothing, but that's starting now :-)
alcazar has joined #asahi-re
<kameks>
what about Vulkan, do you need help there?
<marcan>
(the userspace side is tested on macos, with the ABI compat caveats of course; we already had to forward-port it once)
<marcan>
yeah, there's nobody working on vulkan yet
<kameks>
(yup, this is what I read from Alyssa's blog posts)
<marcan>
but also we don't really understand the GPU well enough yet to start working on vulkan
<kameks>
hmmm, what kind of areas are still foggy?
<marcan>
I think alyssa can better answer that question :-)
<kameks>
fair enough. I guess it also better belong to gpu channel too. Sorry I drifted the conversation a lot
<kameks>
well, thank you very much for your time and advices
alcazar has quit [Ping timeout: 480 seconds]
amarioguy has quit [Ping timeout: 480 seconds]
amarioguy has joined #asahi-re
<kameks>
one last thing, I saw documentation about the GPU shader assembly, but I didn't find much documentation about the driver itself (how packets are encoded, descriptors, texture tiling and so on). Maybe I could start reading the mesa driver and improve the documentation to familiarize myself if that's useful
<marcan>
might be useful, yeah :)
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
<amarioguy>
kinda thinking about SEP for a second here, but what are the goals we have in mind for using the SEP?
<sven>
i also have some description of the xART format somewhere required to boot it
alcazar has joined #asahi-re
the_lanetly_052 has joined #asahi-re
<amarioguy>
from stuff i've read, iirc the xART stuff and booting sepOS is handled by iBoot before it hands off execution?
<sven>
no
<sven>
iBoot shuts SEP down again and we have to re-start it and handle the xART endpoint
alcazar has quit [Ping timeout: 480 seconds]
<amarioguy>
so the flow i'm seeing here is: handle xART EP and send that to SEP, set up TZ0 again (since SEP is shut down), send boot TZ0, then set up remaining endpoints for things like the key store and such?
<amarioguy>
saw a few SEP EPs being set up by XNU in boot logs (21, and 7 i think?)
<sven>
uh, not really
<sven>
TZ0 is already setup, that's write-only by the time we get control
<sven>
you need to map the firmware, boot SEP, and then handle requests from the xart endpoint
<j`ey>
amarioguy: proxyclient/m1n1/hw/sep.py
<sven>
erm, s/write-only/read-only/ :D
<sven>
ohh... right, i actually wrote something to boot sep from m1n1
<sven>
that doesn't do xart though iirc
<j`ey>
xart doesnt appear in that file
<amarioguy>
gotcha, so xART is still something we need to do
<sven>
yeah, it's just one of the SEPOS endpoints
<j`ey>
(or anywhere in m1n1)
<sven>
and you need to handle it to bring up the sks endpoint which handles keys and passwords
<amarioguy>
sbio for biometrics too
<sven>
or maybe those were two endpoints? my memory is rather fuzzy
<sven>
i don't remember if sbio requires xart but i assume so
<sven>
i also remember having worked out how the hashes for the actual SEP communication works but that doesn't seem to be in there as well
<sven>
let me see if I can find that again
alcazar has joined #asahi-re
yuyichao has joined #asahi-re
<marcan>
xart is something to be handled in userspace by sepd
<marcan>
that wouldn't go in the driver
<marcan>
and xart depends on linux-apfs
<amarioguy>
right since the xART partition macOS normally uses is an APFS partition right
<marcan>
and iBoot, yes
<marcan>
it's systemwide and not practical to replace/duplicate
<marcan>
so a prereq here is a bunch of audit/lockdown work on linux-apfs to convince ourselves that either 1) it's stable enough it'll *never* break the iSC, or 2) we can mount it with some kind of restriction option that disables enough things we can consider it stable
<sven>
most things will probably have to be done inside userland
alcazar has quit [Ping timeout: 480 seconds]
<marcan>
xARTS requires a very limited set of operations, namely in-place overwrites of a single file, though with the CoW nature of APFS I'm not sure how overcomplicated even that could turn out to be
<amarioguy>
marcan: i see, so any SEP works relies on the APFS driver not crashing and burning first right?
<sven>
the first step would be to understand SEP and probably implement some features into sep.py
<amarioguy>
sven: right right. any major SEP changes between t8101 and t600x platforms i should be aware of? have a t8101, still waiting on a t6002
<sven>
no idea, i only have t8103 as well
<sven>
but i'd assume they're very similar if not the same
alcazar has joined #asahi-re
<amarioguy>
i'd assume that too, since it's still a14 based at the end of the day
<amarioguy>
marcan: want some clarification on the RE policy since this is kind of an edge case here: with the 12.3 and 12.3.1 IPSWs (all 12.3.x seeds actually), complete dev kernelcaches accidentally ended up shipping with these (the difference between these and the KDK stuff is a number of kexts are shipped as development versions, including apfs)
<amarioguy>
do we treat this as a leak and therefore offlimits since it was an accident (in all 12.3 seeds) or is this development kernelcache fair game for something like m1n1 tracing (definitely not binary RE) since other than the development kexts, it's identical to what the KDK would give and it's in an official IPSW
<marcan>
"leaks" means stolen or similar in the context of the policy; anything officially published does not count, accidental or not
<amarioguy>
gotcha, so sounds like it's fair game since these are in official, final released seeds just accidental releases
<marcan>
correct
<amarioguy>
awesome, APFS stuff just got easier lol
alcazar has quit [Ping timeout: 480 seconds]
<marcan>
when I say leaks I mean things like that iBoot source code leak, or internal tooling, and things like that
<marcan>
technically that would also include leaked schematics, but I make an exception for those because facts are not copyrightable, they are widely available, considered critical for repair, practically useless for evil purposes, and it is almost impossible for us to end up infringing their copyright by merely looking things up in them and using the learned information
<marcan>
(of course, as with other kinds of RE in general, whether you're okay getting ahold of them or not in the first place is a personal decision; I'm strictly talking about the project stance here, not individual liability)
<amarioguy>
fair enough
<amarioguy>
btw how's power management progress?
<marcan>
in what sense? haven't really done much since the release, too busy working around Apple bugs and catching up on everything I slacked off on
<nicolas17>
amarioguy: not the first time things ended up in ipsws that weren't supposed to be there >.>
<amarioguy>
marcan: like an overall picture, just kinda a "what's left" in that regard
<amarioguy>
sounds like it's "almost everything" lol
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
roxfan2 has joined #asahi-re
amarioguy has quit [Remote host closed the connection]
roxfan has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
<marcan>
on the short term list are figuring out how to make sleep work for brcmfmac, whether we can power down/up the PCIe controllers sanely, doing the PSCI-over-UEFI thing for CPU deep idle, shutdown/startup, and deep sleep
<marcan>
and full system sleep via PSCI too
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
doggkruse has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
MajorBiscuit has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
caef^ has joined #asahi-re
alcazar has joined #asahi-re
MajorBiscuit has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
yamii has joined #asahi-re
MajorBiscuit has quit [Ping timeout: 480 seconds]
the_lanetly_052 has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
caef^ has quit [Ping timeout: 480 seconds]
nyx_o has quit [Quit: WeeChat 3.5]
alcazar has joined #asahi-re
roxfan2 is now known as roxfan
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
chengsun has quit [Quit: Quit]
chengsun has joined #asahi-re
nyx_o has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
the_lanetly_052 has joined #asahi-re
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
doggkruse has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
the_lanetly_052__ has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
the_lanetly_052 has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
_whitelogger has joined #asahi-re
phire_ has joined #asahi-re
sirn has joined #asahi-re
the_lanetly_052___ has joined #asahi-re
jakebot has joined #asahi-re
ChaosPrincess has joined #asahi-re
riatre has joined #asahi-re
lethalbit has joined #asahi-re
robinp_ has joined #asahi-re
dougall has joined #asahi-re
abrasive has joined #asahi-re
alcazar has joined #asahi-re
MajorBiscuit has quit [Quit: WeeChat 3.4]
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
caef^ has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
atsalyuk has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Ping timeout: 480 seconds]
alcazar has joined #asahi-re
alcazar has quit [Remote host closed the connection]
<nicolas17>
M1 has the recfg engine thing too right?