marcan changed the topic of #asahi to: Asahi Linux: porting Linux to Apple Silicon macs | General project discussion | GitHub: https://alx.sh/g | Wiki: https://alx.sh/w | Topics: #asahi-dev #asahi-re #asahi-gpu #asahi-offtopic | Keep things on topic | Logs: https://alx.sh/l/asahi
klaus has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
fridtjof[m] has quit [Quit: authenticating]
fridtjof[m] has joined #asahi
balrog has quit [Quit: Bye]
balrog has joined #asahi
BaughnLogBot has quit [Ping timeout: 256 seconds]
BaughnLogBot has joined #asahi
phiologe has quit [Ping timeout: 240 seconds]
phiologe has joined #asahi
mxw39 has quit [Ping timeout: 240 seconds]
mxw39 has joined #asahi
BaughnLogBot has quit [Ping timeout: 256 seconds]
KindOne is now known as KindTwo
KindTwo is now known as 18WABQDLV
mxw39 has quit [Remote host closed the connection]
mxw39 has joined #asahi
marvin24 has quit [Ping timeout: 260 seconds]
marvin24_ has joined #asahi
bgianf has quit [Quit: leaving]
bgianf has joined #asahi
<Glanzmann>
Someone in here disabled the swap in macos, was your system stable, can I do the same?
<Glanzmann>
Because I currently only use macos for three things: ssh, firefox, iina (movie player)
maor26 has joined #asahi
VinDuv has joined #asahi
amw has quit [Ping timeout: 246 seconds]
jaXvi has joined #asahi
raster has joined #asahi
nicolas17 has quit [Quit: Konversation terminated!]
VinDuv has quit [Quit: Leaving.]
amw has joined #asahi
amw has quit [Ping timeout: 240 seconds]
amw has joined #asahi
ephe_meral has joined #asahi
klaus has joined #asahi
18WABQDLV is now known as KindOne
klaus has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
amw has quit [Ping timeout: 265 seconds]
ephe_meral has quit [Ping timeout: 240 seconds]
klaus_ has joined #asahi
TheJollyRoger has quit [Ping timeout: 268 seconds]
<arnd>
I haven't tried that, but I expect it would not be particularly stable. Regarding the heavy swap usage I have two other observations though:
<arnd>
- if I only log in remotely to macos and there is no local user, the 16GB are usually sufficient for what I do, even when running an 8GB linux VM guest, and it doesn't swap at all as long as there is free memory (this is what is supposed to happen, but it wasn't obvious that there was no bug here)
<arnd>
- the swap problem is likely made worse by the use of 16KB pages, compared to the 4KB pages that macos on x86, and (most) Linux on arm use
<arnd>
I did some experiments on Amazon Graviton guests a while ago and found that while the memory usage explosion from 64KB pages compared to 4KB wasn't as bad as I had expected,
<arnd>
the slowdown was worse because of the extra I/O required while paging
<arnd>
16KB obviously won't be as bad as 64KB for either the extra memory usage, or the extra I/O per swap operation, but it will still be very noticeable
<marcan>
arnd, maz: this one has me worried. it's a pre-standard implementation of FEAT_AFP (i.e. the SSE-compatible denormal handling), but with bits in a custom register, not FPCR
<marcan>
it has to be enabled to *work*, but APFCR itself is not locked out if the feature is disabled; the bits just have no effect
<marcan>
accesses do not trap, at least from EL2
<marcan>
if they also don't trap from EL0, then that's a two-bit sidechannel
<marcan>
I'm going to need some kind of call_el0 thing in m1n1 to do EL0 tests...
<marcan>
*covert channel
<never_released>
marcan: AFPCR is writable from EL0
<never_released>
as far as I know
<marcan>
is it writable from EL0 if AFP is disabled?
<marcan>
obviously it has to be writable if AFP is enabled
<marcan>
(e.g. rosetta would use that mode)
<marcan>
also there could be another bit to enable userspace access
<marcan>
my real question is: is there a way to make it trap on userspace access, and is that the default from iBoot
<never_released>
do you know what APFLG are for?
<marcan>
nope
<marcan>
I know there's a bit for that
<never_released>
I'll have to look at AHCR too
<never_released>
it's Apple-specific hypervisor control regs
<marcan>
do you know the encoding for that one? I've seen the name but not the encoding
<never_released>
the hypervisor stuff is censored from OSS XNU
<never_released>
so will have to reverse
<marcan>
I know
<marcan>
I'm looking through the 11.1 binary blob object files right now, since those are fair game because APSL
<never_released>
marcan: those are _very_ partial
<marcan>
I know
<marcan>
but they are worth starting with
<never_released>
a thing that's nice to experiment with, marcan
<never_released>
is that almost all the Apple extensions
<never_released>
are accessible within a VM
<marcan>
and importantly there is no legal risk in doing so :p
<never_released>
including guarded exec
<never_released>
you can experiment from within macOS with GL1 just fine
<marcan>
that's a good point, and if they didn't do anything about this AFPCR in linux VMs then this is a CVE in macOS VMs - and one they might not be able to fix
<marcan>
I guess I should spin up a VM
<never_released>
even AMX is accessible to VMs
<marcan>
how? I haven't even figured out how to turn it on yet, unless it works in EL0 and not EL2
<marcan>
or unless the host turns it on and then VMs inherit that I guess
<never_released>
marcan: you have specific AMX sysregs to toggle
<never_released>
and then you specifically have to arm it with the AMX enable instruction
<marcan>
yeah, I've been using that instruction
<marcan>
but I don't know what sysreg bits enable it yet
<never_released>
the reg for it is amx_config_el1
<marcan>
do you have the encoding?
<never_released>
not yet, didn't want to poke in detail at the binaries themselves, but will have to do it at some point
<marcan>
if you know the bits I can also just try brute-force; I have a list of all sysregs anyway
<marcan>
it can be guessed
<marcan>
probably about time I try toggling all bits in all sysregs and build a blocklist of things that break the world
<marcan>
and then we can automate searches against everything else
<marcan>
we already know large ranges that are unrelated (HID, PMCs, etc)
<never_released>
funnily enough
<never_released>
the hypervisor is present on iOS builds too
<never_released>
but is gated by CONFIG_HYPERVISOR_PUBLIC, which isn't present in the MASTER.arm64 config
<never_released>
(only there in the macOS kernel one, private entitlement on iOS)
thestr4ng3r has quit [Ping timeout: 264 seconds]
thestr4ng3r has joined #asahi
<maz>
marcan: if there is no way to disable it from EL2 when HCR_EL2.TGE==1, we'll need to wipe this register on context switch. really ugly.
<marcan>
yup.
<marcan>
still investigating, I'll let you know what I find
<maz>
for TGE==0, this should already trap to EL2 when HCR_EL2.TIDCP is set.
<marcan>
maz: if this affects linux guests under macos, can we file a CVE and do nothing and let *Apple* send the kernel a patch fixing it?
<marcan>
:-)
<maz>
marcan: here's to hope! :D
<maz>
marcan: what would be interesting is to find out whether HCR_EL2.TIDCP is honored when these non-architectural features are used at EL0/EL1.
<marcan>
yeah
<maz>
I can see why they'd leave that off in their hypervisor.
<maz>
but we definitely want that on for KVM.
<marcan>
ok, mrs from AFPCR traps in docker on macos
<marcan>
so at least it's not haha panic broken
<marcan>
(from EL0)
<maz>
OK, that's somewhat reasuring.
<never_released>
marcan: no reason to make it trap on EL2 in practice anyway... so
<marcan>
never_released: yeah, but also no reason to make it not trap :)
<marcan>
writing some horrible el0 thunk for m1n1 now
<never_released>
marcan: you might want to test at EL1.
<never_released>
more important than EL2 tbh
<never_released>
(for KVM)
<never_released>
because it could allow communication between VMs
<marcan>
there is no EL1 on bare metal :)
<marcan>
but that goes with the HV stuff I want to do later anyway
<marcan>
so I'll punt it to then
<marcan>
for now I just want to make sure EL0 is sane for bare-metal linux
ch has quit [Quit: restart]
<never_released>
marcan: there is, for virtual machines
<never_released>
EL0 is sane
<marcan>
yes, inside VMs
<never_released>
hint: you can just test without m1n1 there, just from a booted up Linux
<marcan>
Linux runs in EL2
<marcan>
E2H is forced to 1
<never_released>
what you care about is two things
<never_released>
marcan: you can drop to EL1
<never_released>
and run Linux there
<never_released>
w/ nothing running at EL
<never_released>
*EL2
<marcan>
does that work properly with E2H forced to 1?
<never_released>
yes
<marcan>
hwh
<marcan>
*heh
<maz>
marcan: just clear TGE, which should be enough.
<maz>
EL1 will magically appear.
<marcan>
right
<never_released>
marcan: you also have the _EL12 variant on those
<never_released>
not even the need for that
<maz>
you will need some sane state at EL2 though.
<never_released>
if you aren't ever going to go back to EL2
<maz>
never_released: the _EL123 only gives you some save/restore facility. being out of context, exceptions doen't apply to them.
<marcan>
let me figure out plain old EL0 first :)
<never_released>
you set interrupt routing in HCR to EL1
<never_released>
and then you lose EL2 forever after dropping to EL1, with no way to come back
<never_released>
(some companies annoyingly do this in the devices they ship)
<never_released>
what really annoys me in the M1 tho
<never_released>
is apparently no nested virt support
<maz>
to my knowledge, only 2 CPUs implement NV: Neoverse V1, and TX3. and the latter is dead.
<never_released>
didn't TX3 implement the lunatic ARMv8.3 variant
<never_released>
of the NV extensions
<maz>
no, it had the full v8.4-NV.
<never_released>
oh nice
<maz>
same as Zeus.
<maz>
though none of the two were actually v8.4 due to the lack of SEL2.
<never_released>
meanwhile, Cortex-A78AE and C have PAC among other backports
<maz>
anyway. one is dead, and the other won't be available for another 2 years, I expect.
<never_released>
but not regular A78 (to match the A55 ISA level I guess)
<never_released>
I wonder if there's an Apple-specific implementation sitting in AHCR
<never_released>
wouldn't bet against it
<never_released>
actlr, acfg, ahcr
BaughnLogBot has joined #asahi
<marcan>
heh, my el0 thunk code worked on the first try other than getting data aborts on everything (bad page attributes making everything RO at EL0 probably)
<marcan>
(code does run)
<marcan>
(and my exception recovery code recovers from the bad writes anyway)
<marcan>
ah right, PAN getting in my way when I mark stuff EL0 accessible
<marcan>
or not... hm
<marcan>
I remember this from back when I was messing with page permissions, but setting bit 6 in the PTEs breaks everything and it doesn't look like it's PAN?
<j`ey>
why dont you think it's PAN?
<j`ey>
so you marked it as accessible at EL0, but are accessing it from EL1/2?
<marcan>
oh wait, PXN
<marcan>
no wait, but I don't set that either
<marcan>
j`ey: yes, but I set PSTATE.PAN to 0 and it still breaks
<marcan>
sec, doing an actual controlled test with only some mappings, not the whole thing
<j`ey>
are you modifying SCTLR? that has a bit SPAN (23) that enables PAN on exceptions
<marcan>
actually, that bit *dis*ables enabling PAN on exceptions
<marcan>
but I am setting it; however, things break before I take an exception normally anyway
<marcan>
yeah, it's execution that breaks
<marcan>
now I wonder if this is an apple-ism somewhere...
<marcan>
is there any mechanism that would forbid EL2/1 execution from EL0-accessible pages, other than the PXN bit in the page tables?
<j`ey>
"c. Not executable, because AArch64 execution treats all regions writable at EL0 as being PXN."
<marcan>
argh, so this is architectural?
<marcan>
hahahaha
<marcan>
lovely
<marcan>
"architecturally secure"
<marcan>
well okay, it's alias time then
<marcan>
maz: okay, yeah, this stuff traps from EL0 if not enabled, even with TGE=1
<marcan>
I guess this makes sense to some extent; it means the kernel save/restore code can just unconditionally save/restore these flags, regardless of whether a given process has the feature enabled
mechpilotace has quit [Quit: WeeChat 2.8]
<marcan>
pushed m1n1 changes for EL0 support
Necrosporus has quit [Killed (barjavel.freenode.net (Nickname regained by services))]
Necrosporus has joined #asahi
Necrosporus has quit [Ping timeout: 256 seconds]
Necrosporus has joined #asahi
Behemoth has joined #asahi
maor26 has quit [Ping timeout: 264 seconds]
maor26 has joined #asahi
tomhv has joined #asahi
jaXvi has quit [Quit: Leaving]
jaXvi has joined #asahi
ephe_meral has joined #asahi
tomhv has quit [Ping timeout: 260 seconds]
tomhv has joined #asahi
tomhv has quit [Client Quit]
<marcan>
uh-oh
<marcan>
maz: I don't know what s3_5_c15_c10_1 is, but it's also 2 bits, and writable by EL0 by default
ephe_meral has quit [Ping timeout: 240 seconds]
<marcan>
aaand that's a macOS CVE. hoo boy.
<davidrysk[m]>
oof.
<davidrysk[m]>
oh yeah, something I noticed is that it does not seem darwin-xnu has the concept of "swappiness"
<eta>
that's fun
odmir has quit [Remote host closed the connection]
devinvs[m] has quit [Quit: Idle for 30+ days]
nutmanja[m] has quit [Quit: Idle for 30+ days]
<marcan>
it does trap in VMs if I'm not mistaken, so at least there's that, but I need to double check
<marcan>
email sent
<davidrysk[m]>
does apple support the standard FEAT_AFT as well as theirs?
<marcan>
nope
<marcan>
that came out last year, so presumably Apple's is basically a prerelease version of the same
<marcan>
wouldn't be surprised if it came from them
<marcan>
amusingly, they both call it "AFP", but I'm pretty sure the "A" stands for "Apple" in Apple's case
<marcan>
maybe someone at ARM went "eh... "Advanced FP" works too" :p
<marcan>
the feature is *very obviously* an x86 emulation feature either way
<marcan>
Apple just blatantly called their bits by the SSE names, ARM integrated it into the existing FPCR in a slightly less obvious way
<j`ey>
the A standards for alternate!
<marcan>
oh right, for some reason I thought Advanced
<j`ey>
Probably cos the SIMD stuff is called Advanced SIMD
<marcan>
either that or Tantatrul got to me
<modwizcode>
heh
<modwizcode>
Well that was a fun few days of chat to catch up on
<modwizcode>
marcan: afaik dmb is only guranteed as a memory ordering op. i.e. do not issue memory ops out of order. It offers no assurances beyond that. ARM docs usually use it as part of locking, but from what I can tell it offers no meaning in the face of io.
<marcan>
modwizcode: my question is whether it guarantees ordering between device memory and normal memory ops
<marcan>
or e.g. only between normal-normal and device-device within a peripheral window (there is a reference to an imp-def window size in the spec)
<marcan>
there is an additional confounding factor though, that this is really a 3-party behavior
<marcan>
i.e. what I *really* want to know is whether A writing memory and writing to peripheral B is ordered wrt C reading from peripheral B and reading the memory that A wrote
<marcan>
it all gets very confusing when you step outside CPUs and plain memory
<marcan>
and that is also where many guarantees can easily break down
<marcan>
I'll probably end up writing a litmus test-ish thing for this to prove it to myself that dmb is enough
<modwizcode>
Yeah I wasn't clear in my answer and even if I was I think I was wrong.
<modwizcode>
The ARM is annoyingly vague on that specific issue too.
ephe_meral has joined #asahi
<modwizcode>
My interpretation seems to indicate that DMB is safe for orderings between both on the same core, but DSB is required for multicore because you need to be sure the writes have "completed" which seems to be the only gurantee of external visibility?
<modwizcode>
wrt device accesses that is.
<modwizcode>
I hope that's wrong though because that contradicts a sane interpretation of DMB that affects outer shareable regions like device memory...