marcan changed the topic of #asahi to: Asahi Linux: porting Linux to Apple Silicon macs | General project discussion | GitHub: https://alx.sh/g | Wiki: https://alx.sh/w | Topics: #asahi-dev #asahi-re #asahi-gpu #asahi-offtopic | Keep things on topic | Logs: https://alx.sh/l/asahi
choozy has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
PhilippvK has quit [Read error: Connection reset by peer]
phiologe has joined #asahi
<marcan> TheLink: dougall is around here too :)
hspak has quit [Quit: bye]
hspak has joined #asahi
inglor has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
inglor has joined #asahi
_whitelogger has joined #asahi
phiologe has quit [Ping timeout: 260 seconds]
phiologe has joined #asahi
marvin24_ has quit [Ping timeout: 240 seconds]
marvin24 has joined #asahi
nicolas17 has quit [Quit: Konversation terminated!]
marvin24 has quit [Ping timeout: 260 seconds]
marvin24 has joined #asahi
odmir has joined #asahi
odmir has quit [Ping timeout: 276 seconds]
VinDuv has joined #asahi
<TheLink> yeah, realized that afterwards
jeff_miw has joined #asahi
<jeff_miw> sven: nice:)  I would like to help to get usb on m1n1, I have a macbook air, let me the best way to help there
<sven> do you have a uart cable and feel like reading cursed usb specs? then we could see if there's a way for you to help write serial code.
<sven> otherwise i'd be happy to have some people test this once i make a little more progress and implement the serial class
<sven> https://www.ti.com/lit/ug/spruhj7a/spruhj7a.pdf this is a user manual for this dwc3 controller fwiw
<jeff_miw> no uart cable with all the usb pd/vdm stuff ... but I can work how to get one if mandatory. no problem to read the cursed specs :)
<jeff_miw> let me know once you have something that makes sense to test
<sven> nah, no need to get one. i hope that i'll be able to finish at least an initial version of the serial part this week. after that most people probably won't need the pd uart cable anymore
<sven> i'll let you know once i have something that is more useful than just printing what commands it receives over uart :)
jeff_miw has quit [Ping timeout: 240 seconds]
VinDuv has quit [Quit: Leaving.]
<marcan> sven: it shouldn't be too hard to do CDC serial AIUI
<sven> yeah, i think once i have enumerating working CDC serial shouldn't be much more complicated
vimal has quit [Ping timeout: 260 seconds]
raster has joined #asahi
mxw39 has quit [Quit: Konversation terminated!]
vimal has joined #asahi
josiahmendes[m] has quit [Quit: Idle for 30+ days]
dyniec[m] has quit [Quit: Idle for 30+ days]
mofux[m] has quit [Quit: Idle for 30+ days]
<maz> marcan: finally ironed out the annoying timer issue over the weekend, though the result isn't pretty at all. guests now work reliably.
<maz> marcan: I'll explore a solution that doesn't involve the magic timer LR though, as it prevents the emulation of some GICv3 features Linux relies on.
mofux[m] has joined #asahi
zkrx has quit [Ping timeout: 245 seconds]
<kettenis> sven: TI probably has slightly different glue logic for the dwc3 controller
<kettenis> so not everything in that document will apply
<sven> yeah, especially the PHY doesn't apply at all.
<sven> but the "USB Device (Peripheral) Programming Model" should be close enough
<sven> if I do the phy init manually from m1n1 and use my dart driver in linux their dwc3 driver also works without any modifications in gadget mode fwiw
<j`ey> their = linux or?
<sven> yes
<kettenis> you don't have to muck around with the i2c type-c controller?
<sven> not for gadget mode
<sven> but I believe that's unfortunately required for host mode
<kettenis> yeah, my attempts to use host mode failed to detect any devices
<sven> yeah, same here
<kettenis> no port status changes
<sven> exactly
<kettenis> that's when I switched to looking at pcie
<sven> :-)
<sven> I then switched to writing usb code for m1n1 because I was tired to wait for my kernels to load over uart
<spikebike> 55dddd:LK;'
zkrx has joined #asahi
<marcan> maz: awesome! what's the exact issue?
<marcan> I think qemu on macos works reliably without that LR stuff, only mask bits for the two timers, though I'm not entirely sure
<marcan> sven: the ports won't go into DFP mode without poking the type C controller
<marcan> same reason you need to be DFP for serial in m1n1
<marcan> the controller doesn't have DFP negotiation enabled by default, so if you present to it as an UFP it just does nothing until macos boots
<marcan> I bet it's not too hard of a dance though, and those things are mostly documented
<marcan> the linux driver for it should mostly just work
<marcan> but that does depend on i2c
<marcan> which I guess is a very good reason to make that my next target
<marcan> (need to poke around the pasemi i2c driver)
Jangal has quit [Remote host closed the connection]
Janhouse has joined #asahi
<maz> marcan: the main problem is that you can't inject a timer interrupt using that magic register. the Pending state is only driven by the timer itself, and only the Active state is writable.
<maz> marcan: so although we can save/restore the timer state correctly, including the GIC Active state, we can't pend an interrupt from SW, and Linux relies on that for interrupt resending.
<marcan> ah, so you can save/restore the timer counter state, but not the pending bit?
<maz> marcan: the GICv3 architecture mandates that you can write to the pending state and generate the corresponding interrupt, so that's pretty annoying.
<marcan> could you force the issue by, say, setting it up to count down 0 ticks?
<maz> marcan: you don't need to save the pending bit itself. the state of the timer is enough to regenerate it.
<marcan> but wouldn't that be enough to inject the FIQ? AIUI it's level-triggered
<maz> setting TVAL to 0 would be observable by the guest, so that's a bit nono.
<marcan> oh wait, you mean in that LR
<marcan> I was talking about the timer IRQ flag itself in the control register
<marcan> yeah. I noticed only one of the bits is writable in the LR
<maz> so short of being able to set the Pending bit in the LR, we can't actually emulate the GICR_ISPENDING register for the virtual timer PPI.
<maz> which is why I'm going to try and switch to just masking/unmasking the interrupt.
<maz> and rely on a normal LR for injection.
<marcan> this is for emulating a normal GIC system, not the FIQ stuff, right?
<maz> we'll get a performance degradation because of the exit on EOI.
<maz> I'm only concerved with GIC emulation, because that's the only thing KVM supports.
<marcan> right
<maz> in the kernel, that is.
<maz> you can obviously plug a userspace interrupt controller.
<marcan> so I actually don't know at all how the timers interact with the vGIC stuff
<marcan> or that magic LR register for that matter
<marcan> so I need a bit of context :)
<marcan> is that magic LR connected to the vGIC stuff?
<marcan> oh, I guess what that HACR bit is redirect the vtimer from FIQ to that LR which then interacts with vGIC?
<marcan> *bit does
<maz> yes. the magic LR takes the timer output as its pending bit, and exposes the resulting interrupt in the guest via ICV_IAR*_EL1.
<marcan> makes sense
<marcan> and what you're saying is we can't make the timer drive a pending state?
<maz> it's a really nice shortcut in the sense that it doesn't generate an exit from the VM when the timer fires.
vimal has quit [Quit: Leaving]
<maz> because that pending state is *only* drive by the timer, SW can't inject an interrupt in that LR despite the timer not firing, which the GICv3 architecture allows.
<maz> driven*
<marcan> how would that be used? wouldn't the guest always expect the timer to be pending when it receives an IRQ?
<maz> in the case of a level interrupt, that's not a huge problem. it's more for edge interrupt that it is problematic.
<marcan> right, but these timers are level, right?
<maz> but in general writing to GICx_ISPENDING must generate an interrupt, full stop.
<maz> and we can't do that for the timer.
<marcan> oh, you mean if the guest writes that
<maz> yup.
<marcan> argh, okay
<maz> it really is a corner case, but deviating from the architecture is in general a very bad idea.
<marcan> can we use a regular LR for that?
<maz> yup, at the expense of using mask/unmask to prevent the timer from firing.
<maz> which we need to do anyway for the physical timer, which doesn't have a magic LR.
<marcan> right
<maz> the main disadvantage is that you need to force an exit whenever the guest EOI a timer interrupt in order to unmask the physical interrupt.
<marcan> could we use both, somehow? use a regular LR to inject pending states if the guest asks us to, otherwise let the timer use the shortcut LR
<maz> which we don't have to do on normal GIC implementations, as the HW knows how to link the virtual interrupt to the physical one.
<maz> marcan: it is illegal to have two LRs with the same INTID...
<marcan> yes, but a lot of things are illegal about this architecture already...
<marcan> worth trying it :)
<marcan> I mean that magic register is already way out in IMP-DEF territory anyway
<maz> sure, but if I can get decent perf with just mask/unmask, I'd be happy to just ditch the IMPDEF stuff.
<marcan> sure, works for me
<maz> becase this is a major hack in the vgic implementation, which is already a very complicated beast.
<marcan> especially for a first implementation
<marcan> we can worry about micro-optimizing this later if we need to
<marcan> when we have a use case
<maz> yup.
<sven> marcan: nice, I'm pretty sure usb host most is just going to work as well then once you have that i2c figured out
klaus has joined #asahi
jaalsa has quit [Quit: jaalsa]
jaalsa has joined #asahi
raster has quit [Quit: Gettin' stinky!]
raster has joined #asahi
jeff_miw has joined #asahi
nmg_ has quit [Ping timeout: 246 seconds]
<jeff_miw> I tried to mess around to understand how macOS is managing the USB port but I failed miserably to have the boot/kernel to take my kext :(. if anyone has a good pointer to replace an existing original macOS by one hacked, I'm interested :)
nmg has joined #asahi
<sven> there are really three components to it: That TI chip connected over i2c to select the mode (DFP,UFP), the PHY (apple calls this one "ATC" I think, essentially magic register pokes required before accessing the controller), and this designware did controller (xhci + configuration for host mode, custom stuff for device mode)
<sven> part three is well documented if you just look hard enough. part one is mostly documented as well I think (except for the i2c part) and part 2 is completely custom apple stuff afaict
jeff_miw has quit [Ping timeout: 240 seconds]
<sven> *designware drd
raster has quit [Quit: Gettin' stinky!]
<arnd> maz: how does qemu/hvf do it?
<arnd> does it need a similar hack?
tomtastic_ has joined #asahi
tomtastic has quit [Ping timeout: 240 seconds]
Spiffy has quit [Quit: BRB]
Spiffy has joined #asahi
<maz> arnd: I have no idea, and to be honest I don't really want to know what macos does...
<maz> arnd: from what I understand, HVF gives you a prety high-level interface to the vcpu, and can do a lot of stuff in the hypervisor layer.
<arnd> ok, so it's not just emulated in qemu then
<maz> arnd: I doubt it. performance would be absolutely terrible, similar to what we end-up with running KVM on a RPi3...
<marcan> arnd, maz: as far as I know hypervisor.framework only deals with IRQ/FIQ, it does not do vGIC
<marcan> so it should all be emulated in userspace
<marcan> all you get is management of the pending IRQ state and the vtimer mask
<marcan> "After receiving a HV_EXIT_REASON_VTIMER_ACTIVATED exit reason, the caller of hv_vcpu_run() needs to make the interrupts to the VTimer pending in the guest’s virtual interrupt controller."
odmir has joined #asahi
lethalbit has quit [Read error: Connection reset by peer]
lethalbit has joined #asahi
vimal has joined #asahi
<maz> marcan: it has to do *some* vgic in the kernel, as the LRs cannot be populated from EL0. from what I see, their interface is exactly the same as KVM, except for the timer (and I don't really understand why).
<marcan> maz: I think they don't do vgic at all, and it's all emulated in userspace?
<maz> marcan: to do the GIC in userspace, you'd need to pass the trapping of all ICV_* regs to userspace.
<maz> plus the distributor and RD MMIO as well.
<maz> are they actually doing that?
<maz> that'd seem... silly.
<maz> why would they build a vgic the first place if they aren't using it...
<marcan> because this launch was very, *very* obviously rushed
<marcan> and they probably never got around to implementing it
<marcan> but there is nothing on IRQs in the API other than "IRQ, FIQ"
<maz> they don't have any exit reason for a trapped sysreg.
<maz> I think you are confusing two different things: injecting a IRQ/FIQ directly, and injecting a GIC interrupt.
<marcan> probably HV_EXIT_REASON_EXCEPTION?
<maz> hv_vcpu_set_pending_interrupt clearly injects a GIC interrupt, because there is an interrupt value provided to it.
<maz> IRQ?FIQ is just the grouping here.
<marcan> ah, I think you're right
<maz> (grouping in the GIC sense)
<marcan> I skimmed too quickly there
<maz> I think they have the distributor in userspace, while KVM has it in the kernel.
<marcan> yeah
<marcan> and in that case they probably don't use that magic LR
<maz> and that explains why userspace needs to mediate the vtimer as well.
<marcan> wait, no, are you sure?
<marcan> the structure is called hv_interrupt_type_t
<marcan> I think this is just an enum wrapper
<marcan> and it really is just IRQ and FIQ
<marcan> rawValue being the enum value
<maz> what is rawValue then?
<marcan> that
<marcan> IRQ, or FIQ
<marcan> that's really all there is to it
<maz> then I don't understand how this stuff works.
<marcan> I think all the registers trap...
<marcan> agraf would know :)
<maz> great for performance... I'll stay with my in-kernel, non-trapping handling, thank you very much.,
<marcan> maz: I'm really looking forward to this, running a few benchmarks side by side on qemu/macos vs qemu/kvm will make for good clickbait ;)
<maz> marcan: qemu needs fixing, there is a stupid but with the reduce IPA space.
<maz> reduced*
<marcan> ah, heh
<maz> kvmtool and crossvm are equially broken.
<j`ey> qemu on linux on m1 you mean?
<maz> yup.
raster has joined #asahi
<agraf> maz: I'm not aware of any hw backed GICv3 MSRs, but I haven't looked hard from EL2 either. HVF definitely just traps and delegates emulation to EL0 for everything. The only remotely irq related thing that's handled in-kernel is the arch timer, and that also just tells user space when it crossed cval.
<maz> agraf: KVM directly uses all the vgic registers without any trapping.
<maz> agraf: all the standard GICv3 stuff is there and seems functionnal.
<agraf> maz: oh, so they actually implemented all of it?
<agraf> That's funny - in a way :)
<maz> agraf: just enough to have a guest that doesn't suck too much.
<maz> agraf: they of course miss all the linked deactivation, but hey, sub-standard HW... ;-)
<agraf> maz: I'm definitely eager to see some comparison on native vgic vs the el0 loop for real world use cases. Probably has quite a nasty impact on IPI heavy bits
<agraf> maz: macos doesn't use any of the gic parts, so expect pitfalls as large as TX1 ;)
<agraf> I'm fairly confident it's not fully verified ip
<maz> agraf: running hackbench (as IPI-heavy as it gets) in a guest, I get a 6/7% overhead.
<maz> overhead compared to running on bare-metal.
<maz> agraf: as for bugs, I'm planning to taint the kernel with CPU_OUT_OF_SPEC at KVM-init time.
Hetflik[m] has joined #asahi
never_released has quit [Changing host]
never_released has joined #asahi
<marcan> ha
<marcan> OUT_OF_SPEC?
<marcan> what spec?
<marcan> :D
<never_released> maz: so there is a GICv3?
<never_released> in addition to AIC?
ldhacker[m] has quit [Quit: Idle for 30+ days]
<maz> never_released: no. what's available is the ICH_*_EL2 and the ICV__EL1 registerse.
<maz> registers*
<maz> never_released: you get enough to *virtualise* a GICv3.
<never_released> makes sense
<never_released> maz: pretty odd that macOS doesn't use that at all
<never_released> and just traps all the regs to the VMM through Hypervisor.framework
<maz> never_released: as agraf said, it's possible that it is totally broken.
<maz> never_released: but as long as the traps work, we can deal with that in the kernel, where we have all the required emulation already.
<never_released> knowing Apple, it's probably fully functional
<never_released> I wonder if the reg to trap the GIC regs is known though
zkrx has quit [Ping timeout: 245 seconds]
<maz> never_released: ICH_HCR_EL2.T{ALL0,ALL1,C}.
marvin24 has quit [Ping timeout: 260 seconds]
modwizcode has quit [Ping timeout: 246 seconds]
vimal has quit [Ping timeout: 246 seconds]
zkrx has joined #asahi
modwizcode has joined #asahi
nmg has quit [Quit: ZNC 1.7.2+deb3 - https://znc.in]
nmg has joined #asahi
nicolas17 has joined #asahi
marvin24 has joined #asahi
marvin24 has quit [Ping timeout: 240 seconds]
marvin24 has joined #asahi
vimal has joined #asahi
<marcan> I still bet it works ~fine and they just didn't make shipping for macos
<marcan> I mean ffs, the version of macos that shipped on these things had some hideous brokenness in the boot process; everyone was screaming "update before you even dare do anything"
marvin24 has quit [Ping timeout: 240 seconds]
marvin24 has joined #asahi
TheJollyRoger has quit [Quit: TheJollyRoger]
VinDuv has joined #asahi
mxw39 has joined #asahi
aratuk has joined #asahi
aratuk has quit [Remote host closed the connection]
<marcan> writing blog posts takes a while... I'm 3500 words into the progress report and I haven't even gotten to the linux kernel yet
<marcan> hopefully I can mostly wrap that up tomorrow
KindTwo has joined #asahi
KindOne has quit [Ping timeout: 272 seconds]
KindTwo is now known as KindOne
TheJollyRoger has joined #asahi
<jn__> wow, excellent
klaus has quit [Ping timeout: 272 seconds]
choozy has joined #asahi
aratuk has joined #asahi
aratuk has quit [Ping timeout: 245 seconds]
raster has quit [Quit: Gettin' stinky!]
amw has joined #asahi
marvin24 has quit [Ping timeout: 240 seconds]
marvin24 has joined #asahi
eta has quit [Ping timeout: 245 seconds]
artemist has quit [Ping timeout: 240 seconds]
VinDuv has quit [Quit: Leaving.]
eta has joined #asahi
Guest56 has joined #asahi
choozy has quit [Remote host closed the connection]
richbridger has quit [Quit: Leaving]
richbridger has joined #asahi