uniq has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
thelounge757 has quit [Remote host closed the connection]
<skmp__>
which seems to be highly informative and a pretty good read, if the codebase is still similar
thelounge757 has joined #asahi-dev
DarkShadow44 has joined #asahi-dev
thelounge7571 has joined #asahi-dev
amarioguy has joined #asahi-dev
thelounge757 has quit [Ping timeout: 480 seconds]
e1eph4nt has quit [Ping timeout: 480 seconds]
e1eph4nt has joined #asahi-dev
e1eph4nt has quit [Ping timeout: 480 seconds]
e1eph4nt has joined #asahi-dev
riker77_ has joined #asahi-dev
riker77_ is now known as riker77
thelounge7571 has quit [Remote host closed the connection]
uniq has joined #asahi-dev
thelounge7571 has joined #asahi-dev
uniq has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
e1eph4nt has quit [Ping timeout: 480 seconds]
thelounge7571 has quit [Remote host closed the connection]
thelounge7571 has joined #asahi-dev
e1eph4nt has joined #asahi-dev
e1eph4nt has quit [Ping timeout: 480 seconds]
claraaudbtc[m] has joined #asahi-dev
e1eph4nt has joined #asahi-dev
amw has quit [Ping timeout: 480 seconds]
thelounge7571 has quit [Remote host closed the connection]
thelounge7571 has joined #asahi-dev
uniq has joined #asahi-dev
amw has joined #asahi-dev
SSJ_GZ has joined #asahi-dev
<marcan>
skmp__: I honestly have no idea if you're looking to add "real" 4K support to the kernel. I imagine it won't be easy, but probably not much better resource than reading the code itself.
<marcan>
one thing that did get merged semi recently is page folios, which are used in the other direction (higher order pages), but might be of use here?
uniq has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
e1eph4nt has quit [Ping timeout: 480 seconds]
thelounge7571 has quit [Remote host closed the connection]
thelounge7571 has joined #asahi-dev
<marcan>
maybe the ticket is to go the other way and start from a nominally 4K kernel, then change the allocation/tracking to always use 16K page folios by default, then figure out how to turn the kernel address space and a subset of process address spaces into 16K mode
e1eph4nt has joined #asahi-dev
c10l has joined #asahi-dev
claraaudbtc[m] has quit [autokilled: This host violated network policy and has been banned. Mail support@oftc.net if you think this is in error. (2022-09-08 05:01:58)]
clararussel[m] was banned on #asahi-dev by ChanServ [*!~clararus_@2001:470:1af1:101::*]
clararussel[m] was kicked from #asahi-dev by ChanServ [You are not permitted on this channel]
thelounge7571 has quit [Remote host closed the connection]
thelounge7571 has joined #asahi-dev
uniq has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
e1eph4nt has quit [Ping timeout: 480 seconds]
e1eph4nt has joined #asahi-dev
<skmp__>
@marcan: I'd be interested for this use case (emulation in a system that the 'best' page size isn't 4k) for now, so not sure if it needs to be 'real' 4k or not
<skmp__>
yeah, that documents some form of higher order pages and also gives context on how things fit together as well as gotchas
<marcan>
If you just want to add support to FEX for 16K systems without kernel changes, then it's mostly just about playing with the mmap/etc wrappers and doing some ugly tricks when you get unaligned ELF sections
<skmp__>
I'm familiar with the vm 'frontend' (application <-> vma ranges interactions) as that was needed to cross check that the userspace vma tracking matched kernel
thelounge7571 has quit [Remote host closed the connection]
<skmp__>
yeah I recall that, I took some notes
<skmp__>
(though i'm no longer directly involved in fex)
<skmp__>
there's one complication of mremap that wasn't accounted in our previous discussion there
<skmp__>
but nothing that can't be worked around - more so since userspace vma mappings are used
<skmp__>
s/used/kept
<skmp__>
though, I'm very uneasy of how many unhandled cases that approach has
<marcan>
All approaches to this problem are going to be either incomplete or very difficult to cover all the cases :)
<marcan>
it's a difficult problem
<skmp__>
but then again, I've been working in user space that intimetally plays with specific page sizes and semantics since for ever at this point
<skmp__>
so maybe I overvalue how much games do the same
<skmp__>
it's a fun little problem to think before bedtime :P
<skmp__>
there's also the extra complication of interworking, even if the guest is happy and all
<skmp__>
we still need to be able to use host libraries in the same address space - say host opengl driver stack
<skmp__>
to transparently forward
<skmp__>
I guess I also see the value of the NT kernel only supporting '64k' segments for maps even on '4k' page hosts - as much as I hated it back then I had to work around it
<skmp__>
(s/supporting/exposing IG)
<skmp__>
I also poked quite a bit in that userspace paging idea - which I find much, much more exciting and extendable
<skmp__>
I'm not sure a fix specifically for x86/64 emulation for linux on m1 would justify inclusion in the kernel
thelounge7571 has joined #asahi-dev
<skmp__>
maybe something more exotic that opens new frontiers would, but then again maybe the scope is much better so it won't
<skmp__>
though, conceptually, it would avoid (forward to userspace) all of those concerns
<skmp__>
another semi-programmable middle ground could be to re-repurpose BPFs and give hooks where userspace can do what it wants, akin to how seccomp does
<skmp__>
though, honestly, this 'bpf as a secure vm in kernel space' thing is something I don't like so much
<skmp__>
I like* so much (damn today I keep mistyping, brb coffee + cat petting time)
e1eph4nt has quit [Ping timeout: 480 seconds]
mps has joined #asahi-dev
e1eph4nt has joined #asahi-dev
thelounge7571 has quit [Remote host closed the connection]
e1eph4nt has quit [Ping timeout: 480 seconds]
thelounge7571 has joined #asahi-dev
amw_ has joined #asahi-dev
amw has quit [Ping timeout: 480 seconds]
e1eph4nt has joined #asahi-dev
uniq has joined #asahi-dev
thelounge7571 has quit [Remote host closed the connection]
<marcan>
skmp__: for 4K apps running with native kernel support the whole process would be 4K. system libraries built to support 16K systems can run fine on 4K kernels.
<marcan>
you'd only have an interworking problem if those system libraries for some reason used shared memory / protocols that depend on the page size
<marcan>
but that can probably be dealt with on a case by case basis
uniq has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<marcan>
so really there's two approaches here, just run in 16K mode and play around with syscalls/loader stuff to fake-support 4K as much as possible, or do the hard work of making the kernel support mixed page sizes and just run every FEX process in 4K mode (which is what Rosetta does).
<marcan>
I personally don't think userspace paging makes any sense, since getting it right involves as much complexity in the kernel as just supporting 4K pages, and getting it wrong means massive security vulnerabilities.
thelounge7571 has joined #asahi-dev
<j`ey>
you'd probably want to talk to upstream maintainers about mixed page sizes too, it's a lot of work and you want to find out if they're open to it first?
uniq has joined #asahi-dev
<j`ey>
jannau: I just realised that robin ack'd the v3, so I think that means it's waiting for joerg to pick it up now?
e1eph4nt has quit [Ping timeout: 480 seconds]
e1eph4nt has joined #asahi-dev
thelounge7571 has quit [Remote host closed the connection]
<jannau>
has someone recently seen spi hid probe errors with "waiting for device info failed: -19" (mb pro 14" m1 max)? might be related to having a usb-c cable conencted to another apple silicon machine
<jannau>
on both machines the blessed port with uart
<jannau>
appears to a "hardware" issue as u-boot is affected as well. differently though, auto-boot stops with a few random characters on the prompt
<jannau>
not reproducible anymore
<maz>
jannau: pinging Joerg to make sure that this is on his radar wouldn't hurt.
chadmed has quit [Read error: Connection reset by peer]
e1eph4nt has quit [Ping timeout: 480 seconds]
thelounge7571 has quit [Remote host closed the connection]
e1eph4nt has joined #asahi-dev
Race has joined #asahi-dev
chadmed has joined #asahi-dev
e1eph4nt has quit [Ping timeout: 480 seconds]
<sven>
yeah, would be nice if we finally get t6k dart support in mainline
thelounge7571 has joined #asahi-dev
thelounge7571 has quit [Ping timeout: 480 seconds]
pjakobsson has joined #asahi-dev
e1eph4nt has joined #asahi-dev
rpirea has joined #asahi-dev
<jannau>
done
thelounge7571 has joined #asahi-dev
<skmp__>
> @marcan you'd only have an interworking problem if those system libraries for some reason used shared memory / protocols that depend on the page size
<skmp__>
opengl, vulkan
<skmp__>
that's all I care about
<skmp__>
@j`ey yeah, I'm just trying to figure out the upstreams right now
<skmp__>
and the relevant code / people to discuss
Race has quit [Ping timeout: 480 seconds]
<skmp__>
there's some tricks that can be used to make it less work
<skmp__>
but one way or the other, if people want to have x86 emulation, 4K pages are a given
<skmp__>
if upstream doesn't want to collaborate with the end users, people will end up keeping patchsets
<skmp__>
and, m1 takes a big (iirc 12%ish) perf hit from using 4k pages, so it will be very hard to convince a distro to do that
<marcan>
skmp__: I don't think that's going to be a common problem, realistically, and it would probably be something that those upstream libraries would accept fixes for (e.g. some way of determining the system *max* page size, not the *current* page size, and using that)
<skmp__>
oh possibly
<skmp__>
I'm an emulator author, my job is to predict how people will misasume things and then it'll end up on my bug tracker as a bug on my project
<marcan>
opengl and vulkan don't care, that all goes through GEM/whatever on the kernel side and the driver there would allocate things using the right page size
<skmp__>
which, technically it is, because i'm providing the vm
<marcan>
if anything I expect some corner cases with, like, pulseaudio and/or pipewire, that kind of thing
<jannau>
skmp__: I think it will not hard to convince distro's to enable apple silicon support in their standard arm64 4k kernels
<marcan>
they are already doing that
<skmp__>
maybe
<jannau>
probably harder to convince them to carry a 16k special kernel for apple silicon
<skmp__>
branding wise, it will look really bad
<marcan>
it's going to be harder to convince them to have 16K kernels as an alter- yeah what jannau said
<skmp__>
'ubuntu is 10% slower than osx - apple is right osx is best'
<skmp__>
no one likes that narative
<marcan>
well, linux is already 10% faster than osx because osx is bad
<marcan>
so :)
<skmp__>
rotfl
<maz>
also, if you think os-x is better, go use it. we're not presecriptive.
<skmp__>
well, 4k standar page size isn't really a 'standard'
<jannau>
once 4k iommu support is upstream of course
<skmp__>
it's only been standard for a decade or so at best
<skmp__>
plus, 16k is the right size
<skmp__>
apple is right there
<skmp__>
4k is too small for the amount of ram we have
<skmp__>
and 16K is just so much more power efficient at the tlb cache level
<marcan>
a more practical thing might be to teach linux how to decide on the page size at *boot time*
<skmp__>
(because it holds 4x as much mapped area)
<marcan>
it's going to involve making a huge amount of changes to compile-time constants but at least it wouldn't require a massive rework of the VM subsystem
<marcan>
and then maybe we can have multi-pagesize kernels and distros just ship that
<mps>
uhm, so I have to make 64K kernel for neoverse-n1 machine with 512GB ram
<marcan>
(and it could still be a stepping stone towards mixed page sizes)
<skmp__>
@jannau The iommu has 4k page support?
<mps>
jk
<skmp__>
@mps that's how it used to be up to early 2000s
<skmp__>
IBM always used random page sizes
<skmp__>
microsoft used 1K for wince
<marcan>
skmp__: no, that's what the 4K support patch is for, working around that
<skmp__>
everyone just used 'whatever was best'
<mps>
skmp__: yes, I rmember
<skmp__>
I member
<skmp__>
:p
<mps>
:)
<skmp__>
marcan: so it is not a strict hardware limit?
<skmp__>
if so, a boot option is the best 'hack' right now
<marcan>
it is a strict hardware limit, but DMA is special anyway
<marcan>
it is much easier to teach the kernel to allocate 16K pages for DMA than it is to teach it mixed page sizes with userspace involved
<skmp__>
Well, one could say that says more about the kernel, than the feature
<skmp__>
but anyways
<skmp__>
the point is getting this in front of the users atm
<skmp__>
not discussing how we got into this mess
<marcan>
I say just fake 4K in userspace first, see how far that gets you
<marcan>
it will be a lot easier and get you results much faster than any attempt at messing with the kernel
<skmp__>
uh, maybe
<marcan>
if that's unsatisfactory, we can start talking about kernel changes, but any reasonable approach there will likely take a very long time to settle on and be implemented, if practical at all
<skmp__>
but doing it in userspace is a waste of time as well
<skmp__>
as it won't work for all solutions anyway
<skmp__>
so we'll have to dive into the kernel
<skmp__>
and after 16 hours of emulator programming, i'd rather chillax with some easy kernel work
<skmp__>
rather than this
<marcan>
if it gets you 90% of the way there now it's worth it over being 0% of the way for years while someone figures out how to do this in the kernel, if it's even practical
<marcan>
"easy" kernel work lol
<marcan>
I still think you are vastly underestimating how hard this is on the kernel side
<skmp__>
have you seen what is involved in the emulation side?
<mps>
hm, F2FS still works only on 4K kernels
<marcan>
have you seen what is involved in the kernel side?
<marcan>
:)
<skmp__>
yeah
<skmp__>
i've read large parts
<skmp__>
mostly to work around kernel bugs
<skmp__>
it's pretty easy to understand, simple code
<skmp__>
for the most part
<skmp__>
a bit love for gotos and labels but oh well
<marcan>
8573 instances of "struct page", all of which could assume pages are fixed size :)
<skmp__>
that reminds me I need to notify someone about that signal handling bug I found
<marcan>
but hey, if you think it will be easy to teach Linux multiple page sizes, go for it!
<skmp__>
Tech is easy man
<skmp__>
it's been like that for a long time now
<skmp__>
the politics make it hard
<skmp__>
code is ephemeral anyway
<skmp__>
you write it now, delete it tomorrow
<skmp__>
that's the nature of it
<marcan>
that's definitely another aspect, you're going to spend a lot more time in politics and a lot less time writing code on the kernel side
<skmp__>
yea that's what I'm afraid there
<marcan>
you should be
<skmp__>
Well, I disagree
<skmp__>
A project where contributors are afraid of politics is uh
<skmp__>
a project
<skmp__>
I guess
<marcan>
I mean given the reality of how Linux is developed
<skmp__>
but that doesn't change reality much
<marcan>
I'm not saying it's the best state of affairs
<marcan>
I'm saying you should be scared given the current state of affairs :)
<marcan>
so I would highly suggest having a long discussion with the mm folks about this
<marcan>
about what is practical and what isn't
<marcan>
before trying anything
<skmp__>
yes, possibly
<skmp__>
well, I want to do it mostly for the lulz tbh
<skmp__>
You know my wish for userspace paging anyway
<marcan>
yeah lol
<skmp__>
though possibly a BSD variant is best
<skmp__>
for that
<skmp__>
(or fuscia)
<skmp__>
(but eh, google, ...)
<marcan>
well, thing is here we actually want things upstream so... "for the lulz" isn't really useful unless it's actually a good solution that convinces upstream to change their opinion
rpirea has quit [Quit: rpirea]
<skmp__>
well, that's why I'm here
<skmp__>
I want to have popcorn time for me
<skmp__>
and also maybe be of use
rpirea has joined #asahi-dev
<marcan>
I'm not going to stop you from doing whatever you want with the kernel, but don't expect us to merge it either even in our tree unless it's seriously beautiful and clearly correct :p
<marcan>
because I am not qualified to evaluate a deep VM change like this for security impact, and I am not risking that
<marcan>
(thus: you need to talk to mm folks)
e1eph4nt has quit [Ping timeout: 480 seconds]
<skmp__>
Well, if I really wanted to have to run, I can always just patch the kernel at runtime to do what I want
<skmp__>
no need for source or anything
<skmp__>
;)
<skmp__>
will hardly be the first time
<skmp__>
but I think the point is more of how to not waste time and energy on fun and possibly meaningless endeavours
<skmp__>
got any more specific contact for the 'mm folks' ?
<marcan>
talking to the mm folks would be a good idea to figure out what is meaningful ;)
<skmp__>
Dear mm folks, .. :P
<marcan>
not really, but MEMORY MANAGEMENT in MAINTAINERS will get you a name and a special purpose mailing list ;)
<marcan>
(feel free to cc asahi@lists.linux.dev, we care about this!)
<skmp__>
alright
<skmp__>
thanks a ton
<skmp__>
in another note, how's the gpu side of things coming along?
<skmp__>
I'd really love to take a look there
<skmp__>
the name (AGX?) is peculiar
<skmp__>
it kind of implies some imgtec lineage
<marcan>
you want to talk to Lina about that (she's in #asahi-gpu)