marcan changed the topic of #asahi to: Asahi Linux: porting Linux to Apple Silicon macs | "Does XXX work yet?": https://alx.sh/fs | GitHub: https://alx.sh/g | Wiki: https://alx.sh/w | Topics: #asahi-dev #asahi-re #asahi-gpu #asahi-alt #asahi-stream #asahi-offtopic | Keep things on topic | Logs: https://alx.sh/l/asahi
mikelee has quit [Ping timeout: 480 seconds]
ourdumbfuture has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
ourdumbfuture has joined #asahi
mikelee has joined #asahi
Poplar has joined #asahi
darkapex has quit [Remote host closed the connection]
darkapex has joined #asahi
fenix144 has joined #asahi
mikelee has quit [Ping timeout: 480 seconds]
ourdumbfuture has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
roxfan2 has joined #asahi
roxfan has quit [Ping timeout: 480 seconds]
ourdumbfuture has joined #asahi
fenix144 has quit [Remote host closed the connection]
fenix144 has joined #asahi
zeroadrenaline has joined #asahi
<zeroadrenaline>
Hello from Argentina, new joiner to the channel and project. \O
fenix144 has quit [Ping timeout: 480 seconds]
jn has joined #asahi
jnn has quit [Ping timeout: 480 seconds]
<hgw7>
ok
mikelee has joined #asahi
gabuscus has quit []
mikelee has quit [Ping timeout: 480 seconds]
mrhh has joined #asahi
Brainium has quit [Quit: Konversation terminated!]
mrhh has quit []
mrhh has joined #asahi
zeroadrenaline has quit [Quit: Leaving]
zeroadrenaline has joined #asahi
zeroadrenaline has quit [Quit: This computer has gone to sleep]
mikelee has joined #asahi
gabuscus has joined #asahi
Hibyehello has joined #asahi
mikelee has quit [Ping timeout: 480 seconds]
ourdumbfuture has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
jeisom has quit [Ping timeout: 480 seconds]
mikelee has joined #asahi
mrhh1 has joined #asahi
mrhh has quit [Ping timeout: 480 seconds]
mrhh1 is now known as mrhh
fenix144 has joined #asahi
ourdumbfuture has joined #asahi
mikelee has quit [Ping timeout: 480 seconds]
mrhh has quit [Read error: Connection reset by peer]
fenix144 has quit [Ping timeout: 480 seconds]
mikelee has joined #asahi
Hibyehello has quit [Ping timeout: 480 seconds]
c10l563 has joined #asahi
dsrt^ has quit [Ping timeout: 480 seconds]
mikelee has quit [Ping timeout: 480 seconds]
c10l56 has quit [Ping timeout: 480 seconds]
ec0 has quit [Ping timeout: 480 seconds]
mrhh has joined #asahi
ourdumbfuture has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
mrhh has quit [Quit: mrhh]
mikelee has joined #asahi
mikelee has quit [Ping timeout: 480 seconds]
tristan2_ has joined #asahi
tristan2 has quit [Ping timeout: 480 seconds]
mikelee has joined #asahi
mikelee has quit [Ping timeout: 480 seconds]
leo60228- has joined #asahi
cylm has quit [Ping timeout: 480 seconds]
leo60228 has quit [Ping timeout: 480 seconds]
mikelee has joined #asahi
mikelee has quit [Ping timeout: 480 seconds]
mikelee has joined #asahi
mikelee has quit [Remote host closed the connection]
mikelee has joined #asahi
marvin24 has quit [Quit: marvin24]
mikelee has quit [Remote host closed the connection]
<rycloud>
The same actually works fine on Ubuntu 22.04 arm64 on a Raspberry Pi4 - so I'm not sure if it's an issue with qemu or with the kernel
<rycloud>
For my purpose, I'm trying to get Apache CloudStack to support Asahi Linux kernel with M1 Mac mini; so I need to use libvirtd and can't use qemu directly
<maz>
it could also be an old QEMU build that doesn't grok the IPA limitations of M1.
<maz>
rycloud: what is the version of your QEMU?
<maz>
(run "qemu-system-arm --version")
<maz>
or "qemu-system-aarch64 --version", which hopefully returns the same thing.
<dottedmag>
This sounds more like libvirt issue, really. IIRC it runs qemu-system-$arch at startup trying to guess the capabilities, and it might be passing an argument that is not supported on aarch64.
<maz>
OMG: "QEMU 6.2.0"
<dottedmag>
Though my memory is hazy, I haven't seen that code for a long time.
<maz>
there is no way this is going to run on M1.
<waldi>
even the 8.0.3 from arch-arm only gives me some IPA error
<maz>
waldi: if you get errors, please give me a detailed bug report.
<maz>
I run KVM 24/7 on M1/M2, and if things don't work, I really want to know.
<maz>
for the record, the first version of QEMU that has a chance to run with KVM on M1 is 7.0.
<maz>
anything before that is busted.
<rycloud>
I got this - root@macmini:~# qemu-system-arm --version QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.12) Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers root@macmini:~# qemu-system-aarch64 --version QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.12) Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
<rycloud>
Thanks maz dottedmag
<rycloud>
root@macmini:~# virsh version Compiled against library: libvirt 8.0.0 Using library: libvirt 8.0.0 Using API: QEMU 8.0.0 Running hypervisor: QEMU 6.2.0
<maz>
rycloud: you need to upgrade your QEMU. this version is too buggy to support the small IPA limit on M1 mini (it would work on an M1 pro though).
<rycloud>
Is there a known version/build of qemu working?
<maz>
rycloud: see above. at least 7.0.
<rycloud>
ah okay maz - is there a specific version I should try and try - copy
<waldi>
rycloud: qemu is not longer supported on ubuntu? this package is in universe
<rycloud>
root@macmini:~# virsh define s.xml error: Failed to define domain from s.xml error: unsupported configuration: Emulator '/usr/bin/qemu-system-aarch64' does not support virt type 'kvm'
mikelee has quit [Remote host closed the connection]
<maz>
OK, so you QEMU is OK. this looks like a libvirt problem, for which I'm utterly incompetent.
mikelee has joined #asahi
mikelee has quit [Remote host closed the connection]
<rycloud>
ah :) let me share the qemu commands it's running. It's possible a libvirt issue.
mikelee has joined #asahi
<rycloud>
I suppose I can try Debian and see it works, thanks maz
mikelee has quit [Remote host closed the connection]
<maz>
for a start, you should remove this "kvmclock" from your XML. it has no purpose on arm64, and may well be the cause of the error you're seeing
mikelee has joined #asahi
mikelee has quit [Remote host closed the connection]
<Guest6702>
could it be a permissions issue on /dev/kvm if libvirt isn’t running qemu as root?
<maz>
rycloud: let's not create additional problems by switching distro. we want to understand the issue, not paper over it.
mikelee has joined #asahi
<j`ey>
rycloud: its very likely just libvirt running qemu incorrectly
<maz>
Guest6702: no. we already know /dev/kvm accessible, since QEMU is able to open it (and rycloud is running as root...)
<waldi>
rycloud: you could use strace to get the correct command line libvirtd is about to execute. strace -fe execve -p $pid
<maz>
Guest6702: ah, I see what you mean now.
<Guest6702>
yeah but libvirt can setuid down i believe
<maz>
yeah, that's a potential issue. burn libvirt.
<maz>
(or set permissive permissions on /dev/kvm)
<Guest6702>
or maybe not setuid, but the daemon can be low privileged while controlled by a virsh running as root
<maz>
rycloud: "ls -l /dev/kvm"?
<rycloud>
yes it's available maz root@macmini:~# kvm-ok INFO: /dev/kvm exists KVM acceleration can be used root@macmini:~# ls -l /dev/kvm crw-rw---- 1 root kvm 10, 232 Jul 24 08:46 /dev/kvm root@macmini:~# lsmod | grep kvm root@macmini:~# modprobe kvm
<rycloud>
(I was running everything as root btw)
<rycloud>
waldi: sorry the process exits very quickly how do I run the strace command?
<waldi>
rycloud: you use the main libvirtd process
<rycloud>
ah okay :) I can also try to get the libvirtd logs if they help
<waldi>
the pid of that process. and hopefully it'll catch the exec of qemu
mikelee has quit [Remote host closed the connection]
<waldi>
this ought to read "virt" or so
<waldi>
rycloud: okay, so the log is enough in this case
mikelee has joined #asahi
<rycloud>
I got this in the xml though - <cpu mode='custom' match='exact' check='none'> <model fallback='forbid'>cortex-a57</model> </cpu>
<waldi>
rycloud: <model> sounds like what you want. but "virt"
mikelee has quit [Remote host closed the connection]
<rycloud>
I got a test VM xml that works on this machine with emulator qemu but when I change the domain type to 'kvm' it fails
<rycloud>
I got a test VM xml that works on this machine with emulator qemu but when I change the domain type to 'kvm' it fails
mikelee has joined #asahi
<rycloud>
waldi maz this domain/xml works but with qemu - https://pastebin.pl/view/9387ac0a but if I change the top xml <domain type='qemu'> to <domain type='kvm'> I get the error
mikelee has quit [Remote host closed the connection]
mikelee has joined #asahi
<rycloud>
(taking a break)
roxfan has joined #asahi
roxfan2 has quit [Ping timeout: 480 seconds]
rycloud has quit [Remote host closed the connection]
roxfan2 has joined #asahi
roxfan has quit [Ping timeout: 480 seconds]
rycloud has joined #asahi
mikelee has quit [Remote host closed the connection]
mikelee has joined #asahi
<rycloud>
So it seems instead of a host-passthrough or cpu model, if I use host it works on cmd line;
<rycloud>
/usr/bin/qemu-system-aarch64 -accel kvm -M virt -cpu host -m 2G -smp 2 -bios /usr/share/AAVMF/AAVMF_CODE.fd -drive if=none,file=/export/primary/disk,id=hd0 -vnc :1 -monitor stdio QEMU 7.2.0 monitor - type 'help' for more information (qemu) info kvm kvm support: enabled
<waldi>
rycloud: it misses the accel=kvm
<waldi>
so runs tcg emulated
<dottedmag>
Good luck figuring out how libvirt converts domain properties to QEMU command line. I have read this code some time ago, and it took me several days to stop weeping.
<maz>
waldi: "-accel kvm". old style QEMU command line.
<waldi>
maz: ups
mikelee has quit [Remote host closed the connection]
mikelee has joined #asahi
<maz>
dottedmag: I understand that people want a "universal" virt framework, but it really feels that libvirt is a horrible ball of tangled wires... I tried to use it twice, and wrote my own, KVM-specific ball of wires.
<dottedmag>
Yeah. I work in a company that makes a specialized cloud, and we are 1 step from ditching libvirt and making our own replacement.
<maz>
it's pretty funny how far you can go with just a couple of shell scripts and screen...
<dottedmag>
Exactly.
<dottedmag>
Even if one needs an abstraction over the virtualization, libvirt's abstraction is horrifyingly leaky.
<Guest6702>
absolutely, i wish i could have virt-manager but directly on qemu
roxfan has joined #asahi
leitao has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
roxfan2 has quit [Ping timeout: 480 seconds]
* mps
tried once about 15-20 (forgot exact year) to use libvirt and similar things with qemu and happily switched to shell scripts. never looked back
nsklaus has joined #asahi
<mps>
`qemu-system-aarch64 -bios QEMU_EFI.fd -machine virt -m 1G -cpu host -smp cores=4 -accel kvm -nographic -cdrom alpine-standard-3.17.2-aarch64.iso` is cli example how I run alpine alpine on different arm63 boxes
mikelee has quit [Remote host closed the connection]
mikelee has joined #asahi
mikelee has quit [Remote host closed the connection]
leitao has joined #asahi
Z750 has quit [Quit: Ping timeout (120 seconds)]
Z750 has joined #asahi
gladiac has joined #asahi
mikelee has joined #asahi
mikelee has quit [Remote host closed the connection]
gladiac has quit [Quit: k thx bye]
gladiac has joined #asahi
chadmed has quit [Ping timeout: 480 seconds]
ourdumbfuture has joined #asahi
mikelee has joined #asahi
chadmed has joined #asahi
mikelee has quit [Remote host closed the connection]
ourdumbfuture has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
leitao has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<rycloud>
Thanks mps, maz, waldi and all - I did several permutations, while I got it to sort of work directly using qemu but not via libvirtd yet. I'll explore further if upgrading libvirtd helps.