2024-01-04

<austriancoder> pitust: ane is tested/developed only for M1 based systems (I was told)
<pitust> ane->engine + ANE_TM_BASE + TM_STATUS reads to zero
<leio> cy8aer: I was told MLX is GPU stuff, not ANE stuff
<pitust> specifically, i get [ 1447.629031] ane 26bc04000.ane: tm execution failed w/ -110
<pitust> cool, now the ane userspace stuff... maybe works?
<pitust> what's the correct DT for the ANE driver? (@eiln)

2024-01-03

<sven> I think right now you’d have to write that file sent to ANE manually
<sven> keep in mind the ANE is essentially a fixed function thing that does matrix multiplication really well
<pitust> and how can i convert my own model to ANE stuff?
<pitust> j`ey: that's just the "send blobs built on macos" part of ane
<pitust> is the ANE's functioning understood, like, at all?

2023-12-19

<balrog> oh correction, I don't think it uses ANE, just CPU or GPU
<leio> (I haven't actually checked if that stuff is using ANE or GPU)
<leio> regarding ANE, apple open sourced this a few days ago (or the news circled then): https://github.com/ml-explore/mlx
<marcan> I'm actually curious whether I can use ANE to run some kind of voice recog/assistant model to replace Google Home (which I use as a Home Assistant frontend only these days)
<marcan> also I hear OpenCL on the GPU is not far off (which is more broadly useful for ML training and things that use fp32 and whatnot, while ANE is for more limited but efficient use cases)
<ydalton> oh that's useful, then some apps can integrate ANE support
<marcan> ydalton: eiln has a basic python toolset for "compiling" for ANE
<ydalton> does ane count? it's not in tree yet

2023-09-14

<ydalton> eiln: thanks for your answer! is there some kind of conformance suite i could use to see if ane is working properly?
<eiln> as for the engine itself, there's a pretty amusing story behind it. tldr they tried to maxx FOPS by literally stacking two identical t8103 ane's next to eachother, without any further effort, thus making no way for cross-device communication or memory sharing. now the m1 max has a fused-off (i.e. dead) ane. double for ultra
<ydalton> eiln: maybe a dumb question, but how different are the m1 neural engine and the m2 neural engine? you can't just slap "apple,t8112-ane" in ane_of_match?

2023-07-06

<ydalton> but ANE does have an rtkit right?
<ChaosPrincess> most of them do, i think ANE's one bypasses rtkit and directly pokes the hw registers

2023-06-29

<marcan> the Apple proprietary API for ANE (and other stuff) is CoreML. notably, it is fairly high level. chances are you will be able to do more stuff with our low level API, including non-ML stuff.
<marcan> I'm not aware of any particularly popular generic standard for ML accelerators, so ANE will probably get a bespoke API that various frameworks can add as a backend or whatever
<ktz> so practically it won't be possible to utilize both the ane and the gpu as seen on llama.cpp for example since it relies on metal which is a gpu API that is irrelevant in the linux context?
<ktz> ok apparently metal refers to the GPU, not the ANE. I've seen people mention metal in the context of llms so I assumed it had to do with the ANE.
<_jannau__> I don't think ANE has anything to do with metal, you want to look at https://github.com/eiln/anecc and https://github.com/eiln/ane
<ktz> ChaosPrincess: very interesting, so ANE is already partially functional more or less?
<ChaosPrincess> specifically for ggml, there is a open pr for asahi ane support
<ktz> I'm wondering, when ANE support comes online, will the API be combatible with apple metal?

2023-05-14

<dottedmag> Well, even when ANE is available, #include <Accelerate/Accelerate.h> in ggml.c won't work

2023-03-22

<blargh> Has anyone had a chance to look at the M2 max device tree and compare it with that of M1 max? Wondering if M2 max also has a hidden second ANE like M1 max had

2022-09-01

<fej487sn[m]> it seems ane is embedded into the fw instructions
<fej487sn[m]> ANE is triggered anytime the webcam is (even for non-apple sw like zoom) to enhance camera output, and im not sure if it can be separated

2022-08-20

<qcyrdqcpzg[m]> hey im new here so sorry if its already been asked but is there a team for the apple neural engine (ANE)? i used to work in CV so im really interested in it. i cant find anything much other than https://github.com/geohot/tinygrad/tree/master/accel/ane

2022-06-06

<opticron> kaprests, it might be nice if that type of functionality was expanded to more than just the camera, imagine if you could use a larger selection of hardware resources like that, CPU, ANE, GPU, camera, mics, speakers, etc. it's a lot easier with a nearly homogenous ecosystem

2022-03-09

<PthariensFlame[m]> Does the fact that having two ANEs hooked up at once is apparently not a problem tell us anything about why one (or two) of them are still disabled?
<PthariensFlame[m]> So what does the M1 Ultra officially having "32 Neural Engine core" mean for the ANE@1 on each die?

2022-03-08

<jannau> still only 1 ANE per m1 max

2021-11-18

<marcan> _jannau_: the second ane definitely exists, and is definitely disabled for some reason
<mini> Is the ANE the same between M1, M1 Pro and M1 Max?
<_jannau_> the second ane doesn't exists. I doubt anyone has drivers or uses a M1 pro/max in way that the differences would be noticeable
<Dcow> is it same state except the second ANE or something else is there?

2021-11-17

<balrog> marcan: is it possible that the ane that appears is determined by firmware and calibration data, so it remaps the device to ane0 regardless of which one it really is?
<opticron> yeah...just seems like a waste if the ANE is the only thing that's bad, but then again they don't have a secondary market for those chips like Intel or AMD
<opticron> re dual ANEs on the max: if it really is for binning, I wonder if they're just throwing away the M1Ps that have a dead ANE

2021-07-28

<sven> grep -lr RTKit /System/Volumes/Preboot/*/boot/*/usr/standalone/firmware/FUD/ results in SIO, ISP, AVE, DCP, AOP, ANE, GFX, DCP2 (???) and PMP

2021-02-01

<segher> jix, arnd: so AMX is for training neural nets, and ANE is for running them, it seems
<arnd> ANE seems to be its own processor core with shared memory, while AMX is the instruction set extension for the CPU itself and operates on vector registers
<arnd> It looks like ANE (apple neural engine) is bfloat16 and float32, while AMX is int16/float32/float64