I'm guessing that the "ML accelerator" in the CPU cores means one of ARM's SME extensions for matrix multiplication. SME in ARM v8.4-A adds dot product instructions. v8.6-A adds more, including BF16 support.
Apple has the NPU (also called Apple Neural Engine), which is specific hardware for running inference. Can't be used for LLMs though at the moment, maybe the M4 will be different. They also have a vector processor attached to the performance cluster of the CPU, they call the instruction set for it AMX. I believe that that one can be leveraged for faster LLM inferencing.
Clever wording on their part: 2x performance per watt over M2. Took me a minute, had to reason through this is their 2nd generation 3nm chip, so it wasn't from a die shrink, then go spelunking.
This claim can only be evaluated in the context of a specific operating point. I can 6x the performance per watt of the CPU in this machine I am using by running everything on the efficiency cores and clocking them down to 1100MHz. But performance per watt is not the only metric of interest.
It surprised me they called it an M4 vs an M3 something. The display engine seems to be the largest change I don't know what that looked like on previous processors. Completely hypothesizing but could be a significant efficiency improvement if its offloading display stuff.
I'd rather they just keep counting up than some companies where they get into wonky product line naming convention hell.
It's ok if 3 to 4 is or isn't a big jump, it's the next one is really all I want to know. If I need to peek at the specs, the name really won't tell me anything anyhow and I'll be on a webpage.
