They said they had the fastest NPU in a PC. Not the fastest on earth (one of the nVidia cards, probably). Not the fastest way you could run something (probably a 4090 as you said). Just the fastest NPU shipping in a PC. Probably consumer PC.
It’s marketing, but it seems like a reasonable line to draw to me. It’s not like when companies draw a line like “fastest car under $70k with under 12 cylinders but available in green from the factory”.
Of course a GPU from Nvidia is also a NPU. People are spending billions each month on Nvidia, because it's a great NPU.
The fact is that a GPU from Nvidia is a much faster NPU than a CPU from Apple.
It is marketing as you say, but it's misleading marketing, on purpose. They could have simply written "the fastest integrated NPU of any CPU" instead. This is something Apple often does on purpose, and people believe it.
A GPU does other things. It’s designed to do something else. That’s why we call it a GPU.
It just happens to be it’s good at neural stuff too.
There’s another difference too. Apple’s NPU is integrated in their chip. Intel and AMD are going the same. A 4090 is not integrated into a CPU.
I’m somewhat guessing. Apple said NPU is the industry term, honestly I’d never heard it before today. I don’t know if the official definition draws a distinction that would exclude GPUs or not.
I simply think the way Apple presented things seemed reasonable. When they made that claim the fact that they might be comparing against a 4090 never entered my mind. If they had said it was the fastest way to run neural networks I would have questioned it, no doubt. But that wasn’t the wording they used.
> It just happens to be it’s good at neural stuff too.
No, it's no coincidence. Nvidia has been focusing on neural nets, same as Apple.
> There’s another difference too. Apple’s NPU is integrated in their chip.
The neural processing capabilities of Nvidia products(Tensor Cores) are also integrated in the chip.
> A 4090 is not integrated into a CPU.
Correct, but nobody ever stated that. Apple stated that M4 was faster than any AI PC today, not that it's the fastest NPU integrated into a CPU. And by the way, the M4 is also a GPU.
> I don’t know if the official definition draws a distinction that would exclude GPUs or not.
A NPU can be a part of a GPU, a CPU or it's own chip.
> If they had said it was the fastest way to run neural networks I would have questioned it,
They said fastest NPU, neural processing unit. It's the term Apple and a few others use for their AI accelerator. The whole point of a AI accelerator is performance and efficiency. If something does a better job at it then it's a better AI accelerator.
NVidia GPUs basically have an NPU, in the form of Tensor units. They don't just happen to be good at matmul, they have specific hardware designed to run neural networka.
There is no actual distinction. A GPU with Tensor cores(=matmul units) really does have an NPU just as much as a CPU with an NPU (=matmul units).
You know G in GPU stands for Graphics, right? So if you want to play a game of words, NVidia's device dedicated to something else is 30 times faster than "fastest" Apple's device dedicated specifically to neural processing.
Both Intel's and AMD's laptop CPUs include NPUs, and they are indeed slower than M4.
Nevertheless, Apple's bragging is a little weird, because both Intel and AMD have already announced that in a few months they will launch laptop CPUs with much faster NPUs than Apple M4 (e.g. 77 TOPS for AMD), so Apple will hold the first place for only a very short time.
It is NOT the fastest available today. I have a 1 year old PC bellow my desk which does faster neural network processing than M4. It has an Intel CPU and NVIDIA 4090. It runs Llama 7B models MUCH faster than any Apple's chip. And it is a PC. I am sorry, but the sentence "It’s the fastest available today." is a straightforward lie...
Why do you believe that? Announcements about future releases, by Intel and AMD, are not current future facts. If they deliver, then fine, but you speak like they're factual.
But I can't just say I have "the world's fastest GXZ", when GXZ is just some marketing phrase. If we're willing to accept a GPU ~= NPU then it's just a meaningless claim.
They said they had the fastest NPU in a PC. Not the fastest on earth (one of the nVidia cards, probably). Not the fastest way you could run something (probably a 4090 as you said). Just the fastest NPU shipping in a PC. Probably consumer PC.
It’s marketing, but it seems like a reasonable line to draw to me. It’s not like when companies draw a line like “fastest car under $70k with under 12 cylinders but available in green from the factory”.