Where is that low-power x86 and its fleet of low-power support chips to do what something like an Armada or EnergyCore can do on its own? They have yet to come up with one.
As for your 8051 example, it's bogus because that chip simply can't do the work. It can't run a real OS, and even if it could do that it couldn't keep even a single Ethernet or SATA port busy. Therefore you'd need a lot more nodes, each with their own network/storage/memory that don't come for free, rapidly wiping out any savings on the CPU alone before you even get to the high-node-count coordination problems that would make the whole thing fall flat on its face.
The whole issue here is not just absolute minimum power but balance. In a server environment, where the processor's job is about keeping ports full more than about pure number-crunching, you have to start with what kinds of ports you have. What processor and memory most exactly matches a commodity I/O profile, neither running over nor falling short, while consuming the fewest watts? Modern ARM chips are often a better answer to that question than anything Intel makes. It's a shame that some people who've invested many years in x86-specific expertise might find the market for those skills eroding as a result, but that's the harsh reality.
A quad-core Xeon is overkill for a static content webserver. An ARM is overkill for an embedded controller. If you specify inappropriate hardware then you'll end up with an inappropriate power/performance ratio.
As for low-power x86 - HP's Moonshot is in the ballpark of ARM blade devices, and Baytrail pushes Intel even closer. ARM probably still wins, but the figures are nothing like 15x. And once you take fixed costs like disk and RAM into account, the difference ends up being even smaller.
ARM have done a great job of improving the performance of their cores. Intel have done a great job of cutting the x86 power budget. Given that nobody's really shipping ARM servers yet, it's still not clear who's going to come up with the better product. The problem that ARM face is that they not only have to be better, they have to be sufficiently better that it's worth the cost of porting in-house applications to a new architecture.
As for your 8051 example, it's bogus because that chip simply can't do the work. It can't run a real OS, and even if it could do that it couldn't keep even a single Ethernet or SATA port busy. Therefore you'd need a lot more nodes, each with their own network/storage/memory that don't come for free, rapidly wiping out any savings on the CPU alone before you even get to the high-node-count coordination problems that would make the whole thing fall flat on its face.
The whole issue here is not just absolute minimum power but balance. In a server environment, where the processor's job is about keeping ports full more than about pure number-crunching, you have to start with what kinds of ports you have. What processor and memory most exactly matches a commodity I/O profile, neither running over nor falling short, while consuming the fewest watts? Modern ARM chips are often a better answer to that question than anything Intel makes. It's a shame that some people who've invested many years in x86-specific expertise might find the market for those skills eroding as a result, but that's the harsh reality.