Why can't you have black box abstractions in things that "have to work"??
And even "has to work" can be relative, as if you can prove that something will work for a p value small enough to guarantee risk of failure is small enough in the next 100 years for a device/system that will at most stay in use for 10 it good enough... And if you're not constrained by power use or computing power, you can also use redundant subsystems: I imagine something like a satellite, receiving high radiation that consistently corrupts cache or RAM but runt 5 identical OSs on five identical subsystems and on every "high stakes" decision it does a "voting session", choosing the result of a function with the most out of 5 votes and resetting the state of the disagreeing systems with the winning ones, and a similar type of reliabilty by duplication for medical devices or aircraft systems... And going further, you can always replace an expensive reliable satellite with a swarm of cheaper ones that can fail. I think we'll choose the "swarm way" for more and more things and this will relax reliability requirements for individual components and at the whole swarm level we'll have "good enough" statistical reliability instead.
And any "swarm" can conceptually be seen as a black box, once you accept its probabilistic nature and the low but measurable likelihood of it being wrong, and maybe using a "swarm of swarms" if one swarm's p value of being true is not good enough for your application.
The "fractal attention to detail" seems needed when you need both reliability and you have serious power or computing resources constraints. And since power usage per flops is always decreasing, the number of such "dual constraint" cases should get smaller...
EDIT: typos and + pre-last paragraph to clarify what I meant
There are things you can't swarm. Security, for instance; a breach is a breach. Put the hardware you're trying to secure in the hands of the consumer -- the nominal attacker -- and this becomes Really Hard.
Swarms work great for server farms, or bespoke projects where cost is not an issue (e.g., that spacecraft, where you can afford multiple implementations of the system).
I can't see anyone throwing a swarm at a consumer product. Why would you waste the money? The pressure is to go to the edge and make it more reliable, and cheaper.
Black box abstractions that allocate memory are a problem. Presently, if you want to prove you will not run out of memory, you do it by not allocating memory or using a special purpose allocator.
point taken, the "p value" way of talking is confusing when talking to people doing real work and not reading articles, you're right (I was with my mind deep in reading some medical research articles actually so my mind "stuck" on this wording...)
but it's just about: probability of scooter A and B failing at the same time is their probabilities of failure multiplied and for a "swarm" the probability of everything failing at the same time decreases exponentially with the swarm size - eg. the probability of 10 scooters with a 50% each probability of failure failing at the same time is less than 0.01% ...thinking probabilities is what you end up doing even if you do smth like devops and try to do a good job at it when stretching resources thin :)
And even "has to work" can be relative, as if you can prove that something will work for a p value small enough to guarantee risk of failure is small enough in the next 100 years for a device/system that will at most stay in use for 10 it good enough... And if you're not constrained by power use or computing power, you can also use redundant subsystems: I imagine something like a satellite, receiving high radiation that consistently corrupts cache or RAM but runt 5 identical OSs on five identical subsystems and on every "high stakes" decision it does a "voting session", choosing the result of a function with the most out of 5 votes and resetting the state of the disagreeing systems with the winning ones, and a similar type of reliabilty by duplication for medical devices or aircraft systems... And going further, you can always replace an expensive reliable satellite with a swarm of cheaper ones that can fail. I think we'll choose the "swarm way" for more and more things and this will relax reliability requirements for individual components and at the whole swarm level we'll have "good enough" statistical reliability instead.
And any "swarm" can conceptually be seen as a black box, once you accept its probabilistic nature and the low but measurable likelihood of it being wrong, and maybe using a "swarm of swarms" if one swarm's p value of being true is not good enough for your application.
The "fractal attention to detail" seems needed when you need both reliability and you have serious power or computing resources constraints. And since power usage per flops is always decreasing, the number of such "dual constraint" cases should get smaller...
EDIT: typos and + pre-last paragraph to clarify what I meant