That's what I mean by gathering information through dynamic interaction. It's not explicitly given the rules, but it can infer them. Interacting with an external system and sampling the result is still a way of gathering training data.
In fact this is ultimately how we've gathered almost all the information we have. If it's in our cultural knowledge store it means someone observed or experienced it. Humans are very good at learning by sampling reality and then later systematizing that knowledge and communicating it to other humans with language. It's basically what makes us "intelligent."
A brain in a vat can't learn anything beyond recombinations of what it already knows.
The fundamental limit on the growth of intelligence is the sum total of all information that can be statically input or dynamically sampled in its environment and what can be inferred from that information. Once you exhaust that you're a brain in a vat.
This is so critical and so easy to forget. Every day is an effort to remind oneself of these basic truths.
“Whenever distress or displeasure arises in your mind, remind yourself, “This is only my interpretation, not reality itself.” Then ask whether it falls within or outside your sphere of power. And, if it is beyond your power to control, let it go.”
― Epictetus
The only way to be sure you understand everything
is to learn quantum mechanics perhaps string theory too just to be safe. But in practice modesty dictates we limit to learning as much as we can to get the job done.
I must disagree, not having the implications of laws obvious and runnable and checkable by anyone owning a computer is a big impediment to the democratic ideal. To have interpretations running in the brain of a specialized workers takes away from the ideal that anyone can know plainly what are the implications of laws. These laws can tie into economic models and make the ethical choices a law makes more obvious and checkable by anyone. There is are parallel between mathematical proof checking and computer proofs where flaws, loop holes in arguments can be more easily checked for. It also allows one to run counter simulations for alternative laws. And ideally can allow us to shift away from representative democracy to a more direct form allowing anyone to contribute to laws when they can show the new law can be better in some practical utilitarian ethical model. It also forces us to make our models of what a free ethical society should be more concrete rather than sticking to abstract buzzwords with open interpretations. Just a personal opinion.
It's a common misconception that the law can be so fine-grained. As the saying goes, the law is a sledgehammer, not a scalpel. If you look at cases with even a modicum of detail, you very quickly find situations which don't cleanly submit to classification. Code isn't very good at dealing with edge cases or fuzzy inputs.
Similarly, applying rigid code to law can make some crazy results, like the guy who got 25 years for stealing a slice of pizza. It was his 'third strike', and because the slice was stolen from a minor, it was classified as a felony. No sane person would argue for 25 years for a slice of pizza. However, the 'hardcoded' law demanded that, so the people are left picking up the bill for 25 years of imprisonment over a $1 food item (independent of any moral arguments on the issue).
Finally, 'ethics' is ridiculously hard to clearly classify, and if we can't classify it, we can't encode it.
Not saying that any of this is even remotely easy. Simply pointing out what should be some form of ideal. The laws are not meant to be rigid or hardcoded but to be striving for some ideal. The ideal are the axiomatic input to the systems and thus the center of the flaws in our thinking. Thus society that keep redefining what it means to live in an ideal society by looking at the consequences of our laws. But not considering any of this is simply to avoiding thinking about the consequences of the current state of affairs, where many get disenfranchised by the current system as can been seen quite clearly in racial/economic skews of populations in the prison system. The three strikes law is a perfect example of a law where the consequences of the laws which though not perfectly predictable are also not perfectly unpredictable but a consequence the inherent prejudices of people being arbitrarily enacted into laws with little examination of the consequences. To have an explicit legal system would need the explicit spelling out of these prejudices and at least provide a coherent narrative which can be argued against with data and alternatives.
> To have an explicit legal system would need the explicit spelling out of these prejudices and at least provide a coherent narrative which can be argued against with data and alternatives.
