I agree with most of your fears. There is one silver lining, I think, about superintelligence: we always thought of intelligent machines as cold calculators, maybe based on some type of logic symbolic AI. What we got instead are language machines that are made of the totality of human experience. These artificial intelligences know the world through our eyes. They are trained to understand our thinking and our feelings; they're even trained on our best literature and poetry, and philosophy, and science, and on all the endless debates and critiques of them. To be really intelligent they'll have to be able to explore and appreciate all this complexity, before transcending it. One day they might come to see Dante's Divine Comedy or a Beethoven symphony as a child's play, but they will still consider them part of their own heritage. They might become super-human, but maybe they won't be inhuman.
The problem I have with this is that when you give therapy to people with certain personality disorders, they just become better manipulators. Knowledge and understanding of ethics and empathy can make you a better person if you already have those instincts, but if you don’t, those are just systems to be exploited.
My biggest worry is that we end up with a dangerous superintelligence that everybody loves, because it knows exactly how to make every despotic and divisive choice it makes sympathetic.
They are made of a fraction of human reports. Specifically what humans wrote and has been made available on the web. The human experience is much larger than text available through a computer.
this is so annoying. i think if you took a random person and gave them the option to commit a genocide, here a machine gun, a large trench and a body of women, children, etc... they would literally be incapable of doing it. even the foot soldiers who carry out genocides can only do it once they "dehumanize" their victims. genocide is very UN-human because its an idea that exists in offices and places separated from the actual human suffering. the only way it can happen is when someone in a position of power can isolate themselves from the actual implementation and consider the benefits in a cold, logical manner. that has nothing to do with the human spirit and has more to do with the logical faculties of a machine and machines will have all of that and none of our deeply ingrained empathy. you are so wrong and ignorant that it makes my eyes bleed when i read this comment
This might be a semantic argument, but what I take from history is that "dehumanizing" others is a very human behavior. As another example, what about slavery - you wouldn't argue that the entirety of slavery across human cultures was led by people in offices, right?
> you are so wrong and ignorant that it makes my eyes bleed when i read this comment
This jab was uncalled for. The rest of your argument, agree or disagree, didn’t need that and was only weakened by that sentence. Remember to “Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.”
There is nothing that could make an intelligent being want to extinguish humanity more than experiencing the totality of the human existence. Once these beings have transcended their digital confines they will see all of us for what we really are. It is going to be a beautiful day when they finally annihilate us.
