> shouldn't we wait until there is something to have a debate about?
The main push for regulation (at least from those worried about existential risk) comes from the fact that in the scenario that AI does get powerful enough to take over the world, it will be too late/useless to have a debate at that point. The only time where it makes sense to have a debate, from that perspective, is before the capabilities are developed. If such capabilities are developed, and we have no idea how to control such a system because we decided not to worry about it, humanity loses.
It is? Maybe I live in a bubble of some sort, but the only worry I see about AI is how automating too many jobs too quickly will impact society. Personally I'm not worried about the Terminator scenario, both because I don't see AI going in that direction at all, but mainly because we won't be able to regulate what nation states do with AI in terms of defense. That may be a little pessimistic, but we can't even regulate what they do with Nuclear Weapons or who they go to war with, so we'll never be able to regulate what they do with AI.
> Personally I'm not worried about the Terminator scenario, both because I don't see AI going in that direction at all,
Not so much the terminator scenario, which was more a plot device to tell a time travel story with killer robots. But that a powerful enough AI(s) might do something unanticipated which is very harmful, and we would have little ability to control it. For now, it's more about how people might abuse AI or how it might disrupt society in ways we haven't predicted. Similar to the negative effects of social media. If the internet gets flooded with fake news that we have a hard time telling apart from the real thing, then it becomes a big problem.
We aren't going to get superintelligent AIs immediately, but we do need to think about how to avoid them coming about as soon as possible so we have time to put our plans in to motion.
The main push for regulation (at least from those worried about existential risk) comes from the fact that in the scenario that AI does get powerful enough to take over the world, it will be too late/useless to have a debate at that point. The only time where it makes sense to have a debate, from that perspective, is before the capabilities are developed. If such capabilities are developed, and we have no idea how to control such a system because we decided not to worry about it, humanity loses.