There are already above intelligence computer programs and we have subjugated each one.
There are already greater than human organisms that are reliably the biggest danger and boon to humanity but are also composed of humans; i.e. societies
The effort to build a program which beats defeats a specific program (as opposed to a general case) is significantly easier than maintaining a general lead. i.e. easier to kill a bad head of state than to run one, but this is also true of "AI" models now
There are already complex systems vastly beyond human understanding, and certainly our control which are indifferent to our survival and yet we persist.
It's not that "AI" isn't a potential danger, just that it's one of many we are already enduring. Personally there are far more present and likely hazards I am focusing on before getting to the hypothetical ones. Even if humanity fails to fumble its way through this hazard, it's not an end but a new beginning.
I believe that you have a fine understanding of the issues involved. The risk is two-sided. One part of it is that AI may somehow become smarter than us. The other side is that humans have developed mathematics, which encourages us to treat the universe as an optimization problem, which encourages us to think that the best optimizers are the best people and that they deserve great rewards for optimizing, which leads to competition at optimizing optimization, serious negative feedback for the losers, single winner systems, and all the winners realizing that the more they resemble their enemies, the less likely they are to be targeted as a resource to plunder. Perhaps we are not able to figure out if any of this is wrong, but the emphasis on convergent thinking will make our species easy to fool and sabotage, if our species doesn't win a Darwin award first. AI will be able to avoid the blame. It may be optimizing the fire department and selling fire insurance when civilization burns down, but the fire started millennia ago.
There are already greater than human organisms that are reliably the biggest danger and boon to humanity but are also composed of humans; i.e. societies
The effort to build a program which beats defeats a specific program (as opposed to a general case) is significantly easier than maintaining a general lead. i.e. easier to kill a bad head of state than to run one, but this is also true of "AI" models now
There are already complex systems vastly beyond human understanding, and certainly our control which are indifferent to our survival and yet we persist.
It's not that "AI" isn't a potential danger, just that it's one of many we are already enduring. Personally there are far more present and likely hazards I am focusing on before getting to the hypothetical ones. Even if humanity fails to fumble its way through this hazard, it's not an end but a new beginning.
Also, it's great satire.