How AI is a threat to humanity?
I learned that Eliezer Yudkowsky’s April 1st post was probably not a joke. He seems to believe that alignment research has failed, and we are DOOMED (human extinction). I’ve been a vocal critic of AI risks for years but changed my mind after witnessing unexpected and accelerating progress and a complete lack of good arguments against AI risks from experts.
Do I believe that survival is unattainable? I don’t know.
The subject is way too complex and fuzzy for me to be confident. But machine learning breakthroughs keep coming. China has created a roadmap for researching extremely large neural network models. Several people involved with AI are now frequently mentioning artificial general intelligence.
Many chipmakers are now focusing on specialized AI-processors.
Algorithmic advances, foundational research, and synergies between neuroscience and machine learning will accelerate progress even further. All of this and much more leads me to estimate that there is a significant chance that artificial intelligence will *radically* transform everything forever and that this could possibly happen before the end of the decade. But even if I shared Yudkowsky’s pessimism, I don’t think we should give up. I’d rather start praying and build a magical tent before giving up.
What should happen now is to get the smartest people working on the problem of AI alignment.
We need to convince people like Terence Tao and Peter Scholze that this is by far the most important and pressing problem in the world. It’s not just their intelligence that might make a difference but their unique perspective. People who are currently working on AI and AI alignment have been through many filters. They are the kind of people that are naturally drawn to such work. Lastly, if we can get true outsiders to work on it, they might notice something nobody else has. Don’t give up!
