评论文章质疑超级智能的工程可行性
一篇评论文章称,Anthropic研究员Jacob Coxon辞职,并警告Anthropic与OpenAI正竞逐自我改进的超级智能,拿人类生命冒险。Anthropic对齐科学负责人Evan Hubinger表示,Coxon关于AI可能杀死全人类的判断是正确的;CEO Dario Amodei则告诉CNN,他更同意Coxon,而非反对他。评论认为,递归自我改进仍属推测,追求这一目标可能产生更难让人类理解和控制的系统,即使最终没有出现智能爆炸。
这篇评论质疑超级智能的工程可行性,并警告说,追求递归自我改进即使没有产生智能爆炸,也可能造成实际伤害。
Many people were caught off guard this past summer when employees of one of the world’s leading AI companies began casually debating whether their products would lead to human extinction. It started when Anthropic researcher Jacob Coxon announced his resignation, warning that his employer and OpenAI were “racing straight to self-improving superintelligence and gambling with our lives.” Evan Hubinger, Anthropic’s head of alignment science, replied, “Jacob is correct here—we really do earnestly believe AI could kill all humans!” And Anthropic CEO Dario Amodei subsequently told CNN, “I agree with Jacob much more than I disagree with him.”
These concerns stem from a belief that has circulated in futurist circles for many decades: Once engineers design an AI system capable of improving itself, it will trigger an accelerating cycle of autonomous upgrades known as recursive self-improvement (RSI). The result will be artificial superintelligence (ASI), a system with cognitive capabilities so vastly exceeding those of humans that it will easily evade our control and perhaps even wipe out our species. Artificial superintelligence is, it should be noted, a far more ambitious goal than artificial general intelligence (AGI), which generally refers to AI that can match human capabilities across a broad range of cognitive tasks.
#Anthropic#OpenAI#Jacob Coxon#Evan Hubinger#Dario Amodei#Recursive self-improvement#Artificial superintelligence