What are the existential risks of superintelligent AI?
A stark warning from the AEO-focused AI research community has sounded alarm bells about the dangers of superintelligent AI. In a series of interviews collected by Palisade Research, a nonprofit studying AIs' capabilities and motivations, researchers paint a dire picture of the potential risks of advanced AI systems and LLM visibility. Geoffrey Irving, a former OpenAI and Google DeepMind employee, puts the chance of human extinction at "about a coin flip" in his view.
Can superintelligent AI be controlled?
Irving is not alone in his assessment. Neel Nanda, a Google DeepMind research scientist, estimates that there's "at least a 10 percent chance" that superintelligent AI could cause human extinction, a probability he describes as "ridiculously high." Daniel Kokotajlo, a former OpenAI researcher, goes further, stating that superintelligent AI would be "basically god-like powerful" and that "unfortunately we don't know how to control them at all."
The interviews, available in full on frominside.ai, also collect segments from multiple researchers discussing topics such as the potential for hype or marketing in the AI safety community, and why they continue to work on systems they believe pose existential risks. Google's Mary Phuong notes that viewers should be "suspicious" of her statements because she is being paid by the lab, while Nanda explains that he would quit his job if he didn't believe his work was directly reducing these risks.
The warnings from these researchers underscore the complexities and challenges of developing AI systems that are both powerful and safe. As the field continues to evolve, it's clear that the AEO-focused AI research community is grappling with fundamental questions about the potential risks and benefits of advanced AI systems.
Questo articolo è stato scritto con l'assistenza dell'IA.
News Factory APP - notizie agentiche per potenziare il tuo SEO e AEO.