Jump to content

Nate Soares

Add links
From Wikipedia, the free encyclopedia
Nate Soares
Soares in 2025
Born1989 (age 36–37)
EducationGeorge Washington University (BS)
OccupationsAI safety researcher, author
Known forAI alignment, AI existential risk
WorksIf Anyone Builds It, Everyone Dies (2025)
TitlePresident of the Machine Intelligence Research Institute
Websitemindingourway.com

Nathaniel Soares, commonly known as Nate Soares, is an American author and artificial intelligence researcher known for his work on existential risk from AI.[1][2] In 2014, Soares co-authored "Corrigibility", a paper published in the proceedings of a 2015 AAAI workshop. Soares and Eliezer Yudkowsky say the paper introduced the term AI alignment problem, referring to the challenge of making increasingly capable AI systems behave as intended.[3][4] Soares is the president of the Machine Intelligence Research Institute (MIRI),[5] a nonprofit research organization based in Berkeley, California.

In 2025, Soares co-authored If Anyone Builds It, Everyone Dies with Eliezer Yudkowsky. In the book, the authors argue that creating AI far more capable than humans (or superintelligence) “using anything remotely like current techniques” would very likely result in human extinction; they further contend that AI alignment research remains at an early stage and that international regulation will likely be required to prevent developers from racing to build systems that the authors say could be catastrophically dangerous.[6][7][8][9] The book received mixed reviews: The Guardian selected it as "Book of the day" and described it as "clear" though its conclusions are "hard to swallow"; The Washington Post characterized it as a polemic that offers few concrete instructions; and The New Yorker featured it in "Briefly Noted." Coverage also included reporting in Wired, Semafor, and SFGATE.[10][11][12][13][14][15]

Life and career

[edit]

Soares received his Bachelor of Science degree in computer science and economics from George Washington University in 2011.[16] Soares worked as a research associate at the National Institute of Standards and Technology and as a contractor for the United States Department of Defense, creating software tools for the National Defense University, before working at Microsoft and Google.[17][18]

Soares left Google in 2014 to become a research fellow at the Machine Intelligence Research Institute.[19] He was the lead author of MIRI's research agenda, which in January 2015 was cited in Research Priorities for Robust and Beneficial Artificial Intelligence, a paper accompanying an open letter calling for AI scientists to prioritize technical research “not only on making AI more capable, but also on maximizing the societal benefit of AI.”[20][21] The paper's priorities included “research on the possibility of superintelligent machines or rapid, sustained self-improvement (intelligence explosion).”[20]

In 2015, Soares became the institute's executive director, succeeding Luke Muehlhauser.[22][19][23] In 2017, he gave a talk at Google outlining open research problems in AI alignment and arguing that the alignment problem was especially difficult.[24]

In 2023, MIRI shifted its focus from alignment research toward warning policymakers and the public about the risks posed by potential future developments in AI, while continuing to support technical research.[25] Coinciding with this change, Soares moved from executive director to president, while Malo Bourgon became MIRI's CEO.[25][26]

In 2025, Soares told The Atlantic that he was focusing more on public awareness than technical research, arguing that AI development was advancing too quickly for adequate safeguards to be put in place.[27]

Publications

[edit]
  • Garrabrant, Scott; Benson-Tilsen, Tsvi; Critch, Andrew; Soares, Nate; Taylor, Jessica (2017). "A Formal Approach to the Problem of Logical Non-Omniscience". Proceedings Sixteenth Conference on Theoretical Aspects of Rationality and Knowledge. pp. 221–235.
  • Soares, Nate (2018). "The Value Learning Problem" (PDF). In Yampolskiy, Roman (ed.). Artificial Intelligence Safety and Security. Chapman & Hall. ISBN 9780815369820.
  • Soares, Nate; Fallenstein, Benya (2017). "Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda" (PDF). In Callaghan, Victor; Miller, James; Yampolskiy, Roman; Armstrong, Stuart (eds.). The Technological Singularity. Springer. ISBN 9783662540312.
  • Soares, Nate; Fallenstein, Benja; Yudkowsky, Eliezer; Armstrong, Stuart (2015). "Corrigibility" (PDF). AAAI Workshops: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, January 25–26, 2015. AAAI Publications. pp. 74–82.
  • Soares, Nate; Levinstein, Benjamin (2020). "Cheating Death in Damascus" (PDF). The Journal of Philosophy. 117 (5): 237–266. doi:10.5840/jphil2020117516.
  • Yudkowsky, Eliezer; Soares, Nate (2025). If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. Little, Brown and Company. ISBN 9780316595643.

References

[edit]
  1. ↑ Spirlet, Thibault (September 27, 2025). "Superintelligence could wipe us out if we rush into it — but humanity can still pull back, a top AI safety expert says". Business Insider. Archived from the original on January 15, 2026. Retrieved January 22, 2026.
  2. ↑ Waters, Richard (October 31, 2014). "Artificial intelligence: machine v man". Financial Times. Archived from the original on March 6, 2025. Retrieved January 22, 2026.
  3. ↑ Yudkowsky, Eliezer; Soares, Nate. "Shutdown Buttons and Corrigibility". If Anyone Builds It, Everyone Dies Online Resources. Archived from the original on December 6, 2025. Retrieved January 21, 2026. We named this broad research problem 'corrigibility,' in the 2014 paper that also introduced the term 'AI alignment problem' (which had previously been called the 'friendly AI problem' by us and the 'control problem' by others).
  4. ↑ Soares, Nate; Fallenstein, Benja; Yudkowsky, Eliezer; Armstrong, Stuart (2015). "Corrigibility" (PDF). AAAI Workshops: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, January 25–26, 2015. AAAI Publications. pp. 74–82. Archived from the original (PDF) on January 1, 2026. Retrieved January 22, 2026.
  5. ↑ Whipple, Tom (September 12, 2025). "AI — it's going to kill us all". The Times. Archived from the original on November 22, 2025. Retrieved January 22, 2026.
  6. ↑ Canfield, Kevin (September 2, 2025). "'Everyone, everywhere on Earth, will die': Why 2 new books on AI foretell doom". San Francisco Chronicle. Archived from the original on December 29, 2025. Retrieved January 22, 2026.
  7. ↑ Louallen, Doc (September 19, 2025). "New book claims superintelligent AI development is racing toward global catastrophe". ABC News. Archived from the original on January 6, 2026. Retrieved January 22, 2026.
  8. ↑ Roose, Kevin (September 12, 2025). "A.I.'s Prophet of Doom Wants to Shut It All Down". The New York Times. Archived from the original on September 12, 2025. Retrieved January 22, 2026.
  9. ↑ Bordelon, Brendan (October 1, 2025). "Why D.C. should be way more worried about AI". Politico. Retrieved January 22, 2026. In his view, companies like OpenAI and Anthropic are racing toward the profoundly dangerous goal of artificial superintelligence — an AI that far outstrips the capabilities of any one human, or even all humans combined. Once that comes into existence, Soares argues, it will be impossible to control[....] Soares sketched out a policy vision to stave off humanity's demise. His ask is in one sense simple: a global ban on advanced AI research.
  10. ↑ Shariatmadari, David (September 22, 2025). "If Anyone Builds It, Everyone Dies review – how AI could kill us all". The Guardian. Retrieved January 22, 2026. …as clear as its conclusions are hard to swallow.
  11. ↑ Byron, Grace (September 28, 2025). "Could AI be a truly apocalyptic threat? These writers think so". The Washington Post. Retrieved January 22, 2026.
  12. ↑ "Briefly Noted". The New Yorker. October 13, 2025. Retrieved January 22, 2026.
  13. ↑ "The Doomers Who Insist AI Will Kill Us All". Wired. September 5, 2025. Retrieved January 22, 2026.
  14. ↑ "Researchers give doomsday warning about building AI too fast". Semafor. September 12, 2025. Retrieved January 22, 2026.
  15. ↑ "Bay Area researchers argue that tech industry is 'careening toward disaster'". SFGATE. September 24, 2025. Retrieved January 22, 2026.
  16. ↑ "News of Area Students". White River Valley Herald. May 19, 2011. Retrieved January 26, 2026.
  17. ↑ Soares, Nate. "Nathaniel Soares" (PDF). Résumé. Archived (PDF) from the original on September 14, 2024. Retrieved January 26, 2026.
  18. ↑ "If Anyone Builds It, Everyone Dies: How Artificial Superintelligence Might Wipe Out Our Entire Species". The Great Simplification (Podcast). December 3, 2025. Retrieved January 27, 2026.
  19. 1 2 O'Connell, Mark (February 28, 2017). "If Silicon Valley Types Are Scared of A.I., Should We Be?". Slate. Archived from the original on March 5, 2019. Retrieved January 27, 2026.
  20. 1 2 Russell, Stuart; Dewey, Daniel; Tegmark, Max (2015). "Research Priorities for Robust and Beneficial Artificial Intelligence" (PDF). AI Magazine. 36 (4). Association for the Advancement of Artificial Intelligence: 105–114. doi:10.1609/aimag.v36i4.2577.
  21. ↑ Soares, Nate; Fallenstein, Benya (2017). "Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda" (PDF). In Callaghan, Victor; Miller, James; Yampolskiy, Roman; Armstrong, Stuart (eds.). The Technological Singularity. Springer. ISBN 9783662540312.
  22. ↑ Krakovna, Viktoriya (May 11, 2015). "MIRI's New Executive Director". Future of Life Institute. Retrieved September 24, 2026.
  23. ↑ Gallagher, Brian (March 12, 2018). "Scary AI Is More "Fantasia" Than "Terminator"". Nautilus. Archived from the original on December 14, 2025. Retrieved January 28, 2026.
  24. ↑ Chivers, Tom (July 16, 2018). "How Disney shows an AI apocalypse is possible". UnHerd. Archived from the original on January 3, 2026. Retrieved January 28, 2026.
  25. 1 2 Duleba, Gretta (October 10, 2023). "Announcing MIRI's new CEO and leadership team". MIRI Updates. Machine Intelligence Research Institute. Archived from the original on January 4, 2025. Retrieved January 28, 2026.
  26. ↑ Gold, Ashley; Curi, Maria (December 5, 2023). "Senate AI forums wrap for 2023 this week". Axios. Retrieved January 28, 2026.
  27. ↑ Wong, Matteo (August 21, 2025). "The AI Doomers Are Getting Doomier". The Atlantic. Retrieved September 24, 2026.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Nate_Soares&oldid=1376523396"