A recent survey conducted by AI Impacts reveals that a considerable number of machine learning researchers believe there is a significant risk of human extinction resulting from the development of advanced AI systems. The survey, which serves as a follow-up to a controversial 2016 study, discovered that between 37.8% and 51.4% of respondents gave at least a 10% chance of AI leading to outcomes as catastrophic as human extinction.
The results challenge the prevailing narratives surrounding AI extinction risk. Contrary to popular belief, researchers surveyed did not neatly fall into pessimistic or optimistic categories. Instead, many individuals who acknowledged the potential for negative outcomes also recognized the possibility of positive outcomes. Notably, 57.8% of respondents believed that extremely adverse scenarios, such as human extinction, had a likelihood of at least 5%.
Critics of the 2016 survey had questioned the validity of the findings, suggesting that researchers may have biased the results or that certain experts had not given the matter sufficient consideration. However, when AI Impacts reran the survey in 2022 and recontacted thousands of machine learning researchers, they obtained similar results. The median probability of an “extremely bad” outcome, such as human extinction, remained at 5%. Furthermore, when the question was approached from different angles to mitigate framing effects, the responses consistently fell within the 5-10% range, strengthening confidence in the data.
These findings highlight the growing concerns regarding the potential risks associated with the development of advanced AI technology. As AI systems continue to advance and companies invest significant resources in their development, the prospect of human-level AI leading to outcomes as grave as human extinction no longer seems far-fetched. It is now crucial for experts in the field to further explore and address these risks, fostering a broader conversation to determine the best course of action to mitigate any potential harm.