NEW YORK: OpenAI, the organization behind ChatGPT, has released its latest guidelines aimed at evaluating “catastrophic risks” associated with artificial intelligence (AI) models currently under development. This announcement comes one month after the brief dismissal and reappointment of CEO Sam Altman, a move prompted by internal dissent among staff and investors.
According to reports, Altman faced criticism from board members who were concerned about prioritizing accelerated development at the expense of addressing potential risks associated with OpenAI’s technology.
In the newly published “Preparedness Framework”, OpenAI acknowledges that the scientific study of catastrophic risks from AI has not met the necessary standards. The framework aims to fill this gap by providing a structured approach to assess and mitigate risks associated with cutting-edge AI models.
OpenAI’s Preparedness Framework
The monitoring and evaluations team, established in October, will concentrate on evaluating “frontier models” that surpass the capabilities of the most advanced AI software. Each new model will be categorized based on risk levels ranging from “low” to “critical” across four key domains.
The first category assesses the model’s cybersecurity capabilities, focusing on its potential to execute large-scale cyberattacks. The second category measures the likelihood of the software contributing to the creation of harmful substances, organisms (such as viruses), or even nuclear weapons.
The third category examines the model’s persuasive power, evaluating its influence on human behavior. The final risk category addresses the potential autonomy of the model and its ability to escape the control of its creators.
Models with a risk score of “medium” or lower will be considered deployable, as per the framework. Identified risks will be presented to OpenAI’s Safety Advisory Group, a newly formed body responsible for making recommendations to CEO Sam Altman or a designated representative.
Altman will then decide on any necessary adjustments to mitigate associated risks, with the board of directors staying informed and retaining the authority to overrule management decisions.
