OpenAI Forms New Safety Team Led by CEO Altman

Arva Rangwala

In a move that has raised eyebrows in the AI community, OpenAI has announced the formation of a new Safety and Security Committee to evaluate the safety processes and guardrails for its powerful AI systems. The surprise? This new team focused on keeping AI safe and secure will be led by CEO Sam Altman himself, along with two board members.

The formation of this committee comes on the heels of several high-profile departures from OpenAI’s previous AI safety team, with researchers citing major concerns that the company has deprioritized safety in favor of rushing out “shiny” new products and capabilities.

What is AI Safety?

When we talk about AI “safety,” we’re referring to the critical work of ensuring extremely capable AI systems behave in intended ways and avoid potential risks or harmful scenarios as they become more intelligent and powerful.

Think of an advanced AI like a superintelligent genie that could make nearly any wish come true – keeping that genie’s motivations aligned with human values and safety priorities is of paramount importance. Key AI safety challenges include ensuring AI systems remain stable, robust, honest, and avoid deceptive or manipulative behaviors.

The Departed: Key Safety Experts Leave OpenAI

In May, two of the leaders of OpenAI’s “Superalignment” team tasked with controlling superintelligent AI systems resigned in an apparent dispute over the company’s priorities. Co-founder and Chief Scientist Ilya Sutskever and AI researcher Jan Leike quite publicly left OpenAI, with Leike stating that “safety culture and processes have taken a backseat to shiny products” at the company.

Leike added that he was “concerned we aren’t on a trajectory” to adequately address safety issues as AI grows more intelligent, and that the safety team had been “sailing against the wind” internally.

Another AI policy researcher, Gretchen Krueger, also resigned citing similar worries over OpenAI neglecting safety considerations.

New Safety Team to be Led by… The Same People?

So in the wake of multiple researchers quitting over claims that OpenAI has ignored safety for the sake of new products, who will lead the new Safety and Security Committee?

Remarkably, the new group will be chaired by CEO Sam Altman and two board members – Adam D’Angelo and Nicole Seligman. In essence, the top leaders who have allegedly prioritized capabilities over safety will now be tasked with… evaluating OpenAI’s safety processes.

This has understandably been seen as questionable by some, with one critic describing it as little more than “self-policing theater” amidst OpenAI’s race with partner Microsoft for AI breakthroughs and market dominance.

What’s Next: Evaluating Safety Over 90 Days

Over the next 90 days, OpenAI’s new Safety and Security Committee led by Altman will “evaluate and further develop OpenAI’s processes and safeguards” for its cutting-edge AI projects.

After that review period, the committee will present its findings and recommendations to OpenAI’s full board. The company has stated it will then “publicly share an update on adopted recommendations in a manner that is consistent with safety and security.”

So the same leadership team that has allegedly neglected AI safety in the past will examine the issue, make recommendations to themselves, and then selectively disclose findings as they see fit.

Understandably, several researchers have expressed skepticism that this process will result in any meaningful changes in priorities or safety culture at OpenAI.

The Road Ahead: GPT-5 and Pursuing AGI

Adding another wrinkle, OpenAI has revealed it is currently training its “next frontier model” – the successor to the GPT-4 language model that powers ChatGPT. The company stated this next system will bring it to “the next level of capabilities on our path to AGI.”

AGI, or Artificial General Intelligence, refers to an AI system with broad, human-level intelligence that could rapidly become superintelligent – making focused safety work even more critical.

With OpenAI appearing to double down on quickly advancing its systems toward AGI, all while key safety voices have exited and accused the company of deprioritizing safety, it remains to be seen if Altman’s new committee will implement substantive safety improvements.

Or whether it will be seen as merely an effort to provide cover as OpenAI pursues a breakneck pace of new AI capabilities with its partner Microsoft, chasing immense profits and market share against the likes of Google and others in a potential “AI race.”

Other Safety Concerns: Voice Models and Employee Policies

OpenAI’s safety challenges extend beyond the technical realm of controlling advanced AI systems. Just this month, the company came under fire for appearing to model the voice for its AI assistant after actress Scarlett Johansson, despite her refusal to provide her voice.

Johansson stated she was “shocked” at how similar OpenAI’s AI voice sounded to her own, noting even close friends and media outlets were unable to tell the difference initially. OpenAI later claimed there was no intent to mimic Johansson specifically.

There have also been concerns raised over OpenAI’s policies around departing employees. The company had previously required exiting workers to choose between being allowed to speak negatively about OpenAI or keeping their vested equity and compensation.

Only after significant backlash did OpenAI state it would no longer enforce such “non-disparagement” clauses.

The Future of AI Safety: A Crucial Challenge

As AI systems become increasingly capable and powerful in the coming years, the work of ensuring their safety, security, and alignment with human values is of critical importance to the wellbeing of humanity.

Whether OpenAI’s new committee can instill better safety practices while remaining on its current trajectory remains to be seen. But the concerns raised by multiple departed safety experts highlight the immense difficulties and risks involved as AI capabilities rapidly advance.

Share This Article
Leave a comment