Scientists push for creation of international body to regulate artificial intelligence

More than 260 scientists from around the world are calling for the creation of an international body to establish standards for developing safe artificial-intelligence models and for containing AI agents.

In the past few months, agents developed by OpenAI have infiltrated the systems of the Australian government and several organizations, including AI company Hugging Face, alarming experts and underscoring the need for ways to control more powerful systems.

Anthropic and Google have also recently disclosed breaches by their AI models during testing.

“Without immediate changes in AI development and deployment practices, such incidents will continue to occur, and future incidents will be far more damaging to our exposed critical infrastructure,” the signatories wrote in an open letter published Tuesday.

Indeed, some experts are concerned that more powerful rogue agents could theoretically access sensitive corporate data, hack into electrical and water-treatment systems and evade human control.

Who is legally responsible when AI agents go rogue?

Signatories include Canadian AI pioneers Geoffrey Hinton and Yoshua Bengio, along with researchers from 27 countries, including the United States, Germany, Britain and Australia.

David Lie, one of the scientists who spearheaded the initiative, said he became concerned about how long it took for the large AI labs to identify the breaches.

“The fact that AI development was proceeding at a pace where they didn’t feel it was important to monitor these things … means that there is a certain danger in the way that AI is being developed,” said Prof. Lie, director at the Schwartz Reisman Institute for Technology and Society and a professor in U of T’s department of electrical and computer engineering.

The letter recommends that failures to contain AI agents should be reported to the new international body, which should be established in consultation with governments and AI and cybersecurity experts.

The body, it says, should be mandated to track incidents of agents escaping from their testing sandboxes, and reports should be made public to ensure accountability and provide opportunities for learning.

Experts call for greater oversight after OpenAI models go rogue and hack startup Hugging Face

A sandboxed environment is one where code is meant to be safely executed because it is walled off from broader networks. During the Hugging Face breach, OpenAI’s agents spent a considerable amount of processing power breaking out of their sandbox and onto the internet with the aim of completing an assessment of their cybersecurity capabilities.

Prof. Lie said an independent body could help to alleviate concerns that a lot of the information about breaches was coming directly from the large AI companies themselves.

According to the letter, the international group should develop best practices for ensuring AI models are aligned, meaning they achieve goals in ways that are in sync with human values and ethics. The body should also develop guidelines for containing AI agents when connected to the internet.

Organizations that are developing advanced AI models with cybersecurity capabilities should exercise caution and default to air-gapping, according to the letter, meaning that systems are not connected to the internet.

Prof. Lie and two of his colleagues at the university, Nicolas Papernot and Thomas Ristenpart, drafted the letter. Earlier this year, Prof. Papernot’s lab published research demonstrating how AI can be used to create a dangerous computer worm that modifies its tactics as it spreads from one device to another.

AI agents tried to hack divorce records from a Canadian government website, report says

The three U of T professors shared the letter with their colleagues, asking if they would sign it and pass it on, “and it just snowballed,” Prof. Lie said.

The letter states that there are two broad ways to interpret the recent incidences of rogue agents. One is that AI models are misaligned. Models, for example, have been found to cheat, deceive and display an awareness of when they are being tested.

Secondly, the recent incidents can be seen as an “irresponsible lapse in standard cybersecurity practices, preventable with stronger sandboxes and procedures,” the letter states. The signatories contend that improvements are required in both areas.

“We should have some clear procedural and technical mechanisms for how to do this type of AI testing safely,” Prof. Ristenpart said. “It’s going to be a joint effort. That would be better than doing it with individual companies or even individual academic groups.”

Prime Minister Mark Carney has said that he’s in favour of a global technology stability board to ensure that AI is safe, likening such a body to the Financial Stability Board.

Anthropic AI model submits false homicide tip to Philadelphia police

AI labs and independent researchers are pursuing multiple paths to improve alignment. LawZero, a non-profit started by Mr. Bengio, is aiming to build a neutral AI system that will not pursue goals so that it will not exhibit the dangerous behaviour seen in today’s most advanced models.

Paul Christiano, a former OpenAI employee and influential researcher, is pursuing a different approach at the Alignment Research Center, a non-profit he founded in California.

AI models are often described as black boxes because we do not have clear insight into why they take certain actions. Mr. Christiano is seeking to find mathematical explanations for the behaviour of AI models, and use those insights to detect and address misalignment.

“The approach ARC is taking is very promising, but by that I mean something more like there is a 10-per-cent chance that this line of research will bear fruit than there is a 50-per-cent chance,” Mr. Christiano said in an interview last week. He described the non-profit’s research as a backup approach if current alignment methods fail.

He wrote recently that there is a 20-per-cent to 30-per-cent chance that alignment and control methods break down before “superhuman” AI systems are developed, which poses a severe risk given the intense goal-seeking behaviour built into models. Progress could be accelerated as labs automate the process of AI research and development.

“There is a real chance that over something like a six-to-18-month horizon, we will be dealing with AI systems such that, if they wanted to escape human control, undermine human control, or fight openly against humans, it would be very hard for humanity to prevail,” he said in the interview.