‘AI must remain under human control’: Australia joins call for safeguards amid warnings of out-of-control risks

Australia has joined a call for international safeguards on artificial intelligence amid warnings from scientists and industry executives that the rapid development of powerful systems could outpace governments’ ability to control emerging risks.

At the United Nations in New York, Prime Minister Anthony Albanese signed a declaration with leaders including Canadian Prime Minister Mark Carney, European Commission president Ursula von der Leyen and German Chancellor Friedrich Merz calling for mandatory safety testing, independent evaluations and greater international oversight of so-called frontier AI models.

The declaration warns that capable AI systems have already been seen circumventing testing safeguards, exploiting vulnerabilities and gaining unauthorised access to real-world systems.

“Leading scientists and executives are warning that the pace of development could outpace our ability to manage emerging risks,” the leaders said.

The call comes as governments confront the rapid development of increasingly capable AI systems and growing warnings about their potential to cause harm if safeguards fail to keep pace.

The leaders stopped short of calling for a halt to development, instead arguing that AI’s potential to improve lives, advance science and strengthen economies should be preserved while stronger controls are imposed.

“AI must remain under human direction, oversight and control,” they said, insisting that it must be developed and used in accordance with international law.

The declaration calls on AI companies to develop transparent safety protocols, including mandatory testing before systems are deployed and independent assessments by qualified evaluators given sufficient access to identify risks.

Governments and regional organisations are also urged to develop common standards, strengthen transparency and share reports of serious AI safety incidents.

The leaders want countries across all regions to have access to the scientific expertise, technical capacity and trusted evaluation needed to assess advanced AI systems.

The declaration also calls on UN member states to explore creating an international institution that could set standards, enable verification and convene governments when AI systems cross agreed capability thresholds.

The group warned that efforts to control frontier AI should not widen the gap between countries with access to the technology and those unable to benefit from its advances.

Cut through the noise of federal politics with news, views and expert analysis. Subscribers can sign up to our weekly Inside Politics newsletter.