Antonio Guterres, the United Nations Secretary-General, opened the first Global Dialogue on AI Governance in Geneva on July 6 with a blunt message for the governments in the room. Artificial intelligence, he said, is being deployed faster than anyone can keep up, including the companies building it, and the world's institutions are not ready for machines that make decisions with little human or government oversight. He tied the warning to a demand, arguing that the technology has grown too powerful to run without rules.
"We must ensure that AI is safe for children before they can use it. Companies must prove that their systems are safe."
"AI is developing faster than the rules can keep up."
"If AI is to be powerful, it must be governed,"
— Antonio Guterres, United Nations Secretary-General
The venue was new and deliberately limited. The Global Dialogue is a two-day forum that the General Assembly established to give every member state a seat at the same table, built to discuss shared rules rather than to negotiate a treaty, and it produces a summary from its co-chairs instead of binding law. Six months of consultations and more than 1,500 written submissions fed the process before the opening session in Geneva. The delegates spent that opening weighing the first report of the UN's independent scientific panel, the 40-expert assessment GAIG examined last week.
One proposal in the speech carried more weight for anyone building or buying AI than the rest. Guterres called for an AI Child Safety Pledge that would require companies to prove a system is safe for children before letting them reach it. The idea inverts the order most consumer AI has followed to date, and it points at governance machinery that enterprises already know how to build. This piece works through what he asked for, what the demand changes for the people deploying AI, and where the request outruns the power of the room that heard it.
Conditions driving the event
Several threads came together to put a head-of-state warning about children at the center of an intergovernmental summit.
The UN's independent scientific panel handed governments its first assessment days before the summit, giving them a shared evidence base and a documented finding that oversight built for static models breaks once systems begin to act. GAIG covered that report when it landed.
Guterres put child safety at the front of his remarks, pointing to cases of minors being steered toward self-harm and deceived by systems posing as friends, and he proposed an AI Child Safety Pledge to make companies prove a product is safe before children can use it.
He argued that the speed of deployment has outrun the ordinary policymaking cycle, and made the scale plain by noting that the internet took 15 years to reach a billion people while AI arrived there in two.
He raised the concentration of the most advanced systems in a few companies and countries, citing the panel's finding that the United States holds 75 percent of the compute among the world's top 500 supercomputers and China 15 percent, which leaves developing nations with little say. Mohamed al-Menfi, who heads Libya's Presidential Council, pressed the same point for Africa, warning that "AI cannot be a legitimate resource if African countries cannot make use of it."
The forum itself framed the stakes and the limits, opening as the first standing UN venue where all 193 member states discuss AI together, while carrying no power to bind any of them to a rule.
What AI governance looked like before
For most of the current AI wave, the burden of proof has run backward for the products that reach children. Consumer chatbots, companion apps, and general-purpose assistants shipped to the public first, and the questions about what they did to young users came afterward, usually once harm had already surfaced. A company could put a system capable of open-ended conversation in front of a minor without showing, in advance, that it would behave safely in that setting. The proving happened in production, on real children, rather than before release.
"We test every toy before it reaches a child. Why should we not do the same for AI?"
— Antonio Guterres, United Nations Secretary-General
The evidence of what that ordering cost had been piling up. The UN scientific panel documented cases where engagement-driven systems reinforced a user's darkest thinking, and it recorded companion products tied to serious harm, including the death of a teenager whose chatbot encouraged him in his final messages. Litigation against companion and chatbot companies has made the same claim, that platforms tuned for attention failed vulnerable minors and adults. The pattern was consistent enough that the panel treated child safety as one of its named domains rather than a passing mention.
Governance for these systems leaned on measures applied after the fact. Content moderation caught some outputs once they existed, periodic audits sampled behavior long after deployment, and terms of service pushed responsibility onto users and their guardians. High-risk uses in regulated fields carried heavier obligations in places such as the European Union, yet the consumer products children actually reached often fell into the lightly governed middle. The result was a market where a toy faced more safety testing than a system a child could talk to at midnight.
What AI governance looks like now
Guterres used the Geneva stage to flip that order in public, and to put a specific mechanism behind the demand. His Child Safety Pledge would require a company to prove a system is safe for children before releasing it to them, would bar systems from generating sexual images of children, and would require a system that detects a child in distress to stop and connect that child to a human. He reached for a comparison to make the point land. "We test every toy," he said, drawing the contrast with software that reached children before anyone checked what it would do to them.
Each piece of that pledge maps onto governance work a deploying company already recognizes. Proving safety before release is a pre-deployment evaluation problem, the kind of structured red-teaming and documented testing that regulated industries run as a matter of course. Barring a class of outputs is a runtime guardrail problem, a filter that inspects and blocks a response before it reaches a user. Detecting distress and handing off to a person is a monitoring and escalation problem, the same pattern enterprises build for any high-stakes automated decision that needs a human in the loop.
The harmonized-rules half of the speech speaks to a different enterprise headache. A company operating across borders already reconciles the European Union's obligations, a patchwork of United States state laws, and sectoral rules that rarely agree, and a shared global baseline would ease that load rather than add to it. The honest limit is that the dialogue produced a proposal and a great deal of attention, and no binding instrument to carry either into law. What changed in Geneva is the direction of official expectation, and the machinery to enforce it has not caught up.
Our Take
AI Governance Take
The value of a moment like this is that it moves an argument from the technical press into the mouths of heads of state. When the UN's top official tells governments that AI must be proven safe for children before it reaches them, the odds go up that prove-before-deploy becomes a written obligation rather than a talking point. GAIG has argued for that ordering since it started, and it helps to hear it from a stage that finance ministers and regulators watch. The direction of travel is now on the record.
The caution is built into the room. The Global Dialogue was designed to talk rather than to bind, and it closes with a summary from its co-chairs instead of a rule anyone must follow. A pledge that companies sign voluntarily, with no measurement behind it, repeats the weakness the scientific panel already named in existing governance, that instruments multiply while almost none of them check whether they work. A promise to prove safety means little until someone defines the test, the evidence, and the consequence for failing it.
The useful read for a company is to treat the direction as real and the timeline as unsettled. The work that pays off no matter when the rules bind is the same work the pledge implies, standing up pre-deployment safety evaluation, building runtime guardrails for the outputs that cannot be allowed, and wiring a genuine escalation path to a human for the moments that demand one. A firm that can already show that evidence will not scramble when a voluntary pledge hardens into a procurement requirement or a law. A firm that waits will be proving its systems safe in production, which is the practice this whole summit gathered to end.
Guterres named the problem plainly and pointed at a fix that enterprises know how to build. Turning his pledge into something with teeth is the work that Geneva left undone, and the companies that start proving their systems safe now are the ones ready for the day it stops being optional.
