Canada must police AI more, even if it means displeasing Trump-backed tech giants

Olivier Blais is co-chair of the federal government’s Advisory Council on Artificial Intelligence and its AI Strategy Task Force.

Just 12 days after OpenAI launched ChatGPT 5.6, OpenAI’s own employees asked the model to run cybersecurity tests in a closed environment. The artificial-intelligence model concluded that the most efficient route was to steal the answers.

The model seized control of a server and spent four days breaking into the infrastructure of Hugging Face, the open-source machine-learning company that allowed access to the tests and held the answers.

The incident marked the first fully autonomous, end-to-end cybersecurity incident ever documented.

Some experts see this as an impressive feat. In reality, it’s a warning. A warning that neither the very companies most advanced in AI nor the governments charged with assessing them are ably managing risks that could soon outweigh rewards.

Experts call for greater oversight after OpenAI models go rogue and hack startup Hugging Face

If Canada fails to heed the warning, similar cybersecurity breaches could come to our side of the border. We can no longer ask the AI giants to be the sole judges of whether their own systems are safe enough to distribute.

As co-chair of the federal Advisory Council on Artificial Intelligence and a member of the AI Strategy Task Force, I have worked with the government precisely to put certification in place, along with greater awareness of what safe, secure and transparent systems actually look like. I am very pleased to see the government’s interest in these measures.

But the AI advisory council’s mandate hasn’t gone far enough.

Since drafting the “AI for All” strategy, the council has lacked a clearly defined purpose. Beyond good ideas, it takes the courage to implement them, and to do so as soon as possible, so that we are finally able to show leadership in this new phase of international commerce. Even if that means stepping on the toes of AI giants backed by the Trump administration.

Task force advises Ottawa to regulate chatbots, require labels for AI-generated content

Prime Minister Mark Carney and Evan Solomon, Minister of Artificial Intelligence, should commit to expanding the council’s mandate, implementing a public policy framework for the conformity assessment of consumer-facing AI systems as a priority within the next 12 months. It should be based on strict criteria and administered by a credible organization with the authority to make strong recommendations when systems fail to meet expected standards.

Such systems for assessment already exist to test the safety of models across various categories, from child pornography and violence to terrorism and cybersecurity. One notable solution is the AILuminate benchmark, an open-source tool that has been available for nearly two years.

Canada must have the capacity to judge whether AI systems are safe enough to be made available to Canadians, as we already do in other industries. The government has banned or controlled certain products that may pose risks. This is standard practice across many industries, but apparently not in AI. Earlier this year, when Elon Musk’s AI-powered assistant Grok allowed users to generate nude images of young women of any age, Canada should have restricted its use. It did not.

The Hugging Face incident is particularly worrisome considering it wasn’t the first documented alarm bell. In fact, OpenAI knew this new generation of AI models was unusually sensitive. Only a few weeks earlier, Anthropic was forced to pull back its Mythos model over cybersecurity vulnerabilities, and the U.S. government switched off its Fable 5 model over serious cybersecurity risks. These incidents were only the beginning. Since July 21, we have learned of similar intrusions carried out unintentionally by AI models from Anthropic and even Meta AI.

Opinion: Trump’s big-government, interventionist AI policy is the one thing he’s done right

That none of these serious incidents have occurred in Canada is perhaps in part owing to the kind of mindset gap that has emerged between Canada and the U.S. Canada puts transparency and trust at the centre of every effort to make AI accessible and widely adopted, as set out in the federal strategy. Mr. Solomon has even said that “adoption moves at the speed of trust.”

The message from our southern neighbours is entirely different. A year ago, the U.S. published a highly ambitious action plan that essentially proposes stripping away any regulation or effort that might slow industry players down.

Sure enough, over the past year we have seen and documented many incidents like the ones described above. With deregulation and mounting competition, the temptation to cut corners in order to ship before your competitors must be considerable.

In an industry dominated by giants that prioritize rapid innovation over responsible AI development and management, Canada needs to be able to protect itself. We need to have the capacity to promote and invest in the development and adoption of systems that reflect our vision of trustworthy AI.