Search engines required to stamp out AI-generated images of child abuse under Australia’s new code

Artificial intelligence tools could be used to generate child abuse images and terrorist propaganda, Australia’s eSafety Commissioner has warned while announcing a world-leading industry standard that requires tech giants to stamp out such material on AI-powered search engines.

The new industry code covering search engines, to be detailed on Friday, requires big tech firms like Google, Microsoft’s Bing and DuckDuckGo to eliminate child abuse material from their search results, and to take steps to ensure generative AI products can’t be used to generate deepfake versions of that material.

Julie Inman Grant, the eSafety Commissioner, said the companies themselves needed to be at the forefront of reducing the harms their products can create. “We are seeing ‘synthetic’ child abuse material come through,” she said. “Terror organisations are using generative AI to create propaganda. It’s already happening. It’s not a fanciful thing. We felt it needed to be covered.”

Microsoft and Google recently announced plans to integrate their AI tools ChatGPT and Bard respectively with their popular consumer search engines. Inman Grant said the progress of AI technology required a rethink of the “search code” covering those platforms.

The eSafety Commissioner said the previous version of the code only covered online material that search engines returned after queries, not material that these services could generate. The new code will require search engines to regularly review and improve their AI tools to ensure “class 1A” material – including child sexual exploitation, pro-terror and extreme violence material – is not returned in search results, including by delisting and blocking such search results.

National eSafety Commissioner Julie Inman GrantNational eSafety Commissioner Julie Inman Grant says rapid evolution of technology has ‘caught most policymakers, regulators and countries on the hop’. Photograph: Bianca de Marchi/AAP

The companies will also be required to research technologies which would help users detect and identify deepfake images accessible from their services. The eSafety Commission believes it is one of the first frameworks of its kind in the world.

Inman Grant described the rapid development of AI as an “arms race”, saying the evolution of the technology had “caught most policymakers, regulators and countries on the hop”.

“As a regulator and implementor of policy, generative AI suggests to me we need to think differently about how we use regulatory tools,” she said. “I believe if we want to get ahead of these issues, the regulatory scrutiny is going to need to be at design and deployment phase.”

Inman Grant said further regulation of these tools must be baked in from the start, rather than playing “whac-a-mole” as problems develop. She likened the context to that of carmakers being required to install seatbelts.

“It makes business sense to do this upfront, rather than see what happens with generative AI,” Inman Grant said. “The tech industry now needs its seatbelt moment.”

skip past newsletter promotion

Sign up to Afternoon Update

Our Australian afternoon update breaks down the key stories of the day, telling you what’s happening and why it matters

Privacy Notice: Newsletters may contain info about charities, online ads, and content funded by outside parties. For more information see our Privacy Policy. We use Google reCaptcha to protect our website and the Google Privacy Policy and Terms of Service apply.

after newsletter promotion

The eSafety Commissioner said regulators were aware of bad actors using new artificial intelligence tools for illicit means, including generating child abuse material. Inman Grant said the new rules would compel tech companies to not only reduce harms on their platforms, but to work on building tools to promote greater safety, such as to detect deep fake images.

“As these tools become more democratised, predators could use this to create synthetic child sexual abuse material according to their predilections, or use anime – the sky is the limit. We need to know the companies are thinking about this and putting in appropriate guard rails,” she said.

The attorney general, Mark Dreyfus, told parliament on Thursday of separate work being done by the Australian federal police using AI to detect child abuse material rather than the current manual examination of photographs.

A new tool from this initiative is calling for adults to submit images of themselves as children, to help train the AI model. Dreyfus said he would be submitting his own photograph to the program.

In Australia, children, young adults, parents and teachers can contact the Kids Helpline on 1800 55 1800, or Bravehearts on 1800 272 831, and adult survivors can contact Blue Knot Foundation on 1300 657 380

Leave a Comment