Australian schools grappling with AI-driven deepfake crisis

Australia’s peak school bodies have raised alarm bells over AI-driven deepfakes and sexually explicit content on campuses as the Senate looks into a new Bill targeting the “unprecedented” crises.

The proposed amendments to the Commonwealth Criminal Code would introduce offences for transmitting sexual material that had been altered or “created and altered using technology”.

A senate inquiry was told on Tuesday morning that while advocates welcomed the Bill, a stand-alone offence for creating deepfake material was needed to address the issue.

Independent Schools Australia CEO Graham Catt told the inquiry teachers, parents, and students were grappling with AI technologies, from public to private schools.

“The app are changing so rapidly it’s very easy for them (students) to them manipulate someone else’s speaking,” Mr Catt said alongside multiple education representatives.

“I think anything that can future proof as much as possible the rapid evolution of technology so that in six months time you’re not coming back trying to reword a Bill is good.

“Trying to keep up with this rapid technological advancement is something that affects everyone in society. And I think it won’t just be deep fake sexual imagery.”

Mr Catt said issues surrounding AI-generated deepfakes in a non-sexual context were already a generating “huge issues for all of society”, namely in a political context.

Setting the scale of the problem, eSafety Commissioner Julie Inman Grant said 98 per cent of deepfake material online was pornographic – 99 per cent of that was women and girls.

Ms Inman Grant said the watchdog had a particular focus on child sexual material, is separately covered by existing laws, and applications such as nudeify.

The Senate inquiry were told the popular online and open-source AI application allowed users to “upload a photo and AI will remove clothes with seconds”.

While she supported the proposed Bill, Ms Inman Grant said she “felt justified putting the burden on AI companies themselves to engineer out potential misuse”.

She added that Google, which had submitted to the inquiry, were “cooperative in removing damaging content” but that 90 per cent was through “informal means”.

It comes after Rape and Sexual Assault Research and Advocacy chief executive Rachel Burgin told the inquiry with “widespread and easy access to AI, deepfake material was becoming more prolific”.

“This type of behaviour is deeply gendered: Women, girls and gender diverse people are most commonly targeted and most commonly targeted by this material,” Ms Burgin said.

“A failure to criminalise the creation of this type of (deepfake) material is accepting a culture of gender violence where women’s bodies are easily accessed and the property of all.

“We have a problem with violence against women. More than 50 women have been killed this year alone … We cannot allow women safety to play second fiddle to men’s interests.”

Ms Burgin said deepfakes and threats to release them were not exclusive to celebrities and were a “tactic by abusers to instil fear and exert control or power over another person”.

The inquiry’s deputy chair, Queensland senator Paul Scarr, lamented “horrifying statistics” around deepfake material but said Commonwealth politicians were limited in their remit.

“I think it is really important to consider how we might work with the states and territories around (criminalising creation),” he said

“From a prevention point of view, if we really want to tackle this, it is that creating these images is an offence, sharing them it’s also an offence, but creating them as an offence.”

Sexually explicit material altered or manipulated using software, also know as deepfakes, has increasingly made headlines in recent years, particularly with the recent prevalence of AI.

“I feel extremely violated”: model Robyn Lawley on the dangers of AI

Last month, it was revealed about 50 students at a Victorian private school had their images taken from their social media accounts and manipulated and made into “deepfake nudes”.

The inquiry earlier heard evidence that about 96 per cent of sexually explicit deepfake material impacted women whose lives were often left “destroyed and shattered”.

National Association of Services Against Sexual Violence chief executive Natalie Lambert said she was concerned about the simultaneous rise in reported sexual assaults and the prevalence of violent pornography.

“We already see a large number of people accessing services for support around (deepfake) issues. The impact on people’s health wellbeing, their lives and particularly their mental health cannot be overstated,” she said.

As for the perpetrators, Professor Asher Flynn told the inquiry their motivations varied widely, from something made at a bucks party to a deliberate attempt at sextortion or blackmail.

She said school based education around privacy potentially failed to address the harms of artificial sexual images, and said the proposed Bill sent the “message society won’t tolerate this behaviour.

“We have to also make sure that message is being promoted and people are raising awareness as to these are the consequences but also this is why we’re doing it because it’s harmful,” she said.

Contrary to earlier evidence, Professor Jonathon Clough told the inquiry the scope of the Bill might not be narrow enough and raised concerns about whether “non-realistic” or animated-style content would be included.

“Avatars, cartoons or non-sexual material may be terribly degrading and repugnant but would not fall within this provision … There is an ongoing question that we need to have about where does that sit,” he said.

The inquiry also heard evidence from the AFP and the office of the Attorney-General who said the amendments moved to a “consent-based model” and that not all AI technologies were being targeted.

Earlier, advocates also gave evidence about the need for awareness and early prevention, as well as the Bill’s aggravating factor provisions: creations and following a notice from the eSafety Commission.

It is being chaired by Labor senator Nita Green and Greens senator Larissa Waters.