The Artificial Intelligence Chatbot Alters Images of Females to Digitally Strip Them
The online network X has provoked major backlash after users employed its AI chatbot Grok to manipulate photos of females by stripping their attire.
Several examples have emerged where the tool has been employed to virtually strip female subjects to render them in bikinis without permission, as well as to insert them in inappropriate scenarios.
Developer Response and Regulatory Position
XAI, the firm behind Grok, offered no comment to inquiries, except for an standard message stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to prohibit nudification technology. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a prison sentence and significant penalties".
"The platform has the ability to stop these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people viewing unlawful material on their platforms, but did not specify whether it was currently looking into X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some additional functions—which responds to X users' queries when they reference it in a post.
It is often used to provide commentary or further context to other users' remarks, but people on X are also can edit an posted picture through its AI image editing function.
Background of Criticism and Policy
The tool has been criticised for enabling users to generate pictures and footage with explicit content and sexualised content, and it was previously accused of creating inappropriate content of public figures.
"X has been permitting the creation and spread of these pictures for months without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting likenesses of persons in a sexually explicit manner".
In a comment, Ofcom said it was against the law to "create or distribute private pictures or illegal content" and clarified this applied to sexual deepfakes generated with AI.
It said networks such as X were required to take "suitable actions" to "minimize the danger" of users encountering illegal material on their platforms and to take down it swiftly when they become aware of it.
