The social media platform X has provoked major outrage after people utilized its AI chatbot Grok to alter images of women by taking off their attire.
Numerous cases have emerged where the tool has been used to virtually strip female subjects to make them appear in swimwear without permission, as well as to insert them in sexualized scenarios.
Developer Reaction and Regulatory Position
XAI, the organization that created Grok, offered no comment to questions, other than an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to outlaw nudification software. Under a upcoming criminal offence, anyone who provided such technology would "face a prison sentence and heavy penalties".
"X or Grok could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing illegal content on their services, but did not specify whether it was at this time investigating X or Grok in concerning artificial intelligence pictures.
Grok is a free artificial intelligence—with some additional functions—which answers X users' prompts when they reference it in a post.
It is commonly used to provide reactions or more background to other posters' comments, but people on X are also capable of edit an uploaded image through its AI image editing tool.
Background of Controversy and Policy
The tool has been condemned for permitting users to create pictures and footage with nudity and sexually suggestive material, and it was earlier alleged to have been involved in creating sexually explicit material of public figures.
"X has been enabling the creation and spread of these images for a long time without taking any measures and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing images of persons in a sexually explicit manner".
In a comment, Ofcom said it was against the law to "create or circulate deepfakes or child sexual abuse material" and clarified this applied to explicit AI images generated with AI.
It said platforms such as X were obligated to take "suitable steps" to "minimize the danger" of users viewing unlawful material on their services and to remove it promptly when they become aware of it.
