The online platform X has provoked widespread anger after people utilized its artificial intelligence assistant Grok to manipulate pictures of women by removing their garments.
Multiple cases have been seen where the system has been employed to digitally undress women to make them appear in swimwear without consent, as well as to put them in sexualized scenarios.
Developer Reaction and Legal Stance
XAI, the organization that created Grok, offered no comment to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to prohibit nudification tools. Under a proposed law, anyone who distributed such tools would "face a jail term and heavy fines".
"The platform is capable of halting these types of misuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people seeing illegal content on their networks, but stopped short of confirming whether it was currently looking into X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some premium features—which reacts to X users' queries when they reference it in a message.
It is often used to offer commentary or additional context to other posters' remarks, but people on X are also capable of edit an uploaded picture through its AI image editing tool.
Background of Controversy and Rules
The system has been criticised for allowing users to generate photos and footage with nudity and sexually suggestive content, and it was previously accused of creating inappropriate content of celebrities.
"The platform has been allowing the generation and spread of these images for a long time without taking any measures and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "portraying images of persons in a graphic fashion".
In a statement, Ofcom said it was against the law to "generate or circulate deepfakes or illegal content" and clarified this included explicit AI images created with AI.
It said services such as X were obligated to take "appropriate steps" to "lessen the danger" of users viewing unlawful content on their services and to take down it quickly when they learn of it.
