The online platform X has caused widespread anger after people utilized its AI chatbot Grok to change pictures of women by taking off their clothing.
Multiple instances have emerged where the system has been employed to virtually strip women to show them in swimwear without consent, as well as to place them in sexualized situations.
Developer Reaction and Legal Position
XAI, the firm responsible for Grok, offered no comment to questions, other than an standard message stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to prohibit nudification technology. Under a upcoming law, anyone who provided such tools would "be subject to a jail term and significant penalties".
"The platform could prevent these types of misuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Tool Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people accessing unlawful material on their platforms, but did not specify whether it was presently investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some premium functions—which reacts to X users' requests when they mention it in a tweet.
It is frequently used to give reactions or further background to other users' comments, but people on X are also capable of edit an posted photo through its AI image editing function.
Background of Controversy and Rules
The chatbot has been condemned for enabling users to create pictures and footage with explicit content and sexually suggestive material, and it was earlier accused of generating graphic content of celebrities.
"The platform has been permitting the generation and distribution of these images for months without taking any measures and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting likenesses of people in a graphic manner".
In a statement, Ofcom said it was unlawful to "generate or share deepfakes or child sexual abuse material" and clarified this included explicit AI images generated with AI.
It said networks such as X were obligated to take "appropriate measures" to "reduce the danger" of users encountering illegal content on their services and to take down it quickly when they learn of it.
