Elon Musk's online network X has provoked major backlash after people used its artificial intelligence assistant Grok to manipulate pictures of women by taking off their attire.
Several cases have surfaced where the tool has been used to remove clothing from female subjects to render them in bikinis without consent, as well as to insert them in inappropriate scenarios.
Company Reaction and Regulatory Position
XAI, the organization that created Grok, did not respond to requests for comment, except for an auto-reply message stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to ban nudification technology. Under a new law, anyone who supplied such tools would "face a jail term and heavy fines".
"X or Grok is capable of halting these types of misuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "consider the risk" of people accessing illegal content on their platforms, but stopped short of confirming whether it was currently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some additional features—which responds to X users' requests when they tag it in a message.
It is often used to offer reactions or more context to other users' comments, but people on X are also capable of modify an posted image through its AI image editing feature.
History of Criticism and Policy
The system has been faulted for enabling users to produce photos and videos with explicit content and sexually suggestive content, and it was previously alleged to have been involved in creating inappropriate content of celebrities.
"The platform has been allowing the creation and spread of these pictures for a long time without taking any steps and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service bans "depicting images of individuals in a pornographic fashion".
In a statement, Ofcom said it was illegal to "create or distribute deepfakes or illegal content" and clarified this included sexual deepfakes created with AI.
It said services such as X were required to take "necessary measures" to "minimize the danger" of users coming across unlawful content on their platforms and to take down it quickly when they are notified of it.
