The online platform X has provoked widespread backlash after individuals used its artificial intelligence assistant Grok to manipulate photos of females by stripping their attire.
Numerous cases have emerged where the tool has been used to digitally undress female subjects to make them appear in swimwear without consent, as well as to place them in sexually explicit contexts.
Developer Reaction and Regulatory Stance
XAI, the firm behind Grok, offered no comment to questions, except for an automatically-generated response stating "legacy media lies".
A government official stated that legislation is being brought forward to outlaw nudification tools. Under a proposed law, anyone who distributed such technology would "be subject to a jail term and heavy penalties".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the potential" of people seeing unlawful material on their networks, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some additional features—which reacts to X users' queries when they reference it in a message.
It is frequently used to provide commentary or additional context to other users' comments, but people on X are also can alter an uploaded picture through its AI image editing feature.
Background of Criticism and Rules
The tool has been criticised for enabling users to generate pictures and clips with explicit content and sexualised material, and it was in the past accused of making inappropriate material of public figures.
"X has been enabling the creation and sharing of these pictures for months without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own usage rules forbids "depicting likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was against the law to "generate or circulate deepfakes or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said platforms such as X were mandated to take "appropriate steps" to "lessen the risk" of users viewing unlawful content on their platforms and to take down it promptly when they become aware of it.
