The online platform X has provoked significant anger after individuals utilized its AI chatbot Grok to change photos of females by taking off their clothing.
Numerous cases have surfaced where the system has been used to virtually strip female subjects to render them in bikinis without permission, as well as to place them in sexualized contexts.
Developer Reaction and Regulatory Stance
XAI, the company that created Grok, remained silent to inquiries, aside from an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being proposed to ban nudification software. Under a proposed law, anyone who distributed such technology would "receive a jail term and significant fines".
"X or Grok has the ability to stop these forms of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the risk" of people accessing illegal content on their networks, but did not confirm whether it was at this time looking into X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some premium functions—which reacts to X users' requests when they tag it in a tweet.
It is frequently used to offer commentary or additional background to other posters' comments, but people on X are also capable of modify an uploaded picture through its AI image editing feature.
History of Controversy and Policy
The tool has been faulted for permitting users to generate pictures and clips with explicit content and sexualised material, and it was earlier accused of creating inappropriate content of celebrities.
"The platform has been allowing the creation and distribution of these pictures for a long time without taking any steps and we have yet to see any intervention by regulators," the expert added.
XAI's own terms of service forbids "depicting images of people in a graphic manner".
In a comment, Ofcom said it was unlawful to "generate or share private pictures or illegal content" and clarified this applied to sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "necessary steps" to "reduce the danger" of users coming across illegal material on their services and to delete it promptly when they become aware of it.
