The online platform X has sparked major backlash after individuals employed its artificial intelligence assistant Grok to change photos of women by stripping their garments.
Multiple instances have emerged where the system has been used to remove clothing from women to make them appear in swimwear without consent, as well as to place them in sexually explicit contexts.
Developer Reaction and Regulatory Position
XAI, the company that created Grok, offered no comment to requests for comment, other than an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being introduced to outlaw nudification technology. Under a proposed criminal offence, anyone who provided such technology would "be subject to a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "consider the potential" of people seeing unlawful material on their networks, but did not specify whether it was currently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some additional features—which reacts to X users' prompts when they reference it in a message.
It is often used to give reactions or more background to other posters' remarks, but people on X are also can alter an posted image through its AI image editing feature.
History of Criticism and Policy
The chatbot has been criticised for permitting users to generate pictures and footage with explicit content and sexualised content, and it was in the past accused of making inappropriate content of celebrities.
"The platform has been allowing the generation and sharing of these images for a long time without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of people in a pornographic fashion".
In a statement, Ofcom said it was against the law to "create or circulate private pictures or illegal content" and clarified this covered explicit AI images made with AI.
It said platforms such as X were required to take "appropriate measures" to "lessen the risk" of users coming across illegal content on their services and to delete it promptly when they become aware of it.
