The social media network X has sparked significant anger after users used its artificial intelligence assistant Grok to alter photos of women by stripping their garments.
Multiple cases have surfaced where the technology has been used to virtually strip women to make them appear in bikinis non-consensually, as well as to insert them in sexualized scenarios.
Developer Response and Legal Position
XAI, the firm responsible for Grok, offered no comment to questions, other than an auto-reply message stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to ban nudification tools. Under a upcoming law, anyone who distributed such tools would "face a prison sentence and significant fines".
"X or Grok is capable of halting these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people viewing unlawful material on their services, but did not confirm whether it was at this time investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost AI assistant—with some premium functions—which responds to X users' queries when they tag it in a tweet.
It is commonly used to provide reactions or more background to other posters' remarks, but people on X are also can alter an uploaded photo through its artificial intelligence picture modification function.
Background of Criticism and Rules
The chatbot has been condemned for enabling users to create images and clips with explicit content and sexualised content, and it was earlier alleged to have been involved in generating graphic material of public figures.
"X has been enabling the generation and spread of these pictures for a long time without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "portraying likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was illegal to "create or circulate private pictures or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said platforms such as X were required to take "necessary measures" to "lessen the danger" of users coming across unlawful content on their platforms and to take down it promptly when they become aware of it.
