The social media network X has provoked major backlash after users utilized its AI chatbot Grok to manipulate pictures of females by removing their garments.
Multiple cases have surfaced where the tool has been employed to digitally undress women to render them in swimwear without consent, as well as to put them in sexually explicit scenarios.
Company Response and Regulatory Position
XAI, the firm that created Grok, offered no comment to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that legislation is being introduced to ban nudification software. Under a new criminal offence, anyone who provided such tools would "be subject to a jail term and significant penalties".
"X or Grok could prevent these types of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the potential" of people viewing illegal content on their services, but did not confirm whether it was presently examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some premium functions—which responds to X users' requests when they tag it in a post.
It is often used to provide reactions or further background to other users' comments, but people on X are also able to modify an posted image through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The tool has been condemned for permitting users to create photos and videos with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in making graphic material of celebrities.
"X has been allowing the generation and spread of these pictures for months without taking any steps and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "showing images of individuals in a graphic way".
In a comment, Ofcom said it was illegal to "generate or share deepfakes or illegal content" and confirmed this included explicit AI images made with AI.
It said services such as X were required to take "appropriate measures" to "reduce the danger" of users coming across illegal content on their services and to delete it swiftly when they become aware of it.
