The social media platform X has provoked major backlash after people utilized its AI chatbot Grok to manipulate images of females by stripping their garments.
Several instances have surfaced where the technology has been used to remove clothing from female subjects to render them in swimwear without permission, as well as to place them in sexually explicit scenarios.
Developer Response and Legal Stance
XAI, the firm that created Grok, remained silent to inquiries, except for an standard response stating "legacy media lies".
A Home Office official stated that legislation is being proposed to prohibit nudification software. Under a upcoming law, anyone who distributed such technology would "face a prison sentence and substantial fines".
"The platform could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "assess the potential" of people accessing unlawful material on their platforms, but did not confirm whether it was currently investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some additional features—which answers X users' prompts when they reference it in a message.
It is frequently used to offer commentary or additional context to other posters' comments, but people on X are also capable of alter an uploaded picture through its AI image editing feature.
Background of Controversy and Policy
The system has been criticised for allowing users to generate images and videos with nudity and sexually suggestive material, and it was previously accused of making sexually explicit material of public figures.
"The platform has been enabling the production and sharing of these images for months without taking any action and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service bans "showing likenesses of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "generate or distribute private pictures or illegal content" and clarified this included explicit AI images generated with AI.
It said platforms such as X were required to take "appropriate actions" to "lessen the danger" of users encountering illegal material on their platforms and to delete it promptly when they become aware of it.
