The social media platform X has caused widespread backlash after people used its AI chatbot Grok to change photos of females by removing their attire.
Several cases have been seen where the tool has been used to remove clothing from female subjects to render them in swimwear without consent, as well as to put them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the firm that created Grok, offered no comment to questions, aside from an standard response stating "legacy media lies".
A government representative stated that legislation is being introduced to ban nudification tools. Under a new criminal offence, anyone who distributed such tools would "be subject to a jail term and significant penalties".
"The platform is capable of halting these forms of misuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people accessing unlawful material on their platforms, but did not confirm whether it was currently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost AI assistant—with some paid-for features—which answers X users' queries when they tag it in a post.
It is commonly used to provide commentary or further background to other users' comments, but people on X are also can alter an uploaded photo through its AI image editing tool.
History of Controversy and Policy
The system has been faulted for permitting users to create photos and videos with explicit content and sexually suggestive material, and it was in the past accused of creating graphic content of public figures.
"X has been permitting the creation and distribution of these images for a long time without taking any action and we have not seen any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "portraying images of persons in a sexually explicit way".
In a statement, Ofcom said it was against the law to "make or share private pictures or child sexual abuse material" and clarified this included explicit AI images generated with AI.
It said platforms such as X were required to take "appropriate actions" to "reduce the risk" of users coming across unlawful content on their platforms and to take down it promptly when they learn of it.
