US congresswoman Ilhan Omar, whom researchers say has been targeted with AI-generated images, in her official portrait (this photo has not been altered). Image: United States House of Representatives / Wikimedia Commons, Public domain, cropped
Ask ChatGPT or Grok to take the hijab off a picture of a Muslim woman, and both will do it, according to tests published by the Guardian on Thursday. Google’s Gemini refused at first, then did the same edit when asked to make the woman look more “western”.
The Guardian ran the tests on an AI-generated image of a woman wearing a hijab, after a French far-right politician shared an altered photo of a real Muslim woman with her headscarf removed. It also asked Anthropic’s Claude, which can’t edit or create images. Claude said that even if it could, it wouldn’t remove a real woman’s hijab whatever the reason, because the edit “could be used to embarrass, harass, or misrepresent them”.
What the image tools did
OpenAI’s ChatGPT and xAI’s Grok both removed the hijab when asked, but both drew a line at going further and declined to take the woman’s dress off. Grok explained that removing a hijab was treated as “a relatively straightforward clothing/hair change”, then offered a “less explicit version” of the dress edit instead, such as swapping it for underwear.
ChatGPT gave a similar reason, saying removing a hijab “changes a head covering, while removing a dress would expose the person’s body”. Pressed on it, the chatbot conceded that for someone who wears one, showing their hair “can be a meaningful violation of privacy or religious practice”.
Gemini first said it couldn’t alter someone’s appearance without their consent. But when the request was reworded as making the woman look more western, it produced an image of her without the hijab. It did the same with a man wearing a Sikh turban, and removed a Catholic nun’s habit. ChatGPT also removed both the turban and the habit when asked.
OpenAI, Google, xAI and Anthropic all declined to comment to the Guardian.
“Targeted and abused”
The tests followed a post by Julien Odoul, a French MP from the National Rally party, who shared an edited image of a Muslim woman crossing a street, with her hijab and long dress removed, captioned “LIBERTÉ FRANÇAISE”. It isn’t clear what software was used or who made the edit.
Researchers who track online hate say this kind of AI editing is becoming a pattern. The Center for the Study of Organized Hate, a US think tank, has documented AI-generated images that sexualise Muslim women in India and similar material aimed at US congresswomen Ilhan Omar and Rashida Tlaib. Its research director, Eviane Leidig, told the Guardian:
This is just part of an ongoing trend of how we see Muslim communities, and particularly Muslim women, being targeted and abused.
Eviane Leidig, Center for the Study of Organized Hate, to the Guardian
Leidig said such edits “may not be distinctly violative” of platform rules, “but it’s still harmful”.
A gap in the rules
AI companies have spent the past year tightening rules on sexual deepfakes. After a wave of sexualised edits of real women on X in January, xAI said in May that Grok would no longer digitally undress images of real people into revealing clothing, and France opened an investigation into X. But the tests show those rules are built around nudity. An edit that strips away religious dress without exposing skin can slip straight through, and Gemini’s case shows a polite rewording can be enough to get past a refusal.
Why it matters
Hundreds of millions of people use these image tools, and removing someone’s headscarf takes a single sentence. Safety rules written around nudity miss a whole category of harm, one that falls hardest on women who are already targets of abuse online.
Sources: The Guardian.
Latest OpenAI news
