If you want an app you built to be downloadable from the Apple App Store or Google Play Store, it has to pass a slew of criteria, including safety standards.
But a new report on Wednesday alleges that Apple and Google broke their own rules by promoting “nudify” apps that are outlawed in their app store policies.
The Tech Transparency Project, part of a nonprofit tech watchdog, first revealed in January that Apple and Google app stores had over 100 nudify or undressing apps. These are apps with the sole purpose of taking images of people, usually women, and editing them to appear to be that person without clothing, creating what’s called nonconsensual intimate imagery. Many of these apps use generative AI to create deepfakes.
Apple removed some of the prohibited apps at the time. But many are still out there, as evidenced in a subsequent investigation.
In April, TTP found that Apple and Google still allowed users to search for a number of troubling keywords, including “nudify,” “undress” and “deepnude.” After a deep dive on the top 10 apps across both app stores, TTP found that 40% of the apps advertised themselves as able to “render women nude or scantily clad,” according to the report.
The new report also found that Google and Apple actually promoted such apps in their stores, increasing their visibility, with Google in particular creating “a carousel of ads for some of the most sexually explicit apps encountered in the investigation.”
Read More: How to Keep Kids Safe Online? Europe Believes Its Age-Verification App Is the Answer
Apple and Google both have language in their policies that prohibits apps with “overtly sexual or pornographic material” (Apple) and “sexually suggestive poses in which the subject is nude, blurred or minimally clothed” (Google). And they’ve both enforced these policies in the past — particularly by going after porn apps.
But Apple and Google make money from app developers by running advertising and taking a part of paid app subscriptions. Analytics firm AppMagic found that these “nudify” apps were downloaded 483 million times and made more than $122 million in lifetime revenue.
“This revenue stream may be why the two companies have been less than vigilant when it comes to nudify apps that violate their policies,” TTP writes.
Google told CNET that Google Play doesn’t allow apps containing sexual content, and that many of the apps referenced in the report have been suspended for violating its policies.
Apple told CNET that it has removed 15 of the apps flagged in the report and has contacted six other app developers, notifying them that they need to address issues or risk being removed from the store. It also blocked several additional search terms flagged by TTP.
Nonconsensual graphically sexual content is a growing issue, due in part to AI. We saw in startling clarity how apps with AI can be used to make this illegal and abusive content at the beginning of the year, when Grok users made 1.4 million sexualized deepfakes over a nine-day period.
Some US senators at the time called on Apple and Google to remove Grok from their app stores, but neither removed it.
We learned this week that Apple privately reached out to Grok to express its concerns about its abusive AI capabilities and threatened to remove it. Grok is still available in the Apple and Google app stores and is still reportedly able to create abusive AI sexual images, despite the company saying otherwise.
