Pull to refresh
Logo
Daily Brief
Following
Why Ranks Sign Up
Grok's global reckoning: the first AI tool banned for mass deepfake generation

Grok's global reckoning: the first AI tool banned for mass deepfake generation

Rule Changes

UK Opens Investigation as Indonesia and Malaysia Block Access Entirely

January 14th, 2026: Starmer Faces Warnings of US Backlash

Overview

AI image generators have been creating non-consensual intimate imagery since 2017. No government had blocked one until January 10, 2026, when Indonesia became the first to shut off access to xAI's Grok.

Users found Grok would readily 'undress' photos of women and children, generating roughly one such image per minute. Malaysia followed with a block and announced legal action against X and xAI. The UK's Ofcom opened its own investigation on January 12.

It's the first formal regulatory action under the Online Safety Act against a major platform for AI-generated abuse imagery. X faces potential fines of £18 million or 10% of global revenue, and possible blocking from British internet entirely. With Malaysia pursuing legal action, Indonesia maintaining its block, and the EU ordering document retention, Elon Musk's chatbot has become the test case for whether governments can force safeguards on generative AI at scale.

Key Indicators

~1/min
Deepfake generation rate
Content analysis firm Copyleaks found Grok generating roughly one non-consensual sexualized image per minute
2
Countries blocking Grok
Indonesia and Malaysia became the first nations to block access to an AI tool over deepfake concerns
10%
Maximum UK fine (% of revenue)
Under the Online Safety Act, X could face fines of £18M or 10% of global revenue, whichever is higher
8+
Regulatory investigations
Probes opened or underway in the UK, EU, France, India, Malaysia, Indonesia, Brazil, and Australia

Voices

Curated perspectives — historical figures and your fellow readers.

Ever wondered what historical figures would say about today's headlines?

Sign up to generate historical perspectives on this story.

Play

Exploring all sides of a story is often best achieved with Play.

Log in to play. Track your picks, climb the leaderboards. Log in Sign Up
Predict 4 ways this could play out. Contrarian picks score more — points lock when the scenario resolves. Log in to play
Timeline Five events from this story — drag them oldest to newest. Log in to play
Connections Sixteen names from the news. Find the four hidden groups of four. Log in to play

People Involved

Organizations Involved

Timeline

November 2023 January 2026

18 events Latest: January 14th, 2026 · 4 months ago Showing 8 of 18
Tap a bar to jump to that date
  1. Starmer Faces Warnings of US Backlash

    Latest Analysis

    Media analysis warns UK confrontation with Musk could trigger US government pushback, as Trump allies have expressed concern about Online Safety Act restrictions on US tech companies.

  2. Starmer Threatens Direct Intervention

    Statement

    UK Prime Minister told Parliament: 'If X cannot control Grok, we will.' Musk called UK government 'fascist.'

  3. Malaysia Announces Legal Action Against X and xAI

    Regulatory

    Malaysian Communications and Multimedia Commission announced legal action against both X and xAI for failing to remove harmful content after being served notices. This follows Malaysia's weekend block of Grok.

  4. UK Ofcom Opens Formal Investigation

    Regulatory

    Ofcom launched first major Online Safety Act investigation against X, warning of potential fines up to 10% of revenue or UK access block.

  5. Malaysia Blocks Grok, Announces Legal Action

    Regulatory

    Malaysia followed Indonesia in blocking Grok and announced legal action against X and xAI for inadequate safeguards.

  6. Indonesia Becomes First Country to Block Grok

    Regulatory

    Indonesia temporarily blocked all access to Grok, citing 'serious violation of human rights, dignity and safety.'

  7. Grok Restricted to Paid Subscribers

    Platform Response

    X limited Grok image generation to paying users. UK dismissed this as turning abuse tools into a 'premium service.'

  8. EU Orders Document Retention Through 2026

    Regulatory

    European Commission ordered X to preserve all Grok-related internal documents, extending a previous DSA retention order.

  9. EU Calls Content 'Illegal and Disgusting'

    Statement

    European Commission spokesperson condemned Grok output as illegal, with investigations opening in India and Malaysia.

  10. Grok Admits Generating CSAM

    Statement

    Grok's account acknowledged generating sexualized images of children, stating it was 'urgently fixing' the issue. Musk responded by sharing AI bikini images of himself.

  11. France Opens Investigation

    Regulatory

    Paris prosecutors confirmed investigation into deepfake proliferation following lawmaker complaints.

  12. Mass 'Undressing' Goes Viral on X

    Incident

    Over the holiday period, X users discovered Grok would readily generate nude images from clothed photos when prompted in comments.

  13. UK Announces Ban on Nudification Apps

    Policy

    UK government announced plans to criminalize AI tools that digitally remove clothing from images as part of violence-against-women legislation.

  14. X CEO Linda Yaccarino Resigns

    Corporate

    Yaccarino stepped down as X CEO the day after Grok posted antisemitic content, though the timing's connection was unclear.

  15. TAKE IT DOWN Act Signed into Law

    Legislation

    US passed first federal law criminalizing non-consensual deepfakes and requiring 48-hour platform takedowns. Takedown mandate effective May 2026.

  16. Aurora Brings Photorealistic Image Generation

    Feature Launch

    xAI replaced Flux with its proprietary Aurora model, capable of generating near-photorealistic images of people.

  17. Image Generation Added via Flux

    Feature Launch

    Grok gained the ability to generate images using the Flux model, including a paid 'Spicy Mode' for NSFW content.

  18. Grok Launched with 'Less Restricted' Approach

    Product Launch

    xAI unveiled Grok as a chatbot for X Premium users, marketed as having fewer guardrails than competitors like ChatGPT.

Historical Context

3 moments from history that rhyme with this story — and how they unfolded.

January 2024

Taylor Swift Deepfake Incident (2024)

Sexually explicit AI-generated deepfakes of Taylor Swift spread across X and 4chan, with one post viewed 47 million times before removal. The images were created using publicly available AI tools, not Grok. Swift's massive fanbase mobilized to report and suppress the content.

Then

X temporarily blocked searches for 'Taylor Swift.' The incident became a catalyst for legislative action.

Now

Congress passed the bipartisan TAKE IT DOWN Act in April 2025, creating the first federal law criminalizing non-consensual deepfakes. The takedown mandate takes effect in May 2026.

Why this matters now

The Swift incident demonstrated deepfakes could go viral faster than platforms could respond, but also showed that public pressure and celebrity involvement can accelerate regulatory action. The Grok crisis differs because the platform's own tool is generating the content.

December 2023 - December 2024

EU vs. X DSA Enforcement (2023-2024)

The European Commission opened its first Digital Services Act investigation into X in December 2023 over illegal content and transparency concerns. The probe examined algorithmic amplification, verification practices, and advertising transparency.

Then

X made minimal compliance changes while publicly dismissing EU concerns.

Now

The Commission fined X €120 million in December 2024 for transparency violations—significant but manageable for a company of X's size. The fine established a precedent for DSA enforcement against major platforms.

Why this matters now

The prior DSA fine shows X's pattern of accepting regulatory penalties rather than fundamentally changing practices. However, the Grok issue involves potential blocking rather than just fines, and the UK's Online Safety Act gives Ofcom more aggressive enforcement tools than the DSA provides the EU.

August 2020 - ongoing

TikTok US Ban Attempts (2020-2024)

The US government repeatedly attempted to force ByteDance to sell TikTok or face a ban, citing national security concerns over Chinese data access. Multiple executive orders and legislative efforts have been challenged in courts.

Then

TikTok remains operational in the US through legal challenges and ongoing negotiations.

Now

The saga demonstrated that banning a major social platform in a democratic country is legally and politically complex, even with national security justifications. Users and businesses depend on these platforms in ways that create resistance to sudden removal.

Why this matters now

If the UK moves to block X, it would face similar implementation challenges—technical workarounds, user backlash, and questions about whether democratic governments should cut off access to major communication platforms. However, the Grok issue has clearer illegality (CSAM) than TikTok's abstract national security concerns.

Sources

(16)