Scientists reconstruct movies from mouse brain activity alone

Overview

Updated 1 hour ago

A team at University College London rebuilt 10-second videos of what mice watched, using only the firing of neurons in their visual cortices. The reconstructed clips run at 30 frames per second and match the originals at a pixel correlation of 0.57 — roughly double the accuracy of earlier single-trial reconstructions in awake mice.

The method inverts a neural encoding model trained to predict neuron activity from video. Starting from a blank movie, the team optimizes it until the model's predicted activity matches what was actually recorded from about 8,000 neurons per mouse. Because it works neuron-by-neuron, the technique offers a direct way to study how the brain reshapes visual input before it becomes perception.

Why it matters

Decoding what a brain sees from neuron firing gives researchers a direct tool to study perception — and another step toward brain-reading technology.

Questions about this story

Free account needed to ask — your question is kept and asked for you right after sign-up. Answers are public.

No questions yet — be the first to ask.

Key Indicators

0.57

Pixel correlation with original movies

Measured across all pixels and frames of 10-second clips; prior awake-mouse static reconstructions reached 0.238

8,000

Neurons recorded per mouse

Two-photon calcium imaging across a 630-by-630-micron area of visual cortex V1

28%

Quality gain from ensembling seven model instances

Averaging reconstructions across models removed high-frequency noise; most of the gain came from just two models

10 s at 30 Hz

Reconstructed movie length and frame rate

Ten natural movies reconstructed from five mice

Voices

Curated perspectives — historical figures and your fellow readers.

Ever wondered what historical figures would say about today's headlines?

People Involved

Joel Bauer

Senior Research Fellow, Sainsbury Wellcome Centre (UCL)

Lead author of the movie reconstruction study

Troy W. Margrie

Head of the Margrie Lab, Sainsbury Wellcome Centre (UCL)

Co-supervisor of the study

Claudia Clopath

Head of the Clopath Lab, Imperial College London

Co-supervisor of the study

Organizations Involved

Sainsbury Wellcome Centre

Neuroscience research institute

Host institution for the study's lead researcher

University College London neuroscience institute integrating experimental and computational approaches to brain circuit function.

eLife

Open-access scientific journal

Published the study

Peer-reviewed open-access journal for life and biomedical sciences; the reconstruction study appears as article 105081.

Sensorium 2023 Competition

Machine learning competition

Source of the dataset and winning model

2023 competition that challenged teams to predict mouse V1 neuron activity from videos and behavioral data.

Timeline

August 2023 September 2026

3 events Latest: Today

September 2026
News outlets report the results
Today News

ScienceDaily and other science outlets cover the reconstruction of 10-second movies from mouse brain activity alone.
Today
eLife publishes the reconstruction study
Publication

Peer-reviewed paper reports single-trial reconstructions of 10-second movies at 30 Hz from mouse visual cortex activity, reaching 0.57 pixel correlation.
2 days ago
August 2023
Sensorium 2023 competition releases mouse V1 dataset
Data Release

The competition publishes paired natural movies and two-photon calcium recordings from roughly 8,000 neurons per mouse across 10 mice.
August 1st, 2023

Scenarios

UCL team publishes findings on how mouse vision warps reality

Likely Resolves by Sep 16, 2027

Discussed by: The study authors themselves in the eLife paper, which names understanding deviations between neural representation and ground truth as the next goal

The team plans to use the reconstruction method to study how the visual processing pipeline skews and warps sensory input. A follow-up paper could quantify where the brain's representation diverges from the actual stimulus — for example, how arousal, running speed, or attention change what is encoded in V1.

Higher-resolution and wider-field reconstructions arrive

Likely Resolves by Sep 16, 2027

Discussed by: The eLife paper, which states the team plans to improve resolution and visual coverage of reconstructions

The team says the next technical step is data that supports higher resolution and wider coverage of the visual scene. This could involve recording more neurons across a larger area or improving the encoding model, pushing pixel correlation well above 0.57.

Approach extends to human single-cell recordings

Unlikely Resolves by Sep 16, 2028

Discussed by: Not directly predicted by the authors; speculative but supported by the eLife paper's emphasis on single-cell fidelity versus fMRI

The method requires single-neuron recordings, which are routine in mice but rare in humans. If applied to human single-cell or high-density recordings during surgery, it could offer far finer decoding than fMRI-based approaches. This faces clear regulatory and technical hurdles and is the most speculative outcome.

Historical Context

3 moments from history that rhyme with this story — and how they unfolded.

March 2008

Kay et al. identifying natural images from fMRI (2008)

Gallant's Berkeley lab showed that a computational encoding model could identify which of a set of natural images a person was viewing from their fMRI response. It was the first rigorous proof that visual content could be decoded from brain activity.

Then

Established the encoding-model-plus-inversion framework now used across visual decoding.

Now

Laid the methodological foundation that later movie reconstructions in both fMRI and single-cell settings built on.

Why this matters now

The UCL method follows the same framework — train a model to predict brain responses, then invert it — applied at the much finer scale of individual neurons.

October 2011

Nishimoto et al. movie reconstruction from fMRI (2011)

Jack Gallant's lab at UC Berkeley reconstructed continuous movie clips from human fMRI activity using voxel-wise encoding models. Participants watched Hollywood trailers in the scanner, and the team rebuilt approximate moving images from the brain scans.

Then

The Nature paper established that natural visual experiences, not just contrived stimuli, could be decoded from brain activity.

Now

It became the reference point for human fMRI-based visual reconstruction and inspired single-cell approaches.

Why this matters now

The UCL mouse study is the single-cell successor to this fMRI work, trading whole-brain coverage for the far finer detail of neuron-level recording.

2020

Yoshida and Ohki static image reconstruction in mice (2020)

Researchers reconstructed static images from awake mouse V1 neuron responses using two-photon imaging. Single-trial reconstructions reached a pixel correlation of about 0.238 over a roughly 43-degree visual field.

Then

Demonstrated single-cell image decoding in awake mice, though with limited fidelity.

Now

Set the accuracy baseline that later reconstruction work aimed to beat.

Why this matters now

The new movie reconstruction roughly doubles that correlation (0.57) while adding temporal dynamics — a direct, quantifiable advance over the prior state of the art.