Pull to refresh
Logo
What Changed
Following Why Sign Up
Meta releases SAM 3, a segmentation model prompted by text

Meta releases SAM 3, a segmentation model prompted by text

New Capabilities

Open-weights model finds and tracks described objects in images and video; SAM 3D adds 3D reconstruction.

Today: SAM 3 capabilities summarized in industry briefing

Overview

Updated 43 minutes ago

For years, computer vision models recognized a fixed list of objects. Ask for “the striped red umbrella” and they had no label for it. Meta’s SAM 3 takes plain noun phrases and returns masks for every matching object, then follows those objects through video.

The model is open weights, with training code and a benchmark called Segment Anything with Concepts. That lets researchers and companies adapt it. A companion, SAM 3D, reconstructs object shapes and human poses from a single photo. Meta says the tools are coming to Instagram Edits, Vibes, and Facebook Marketplace’s View in Room.

Why it matters

Open-vocabulary segmentation lets software find objects it was never explicitly trained to label, turning text descriptions into editable, trackable visual data.

Questions about this story

Free account needed to ask — your question is kept and asked for you right after sign-up. Answers are public.

No questions yet — be the first to ask.

Key Indicators

4 million
Unique concept labels in training set
Meta built the dataset with human annotators and AI models in the loop.
2×
Performance gain on SA-Co benchmark
Meta reports twice the performance of prior systems for image and video concept segmentation.
16
Objects tracked per forward pass
SAM 3.1 processes up to 16 objects together in one pass.
32 fps
Video throughput on one H100 GPU
SAM 3.1 reaches this rate for videos with a medium number of objects, per Meta.
75–80%
SAM 3 score relative to human annotators
Hugging Face model card estimate on the SA-Co benchmark.

Voices

Curated perspectives — historical figures and your fellow readers.

Sign up to generate historical perspectives on this story.

Log in Sign Up

Organizations Involved

Meta AI
Meta AI
AI Research Lab
Developer and publisher of SAM 3, SAM 3.1, and SAM 3D

Meta’s artificial intelligence research group.
FM
Facebook Marketplace
Shopping platform
Uses SAM 3D in its View in Room feature

Meta’s marketplace for buying and selling goods.
Hugging Face
Hugging Face
AI model hosting platform
Hosts SAM 3 model checkpoints and documentation

A platform for sharing machine learning models and datasets.

Timeline

November 2025 November 2026

4 events Latest: Today
Tap a bar to jump to that date

  1. SAM 3 capabilities summarized in industry briefing

    Today Publication

    SoBasically describes concept prompting and video tracking for an AI technology edition.

  2. SAM 3.1 doubles video throughput

    Update

    Object multiplexing tracks up to 16 objects per pass, reaching 32 frames per second on one H100 GPU.

  3. Meta releases SAM 3

    Product release

    Open-weights model accepts text phrases and image exemplars for concept segmentation.

  4. SA-Co benchmark opens

    Benchmark

    Meta publishes evaluation dataset for promptable concept segmentation.

Scenarios

1

Meta ships SAM 3 features in Instagram Edits and Vibes

Likely Resolves by Q2 2027

Discussed by: Meta AI blog and product announcements

Meta has said SAM 3 is coming to Instagram Edits and Vibes on the Meta AI app. A broad rollout would put text-prompt segmentation in front of consumer users. The trigger is general availability, not a limited test.

2

SAM 3.1 makes real-time tracking practical on a single consumer GPU

Possible Resolves by Q1 2027

Discussed by: Meta AI and independent benchmarkers

SAM 3.1 already reaches 32 frames per second on an H100 for medium object counts. The open question is whether that speed holds on cheaper hardware. Independent benchmarks would settle it.

3

Fine-grained medical segmentation still needs custom training

Likely Resolves by Jan 31, 2027

Discussed by: Meta researchers and medical imaging groups

Meta says SAM 3 struggles with out-of-domain terms like “platelet” in medical imagery. Fine-tuning on small annotated datasets closes some of the gap. A peer-reviewed medical benchmark would show whether zero-shot use is viable.

Historical Context

3 moments from history that rhyme with this story — and how they unfolded.

April 2023

Segment Anything Model (2023)

Meta released SAM with prompts as points, boxes, or masks. It returned segmentation masks for objects without a fixed label set. The model came with a large mask dataset.

Then

Researchers and developers adopted it for interactive labeling and image editing.

Now

SAM 2 added a memory module for video tracking, setting up the architecture SAM 3 extends.

Why this matters now

SAM 3 keeps the promptable approach but replaces geometric prompts with short text concepts.

2020

Detection Transformer (2020)

Facebook AI introduced DETR, an object detector built on transformers. It removed hand-designed components like non-maximum suppression.

Then

DETR showed that transformers could handle object detection end to end.

Now

It became a foundation for later detection systems.

Why this matters now

Meta says SAM 3’s image detector is based on DETR.

2021

CLIP (2021)

OpenAI trained a model to match images with text descriptions. CLIP could classify images by arbitrary text labels without task-specific training.

Then

Open-vocabulary recognition became practical for image classification.

Now

Text-image models became a common building block for vision systems.

Why this matters now

SAM 3 extends text-image matching from whole-image labels to pixel-level masks and video tracks.

Sources

(6)
+2
SoBasically Meta AI MarkTechPost Hugging Face Meta AI Research Meta AI