Skip to content
Ambient Ambient Ambient Docs

Overview

Ambient is the open-source video engine for AI agents. Give it a video (a short clip or hours of footage) and a task. Its agents watch the footage the way a person would, gather evidence, and produce the outcome: an answer with timestamp citations, JSON that matches a schema you supply, or files such as clips and reports. Any video in, any outcome out.

How it works

Section titled “How it works”

Ambient doesn’t flatten a video into one transcript and hope the answer is in it. It runs an agent that calls tools on time windows of the video, the way a person would scrub through footage:

  1. Search. search_clip scans a stretch of the video for the moments that matter.
  2. Focus. focus_clip looks closely at a short window. If the agent model can see video itself, it watches the clip directly with self_focus_clip.
  3. Grab frames. On image-capable models, grab_frames pulls exact frames, and draw_bounding_box / draw_point mark what’s in them.
  4. Verify. The agent checks what it found against the footage before committing.
  5. Answer with citations. The final answer cites timestamps such as 12:05 or ranges such as 12:05::18:20.

The toolset adapts to the model you pick. Text-only models get annotate_frames instead of the image tools, and an optional bash tool lets the agent run ffmpeg and other shell commands.

There is also a Fast mode: one dense-frame vision pass with no tools. It is much quicker and works well for whole-video summaries that don’t need frame-accurate grounding.

Ways to use Ambient

Section titled “Ways to use Ambient”

Studio

A web UI for uploading videos (or importing a YouTube URL), asking questions, and watching the agent work. The one-line installer sets it up for you.

Python SDK

The engine implements the Anthropic Managed Agents API, so the official anthropic SDK drives it directly through client.beta.files, client.beta.sessions, and related endpoints.

Where to go next

Section titled “Where to go next”
Quickstart Install Ambient and get your first answer in a few minutes.
Studio setup Run and configure the Studio web UI.
Studio walkthrough Upload a video, ask questions, and read the agent's work.
SDK quickstart Drive the engine from Python with the Anthropic SDK.
Configuration reference Every setting the engine reads, with defaults.