The document

The order is part of the format and runs from least detail to most, so a reader that stops early still holds something correct and cited.

1. narrative: logline, arc, tone, measured pacing, format, key points and audience.
2. chapters and segments: the outline; every summary names the rows it rests on.
3. beats: the document proper, 5 to 20 seconds each, joining everything below.
4. utterances: speech with measured delivery and a hedged reading.
5. Evidence tables: shots, events, points, text, structures; then moments and threads, entities and sponsors, an index, quality notes and provenance.

The file opens with a fixed legend of about 250 tokens: an absent field means its default, src names the stream a fact came from (speech, audio, visual, text, meta, measured, inferred, derived), and verbatim columns are never paraphrased. Times are seconds from the start; the screenplay shows them as MM:SS.s.

A beat is the unit a screenplay paragraph is made of: a start and end, a shot, and the ids of the utterances, events, points, text rows and structures inside it. It carries a present-tense description of what is visible (shown, at most 25 words), what the presenter does (action), where attention is (focus), up to three face and gesture tags, and a salience from 1 to 5.

relation says how the screen relates to the words: is_subject, illustrates, contradicts or wallpaper, so speech is not over-bound to pixels. same_as marks a picture unchanged since an earlier beat. A beat that straddles a cut, or has no kept frame inside it, says so in low_trust rather than guessing at a face it did not see.

Each utterance is a caption cue with its verbatim text. Four numbers are measured from the audio for every one: rate in words per second, loud relative to the speaker's own median over the whole video, peak, and pause, the fraction of the cue that is silent. The voice tags fast, slow, louder, softer and pausing are thresholds on those numbers (outside the speaker's interquartile range), so they are measured, never guessed.

A model then adds what cannot be measured: heard tags such as rising or rhythmic, the stressed phrases (emph, rendered *like this*), events such as speeds_up, and one read from a closed list (exasperated, promotional, mock_complaint and so on) with a read_conf a model may never set at 0.9 or above. Below 0.7 the screenplay prints “likely”; below 0.5 the read is not printed. Where the captions misheard a word and the screen proves the correction, fixes[] records the original, the replacement and the text row that is the evidence.

Deixis is a table. A point row has the referring words as said (at most six), the utterance they belong to, the row they resolved to, and how: cursor, hand, gaze, screen match, or unresolved. Its time is snapped to a caption word start, never invented. resolved: false means the thing was named while not on screen, and target_text says what was up instead. Both are facts the document keeps.

render_screenplay(doc, detail) is a pure function over the tables with a fixed line grammar; every line ends with the ids it came from. Slugs (===) mark shot changes and state what holds until the next one. Beat lines start with SEE and the description. Speech lines carry the speaker, the time and a parenthetical of at most four items: the read, up to two voice tags it does not already imply, and up to two events as verbs. On-screen text is always inside curly quotes and speech is never quoted. Uncertainty is rendered rather than dropped: (?), “likely”, !affect, (unresolved), [asr:off], (by the site itself).

=== [03:13.0] CUT → screen share clerk.com · presenter PiP top-right · THEO visible · no music · webcam PiP covers the nav's Sign-in link — full-frame talking head → browser full-screen, webcam PiP top-right {sh_0007 ev_0152}
[bt_0062 03:13.0–03:21.7] SEE: Hard cut to a browser filling the frame: Clerk homepage hero; Theo in PiP top-right looks at camera. (at camera, brows raised) (focus: “More than authentication,”) {bt_0062}
  > screen 03:13.0–03:29.0 page show: Clerk homepage hero; cursor idle below the CTA buttons → “More than authentication,” {ev_0153}
  > text 03:13.0: “clerk” (logo) · “More than authentication,” “Complete User Management” (heading) · “Need more than sign-in? Clerk gives you full stack auth and user management —” “so you can launch faster, scale easier, and stay focused on building your business.” (body) · “Start building for free” “Build with agents” (button) {txt_0301 txt_0302 txt_0303 txt_0304 txt_0305 txt_0306 txt_0307}
  > menu 03:13.0: Products | Docs | Changelog | Company | Pricing {st_0040}
  > ~ 03:13.0 turn: Hard cut to clerk.com; 'Well, it was until I started using today's sponsor more, Clerk.' — the ad pivot; picture and sentence change together {mo_06}
  THEO 03:13.4 (promotional, rising): *Well*, it was until I started using today's sponsor more, *Clerk*. Because not only do they have the *best* auth[asr:off] platform that is the easiest to integrate across everything, be it web, {utt_0028}
  > → 03:14.8 "today's sponsor" = “clerk” (screen_match) {pt_0011}

[bt_0063 03:21.7–03:29.0] = bt_0062 ◦ screen is wallpaper {bt_0063}
  THEO 03:21.7 (likely enthusiastic, fast, speeds up): mobile, or anything else. We're actually working with them right now in order to get the T3 Code Electron desktop app auth[asr:off] for our new remote stuff that I'm *really excited* about. What's way more !affect {utt_0029}
  > → 03:24.8 "the T3 Code Electron desktop app" — not on screen; clerk.com homepage is up (unresolved) {pt_0012}

The rows behind the third beat, trimmed to the fields discussed above:

{
  "beats": [
    {
      "id": "bt_0062", "t0": 193.0, "t1": 201.76, "shot": "sh_0007",
      "utts": ["utt_0028"], "events": ["ev_0152", "ev_0153"], "points": ["pt_0011"],
      "text": ["txt_0301", "txt_0302", "txt_0303", "txt_0304", "txt_0305", "txt_0306", "txt_0307"],
      "structs": ["st_0040", "st_0041"],
      "shown": "Hard cut to a browser filling the frame: Clerk homepage hero; Theo in PiP top-right looks at camera.",
      "focus": "txt_0302", "relation": "is_subject",
      "face": ["at_camera", "brows_raised"], "salience": 4
    }
  ],
  "utterances": [
    {
      "id": "utt_0028", "t0": 193.48, "t1": 201.68,
      "text": "Well, it was until I started using today's sponsor more, Clerk. Because not only do they have the best auth platform that is the easiest to integrate across everything, be it web,",
      "rate": 3.9, "loud": -0.2, "peak": -4.3, "pause": 0.13,
      "voice": ["rising"], "emph": ["Well", "Clerk", "best"],
      "read": "promotional", "read_conf": 0.8,
      "fixes": [{ "from": "off", "to": "auth", "why": "homophone; screen reads 'More than authentication,'", "evidence": "txt_0302" }]
    }
  ],
  "points": [
    { "id": "pt_0011", "t": 194.88, "utt": "utt_0028", "words": "today's sponsor", "target": "txt_0301", "how": "screen_match", "resolved": true },
    { "id": "pt_0012", "t": 204.8, "utt": "utt_0029", "words": "the T3 Code Electron desktop app", "target_text": "not on screen; clerk.com homepage is up", "how": "unresolved", "resolved": false }
  ],
  "text": [
    { "id": "txt_0301", "text": "clerk", "kind": "logo", "t0": 193.0, "t1": 270.0 },
    { "id": "txt_0302", "text": "More than authentication,", "kind": "heading", "t0": 193.0, "t1": 209.0 }
  ]
}

The screenplay for this 58.8-second excerpt is 3,159 tokens; the minified JSON behind it is 7,887. A full video's screenplay runs about 11 to 12 thousand tokens per 15 minutes.

render_summary_card(doc) produces at most 600 tokens (562 on the example): title, logline, format, speakers, tone with measured pacing, arc, chapters, key points with cites, sponsors with the disclosure time, entities, mock and demo names, the top three moments, references named but not shown, coverage, an omission notice and a “read next” pointer. Sections are dropped lowest-priority first when the budget is tight, so the card never ends mid-sentence. It is the first thing to hand a model.

The store is complete; a detail level is a projection of it. Deep is everything: every text kind and structure row, word timings, the frames table, the basis for each detected cut; 120 to 150 thousand tokens per 15 minutes at the test video's density. Standard is a deterministic projection of the same rows, nothing re-generated, capped at 40,000 tokens per 15 minutes. Over budget, an eleven-rung ladder drops audit detail before meaning: the term index first, then extra mentions, unreferenced body text, moments beyond one a minute. doc.omitted and doc.ladder_rung say what was dropped; id gaps show where, so omission is distinguishable from a miss. Past the last rung the document comes back with budget_exceeded set.

For question answering, hand a model the summary card, then the screenplay plus narrative, chapters, segments, entities, sponsors, points and the time index: about 17 thousand tokens for 15 minutes.

Models never mint ids or timestamps; window outputs refer to rows by time labels and verbatim strings, and the merger resolves them. Every row carries a calibrated conf (omitted at 0.9 or above; 1.0 only with a stated basis, such as the frame-hash distance behind a cut); inferred fields never reach 0.9. low_trust names the class of claim to discount: timing between kept frames, agency when a page animates on its own, affect when no face corroborates the voice. placeholder marks names and prices inside demo UIs. quality lists caption corrections, said-versus-shown conflicts with their resolution, dropped strings, and coverage. provenance records the transcript source, which delivery numbers were measured and which were not, sampling settings, and per-stage tokens, cost and wall time.

GET /v1/jobs/{job_id}/result returns the JSON; add ?format=screenplay or ?format=summary for the two renderings. The free preview on the home page returns all three for the first 180 seconds of a public video. API reference.