# events.json schema

`events.json` is the structured log produced by `record.js` and consumed by `postprocess.js` + `review.js`. It is a JSON array of events sorted by `t` (seconds since recording start).

## Event types

```ts
type Event =
  | { t: number, kind: "subtitle", label: string }
  | { t: number, kind: "move", x: number, y: number }
  | { t: number, kind: "click", x: number, y: number, label: string }
  | { t: number, kind: "mask_persistent", x: number, y: number, w: number, h: number, label: string }
```

### `subtitle`

Renders a burned-in subtitle from time `t` until the next `subtitle` event (or 3 seconds if it's the last one).

```json
{ "t": 22.51, "kind": "subtitle", "label": "Detail panel opened" }
```

### `move`

Cursor movement target. The synthetic cursor lerps from its current position to `(x, y)` over the interval until the next cursor event.

```json
{ "t": 22.80, "kind": "move", "x": 1022, "y": 748 }
```

`move` events are pushed by the `click` helper *before* the actual click (with a 700ms gap by default), so the cursor visibly arrives at the target before the click fires.

### `click`

Click flash. Triggers the ripple overlay at `(x, y)` and contributes to the cursor lerp endpoint.

```json
{ "t": 23.21, "kind": "click", "x": 1022, "y": 748, "label": "Click upload button" }
```

The `label` is also written to `subs.srt` only if no nearby subtitle exists (cluster collapse keeps subtitle and click labels from stacking).

### `mask_persistent`

A rectangular region blurred for the entire video. Postprocess inserts the blur AFTER subtitle burn, so it covers cursor, ripples, and subtitles — the masked area is unconditionally unreadable.

```json
{ "t": 0, "kind": "mask_persistent", "x": 0, "y": 820, "w": 220, "h": 80, "label": "sidebar-bottom" }
```

`t: 0` because the mask is in effect for the entire video; the field is kept only for schema uniformity. `x, y, w, h` are integer pixels in viewport coordinates.

Generated by `record.js` from the `PERSISTENT_MASKS` config: `selector` entries are resolved against the live DOM (boundingBox at startup), `box` entries are written through as-is.

## How postprocess consumes events

In order:

1. **Subtract `CALIBRATION = 0.65s` from every `t`** — the gap between Playwright's `newPage()` and our `tStart = Date.now()`. Mask events at `t: 0` are unaffected (they apply globally).
2. **Sort by `t`**.
3. **Filter cursor events** = `move` + `click`. Pass to `addRestEvents` which inserts a "rest" entry at `next.t - REST_LEAD` for any gap > 0.9s, so the cursor stays put before lerping to the next position rather than slowly drifting the entire interval.
4. **Build a piecewise lerp expression** for cursor X / Y as a function of video time `t`. Two ghost cursors trail the main cursor with `dt=0.10s`/`dt=0.20s` and reduced alpha.
5. **Build ripple overlay chains** — each `click` event spawns 2 expanding rings at staggered times.
6. **Build subs.srt** from `subtitle` events (cluster-collapsed: drops events with successors within 0.5s).
7. **Apply persistent masks** — for each `mask_persistent` event, split the video stream, crop+boxblur the mask region, overlay back at original coords. Inserted AFTER subtitle burn so masks cover everything below.
8. **Run ffmpeg** with the assembled `filter_complex` graph: `raw.webm + cursor.png + ripple.png` → cursor track → ripple overlays → subtitles → masks → `final.mp4`.

## How review consumes events

`review.js` reads `events.json` and slices `final.mp4` at strategic offsets:

| Pass | Source events | Sample times | Output |
|---|---|---|---|
| **flow** | `click` events | `t-0.20s` (pre), `t+0.05s` (mid), `t+0.35s` (react) | 3 frames per click |
| **visual** | `subtitle` events | `t+0.50s` (mid-display) | 1 frame per subtitle |
| **coverage** | `subtitle` events | `t+1.00s` (UI settled) | 1 frame per stage |
| **sensitive** | `mask_persistent` events + global timeline | per-mask: 10/50/90% of video (cropped to mask region); full-frame scans every 10s | 3 frames per mask + ⌈videoLen/10⌉ scans |

All sample times are **video time** (after `CALIBRATION` subtraction). See [ffmpeg-pipeline.md](ffmpeg-pipeline.md) for the rationale.

## Inspecting events.json manually

For debugging, the file is human-readable:

```bash
cat events.json | jq '.[] | select(.kind == "click") | {t, label}'
```

Lists all clicks with their labels and timestamps.

```bash
cat events.json | jq '[.[] | select(.kind == "subtitle")] | length'
```

Counts subtitle events (should match the subtitle count printed at end of `npm run record`).

```bash
cat events.json | jq '.[] | select(.kind == "click" and (.y > 800))'
```

Finds clicks whose `y` is suspiciously close to viewport bottom (these are the candidates for "cursor went off-frame" bugs — see [known-pitfalls.md](known-pitfalls.md#cursor-off-frame)).