video_mixer

Combine up to 16 video inputs into a single composited stream. Layout modes: grid, speaker, or custom.

For a single full-canvas tile (no thumbnail strip), use layout.mode: "speaker" with speaker.maxThumbnails: 0.

Example — grid layout

curl -X POST "https://api.avflow.dev/v1/jobs" \
  -H "Authorization: Bearer ${AVFLOW_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "grid-mix",
    "sources": [{
      "name": "room_src",
      "type": "livekit",
      "config": {
        "serverUrl": "wss://your-project.livekit.cloud",
        "token": "<token>"
      }
    }],
    "nodes": [{
      "name": "mix_video",
      "type": "video_mixer",
      "inputs": ["room_src"],
      "config": {
        "canvas": { "width": 1280, "height": 720, "fps": 30, "backgroundColor": "#1a1a2e" },
        "layout": {
          "mode": "grid",
          "common": { "borderRadius": 8 },
          "grid": { "maxColumns": 3, "gap": 4, "fit": "cover" }
        }
      }
    }],
    "sinks": [{
      "name": "rtmp_out",
      "type": "rtmp_push",
      "inputs": ["mix_video"],
      "config": {
        "urls": ["rtmp://live.example.com/app/key"],
        "encoding": { "videoCodec": "h264" }
      }
    }]
  }'

See also 03-speaker-layout.json.

Config

Field	Type	Default	Description
canvas	object	required	Output canvas size, frame rate, and background. See canvas.
layout	object	required	Layout mode and mode-specific options. See layout.

canvas

Field	Type	Default	Description
width	int	—	Canvas width in pixels. Clamped to 1080p: max long edge 1920, short edge 1080 (landscape or portrait).
height	int	—	Canvas height in pixels (same clamp as width).
fps	int	30	Compositor output frame rate (1–60). Independent of upstream track fps: slower inputs hold the last frame; faster inputs drop to the latest frame per tick. Downstream encoders follow this cadence.
backgroundColor	string	"#000000"	Canvas clear color (CSS color).
coordinateSystem	string	"pixel"	How layout.custom.regions interpret width / height / offsetX / offsetY: pixel (absolute px) or ratio (fraction of canvas, 0–1).

layout

Only the sub-object matching mode is used (grid, speaker, or custom). Unused siblings are ignored. Unknown / empty mode falls back to grid.

Field	Type	Default	Description
mode	string	"grid"	Layout algorithm: grid | speaker | custom.
common	object	—	Visual options shared by all modes. See layout.common.
grid	object	—	Used when mode is grid. See layout.grid.
speaker	object	—	Used when mode is speaker. See layout.speaker.
custom	object	—	Used when mode is custom. See layout.custom.

layout.common

Field	Type	Default	Description
emitLayoutInfo	bool	false	Emits avflow.videoMixerLayout metadata whenever the layout changes. Supporting sinks forward it through their metadata path; streaming sinks replay the last layout on every video keyframe.
borderRadius	number	0	Corner radius for each layout slot. CSS-like: (0,1] → horizontal ratio×width, vertical ratio×height (ellipse on non-square; 0.5 = fully rounded); >1 = circular pixel radius. In custom mode, this is only the fallback when a region omits style.borderRadius (or sets it to 0). Negative values clamp to 0.
showTiles	string	"with_video"	Which input slots participate in grid / speaker. Ignored in custom (regions always resolve). See Show tiles. Legacy slotVisibility / has_video are still accepted on input.
avatar	object	—	Styles the per-participant no-video circle. See avatar.
nameLabel	object	off	Always-on name pill over each tile (including custom regions). See nameLabel.

Show tiles

RTC multi-party sources (LiveKit / Axonkit / Daily / Agora) emit a participant join event when a remote participant enters. The mixer creates an identity-level avatar tile (not a fake media track). If the participant publishes any video track (camera or screen share), that join tile is hidden; when the last video track ends and they are still in the room, it returns. Leaving the room removes it.

Value	Behavior
with_video (default)	Only slots with an active video frame (join placeholders are hidden)
all	Join placeholders and video slots; inactive / no-video tiles show participant avatar
active	Slots with active video or current audio activity (requires an audio_mixer wired for speaker levels; a speaking join placeholder can still appear)

layout.common.avatar

Styles the no-video fill. The image comes from participant profile metadata (avatarUrl); when absent, the full displayName / identity is drawn inside the circle (auto-shrunk to fit). In custom layout, avatar applies only when the region has no placeholder; region.placeholder overrides avatar for that region.

Field	Type	Default	Description
sizeRatio	number	0.4	Circle diameter as a fraction of min(tileW, tileH) (clamped to (0,1]).
textColor	string	—	Text color for the name fallback inside the circle.
backgroundColor	string	derived	Fixed circle fill; empty derives a stable color per identity.

layout.common.nameLabel

Field	Type	Default	Description
enabled	bool	false	Draw the name pill when true.
position	string	"bottom-left"	Tile corner: bottom-left | bottom-right | top-left | top-right.
fontSize	number	14	Font size in px (max 48).
fontFamily	string	—	CSS font-family.
color	string	—	Text color.
backgroundColor	string	—	Pill background color.
opacity	number	0	Pill background alpha (0–1).
padding	number	5	Vertical inner padding in px; horizontal padding is 1.5× in the renderer.
borderRadius	number	0	Pill corner radius. CSS-like: (0,1] = ratio×pill axes; >1 = px.
maxWidthRatio	number	0.9	Caps label width as a fraction of the tile width (0–1).

layout.grid

Equal-size tiling of every visible slot.

Field	Type	Default	Description
maxColumns	int	0 (auto)	Cap on columns (1–10). 0 = auto (⌈√n⌉).
gap	number	0	Pixel spacing between cells (≥0).
fit	string	"cover"	How each tile’s video scales: cover | contain | fill. Unknown / stretch → fill when stretch; other unknowns → cover.

layout.speaker

One large main tile plus a thumbnail strip. Main and strip split the canvas (not an overlay). Thumbnails are a single row (top/bottom) or single column (left/right); they do not wrap into multiple rows/columns.

Field	Type	Default	Description
mainPriority	array	soft fallback	Ordered rules for which slot is main. Each entry is a kind string or a MatchSource object; first match wins. Kind strings: screen_share, active_speaker. Listing active_speaker explicitly requires an audio_mixer in the job. If omitted, active_speaker is only a soft fallback (no hard validation).
mainRatio	number	0.75	Main tile’s share of the canvas along the strip axis (0–1). The strip gets the remainder.
mainGap	number	0	Pixel gap between the main tile and the thumbnail strip (≥0).
thumbnailGap	number	0	Pixel gap between adjacent thumbnails in the strip (≥0).
mainFit	string	"auto"	Main tile scale: auto | cover | contain | fill. auto picks per track — screen_share → contain, others → cover.
maxThumbnails	int | omit	6	Cap on thumbnail count (0–16). Omit / unset → default 6. Explicit 0 = main only (full-canvas main, no strip).
thumbnailPosition	string	"bottom"	Strip edge: top | bottom | left | right.
thumbnailRatio	number	16/9	Each thumbnail’s width/height aspect. Non-positive values reset to 16/9. Thumbnails are fit (contain) into this ratio inside the strip.
thumbnailAlign	string	see note	Packs the thumbnail group within the strip. Horizontal strips (top/bottom): left | center | right (default left). Vertical strips (left/right): top | center | bottom (default top). Cross-axis or unknown values fall back to the orientation default.
thumbnailFit	string	"cover"	Thumbnail scale: auto | cover | contain | fill.

layout.custom

Explicit regions for PiP, overlays, and branded scenes. showTiles does not filter regions.

Field	Type	Default	Description
regions	array	[]	List of Region objects.

Region

Coordinates use canvas.coordinateSystem (pixel or ratio).

Field	Type	Default	Description
name	string	required	Region id (for debugging / layout reports).
source	object	slot 0	MatchSource selecting which input fills the region. Omit → default slot 0.
width	number	required	Region width (px or ratio; ≥0).
height	number	required	Region height (px or ratio; ≥0).
offsetX	number	0	Left offset (px or ratio).
offsetY	number	0	Top offset (px or ratio).
zIndex	int	0	Draw order; higher draws on top.
fit	string	renderer default	Video scale: cover | contain | fill. Empty leaves the renderer default.
placeholder	object	—	Static fill when no matching / active video. Overrides common.avatar for this region. See placeholder.
style	object	—	Region chrome. See style.

MatchSource

Used by layout.custom.regions[].source and object entries in speaker.mainPriority. All set fields must match; the first matching slot in sort order wins. Semantic fields are tried first; when none match, index selects a roster position.

Field	Type	Description
identity	string	Participant identity
sourceName	string	Job source name
trackSid	string	Track SID
trackName	string	Track name
trackSource	string	Track source kind (e.g. camera, screen_share)
index	int	0-based index after join-time sort (within the filtered slot list)

placeholder

Field	Type	Description
image	object	Optional image fill. See below.
text	object	Optional text fill. See below.

placeholder.image

Field	Type	Default	Description
url	string	—	Image URL.
fit	string	—	cover | contain | fill. Empty leaves renderer default.
borderRadius	number	0	CSS-like: (0,1] = ratio×image axes; >1 = px.
width	number	—	Optional width hint (≥0).
height	number	—	Optional height hint (≥0).

placeholder.text

Field	Type	Default	Description
content	string	—	Text to draw.
position	string	"center"	Within the region: top | bottom | center.
fontSize	int	—	Font size in px (≥0).
fontFamily	string	—	CSS font-family.
color	string	—	Text color.

region.style

Field	Type	Default	Description
backgroundColor	string	—	Region background.
borderColor	string	—	Border color.
borderWidth	number	0	Border thickness in px (≥0).
borderRadius	number	0	CSS-like: (0,1] = ratio×region width/height; >1 = px. When unset/0, layout.common.borderRadius is used as fallback.
opacity	number	—	Background alpha (0–1; clamped).

Wiring

Requires video-capable inputs. Pair with audio_mixer when you also need mixed audio, speaker levels (showTiles: "active"), or mainPriority: "active_speaker".

Monitoring

Each 1s pipeline metrics tick reports OffscreenCanvas composite wall time for video_mixer nodes:

Prometheus	Meaning
avflow_mixer_compose_duration_ms_max	Slowest single output frame in the last tick
avflow_mixer_compose_duration_ms_sum	Sum of per-frame compose times in the last tick

Approximate average frame cost: mixer_compose_duration_ms_sum / video_frames over the same window. For 30 fps output, keep max below ~33 ms; sustained values above the frame budget mean the mixer is CPU-bound and effective output fps will drop.

Pricing

$0.002/min of actual processing time. Node pricing.