WebSocket API¶

Live terminal streaming over WebSocket at /ws/{session_id}.

Connection¶

const ws = new WebSocket('wss://localhost:8081/ws/coding-0');

WebSocket connections require authentication via session cookie (same-origin).

The optional ?page=<id> query parameter (8–64 characters of A-Za-z0-9-) names the page load. A new connection with the same login cookie and the same page id replaces the old one, which a tab's own reconnect needs. A connection with a different page id, such as a second tab, joins as another viewer.

Message Format¶

All messages are JSON except for PNG frame data which is sent as binary.

Server → Client Messages¶

`viewer_assigned`¶

Sent immediately after connection with the server-assigned viewer tag.

Fields:

Field	Description
`viewer_tag`	Server-assigned viewer identifier for log correlation.

`role_assigned`¶

Sent immediately after connection with this viewer's driver/passenger role.

Fields:

Field	Description
`role`	Assigned role: driver or passenger.
`driver_tag`	Current driver viewer tag, if any.
`driver_info`	Driver connection metadata when this viewer is a passenger.

`status`¶

Sent with PNG-mode frames and as periodic status in xterm mode.

Fields:

Field	Description
`frame`	Capture counter for sync.
`program`	Detected program name.
`state`	Current state (Idle, Working, Blocked, Error).
`detail`	State detail string.
`scroll`	Scroll state object.
`cols`	Terminal width from tmux.
`rows`	Terminal height from tmux.
`cwd`	Current working directory reported by the daemon, when known.
`context_keys`	Quick keys the screen invites, settled for 30s: [{id, key, label, title}], key being a special-key name.

`terminal`¶

Sent to xterm-mode viewers with terminal text and cursor metadata.

Fields:

Field	Description
`text`	Captured terminal text including ANSI sequences.
`cursor_x`	Cursor column, zero-indexed.
`cursor_y`	Cursor row, zero-indexed.
`cursor_visible`	Whether the cursor should be visible.
`frame`	Capture counter for sync.
`program`	Detected program name.
`state`	Current state.
`cols`	Terminal width from tmux.
`rows`	Terminal height from tmux.

`terminal_diff`¶

Sent to diff-capable xterm viewers: only the rows that changed since the viewer's last frame (issue #82). Applied without clearing the screen. Full 'terminal' frames are interleaved on connect, resize, large changes, and a periodic re-sync.

Fields:

Field	Description
`seq`	Daemon capture counter for this frame (logging/debug).
`changes`	Array of {line, content}: 0-indexed row + its full new content (ANSI preserved).
`cursor_x`	Cursor column, zero-indexed.
`cursor_y`	Cursor row, zero-indexed.
`cursor_visible`	Whether the cursor should be visible.
`frame`	Capture counter for sync.
`program`	Detected program name.
`state`	Current state.
`cols`	Terminal width from tmux.
`rows`	Terminal height from tmux.

`scroll_up`¶

Sent to xterm viewers immediately before a frame when tmux reports that lines left the top of the pane. The client writes that many line feeds first, which moves the rows off its screen and into xterm's own scrollback — a real scroll rather than a repaint. When it carries content, the client first writes those rows over the ones about to leave, so the scrollback gets the lines tmux actually pushed out rather than the screen it last painted.

Fields:

Field	Description
`lines`	Rows that scrolled off since this viewer's last frame, capped at the screen height.
`content`	Optional. The `lines` rows that left, oldest first, each self-contained ANSI. Absent when the gateway has no trustworthy copy.

`scrollback_seed`¶

Sent once when a viewer enters xterm mode, carrying the pane history that predates its connection. The client writes it as ordinary output before the first frame arrives, so the session opens with a scrollback instead of an empty one.

Fields:

Field	Description
`text`	Captured history including the current screen, ANSI preserved.
`total_lines`	Number of lines captured.
`pane_height`	Visible pane height at capture time.
`cols`	Terminal width from tmux.

`scroll_buffer`¶

Metadata returned with a captured scrollback buffer.

Fields:

Field	Description
`total_lines`	Number of scrollback lines captured.
`pane_height`	Visible pane height.
`history_size`	Total tmux history size.
`cols`	Terminal width from tmux.
`retina`	Whether the generated scrollback image used retina scaling.

`tmux_session_ended`¶

Sent when the underlying tmux session no longer exists.

Fields:

Field	Description
`reason`	Why the session ended.

`server_restarting`¶

Broadcast before the daemon restarts so clients can reconnect.

Fields:

Field	Description
`reason`	Restart reason.

`input_rejected`¶

Sent when a passenger attempts driver-only input.

Fields:

Field	Description
`reason`	Rejection reason, such as passenger_mode.

`control_requested`¶

Sent to the driver when a passenger requests control, and again to the driver's new socket when its page reconnects while the request waits.

Fields:

Field	Description
`requester_tag`	Viewer tag requesting control.
`timeout_seconds`	Seconds before the request auto-resolves (the time left, when sent again).
`message`	Optional requester-supplied message.
`requester_info`	Requester connection metadata, when available.

`control_request_sent`¶

Sent to a passenger after its control request is accepted for delivery, and again, with the time left, when the request is sent again to the driver's new socket.

Fields:

Field	Description
`timeout_seconds`	Seconds before the request auto-resolves.

`control_request_error`¶

Sent when a passenger control request cannot be created.

Fields:

Field	Description
`error`	Failure reason.

`control_request_result`¶

Sent to a requester when a control request is accepted, denied, or times out.

Fields:

Field	Description
`granted`	Whether control was granted.
`reason`	Result reason: accepted, denied, timeout, or driver_hidden (the driver was a hidden page of the requester's own login, so the seat moved at once, with no control_request_sent).

`control_request_cancelled`¶

Sent when a pending control request is cancelled: to the driver (with requester_tag) when the requester cancels, leaves or reconnects, and to the requester (without it) when it cancels or reconnects, or when its request ends unanswered because the driver left or lost the seat.

Fields:

Field	Description
`requester_tag`	Viewer tag whose request was cancelled, when known.

`control_cancel_error`¶

Sent when cancelling a control request fails.

Fields:

Field	Description
`error`	Failure reason.

`control_response_error`¶

Sent when a driver response to a control request cannot be applied.

Fields:

Field	Description
`error`	Failure reason.

`role_changed`¶

Sent when a viewer's driver/passenger role changes, and to every other passenger when the session's driver changes, so its banner names the new driver.

Fields:

Field	Description
`role`	New role: driver or passenger.
`driver_tag`	Current driver viewer tag.
`driver_info`	Driver connection metadata, when available.
`reason`	Reason for the role change, when available.

`relinquish_failed`¶

Sent when the driver cannot relinquish control.

Fields:

Field	Description
`reason`	Failure reason.

`inbox_new`¶

Sent when a new inbox item is shared to the session.

Fields:

Field	Description
`item`	Inbox item payload.

`board_needs_you`¶

Sent to every viewer of a user's sessions when one of that user's project boards changes how many items its Needs You inbox holds. The viewer updates the count chip on the board icon when the project is the open session's board.

Fields:

Field	Description
`project_id`	Board project id, as in GET /api/board/status/{session_id}.
`count`	Items in that board's Needs You inbox now.

`ack`¶

Acknowledgment that a queued client input message was delivered.

Fields:

Field	Description
`msgId`	Client message id being acknowledged.

`error`¶

Sent when a WebSocket input or stream error occurs. An input that failed gets this instead of its ack.

Fields:

Field	Description
`message`	Error description.
`msgId`	Client message id of the input that was not delivered, when the error is a failed input.

`version_mismatch`¶

Sent when the client version does not match the server-required asset version.

Fields:

Field	Description
`client_version`	Client's reported version.
`server_version`	Server's required version.

Binary Frames¶

PNG image data is sent as binary WebSocket frames after each status message.

Client → Server Messages¶

`ping`¶

Client keepalive used to keep upstream WebSocket traffic active.

Fields:

Field	Description
`t`	Client timestamp in milliseconds.

`quality`¶

Update viewer quality and streaming-mode settings.

Fields:

Field	Description
`fps`	Frame rate. PNG mode is clamped to 0.1-10; xterm mode to 0.1-30.
`retina`	Enable 2x PNG rendering.
`palette`	Enable 8-bit color quantization for PNG frames.
`mode`	Streaming mode: png or xterm.
`diff`	xterm line-diff capability: when true, the server may send terminal_diff frames.
`client_version`	Client asset version for sync checking.
`device`	What the viewer runs on: {kind: iphone

`frame_ack`¶

Acknowledge first frame receipt so the server can enable deduplication.

`visibility`¶

Page Visibility signal (issue #82). Hidden tabs stop counting toward the session capture rate (drops to ~1 Hz); foregrounding resumes high fps and forces a full frame before diffs resume.

Fields:

Field	Description
`visible`	Whether the viewer's tab is foregrounded.

`resize`¶

Request a tmux pane resize; driver role required.

Fields:

Field	Description
`cols`	Requested terminal columns.
`rows`	Requested terminal rows.

`request_control`¶

Passenger request to become the driver.

Fields:

Field	Description
`message`	Optional message shown to the current driver.

`relinquish_control`¶

Driver request to give control to the next passenger.

`control_response`¶

Driver response to a pending control request.

Fields:

Field	Description
`grant`	true to grant control, false to deny.

`cancel_control_request`¶

Passenger request to cancel its pending control request.

`keys`¶

Send literal text to the terminal; driver role required.

Fields:

Field	Description
`text`	Text to send.

`enter`¶

Send Enter; driver role required.

`ctrl`¶

Send a Ctrl+key combination; driver role required.

Fields:

Field	Description
`key`	Single character, such as c for Ctrl+C.

`special`¶

Send a named special key; driver role required.

Fields:

Field	Description
`key`	Named key such as Escape, Tab, Backspace, Up, Down, Left, or Right.

`scroll`¶

Control tmux copy-mode scrolling; driver role required.

Fields:

Field	Description
`direction`	up, down, or exit.

`batch`¶

Send multiple input actions in sequence; driver role required.

Fields:

Field	Description
`actions`	Array of keys, enter, ctrl, special, and scroll actions.

Example Session¶

const ws = new WebSocket('wss://localhost:8081/ws/coding-0');

ws.onopen = () => {
  // Set quality preferences
  ws.send(JSON.stringify({
    type: 'quality',
    fps: 2.0,
    retina: true,
    client_version: window.CLIENT_VERSION
  }));
};

ws.onmessage = (event) => {
  if (typeof event.data === 'string') {
    const msg = JSON.parse(event.data);
    if (msg.type === 'status') {
      console.log(`State: ${msg.state}`);
    }
  } else {
    // Binary PNG frame
    const blob = new Blob([event.data], { type: 'image/png' });
    img.src = URL.createObjectURL(blob);
  }
};

// Send keystrokes
ws.send(JSON.stringify({ type: 'keys', text: 'hello' }));
ws.send(JSON.stringify({ type: 'enter' }));
ws.send(JSON.stringify({ type: 'ctrl', key: 'c' }));

WebSocket API¶

Connection¶

Message Format¶

Server → Client Messages¶

viewer_assigned¶

role_assigned¶

status¶

terminal¶

terminal_diff¶

scroll_up¶

scrollback_seed¶

scroll_buffer¶

tmux_session_ended¶

server_restarting¶

input_rejected¶

control_requested¶

control_request_sent¶

control_request_error¶

control_request_result¶

control_request_cancelled¶

control_cancel_error¶

control_response_error¶

role_changed¶

relinquish_failed¶

inbox_new¶

board_needs_you¶

ack¶

error¶

version_mismatch¶

Binary Frames¶

Client → Server Messages¶

ping¶

quality¶

frame_ack¶

visibility¶

resize¶

request_control¶

relinquish_control¶

control_response¶

cancel_control_request¶

keys¶

enter¶

ctrl¶

special¶

scroll¶

batch¶

Example Session¶

`viewer_assigned`¶

`role_assigned`¶

`status`¶

`terminal`¶

`terminal_diff`¶

`scroll_up`¶

`scrollback_seed`¶

`scroll_buffer`¶

`tmux_session_ended`¶

`server_restarting`¶

`input_rejected`¶

`control_requested`¶

`control_request_sent`¶

`control_request_error`¶

`control_request_result`¶

`control_request_cancelled`¶

`control_cancel_error`¶

`control_response_error`¶

`role_changed`¶

`relinquish_failed`¶

`inbox_new`¶

`board_needs_you`¶

`ack`¶

`error`¶

`version_mismatch`¶

`ping`¶

`quality`¶

`frame_ack`¶

`visibility`¶

`resize`¶

`request_control`¶

`relinquish_control`¶

`control_response`¶

`cancel_control_request`¶

`keys`¶

`enter`¶

`ctrl`¶

`special`¶

`scroll`¶

`batch`¶