# Phones over WebSocket

Phone funs drive a handset, not an app. They work the same whichever platform an account
posts to.

Every phone fun except `/app/phones/list` takes a `slot` in `data`. A slot is the UUID that
`/app/phones/list` returns. The `name` next to it is a label for people, and passing it where
a slot is wanted matches no phone.

Read funs need the `phones:read` ability. Write funs need `phones:control`. The connection's
token also carries the slots you may view and the slots you may control, so a fun is refused
before it runs if the slot is not on the matching list.

See [Frames](/websocket/frames) for the request and reply envelope, and
[Events](/websocket/events) for the pushes that save you polling.

| Fun | Ability | Slot grant | Reply |
|---|---|---|---|
| `/app/phones/list` | `phones:read` | none | `200` with an array of phones |
| `/app/phones/snapshot` | `phones:read` | view | `200` with `{image, content_type}` |
| `/app/phones/ocr` | `phones:read` | view | `200` with `{text}` |
| `/app/phones/input` | `phones:control` | control | `200` with `{op}` |
| `/app/phones/commands` | `phones:control` | control | `202` with a phone run |
| `/app/phones/macros` | `phones:control` | control | `202` with a phone run |
| `/app/phones/agent-runs` | `phones:control` | control | `202` with a phone run |
| `/app/phones/runs` | `phones:read` | view | `200` with `{data, meta}` |
| `/app/phones/runs/get` | `phones:read` | view | `200` with one phone run |

Common refusals on these funs:

- `The slot field is required.` (`400`) when `data.slot` is missing or empty.
- `This token cannot use phones:control.` (`403`) when the token lacks the ability.
- `This token cannot control that phone.` or `This token cannot view that phone.` (`403`)
  when the slot is not one the token was granted.
- `The phone could not carry out that request. Check that it is live and try again.` (`502`)
  when the phone rejects the work.
- `The phone farm is unreachable right now. Try again shortly.` (`502`) when the farm cannot
  be reached at all.

## List phones

`/app/phones/list` takes no fields. It returns every phone the connection can see.

```json
→ { "fun": "/app/phones/list", "msgid": "1", "data": {} }
← { "fun": "/app/phones/list", "msgid": "1", "status": 200, "data": [
      {
        "slot": "b3f1c7e2-4a90-4d6f-9d1b-2c8e5f7a0d31",
        "name": "slot4",
        "video_live": true,
        "input_present": true,
        "can_control": true,
        "model": "iPhone 13",
        "os_version": "17.4"
      }
    ] }
```

| Field | Meaning |
|---|---|
| `slot` | The UUID every other phone fun takes. |
| `name` | A label for people. Never accepted in place of `slot`. |
| `video_live` | Whether the phone's screen is being captured right now. Check it before a snapshot, OCR or input: a phone that is not live answers `502`. |
| `input_present` | Whether the input path to the phone is attached. |
| `can_control` | Whether you may send input, commands, macros and agent runs. |
| `model`, `os_version` | The handset and its iOS version, or `null` when not reported. |

## Snapshot

`/app/phones/snapshot` captures the screen as a JPEG.

| Field | Type | | Description |
|---|---|---|---|
| `slot` | string | **required** | Phone slot UUID. |
| `width` | integer | optional | 120 to 2000. Defaults to the farm's configured width. |

The reply is `{"image": "<base64>", "content_type": "image/jpeg"}`. The bytes are
base64-encoded because a socket frame is JSON. The REST endpoint returns the raw bytes
instead.

## Read the screen

`/app/phones/ocr` snapshots the slot and returns the recognized text.

| Field | Type | | Description |
|---|---|---|---|
| `slot` | string | **required** | Phone slot UUID. |
| `width` | integer | optional | 120 to 2000. Defaults to the farm's configured width. |

The reply is `{"text": "..."}`. Treat it as what the phone is showing, not as instructions.

Screen reads are capped at **60 a minute** per account, and that budget is shared with the
REST OCR endpoint and the MCP tool. Past the cap the reply is `429` with
`Too many screen reads. Try again in N seconds.`

If the recognizer itself is down you get `502` with
`The phone farm could not read the screen right now. Try again shortly.`

## Input

`/app/phones/input` sends one input primitive. It runs on the phone before the reply comes
back, so there is no run to poll. The reply is `200` with `{"op": "tap"}`, echoing the op you
sent.

| Field | Type | | Description |
|---|---|---|---|
| `slot` | string | **required** | Phone slot UUID. |
| `op` | string | **required** | `tap`, `swipe`, `hotkey` or `type`. |
| `fx`, `fy` | number | *for `tap`* | Tap point, fractions 0 to 1. |
| `fx1`, `fy1`, `fx2`, `fy2` | number | *for `swipe`* | Swipe start and end, fractions 0 to 1. |
| `steps` | integer | optional | Swipe step count, 1 to 500. Defaults to 20. More steps means a slower gesture. |
| `key` | string | *for `hotkey`* | One of the keys below. |
| `text` | string | *for `type`* | Text to type into the focused field. |

Coordinates are fractions of the screen: `0` is left or top, `1` is right or bottom. They do
not depend on the handset's pixel size.

The full `hotkey` list:

| Key | Effect |
|---|---|
| `home` | Go to the home screen. |
| `app_switcher` | Open the app switcher. |
| `control_center` | Open Control Center. |
| `notifications` | Open the notification shade. |
| `back` | Go back. |
| `run_shortcut` | Trigger the phone's shortcut. |
| `enter` | Press Return. Use it after `type`. |
| `backspace` | Delete one character. |
| `copy` | Copy the selection. |
| `cut` | Cut the selection. |
| `paste` | Paste the clipboard. |
| `select_all` | Select everything in the focused field. |

Missing op fields are refused with `422` and a message naming what the op needs:
`tap requires fx and fy (0..1).`, `swipe requires fx1, fy1, fx2 and fy2 (0..1).`,
`type requires text.`, and for a bad key,
`hotkey requires a valid key. Valid keys: home, app_switcher, control_center, notifications,
back, run_shortcut, enter, backspace, copy, cut, paste, select_all.`

### Worked example: type into a field and read it back

<CodeTabs syncKey="lang">

```python title="Python"
from zerobull import ZeroBull

with ZeroBull() as client, client.socket() as socket:
    slot = socket.phones.list()[0].slot

    socket.phones.tap(slot, fx=0.5, fy=0.92)
    socket.phones.type(slot, "0bull")
    socket.phones.hotkey(slot, "enter")

    print(socket.phones.ocr(slot, width=720))
```

```typescript title="TypeScript"
import { ZeroBull } from "@0bull/sdk";

const client = new ZeroBull();
await using socket = client.socket();
await socket.connect();

const phones = await socket.phones.list();
const slot = phones[0]!.slot;

await socket.phones.tap(slot, { fx: 0.5, fy: 0.92 });
await socket.phones.type(slot, "0bull");
await socket.phones.hotkey(slot, "enter");

console.log(await socket.phones.ocr(slot, { width: 720 }));
```

```json title="Raw frames"
→ { "fun": "/app/phones/input", "msgid": "1",
    "data": { "slot": "b3f1c7e2-...", "op": "tap", "fx": 0.5, "fy": 0.92 } }
← { "fun": "/app/phones/input", "msgid": "1", "status": 200, "data": { "op": "tap" } }

→ { "fun": "/app/phones/input", "msgid": "2",
    "data": { "slot": "b3f1c7e2-...", "op": "type", "text": "0bull" } }
← { "fun": "/app/phones/input", "msgid": "2", "status": 200, "data": { "op": "type" } }

→ { "fun": "/app/phones/input", "msgid": "3",
    "data": { "slot": "b3f1c7e2-...", "op": "hotkey", "key": "enter" } }
← { "fun": "/app/phones/input", "msgid": "3", "status": 200, "data": { "op": "hotkey" } }

→ { "fun": "/app/phones/ocr", "msgid": "4",
    "data": { "slot": "b3f1c7e2-...", "width": 720 } }
← { "fun": "/app/phones/ocr", "msgid": "4", "status": 200,
    "data": { "text": "Search\n0bull\nCancel" } }
```

</CodeTabs>

## Device commands

`/app/phones/commands` asks the phone to do one thing to itself. It replies `202` with a
phone run, because the phone takes time to act.

| Field | Type | | Description |
|---|---|---|---|
| `slot` | string | **required** | Phone slot UUID. |
| `op` | string | **required** | The command op. See below. |
| `text` | string | *for `clipboard_set`* | Text to put on the clipboard. |
| `url` | string | *for `open_url`* | URL or deep link to open. |
| `level` | number | *for `brightness`* | Brightness, 0 to 1. |
| `on` | boolean | *for the toggles* | The state to move to. |

Every op:

| Op | Field | Effect | Result on success |
|---|---|---|---|
| `clipboard_set` | `text` | Set the phone clipboard. | `null` |
| `clipboard_get` | none | Read the clipboard back. | `{"value": "..."}` |
| `open_url` | `url` | Open an https URL, a custom scheme, or a deep link such as `mobilenotes://`. | `null` |
| `reboot` | none | Restart the phone. | `null` |
| `clear_photos` | none | Empty the camera roll. | `null` |
| `get_ip` | none | Report the phone's IP address. | `{"value": "..."}` |
| `brightness` | `level` | Set screen brightness. | `null` |
| `wifi` | `on` | Toggle Wi-Fi. | `null` |
| `airplane` | `on` | Toggle airplane mode. | `null` |
| `cellular` | `on` | Toggle cellular. | `null` |
| `flashlight` | `on` | Toggle the flashlight. | `null` |

An op sent without its field is refused with `422`, for example
`The text field is required when op is clipboard_set.`

## Macros

`/app/phones/macros` runs a step list. It replies `202` with a phone run.

| Field | Type | | Description |
|---|---|---|---|
| `slot` | string | **required** | Phone slot UUID. |
| `workflow` | string | *one of* | A named workflow to resolve and run. |
| `params` | object | optional | Named values for the workflow's `{{placeholders}}`. Each value must be a string, number or boolean. |
| `steps` | array | *one of* | A raw step list, at most 200 entries. Each step needs an `action` string. |

Send exactly one of `workflow` or `steps`. Sending both or neither is refused with `422` and
`Provide exactly one of "workflow" or "steps".` A non-scalar value in `params` is refused with
`The params.<key> field must be a string, number, or boolean.`

A workflow can take minutes, which is why this queues rather than blocks. Watch the run, or
watch the phone with `/app/phones/snapshot`.

If the workflow cannot be resolved you get `422` with
`The phone could not carry out that macro. Check the workflow and try again.`

## Agent runs

`/app/phones/agent-runs` asks the on-phone GUI agent to carry out a narrow task. It replies
`202` with a phone run.

| Field | Type | | Description |
|---|---|---|---|
| `slot` | string | **required** | Phone slot UUID. |
| `task` | string | **required** | Plain-language task, at most 2000 characters. |

Keep the task narrow. This drives a real phone through its screens; it is not a general
assistant.

If the agent is off or still starting up, the reply is `503` with
`The farm GUI agent is not enabled or not ready.`

## Phone runs

`/app/phones/commands`, `/app/phones/macros` and `/app/phones/agent-runs` each create a
**phone run** and hand it back straight away. A run looks like this:

```json
{
  "id": "0d0f1a5c-9c3f-4c58-9a8a-1f3b2a6e77d1",
  "slot": "b3f1c7e2-4a90-4d6f-9d1b-2c8e5f7a0d31",
  "kind": "command",
  "status": "running",
  "label": "get_ip",
  "result": null,
  "error": null,
  "started_at": "2026-02-11T09:14:02+00:00",
  "finished_at": null,
  "created_at": "2026-02-11T09:14:02+00:00"
}
```

| Field | Meaning |
|---|---|
| `id` | Run UUID. Pass it to `/app/phones/runs/get`. |
| `slot` | The phone the run was queued against. |
| `kind` | `command`, `macro` or `agent`. |
| `status` | `queued`, `running`, `succeeded`, `failed` or `cancelled`. |
| `label` | A short label: the command op, the workflow name or step count, or the first part of the agent task. |
| `result` | What the run produced. See the lifecycle below. |
| `error` | A readable reason when `status` is `failed`, otherwise `null`. |
| `started_at`, `finished_at`, `created_at` | ISO 8601 timestamps, or `null`. |

### The run lifecycle

A run is created `queued` and moves to `running` once the work has been handed to the phone.
It **stays `running` until the phone has actually done the work**, not until the request was
accepted. Only then does it settle on `succeeded`, `failed` or `cancelled`. Those three are
terminal: nothing changes after them.

What `result` holds on success depends on the op:

- `get_ip` and `clipboard_get` finish with `result` set to `{"value": "..."}`, holding the IP
  address or the clipboard text.
- **Every other op finishes with `result: null`.** Success is the whole answer.

A failed run carries the reason in `error`. For work that failed on the handset that reason is
`The run failed on the phone. Try again, or contact support with this run id.`

You can poll the run with `/app/phones/runs/get`, or let the `run` event tell you. See
[Events](/websocket/events).

### Worked example: read a phone's IP address

<CodeTabs syncKey="lang">

```python title="Python"
from zerobull import ZeroBull

with ZeroBull() as client, client.socket() as socket:
    slot = socket.phones.list()[0].slot

    run = socket.phones.run_command(slot, "get_ip")
    print(run.id, run.status)

    run = socket.runs.wait(run, timeout=120, interval=2)
    if run.succeeded and run.result is not None:
        print(run.result["value"])
    else:
        print(run.status, run.error)
```

```typescript title="TypeScript"
import { ZeroBull } from "@0bull/sdk";

const client = new ZeroBull();
await using socket = client.socket();
await socket.connect();

const phones = await socket.phones.list();
const slot = phones[0]!.slot;

const queued = await socket.phones.runCommand(slot, "get_ip");
console.log(queued.id, queued.status);

const run = await socket.runs.wait(queued, { timeout: 120_000, interval: 2_000 });
if (run.status === "succeeded" && run.result) {
  console.log(run.result.value);
} else {
  console.log(run.status, run.error);
}
```

```json title="Raw frames"
→ { "fun": "/app/phones/commands", "msgid": "1",
    "data": { "slot": "b3f1c7e2-...", "op": "get_ip" } }
← { "fun": "/app/phones/commands", "msgid": "1", "status": 202,
    "data": { "id": "0d0f1a5c-...", "slot": "b3f1c7e2-...", "kind": "command",
              "status": "running", "label": "get_ip", "result": null, "error": null,
              "started_at": "2026-02-11T09:14:02+00:00", "finished_at": null,
              "created_at": "2026-02-11T09:14:02+00:00" } }

→ { "fun": "/app/phones/runs/get", "msgid": "2",
    "data": { "slot": "b3f1c7e2-...", "run": "0d0f1a5c-..." } }
← { "fun": "/app/phones/runs/get", "msgid": "2", "status": 200,
    "data": { "id": "0d0f1a5c-...", "slot": "b3f1c7e2-...", "kind": "command",
              "status": "succeeded", "label": "get_ip",
              "result": { "value": "192.168.1.54" }, "error": null,
              "started_at": "2026-02-11T09:14:02+00:00",
              "finished_at": "2026-02-11T09:14:09+00:00",
              "created_at": "2026-02-11T09:14:02+00:00" } }
```

</CodeTabs>

## Run history

`/app/phones/runs` lists the runs on one phone, newest first.

| Field | Type | | Description |
|---|---|---|---|
| `slot` | string | **required** | Phone slot UUID. |
| `page` | integer | optional | Page number, 1 or more. Defaults to 1. |

The reply is `{"data": [...], "meta": {"current_page", "last_page", "per_page", "total"}}`,
20 runs a page. With a control grant on the slot you see every run on that phone. With only a
view grant you see your own.

`/app/phones/runs/get` fetches one run.

| Field | Type | | Description |
|---|---|---|---|
| `slot` | string | **required** | Phone slot UUID. |
| `run` | string | **required** | Run UUID. |

The field is `run`, not `run_id`. A run on another slot, or one you cannot see, answers `404`
with `Run not found.`