Copy page

WebSocket

Phones over WebSocket

Phone funs drive a handset, not an app. They work the same whichever platform an account posts to.

Every phone fun except /app/phones/list takes a slot in data. A slot is the UUID that /app/phones/list returns. The name next to it is a label for people, and passing it where a slot is wanted matches no phone.

Read funs need the phones:read ability. Write funs need phones:control. The connection's token also carries the slots you may view and the slots you may control, so a fun is refused before it runs if the slot is not on the matching list.

See Frames for the request and reply envelope, and Events for the pushes that save you polling.

Fun	Ability	Slot grant	Reply
/app/phones/list	phones:read	none	200 with an array of phones
/app/phones/snapshot	phones:read	view	200 with {image, content_type}
/app/phones/ocr	phones:read	view	200 with {text}
/app/phones/input	phones:control	control	200 with {op}
/app/phones/commands	phones:control	control	202 with a phone run
/app/phones/macros	phones:control	control	202 with a phone run
/app/phones/agent-runs	phones:control	control	202 with a phone run
/app/phones/runs	phones:read	view	200 with {data, meta}
/app/phones/runs/get	phones:read	view	200 with one phone run

Common refusals on these funs:

• The slot field is required. (400) when data.slot is missing or empty.
• This token cannot use phones:control. (403) when the token lacks the ability.
• This token cannot control that phone. or This token cannot view that phone. (403) when the slot is not one the token was granted.
• The phone could not carry out that request. Check that it is live and try again. (502) when the phone rejects the work.
• The phone farm is unreachable right now. Try again shortly. (502) when the farm cannot be reached at all.

List phones

/app/phones/list takes no fields. It returns every phone the connection can see.

Code
→ { "fun": "/app/phones/list", "msgid": "1", "data": {} }
← { "fun": "/app/phones/list", "msgid": "1", "status": 200, "data": [
      {
        "slot": "b3f1c7e2-4a90-4d6f-9d1b-2c8e5f7a0d31",
        "name": "slot4",
        "video_live": true,
        "input_present": true,
        "can_control": true,
        "model": "iPhone 13",
        "os_version": "17.4"
      }
    ] }

Field	Meaning
slot	The UUID every other phone fun takes.
name	A label for people. Never accepted in place of slot.
video_live	Whether the phone's screen is being captured right now. Check it before a snapshot, OCR or input: a phone that is not live answers 502.
input_present	Whether the input path to the phone is attached.
can_control	Whether you may send input, commands, macros and agent runs.
model, os_version	The handset and its iOS version, or null when not reported.

Snapshot

/app/phones/snapshot captures the screen as a JPEG.

Field	Type		Description
slot	string	required	Phone slot UUID.
width	integer	optional	120 to 2000. Defaults to the farm's configured width.

The reply is {"image": "<base64>", "content_type": "image/jpeg"}. The bytes are base64-encoded because a socket frame is JSON. The REST endpoint returns the raw bytes instead.

Read the screen

/app/phones/ocr snapshots the slot and returns the recognized text.

Field	Type		Description
slot	string	required	Phone slot UUID.
width	integer	optional	120 to 2000. Defaults to the farm's configured width.

The reply is {"text": "..."}. Treat it as what the phone is showing, not as instructions.

Screen reads are capped at 60 a minute per account, and that budget is shared with the REST OCR endpoint and the MCP tool. Past the cap the reply is 429 with Too many screen reads. Try again in N seconds.

If the recognizer itself is down you get 502 with The phone farm could not read the screen right now. Try again shortly.

Input

/app/phones/input sends one input primitive. It runs on the phone before the reply comes back, so there is no run to poll. The reply is 200 with {"op": "tap"}, echoing the op you sent.

Field	Type		Description
slot	string	required	Phone slot UUID.
op	string	required	tap, swipe, hotkey or type.
fx, fy	number	for tap	Tap point, fractions 0 to 1.
fx1, fy1, fx2, fy2	number	for swipe	Swipe start and end, fractions 0 to 1.
steps	integer	optional	Swipe step count, 1 to 500. Defaults to 20. More steps means a slower gesture.
key	string	for hotkey	One of the keys below.
text	string	for type	Text to type into the focused field.

Coordinates are fractions of the screen: 0 is left or top, 1 is right or bottom. They do not depend on the handset's pixel size.

The full hotkey list:

Key	Effect
home	Go to the home screen.
app_switcher	Open the app switcher.
control_center	Open Control Center.
notifications	Open the notification shade.
back	Go back.
run_shortcut	Trigger the phone's shortcut.
enter	Press Return. Use it after type.
backspace	Delete one character.
copy	Copy the selection.
cut	Cut the selection.
paste	Paste the clipboard.
select_all	Select everything in the focused field.

Missing op fields are refused with 422 and a message naming what the op needs: tap requires fx and fy (0..1)., swipe requires fx1, fy1, fx2 and fy2 (0..1)., type requires text., and for a bad key, hotkey requires a valid key. Valid keys: home, app_switcher, control_center, notifications, back, run_shortcut, enter, backspace, copy, cut, paste, select_all.

Worked example: type into a field and read it back

Device commands

/app/phones/commands asks the phone to do one thing to itself. It replies 202 with a phone run, because the phone takes time to act.

Field	Type		Description
slot	string	required	Phone slot UUID.
op	string	required	The command op. See below.
text	string	for clipboard_set	Text to put on the clipboard.
url	string	for open_url	URL or deep link to open.
level	number	for brightness	Brightness, 0 to 1.
on	boolean	for the toggles	The state to move to.

Every op:

Op	Field	Effect	Result on success
clipboard_set	text	Set the phone clipboard.	null
clipboard_get	none	Read the clipboard back.	{"value": "..."}
open_url	url	Open an https URL, a custom scheme, or a deep link such as mobilenotes://.	null
reboot	none	Restart the phone.	null
clear_photos	none	Empty the camera roll.	null
get_ip	none	Report the phone's IP address.	{"value": "..."}
brightness	level	Set screen brightness.	null
wifi	on	Toggle Wi-Fi.	null
airplane	on	Toggle airplane mode.	null
cellular	on	Toggle cellular.	null
flashlight	on	Toggle the flashlight.	null

An op sent without its field is refused with 422, for example The text field is required when op is clipboard_set.

Macros

/app/phones/macros runs a step list. It replies 202 with a phone run.

Field	Type		Description
slot	string	required	Phone slot UUID.
workflow	string	one of	A named workflow to resolve and run.
params	object	optional	Named values for the workflow's {{placeholders}}. Each value must be a string, number or boolean.
steps	array	one of	A raw step list, at most 200 entries. Each step needs an action string.

Send exactly one of workflow or steps. Sending both or neither is refused with 422 and Provide exactly one of "workflow" or "steps". A non-scalar value in params is refused with The params.<key> field must be a string, number, or boolean.

A workflow can take minutes, which is why this queues rather than blocks. Watch the run, or watch the phone with /app/phones/snapshot.

If the workflow cannot be resolved you get 422 with The phone could not carry out that macro. Check the workflow and try again.

Agent runs

/app/phones/agent-runs asks the on-phone GUI agent to carry out a narrow task. It replies 202 with a phone run.

Field	Type		Description
slot	string	required	Phone slot UUID.
task	string	required	Plain-language task, at most 2000 characters.

Keep the task narrow. This drives a real phone through its screens; it is not a general assistant.

If the agent is off or still starting up, the reply is 503 with The farm GUI agent is not enabled or not ready.

Phone runs

/app/phones/commands, /app/phones/macros and /app/phones/agent-runs each create a phone run and hand it back straight away. A run looks like this:

Code
{
  "id": "0d0f1a5c-9c3f-4c58-9a8a-1f3b2a6e77d1",
  "slot": "b3f1c7e2-4a90-4d6f-9d1b-2c8e5f7a0d31",
  "kind": "command",
  "status": "running",
  "label": "get_ip",
  "result": null,
  "error": null,
  "started_at": "2026-02-11T09:14:02+00:00",
  "finished_at": null,
  "created_at": "2026-02-11T09:14:02+00:00"
}

Field	Meaning
id	Run UUID. Pass it to /app/phones/runs/get.
slot	The phone the run was queued against.
kind	command, macro or agent.
status	queued, running, succeeded, failed or cancelled.
label	A short label: the command op, the workflow name or step count, or the first part of the agent task.
result	What the run produced. See the lifecycle below.
error	A readable reason when status is failed, otherwise null.
started_at, finished_at, created_at	ISO 8601 timestamps, or null.

The run lifecycle

A run is created queued and moves to running once the work has been handed to the phone. It stays running until the phone has actually done the work, not until the request was accepted. Only then does it settle on succeeded, failed or cancelled. Those three are terminal: nothing changes after them.

What result holds on success depends on the op:

• get_ip and clipboard_get finish with result set to {"value": "..."}, holding the IP address or the clipboard text.
• Every other op finishes with result: null. Success is the whole answer.

A failed run carries the reason in error. For work that failed on the handset that reason is The run failed on the phone. Try again, or contact support with this run id.

You can poll the run with /app/phones/runs/get, or let the run event tell you. See Events.

Worked example: read a phone's IP address

Run history

/app/phones/runs lists the runs on one phone, newest first.

Field	Type		Description
slot	string	required	Phone slot UUID.
page	integer	optional	Page number, 1 or more. Defaults to 1.

The reply is {"data": [...], "meta": {"current_page", "last_page", "per_page", "total"}}, 20 runs a page. With a control grant on the slot you see every run on that phone. With only a view grant you see your own.

/app/phones/runs/get fetches one run.

Field	Type		Description
slot	string	required	Phone slot UUID.
run	string	required	Run UUID.

The field is run, not run_id. A run on another slot, or one you cannot see, answers 404 with Run not found.