Skip to content

PhysiCar API

How to call PhysiCar AI's cloud AI services from your own code. There are two: chat (text, POST https://api.physicar.ai/chat) and realtime (voice, wss://api.physicar.ai/realtime). Both speak one common format and route to multiple providers behind the scenes, and both take the same prompt shape — instructions + tool definitions — so one "brain" can serve a text agent and a voice agent alike.

A complete working example is the workspace notebook examples/agent.ipynb (kernel Python 3 (PhysiCar AI)): a MyApp page that runs both agents in the browser and drives the robot through tool calls.

Authentication

  • Authenticate with your session token. HTTP requests send it as Authorization: Bearer <token>; WebSockets pass it as the subprotocol token.<token> (browsers cannot set WS headers).
  • On any /myapp/ page, physicarSession.token() is auto-injected by nginx and returns the signed-in user's token — no setup code. Read it per request: on a shared robot, caching one token globally would put every user on one account.
  • /chat also accepts guest requests (no token) up to a small daily limit per network; signed-in usage draws on your credits.

Chat

POST https://api.physicar.ai/chat — one request per turn. The prompt (model + instructions + tools) is sent inline on every request; the server keeps only the conversation, addressed by chat_id/turn and retained for 7 days.

GET /chat/models lists the available models with their price per 1M tokens (and, for classroom students, whether the teacher allows them).

Request body

{
  "user_message": { "contents": [ { "type": "text", "text": "Hello!" } ] },
  "prompt":  { "model": "...", "instructions": "...", "tools": [ ... ] },
  "stream":  true,
  "audio":   false,
  "chat_id": "…",
  "turn":    0
}
  • user_message.contents — text and/or image ({ "type": "image", "mime": "image/jpeg|image/png", "base64": "…" }) parts.
  • user_message.tool_call_outputs — results of the previous turn's tool calls (see the tool loop below).
  • prompt — model, instructions, tools, reasoning_effort, max_completion_tokens. The server caps max_completion_tokens to what your balance affords. Prompts can also be stored server-side (/chat/prompt CRUD) and referenced with prompt_id.
  • chat_id + turn — continue an existing conversation. Omit both on the first turn; the done event returns the issued chat_id. Send turn + 1 next time (or omit turn to continue at the end). Sending an earlier turn rewinds the conversation to that point.

Tool definitions

{
  "name": "drive",
  "description": "Drive the robot.",
  "properties": [
    { "name": "speed", "type": "number", "description": "m/s", "required": false }
  ]
}

Stream events

With "stream": true the response is Server-Sent Events; each data: line is a JSON object with a type:

type Carries
text a chunk of the reply (content)
tool_call call_id, name, arguments
audio data (base64 PCM) — only with "audio": true
done chat_id, turn, full_text, tool_calls, usage, finish_reason
error message, code

usage is the credits deducted for the turn. finish_reason is provider-neutral: stop (normal, including tool calls), length (cut off), refusal.

Without stream the response is a single JSON object with the same fields (chat_id, turn, text, tool_calls, usage, finish_reason, and audio with {data, duration_ms} when requested).

The tool loop

When done.tool_calls is non-empty, run each {call_id, name, arguments} yourself and send the results as the next request's user_message:

{
  "chat_id": "…", "turn": 1,
  "prompt": { "model": "...", "instructions": "...", "tools": [ ... ] },
  "user_message": {
    "contents": [],
    "tool_call_outputs": [
      { "call_id": "…", "name": "drive",
        "contents": [ { "type": "text", "text": "speed 0.5 m/s, drove 2 s" } ] }
    ]
  },
  "stream": true
}

A tool result's contents may include images — that is how a camera tool puts its photo into the conversation for the model to describe.

From the browser

const res = await fetch("https://api.physicar.ai/chat", {
  method: "POST",
  headers: { "Content-Type": "application/json",
             "Authorization": "Bearer " + physicarSession.token() },
  body: JSON.stringify({
    chat_id: chatId,            // undefined on the first turn
    turn,                       // 0 on the first turn
    prompt: { model, instructions, tools },
    user_message: { contents: [{ type: "text", text: "Hello!" }] },
    stream: true,
  }),
});

Realtime

wss://api.physicar.ai/realtime — speech-to-speech voice conversation:

mic ──▶ realtime cloud (speech-to-speech LLM) ──▶ speaker
              │            ▲
         tool_call    tool_result
              ▼            │
        your tools ──▶ robot web API (/speed, /camera, ...)

GET https://api.physicar.ai/realtime/models lists the available realtime models. Browsers authenticate with the token.<token> subprotocol; non-browser clients may use ?token= instead (the subprotocol form is recommended — query strings can end up in logs).

The protocol uses JSON text frames only — audio rides inside JSON as base64 pcm16. Sample rates differ per model and are advertised in session.ready's audio_config.

Client → server

Event Carries
session.start prompt (model, instructions, tools, voice), optional chat_id to continue a conversation
audio data — base64 pcm16 at audio_config.input_rate
text text — text input, triggers a turn
image data, mime, turn_complete — explicit image input (e.g. a camera shot)
tool_result call_id, name, output
session.end end the session

Server → client

Event Carries
session.ready model, audio_config (format, input_rate, output_rate), chat_id
audio.delta data — base64 pcm16 at audio_config.output_rate
transcript.delta role (user/assistant), text
tool_call call_id, name, arguments — same shape as chat
turn.complete the turn finished
interrupted barge-in — discard any audio you are still playing
error code, message
session.end code, reason, session_cost (credits)

Sessions

  • A session is capped at 30 minutes and 100 credits; it also ends after 90 seconds of inactivity. The session.end event carries the reason code (CLIENT_END, TIME_LIMIT, CREDIT_LIMIT, IDLE_TIMEOUT, …) and the credits the session consumed. Up to 10 concurrent sessions per user.
  • Transcripts share the conversation store with /chat: pass the session's chat_id to a later session.start — or to a /chat request — and the conversation continues.

From the browser

const ws = new WebSocket("wss://api.physicar.ai/realtime",
                         ["token." + physicarSession.token()]);
ws.onopen = () => ws.send(JSON.stringify({
  type: "session.start",
  prompt: { instructions, tools },
}));

Learn more

AI