Skip to content

Object Detection

examples/object-detection.ipynb keeps the Line Tracing skeleton and swaps the color filter for a neural network: a pretrained YOLOv8 model finds objects in the camera image, and the car chases one — steering toward the biggest detection of a target class (default: sports ball), stopping when close, spinning to search when it's lost. Same perception → control loop; only the seeing changed.

The model ships with the example (examples/assets/object-detection/models/): YOLOv8n trained on COCO's 80 classes, exported to NCNN — the fastest backend on the robot's CPU. No download, no GPU.

Inside the notebook

Cell What happens
1. Setup the camera() and drive() helpers, plus one snapshot
2. Detect load the model, print all 80 class names, run one frame and draw the boxes
3. Chase the live web view with detection overlay, then the chase loop at ~10 Hz

Run it

  1. Open examples/object-detection.ipynb and pick the Python 3 (PhysiCar AI) kernel.
  2. Run the cells top to bottom; the chase cell loops until ⏹ (interrupt), then the car is stopped for you.
  3. Open app.physicar → MYAPP tab to watch the camera with every detection boxed and the target highlighted. Make sure an object of the target class is actually in the world — a soccer ball for the default sports ball.

How it chases

  • Only the target class is requested from the model (classes=[...]) — so near another object, the target can't be crowded out of the results.
  • Biggest box → its center's offset from image center → proportional steering (STEER_GAIN 30).
  • All-or-nothing speed: SPEED 0.8 m/s until the box fills NEAR_HEIGHT (45%) of the frame height — that counts as arrived, and the car stops.
  • Lost? One flickered frame doesn't count — only after LOST_TIMEOUT (1.0 s) does the car spin in place to search.
  • Confidence is deliberately low (conf=0.15) so small, distant objects still register, and imgsz=320 trades a little accuracy for ~4× less compute per frame.

Make it yours

  • TARGET — any of the 80 class names cell 2 prints. person turns it into a follow-me robot on a real kit.
  • NEAR_HEIGHT (0.45) — how close it gets before declaring arrival.
  • conf (0.15) — raise it if random background objects get detected.
  • SPEED / STEER_GAIN — the same P-control feel as Line Tracing.

Watch out

  • No detections at all? The target class must actually be in view — check cell 2's single-frame test before blaming the loop.
  • The first frames after loading the model are slow — the loop settles after a few seconds.
  • If port 5000 is busy, the web view is skipped (with a notice) — chasing still works.

🛠 Mission: in the simulator viewer, drag the ball to a new spot mid-chase and watch the search behavior re-acquire it. Then change TARGET and chase something else.

Learn more

Using pretrained models → AI & LLMs. Camera and sensors → Robot anatomy. Robot endpoints → Web API.

AI