Skip to content

Racing Deep Learning

examples/racing-deeplearning.ipynb is the deep end: one model, two teachers. The same small CNN (PhysicarNet) learns to race in two ways — supervised learning, where you teach by example (drive with buttons; every press stores a labeled photo), and reinforcement learning, where a reward teaches by trial and error in the simulator. Both write the same checkpoint (models/model.pt + models/model.onnx, plus a models/model.json meta with the action table and camera config — everything an external runner like the chat's racing tool needs to drive without the notebook), so you can alternate teachers freely — and the driver that runs at the end is simply whoever taught last.

Inside the notebook

Section What happens
0. Setup the shared ACTIONS table and PhysicarNet; robot helpers; the live dashboard pages; the model.json meta export
1.1 Label drive with on-screen buttons — every press drives and files that moment's photo under the pressed action
1.2 Train (SL) behavior cloning: 20 epochs with brightness/contrast augmentation; saves model.pt + model.onnx
1.3 Drive ONNX inference at ~15 Hz — argmax picks the action
2.1 Environment (RL) a Gymnasium env on the sim API — the reward is centerline proximity; episodes end off-track or on crash
2.2 Train (RL) PPO (Stable-Baselines3) around the same network, with live charts in MYAPP, progress on the /sim overlay, and a per-episode line in the cell output
2.3 Drive the same inference cell as 1.3 — whoever taught last drives

Run it

  1. Open examples/racing-deeplearning.ipynb and pick the Python 3 (PhysiCar AI) kernel. Run section 0 first; the two teachers are then independent.
  2. Supervised: run 1.1, open app.physicar → MYAPP tab, and drive with the three buttons — a few hundred photos per action is a good start. Then 1.2 to train and 1.3 to watch it drive.
  3. Reinforcement: run 2.1, then 2.2 — and let it work. Collecting 30,000 steps takes half an hour of sim time before training even finishes; watch the reward curve in MYAPP and the episode ticker on the /sim overlay.
  4. Driving cells loop until ⏹ (interrupt) — the car is stopped on exit. Interrupted training still saves the checkpoint.

The ideas that carry it

  • One ACTIONS table is the labeling buttons, the classifier's classes, and the RL action space — by default three: left / straight / right at 0.5 m/s, ±20°.
  • The camera pose is fixed (pan 0°, tilt −15°) for collecting, training, and driving alike — the model only works on the view it was taught on.
  • RESUME / WARM_START load the shared checkpoint, so SL can polish what RL learned and vice versa.
  • The reward is the assignment. reward() returns how close the car is to the track centerline — 1.0 at center, 0.0 at the boundary. Every behavior you want lives in that function.
  • Episodes start a little further around the track each time, end when the car leaves the track or crashes (a movable object shifting counts as a crash), and truncate after 150 steps.

Make it yours

  • ACTIONS — add speeds and angles; both teachers retrain onto the new action set.
  • reward() — add a speed bonus, punish zigzag, reward smoothness. Retrain, compare.
  • TOTAL (30,000 steps) — more steps, better driver, more sim time.

Watch out

  • SL runs on the simulator and a real kit; RL is simulator-only — it teleports the car constantly.
  • Collect, train, and drive with the same camera pose — the notebook sets it, so don't pan the camera mid-run.
  • RL training keeps the simulator busy for an hour or more — that time bills credits like any sim usage.
  • Train on too few labeled photos and the SL driver memorizes instead of generalizing — hundreds per action, in varied positions.

🛠 Mission: teach it with SL until it drives one clean lap, then hand the same checkpoint to RL (WARM_START) and see whether the reward improves your lines.

Learn more

How reward-driven learning works → Reinforcement learning. The sim API it drives → PhysiCar SIM.

AI