Synthetic Driving Simulator

This repository contains a deterministic synthetic-driving pipeline built around a small C++ intersection simulator and a PyTorch perception stack. The project generates bird's-eye-view training pairs, trains a compact semantic reconstruction model, compares low-noise and high-noise conditions, and extends the predictions into simple forecasting, planning, and closed-loop control demos.

What Was Built

The implemented system covers the full path from simulation to ML evaluation:

• A deterministic 128x128 four-way intersection world with lane markings, sidewalks, and obstacles.
• Seeded traffic scene generation with cars and pedestrians moving through route-specific state machines.
• Vehicle behaviors for lane following, stop-line handling, yielding, crossing, and completion.
• Pedestrian behaviors for approach, signal waiting, crossing, and completion.
• Export tools for actor logs, clean semantic BEV labels, noisy BEV inputs, metadata snapshots, and HTML scene viewers.
• Python dataset generation with reproducible train/val/test splits and per-sample manifests.
• A semantic denoising/segmentation baseline trained on simulator output.
• Benchmarking for generation throughput, end-to-end determinism, and low-noise versus high-noise ML performance.
• A perception layer that extracts semantic state, forecasts actor motion, plans an ego route, and rolls out simple closed-loop control.

Tech Stack

• C++17 for the simulator, exporters, and CLI runner.
• CMake for builds and test wiring.
• GoogleTest for C++ unit and integration coverage.
• Python 3 for dataset generation, training, evaluation, benchmarking, and autonomy demos.
• PyTorch for the segmentation model, training loop, metrics, and checkpointing.
• JSON and YAML for manifests, metadata, experiment outputs, and noise configuration.
• Self-contained HTML/CSS/JavaScript viewers for visual inspection of simulator and model outputs.

Simulator Scope

The simulator models a single four-way intersection on a 128x128 world grid spanning 128 meters end to end. Cars spawn from all four arms, choose one of two lane offsets per direction, and advance with seeded preferred velocities. Pedestrians spawn along the sidewalks and cross the intersection using their own motion states. The main executable exports:

• CSV actor trajectories over time.
• Clean semantic BEV labels as ASCII PPM.
• Noisy single-channel BEV inputs as ASCII PGM.
• Metadata snapshots in JSON or CSV.
• Browser-based viewers for grid playback and queue behavior inspection.

Dataset Pipeline

The training data is generated by repeatedly calling sim_runner with fixed seeds and writing one sample directory per scene. Each sample contains:

• input.pgm: noisy BEV raster used as the model input.
• label.ppm: clean semantic label map used as supervision.
• metadata.json or metadata.csv: exported actor state for the logged frame.

Two datasets were generated from the same simulator configuration:

• low_noise: 64 scenes split into 45 train, 10 validation, and 9 test samples.
• high_noise: 64 scenes split into 45 train, 10 validation, and 9 test samples.

Both datasets use dt=0.05 seconds and ticks=60, which means each exported sample is a deterministic snapshot of the same simulation horizon under different sensor-noise settings.

Model

The model is a compact U-Net defined in python/models/unet.py.

• Input: 1-channel noisy BEV image.
• Output: 5 semantic classes.
• Classes: drivable, lane, vehicle, pedestrian, obstacle.
• Architecture: 4 downsampling stages, a bottleneck, bilinear upsampling with skip connections, GroupNorm, and ReLU activations.
• Default width: base_channels=32.

The model is used as a semantic reconstruction baseline: it takes a noisy occupancy-like raster and predicts the clean semantic BEV map.

How The Model Was Trained

Training is implemented in python/train_unet.py and the shared helpers under python/training/.

• Optimizer: Adam.
• Learning rate: 1e-3.
• Epochs: 5.
• Batch size: 8.
• Weight decay: 0.0 by default.
• Loss: cross-entropy, with balanced class weighting enabled by default.
• Checkpoint selection: the best checkpoint is chosen by validation mean IoU.
• Evaluation: the saved checkpoint is scored on the held-out test split with pixel accuracy and per-class IoU.

The training loop also saves JSON metrics histories so the benchmark scripts can compare low-noise and high-noise conditions directly.

How Noise Was Added

Noise injection is implemented in src/NoiseInjector.cpp and can be loaded from YAML presets through src/NoiseConfigLoader.cpp.

The noisy BEV image is produced in three stages:

1. Each semantic class is mapped to a base grayscale intensity.
2. Per-pixel jitter is applied with a bounded random offset.
3. Two corruption modes are sampled:
   • dropout, which forces a pixel toward obstacle intensity.
   • speckle, which replaces the pixel with a bright random intensity.

The two shipped presets differ mainly in jitter amplitude and corruption rates:

• Low noise: jitter_amplitude=8, dropout_probability=0.01, speckle_probability=0.005.
• High noise: jitter_amplitude=28, dropout_probability=0.06, speckle_probability=0.03.

That makes the high-noise condition visibly harsher while keeping the underlying scene and labels fixed.

Benchmark Snapshot

The benchmark suite already records the key outcomes of the implemented pipeline:

• Generation benchmark: 8 scenes rendered in about 0.206 seconds, or 38.82 scenes/sec.
• Determinism benchmark: repeated exports for the same seed were byte-identical across CSV, clean BEV, noisy BEV, and metadata artifacts.
• Low-noise training outperformed high-noise training on the held-out test set.

condition	best val mIoU	test mIoU	test pixel accuracy
low noise	0.6670	0.6627	0.9665
high noise	0.5666	0.5714	0.9377

The largest degradation under higher noise showed up in the dynamic classes, especially vehicles and pedestrians, which is consistent with the stronger dropout and speckle corruption.

Beyond Segmentation

The Python perception/ package extends the project past offline segmentation:

• semantic state extraction from predicted class maps
• short-horizon motion forecasting from consecutive semantic frames
• ego-route planning over free space and lane structure
• simple control rollout for a closed-loop driving demo
• HTML viewers for prediction inspection and closed-loop playback

This turns the repository into a compact simulator-plus-perception prototype rather than only a dataset generator.