DeCoR: Design and Control Co-Optimization for Urban Streets Using Reinforcement Learning

📌 Overview

DeCoR is a two-stage reinforcement learning framework for co-optimizing mid-block crosswalk placement and adaptive traffic signal control. It uses pedestrian and vehicle flow observations to generate crosswalk layouts, evaluate them in closed-loop traffic simulation, and learn signal timings that reduce delay for both pedestrians and vehicles.

Overview of DeCoR. A design agent proposes mid-block crosswalk layouts, while a control agent learns adaptive signal timings for each layout through closed-loop traffic simulation with pedestrian and vehicle demand.

---

📊 Data

The default corridor and demand inputs are stored in simulation/ as SUMO-compatible files.

Input	Location	Description
Corridor network	simulation/Craver_traffic_lights_wide.net.xml	SUMO network for the Craver Road study corridor.
Pedestrian demand	simulation/original_pedtrips.xml	2,221 pedestrian <person> / <walk> records over roughly one hour, with 14 TAZ labels and 24 origin/destination edges.
Vehicle demand	simulation/original_vehtrips.xml	200 vehicle <trip> records over roughly one hour, balanced between TAZ 1 -> 7 and 7 -> 1 with one default vehicle type.

The demand XML files encode origin-destination demand, not fully routed paths; realized routes depend on the SUMO network and routing configuration.

Observed pedestrian and vehicle departures, train/evaluation split, and pedestrian OD flows across the 14-zone corridor; 69.6% of pedestrian trips require crossing the corridor.

---

📦 Checkpoints and Results

Artifact	Location	Notes
Pretrained policy	runs/readout_32/May09_11-34-05/saved_policies/policy_at_7603200.pth	Checkpoint used by the default eval_model_path.
Paper evaluation JSONs	runs/readout_32/May09_11-34-05/results/eval_May10_16-16-52/	Includes DeCoR control, fixed-time, unsignalized, and real-world unsignalized evaluation outputs.

---

⚙️ Setup

• Install Python 3.12 and uv. pyproject.toml and uv.lock pin Eclipse SUMO, sumolib, and traci to 1.27.1.
• Sync the project environment, including the SUMO binaries:

  uv sync --frozen

• Verify the project runtime and converter:

  uv run --frozen sumo --version
  uv run --frozen netconvert --version

• The environments use the binaries bundled with the installed eclipse-sumo package, not an unrelated system SUMO_HOME or PATH installation. gui=True uses its sumo-gui binary and requires a working graphical display.
• Historical runs retain their recorded source and simulator versions. The new builder preserves curved road/sidewalk geometry and source sidewalk widths, and the intersection uses conflict-compatible protected greens. Do not reinterpret earlier results as validation of this revised geometry and signal behavior.

---

🚀 Training

For training, set these values in config.py:

"evaluate": False,
"gui": False,

Then run:

uv run python main.py

Training writes a timestamped run folder under runs/<timestamp>/, including config.json, TensorBoard logs, generated SUMO networks, and policies under saved_policies/. If eval_freq > 0, training also writes intermediate evaluation JSONs under runs/<timestamp>/results/train_<timestamp>/.

The design policy proposes a complete layout from the original-crossing reset graph on every round and uses that same context for final layout extraction. New configurations use higher_gamma = 0 for immediate normalized design rewards; the controller retains its temporal PPO objective. Historical runs and their recorded configurations are unchanged.

Controller updates report sampled and exact KL divergence, clipping fraction, maximum absolute log ratio, and pre-update likelihood agreement over the complete collected rollout, respecting each transition's sparse action mask. lower_approx_kl no longer describes only the final minibatch. These diagnostics are recorded by both training entrypoints; the standard entrypoint also sends them to TensorBoard or Weights & Biases.

New controllers retain the hidden layers' random biases and initialize the actor output weights with gain 0.01. This avoids amplified cold-start responses to zero-normalized observations through stacked LayerNorm and starts action probabilities near uniform. Only fresh initialization changes: the architecture, observation version 3, and predictions from loaded saved weights remain unchanged. Record new initial policy hashes when restarting a study.

The controller defaults are LR 1e-4, two PPO epochs, and batch size 1080, matching three complete ten-worker rounds of 36 decisions. These settings reduce policy change in an identical-batch training diagnostic; they are not evidence of convergence or better traffic performance. Validate new studies with full-rollout controller and design gates, preserve failed attempts, and keep tuning separate from held-out evaluation.

Rendered run diagnostics go under runs/<timestamp>/generated/graph_iterations/ and generated/gmm_iterations/. Their serialized graph/GMM source data remain in the run's original graph_iterations/ and gmm_iterations/ directories.

To monitor TensorBoard:

uv run tensorboard --logdir runs

🧠 Trained Design Policy

The design agent learns a Gaussian mixture over crosswalk location and width, where density peaks correspond to selected mid-block crosswalk proposals.

---

📈 Evaluation

Set evaluate=True in config.py and point eval_model_path to a saved checkpoint:

"evaluate": True,
"eval_model_path": "runs/<run_name>/saved_policies/<checkpoint>.pth",

Then run:

uv run python main.py

The active evaluation path in main.py evaluates the trained DeCoR policy over the configured demand scales and writes:

runs/<run_name>/results/eval_<timestamp>/<checkpoint>_ppo.json

Paper comparison results are included under runs/readout_32/May09_11-34-05/results/eval_May10_16-16-52/ for real-world unsignalized, DeCoR unsignalized, DeCoR fixed-time, and DeCoR control settings.

Placement-matched baselines

Prepare from a completed, non-development training.json with the current design readout (version 2). Keep the parent study's study.json and source_snapshot/, plus the recorded checkpoint and final network, available. Replace the example paths below; the destination must be fresh.

uv run python review_validation.py prepare runs/matched_baselines/14100 \
    --training-artifact runs/completed_study/14100/joint/training.json
uv run python review_validation.py search runs/matched_baselines/14100
uv run python review_validation.py matrix runs/matched_baselines/14100

This path uses common actuated control for the original, recorded final, Uniform, and best-of-20 random layouts. Only placement varies: crossing count and west-to-east widths match the recorded final layout. search runs all 120 training-window selection trials (20 candidates, two scales, three seeds), retains failures, and freezes selection before matrix evaluates the study's declared held-out demand. Training provenance is preserved separately from the frozen evaluator sources and inputs. These rows are separate from learned-policy evaluation and random-layout controller training.

Bounded operational-feedback comparison

review_validation.py also compares an explicit placement grid under tuned fixed-time and common actuated control, without historical checkpoints or policy training. Candidate layouts must have the same crossing count and west-to-east normalized widths; normalized locations must be ordered, within [0.01, 0.99], and at least 0.08 apart. Physical proposals and generated networks are recorded in the manifest.

Declare a JSON protocol before running. This small example is an execution smoke, not a research-scale protocol or a scientifically justified one-second practical margin:

{
  "placements": [[[0.25, 0.5], [0.75, 0.5]], [[0.35, 0.5], [0.65, 0.5]]],
  "selection_scales": [0.25],
  "evaluation_scales": [0.25],
  "selection_seeds": [70101, 70102],
  "evaluation_seeds": [70111, 70112],
  "timings": [[15, 10], [30, 20]],
  "practical_margin_s": 1.0
}

Save it as protocol.json, then use a fresh output directory:

uv run --frozen python review_validation.py feedback-prepare runs/feedback_comparison --protocol protocol.json
uv run --frozen python review_validation.py feedback-select runs/feedback_comparison
uv run --frozen python review_validation.py feedback-evaluate runs/feedback_comparison

Each timing pair specifies intersection and mid-block vehicle-green durations in seconds. For each placement, fixed-time tuning minimizes mean pedestrian complete journey time plus mean vehicle time loss and insertion delay, equally weighting the two class means. All rules share those tuned parameters and candidate trials. The three layout-ranking scores use first-approach age, first-approach age plus stopped time on crossing approaches, and those two terms plus vehicle delay. These are score ablations, not procedures wholly isolated from operational information: tuning uses journey outcomes, simulated approach age can depend on control, and the approach cohort can differ by layout.

Selection uniformly averages per-trial scores over the declared scale-by-seed grid; it does not pool travelers across trials.

Trials use 100 s fixed-control warmup, 450 s measurement, then at most 1800 s continued service without later departures. The complete cohort includes warmup trips; journey time begins at scheduled departure. A candidate timing must complete every selection cohort without teleports or collisions. Missing approach observations are undefined, not zero; failures remain in feedback_selection.json, and no eligible choice blocks evaluation rather than manufacturing a winner.

Selection uses the original recording's [0, 2400) window; evaluation uses [2400, 3600) with distinct declared seeds. That evaluation window has already been inspected in earlier studies: this is within-recording exploratory/mechanistic evidence, not a fresh dataset. Source/input/network hashes and selection-result hashes guard reuse. feedback-select cannot overwrite frozen choices. feedback_results.json retains every declared evaluation, simulation accounting, and paired differences in the common journey criterion. Scales are averaged within each evaluation-seed block before computing a conditional Student-t interval across blocks; incomplete pairs are never dropped. Benefit, harm, and practical equivalence use the declared margin and the whole interval. These are per-comparison, not simultaneous, statements conditional on the recording and selected layouts.

Each comparison identifies both selected layouts and flags identical_selection. When both rules select the same layout/controller procedure and service is eligible, their contrast is exactly zero by construction. This is not evidence of selection stability or of feedback being unnecessary on other candidate sets.

Each trial retains raw demand, tripinfo.xml, simulator logs and mechanism.jsonl. Mechanism records cover every warmup, measurement and drain second: signal phases/states, lane membership arrivals, stopped-vehicle counts and distance from the stop line to the farthest stopped vehicle's rear. The latter is a queue-extent proxy, not a verified contiguous queue or a spillback diagnosis; lane membership arrivals include lane changes. Geometry and lane shapes support subsequent mechanism analysis without claiming causal explanations automatically.

📝 Code Structure

├── main.py                  # Training and evaluation entry point
├── config.py                # Runtime configuration and argument grouping
├── utils.py                 # Policy IO, demand scaling, result aggregation
├── pyproject.toml           # uv project metadata and direct dependencies
├── uv.lock                  # Locked dependency resolution
├── images/                  # Tracked README and corridor visual assets
├── plots/
│   ├── training_plots.py             # Training-era control/design result plots
│   ├── result_plots.py               # Graph, distribution, and flow visualizations
│   ├── design_distribution.py        # Design density and reward ablation
│   ├── corridor_demand.py            # Corridor image and observed demand
│   ├── layout_control_comparison.py  # Historical layout/control evaluation
│   ├── pedestrian_flow_allocation.py # Illustrative pedestrian flow allocation
│   ├── training_transfer_comparison.py # Historical rewards and layout transfer
│   └── generated/                   # Ignored standalone plot outputs
├── ppo/
│   ├── models.py            # Lower MLP policy and higher GAT/GMM policy
│   ├── ppo.py               # PPO update implementation
│   └── ppo_utils.py         # Memory, normalizers, graph batching helpers
├── simulation/
│   ├── Craver_traffic_lights_wide.net.xml
│   ├── original_vehtrips.xml
│   ├── original_pedtrips.xml
│   ├── design_env.py        # Higher-level crosswalk design environment
│   ├── control_env.py       # Lower-level TraCI/SUMO control environment
│   ├── worker.py            # Parallel training/evaluation workers
│   ├── sim_setup.py         # Phase definitions and lane/crosswalk metadata
│   └── env_utils.py         # SUMO config, graph, and geometry helpers
└── runs/
    └── readout_32/...       # Included checkpoint and paper result artifacts

Generating plots

Run a descriptive generator without an output argument, for example:

uv run python plots/design_distribution.py

The five descriptive generators write matching PNG filenames under the repository's plots/generated/ directory, independent of the working directory. An optional output path remains available for an explicit export. Legacy plot helpers also write to plots/generated/; animation frames default to plots/generated/gmm_animation_frames/. Animation generation refuses a nonempty directory unless --keep-frames is explicit, preserving unrelated files.

Generators consume retained experiment data, not new simulation results. Some inputs are local research assets rather than files in a clean checkout, including the annotated corridor image from the paper checkout, ablation records and the illustrative flow cache. Existing paper figure copies are not overwritten by default.

plots/generated/ and per-run generated/ directories are ignored by Git.

---

🔧 Important Configuration Values

Key	Default	Notes
evaluate	True	Set to False for training.
gui	True	Set to False for faster headless SUMO runs.
gpu	True	Uses CUDA when available; falls back to CPU otherwise.
total_timesteps	15000000	Total lower-level simulation timesteps for training.
lower_num_processes	10	Parallel lower-level training workers. Adjust to your CPU.
lower_max_timesteps	360	Episode horizon, excluding warmup.
lower_step_length	1.0	SUMO seconds per simulation step.
lower_action_duration	10	Simulation steps per control action.
lower_warmup_steps	[40, 140]	Randomized warmup before policy control.
demand_scale_min/max	1.0 / 2.25	Training demand scale range.
eval_lower_timesteps	450	Evaluation episode horizon, excluding warmup.
eval_lower_workers	10	Parallel evaluation workers.
eval_worker_device	"gpu"	Evaluation policy device preference.
max_proposals	10	Maximum crosswalk proposals from the design agent.
min_thickness/max_thickness	2.0 / 15.0	Crosswalk width bounds in meters.
num_mixtures	7	GMM components for the design policy.

---

⚠️ Debugging

• Running with gui=True is useful for visual checks but substantially slower than headless SUMO.
• On Linux or WSL, if multiprocessing fails because too many files are open, increase the file descriptor limit before training:

  ulimit -n 20000

• If a run fails, inspect components/iteration_*.netconvert.log and sumo_logfile*.txt / sumo_errorlog*.txt in its runs/ subfolder. Converter logs are retained separately for each layout.

---

📖 Citation

If you find this work useful in your own research:

@misc{poudel2026decor,
  title = {DeCoR: Design and Control Co-Optimization for Urban Streets Using Reinforcement Learning},
  author = {Poudel, Bibek and Zhu, Lei and Heaslip, Kevin and Swaminathan, Sai and Li, Weizi},
  year = {2026},
  eprint = {2605.21311},
  archivePrefix = {arXiv},
  note = {Preprint}
}

---

🙏 Acknowledgements

We thank Jakob Erdmann (@namdre) of SUMO for helping with technical issues in simulation.