Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ aisimulate --help

### With Dynamo

Install AISimulate with Dynamo to enable the `dynamo` runner plus Dynamo-owned
Router and Planner configuration adapters:
Install compatible AISimulate and Dynamo releases to enable the `dynamo`
runner and Dynamo-owned configuration adapters:

```bash
python3 -m pip install aisimulate ai-dynamo
Expand All @@ -43,6 +43,28 @@ AISimulate remains the CLI owner in both profiles. Select the integration at
runtime with `--stack dynamo`; installing Dynamo does not add another
simulation command.

**Planner needs additional dependencies.** The two-package installation above
supports basic Dynamo prediction, but does not install the complete Planner
environment. Before using a top-level `planner` section, install Dynamo's
`container/deps/requirements.planner.txt` from the same release tag or commit
as your Dynamo wheels. For example, after installing the Dynamo 1.5.0 RC9
artifacts and their compatible AISimulate wheel:

```bash
# Example for Dynamo 1.5.0 RC9; change this to your installed build's revision.
DYNAMO_REF=ffd7c1a90eb403c0d43911690c5c9b8457acd826
python3 -m pip install -r \
"https://raw.githubusercontent.com/ai-dynamo/dynamo/${DYNAMO_REF}/container/deps/requirements.planner.txt"
python3 -m pip check
```

For release candidates, use the exact release artifacts; a package version
alone may not identify the RC build. Alternatively, use the matching
`dynamo-planner` image, which includes the Planner prerequisites. See the
[Planner installation example](docs/cli/examples/dynamo-planner/README.md)
for a complete CPU-only prediction that loads Planner. `predict --help`
does not verify that optional adapters can load.

### Upgrade from standalone AIConfigurator

Remove the former standalone distributions first so that only AISimulate owns
Expand Down
80 changes: 80 additions & 0 deletions docs/cli/examples/dynamo-planner/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

# Verify the Dynamo Planner installation

This example runs the public AISimulate CLI with Planner enabled. It uses
synthetic traffic, local model metadata, fixed timing, and fixed KV capacity;
no model weights, GPU, or Kubernetes cluster are needed. Its timings are
synthetic and do not measure model performance or qualify scaling accuracy.

## Install the matching dependencies

Use Linux with Python 3.12, matching the release validation environment. From
the AISimulate repository root, place the compatible AISimulate, `ai-dynamo`,
and `ai-dynamo-runtime` wheel artifacts in `./wheels/` (one of each):

```bash
python3 -m venv .venv-planner
source .venv-planner/bin/activate
python3 -m pip install \
./wheels/aisimulate-*.whl \
./wheels/ai_dynamo-*.whl \
./wheels/ai_dynamo_runtime-*.whl
```

For an RC, select the exact release artifacts rather than relying on a
package version shared by multiple builds. RC wheels may not be available
from the public package index.

The basic [With Dynamo installation](../../../../README.md#with-dynamo)
does not install all Planner dependencies. Install the complete Planner
requirements from the **same Dynamo tag or commit as those wheels**:

```bash
# Dynamo 1.5.0 RC9 example; replace with the revision of your installed build.
DYNAMO_REF=ffd7c1a90eb403c0d43911690c5c9b8457acd826
python3 -m pip install -r \
"https://raw.githubusercontent.com/ai-dynamo/dynamo/${DYNAMO_REF}/container/deps/requirements.planner.txt"
python3 -m pip check
```

Use the full requirements file, which includes `scikit-learn` and other
Planner dependencies. The supported prebuilt alternative is the matching
`dynamo-planner` image, which already includes these prerequisites. Use the
image from the same Dynamo release you intend to validate.

## Run a Planner-enabled prediction

From the AISimulate repository root, using the environment above:

```bash
cd docs/cli/examples/dynamo-planner
python3 -m aisimulate predict \
--stack dynamo \
--config prediction.yaml \
--output-dir ./planner-output \
--capture-per-request \
--format json
```

Keep this working directory: `prediction.yaml` resolves its `./model` path
relative to it. The included `model/config.json` is synthetic metadata based
on this repository's unified CLI test fixture, with no model weights.

A successful run exits zero, completes all 12 requests, and writes
`planner-output/prediction.json` plus `planner-output/requests.jsonl`.
The configuration explicitly enables load-based Planner scaling, with a
five-second adjustment interval and a two-GPU simulated budget. The
`--stack dynamo` option is required to resolve its top-level `planner` section.
For another run, choose a new output directory or pass `--overwrite` to
replace the known output files.

If the command fails while loading `dynamo.planner` with
`ModuleNotFoundError: No module named 'sklearn'`, the active Python environment
is missing Planner prerequisites. Install the matching requirements there,
run `python3 -m pip check`, and repeat this prediction. A successful dependency
check, basic prediction without Planner, or `predict --help` alone does not
verify that Planner loads.
20 changes: 20 additions & 0 deletions docs/cli/examples/dynamo-planner/model/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"architectures": ["LlamaForCausalLM"],
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 2048,
"intermediate_size": 5632,
"max_position_embeddings": 2048,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 22,
"num_key_value_heads": 4,
"pretraining_tp": 1,
"rms_norm_eps": 1e-05,
"rope_scaling": null,
"tie_word_embeddings": false,
"torch_dtype": "float32",
"use_cache": true,
"vocab_size": 32000
}
45 changes: 45 additions & 0 deletions docs/cli/examples/dynamo-planner/prediction.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Run from this directory; timings and model metadata are synthetic.
traffic:
source: {type: synthetic, input_tokens: 32, output_tokens: 4}
load: {type: constant_rate, requests_per_second: 1.0}
stop: {requests: 12}

engine:
mode: aggregated
model: ./model
hardware: h200_sxm
backend: vllm
context_length: 2048
workers:
aggregated:
parallelism:
replicas: 2
tensor: 1
pipeline: 1
attention_data: 1
moe_tensor: 1
moe_expert: 1
scheduler: {max_batched_tokens: 2048, max_sequences: 32}
kv_cache:
block_size: 64
prefix_caching: true
capacity: {type: fixed, blocks: 256}
timing: {type: fixed, prefill_ms: 1, decode_ms: 1}

evaluation:
sla: {ttft_ms: 8000.0, itl_ms: 200.0}

planner:
policy: enabled
target: load
enable_throughput_scaling: false
enable_load_scaling: true
throughput_adjustment_interval_seconds: 180
load_adjustment_interval_seconds: 5
load_scaling_down_sensitivity: 80
load_min_observations: 5
max_num_gpus: 2
min_workers: 1