diff --git a/README.md b/README.md index 9c85bcdee..00f878bff 100644 --- a/README.md +++ b/README.md @@ -31,8 +31,8 @@ aisimulate --help ### With Dynamo -Install AISimulate with Dynamo to enable the `dynamo` runner plus Dynamo-owned -Router and Planner configuration adapters: +Install compatible AISimulate and Dynamo releases to enable the `dynamo` +runner and Dynamo-owned configuration adapters: ```bash python3 -m pip install aisimulate ai-dynamo @@ -43,6 +43,28 @@ AISimulate remains the CLI owner in both profiles. Select the integration at runtime with `--stack dynamo`; installing Dynamo does not add another simulation command. +**Planner needs additional dependencies.** The two-package installation above +supports basic Dynamo prediction, but does not install the complete Planner +environment. Before using a top-level `planner` section, install Dynamo's +`container/deps/requirements.planner.txt` from the same release tag or commit +as your Dynamo wheels. For example, after installing the Dynamo 1.5.0 RC9 +artifacts and their compatible AISimulate wheel: + +```bash +# Example for Dynamo 1.5.0 RC9; change this to your installed build's revision. +DYNAMO_REF=ffd7c1a90eb403c0d43911690c5c9b8457acd826 +python3 -m pip install -r \ + "https://raw.githubusercontent.com/ai-dynamo/dynamo/${DYNAMO_REF}/container/deps/requirements.planner.txt" +python3 -m pip check +``` + +For release candidates, use the exact release artifacts; a package version +alone may not identify the RC build. Alternatively, use the matching +`dynamo-planner` image, which includes the Planner prerequisites. See the +[Planner installation example](docs/cli/examples/dynamo-planner/README.md) +for a complete CPU-only prediction that loads Planner. `predict --help` +does not verify that optional adapters can load. + ### Upgrade from standalone AIConfigurator Remove the former standalone distributions first so that only AISimulate owns diff --git a/docs/cli/examples/dynamo-planner/README.md b/docs/cli/examples/dynamo-planner/README.md new file mode 100644 index 000000000..d0ccaeeaa --- /dev/null +++ b/docs/cli/examples/dynamo-planner/README.md @@ -0,0 +1,80 @@ + + +# Verify the Dynamo Planner installation + +This example runs the public AISimulate CLI with Planner enabled. It uses +synthetic traffic, local model metadata, fixed timing, and fixed KV capacity; +no model weights, GPU, or Kubernetes cluster are needed. Its timings are +synthetic and do not measure model performance or qualify scaling accuracy. + +## Install the matching dependencies + +Use Linux with Python 3.12, matching the release validation environment. From +the AISimulate repository root, place the compatible AISimulate, `ai-dynamo`, +and `ai-dynamo-runtime` wheel artifacts in `./wheels/` (one of each): + +```bash +python3 -m venv .venv-planner +source .venv-planner/bin/activate +python3 -m pip install \ + ./wheels/aisimulate-*.whl \ + ./wheels/ai_dynamo-*.whl \ + ./wheels/ai_dynamo_runtime-*.whl +``` + +For an RC, select the exact release artifacts rather than relying on a +package version shared by multiple builds. RC wheels may not be available +from the public package index. + +The basic [With Dynamo installation](../../../../README.md#with-dynamo) +does not install all Planner dependencies. Install the complete Planner +requirements from the **same Dynamo tag or commit as those wheels**: + +```bash +# Dynamo 1.5.0 RC9 example; replace with the revision of your installed build. +DYNAMO_REF=ffd7c1a90eb403c0d43911690c5c9b8457acd826 +python3 -m pip install -r \ + "https://raw.githubusercontent.com/ai-dynamo/dynamo/${DYNAMO_REF}/container/deps/requirements.planner.txt" +python3 -m pip check +``` + +Use the full requirements file, which includes `scikit-learn` and other +Planner dependencies. The supported prebuilt alternative is the matching +`dynamo-planner` image, which already includes these prerequisites. Use the +image from the same Dynamo release you intend to validate. + +## Run a Planner-enabled prediction + +From the AISimulate repository root, using the environment above: + +```bash +cd docs/cli/examples/dynamo-planner +python3 -m aisimulate predict \ + --stack dynamo \ + --config prediction.yaml \ + --output-dir ./planner-output \ + --capture-per-request \ + --format json +``` + +Keep this working directory: `prediction.yaml` resolves its `./model` path +relative to it. The included `model/config.json` is synthetic metadata based +on this repository's unified CLI test fixture, with no model weights. + +A successful run exits zero, completes all 12 requests, and writes +`planner-output/prediction.json` plus `planner-output/requests.jsonl`. +The configuration explicitly enables load-based Planner scaling, with a +five-second adjustment interval and a two-GPU simulated budget. The +`--stack dynamo` option is required to resolve its top-level `planner` section. +For another run, choose a new output directory or pass `--overwrite` to +replace the known output files. + +If the command fails while loading `dynamo.planner` with +`ModuleNotFoundError: No module named 'sklearn'`, the active Python environment +is missing Planner prerequisites. Install the matching requirements there, +run `python3 -m pip check`, and repeat this prediction. A successful dependency +check, basic prediction without Planner, or `predict --help` alone does not +verify that Planner loads. diff --git a/docs/cli/examples/dynamo-planner/model/config.json b/docs/cli/examples/dynamo-planner/model/config.json new file mode 100644 index 000000000..4b87f2150 --- /dev/null +++ b/docs/cli/examples/dynamo-planner/model/config.json @@ -0,0 +1,20 @@ +{ + "architectures": ["LlamaForCausalLM"], + "bos_token_id": 1, + "eos_token_id": 2, + "hidden_act": "silu", + "hidden_size": 2048, + "intermediate_size": 5632, + "max_position_embeddings": 2048, + "model_type": "llama", + "num_attention_heads": 32, + "num_hidden_layers": 22, + "num_key_value_heads": 4, + "pretraining_tp": 1, + "rms_norm_eps": 1e-05, + "rope_scaling": null, + "tie_word_embeddings": false, + "torch_dtype": "float32", + "use_cache": true, + "vocab_size": 32000 +} diff --git a/docs/cli/examples/dynamo-planner/prediction.yaml b/docs/cli/examples/dynamo-planner/prediction.yaml new file mode 100644 index 000000000..e6f165f66 --- /dev/null +++ b/docs/cli/examples/dynamo-planner/prediction.yaml @@ -0,0 +1,45 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Run from this directory; timings and model metadata are synthetic. +traffic: + source: {type: synthetic, input_tokens: 32, output_tokens: 4} + load: {type: constant_rate, requests_per_second: 1.0} + stop: {requests: 12} + +engine: + mode: aggregated + model: ./model + hardware: h200_sxm + backend: vllm + context_length: 2048 + workers: + aggregated: + parallelism: + replicas: 2 + tensor: 1 + pipeline: 1 + attention_data: 1 + moe_tensor: 1 + moe_expert: 1 + scheduler: {max_batched_tokens: 2048, max_sequences: 32} + kv_cache: + block_size: 64 + prefix_caching: true + capacity: {type: fixed, blocks: 256} + timing: {type: fixed, prefill_ms: 1, decode_ms: 1} + +evaluation: + sla: {ttft_ms: 8000.0, itl_ms: 200.0} + +planner: + policy: enabled + target: load + enable_throughput_scaling: false + enable_load_scaling: true + throughput_adjustment_interval_seconds: 180 + load_adjustment_interval_seconds: 5 + load_scaling_down_sensitivity: 80 + load_min_observations: 5 + max_num_gpus: 2 + min_workers: 1