Add Micropolis world generation and evaluation workflows - #9
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Organization main has offline Micropolis helpers but no runnable Micropolis world. This adds
worlds/micropoliswith engine orchestration, question/corpus generation, cached/model response gathering, binary/continuous dataset IO, continuation-stream handling, and raw cached scoring. It imports canonical core and existing benchmark parsers/scorers/converter; no existing world is modified.Ported selectively from Fabio Rocha's
micropolisbranch at8c70972b698f079d54b8a2255c81fbefc55e7d99, preserving GPL text, source hashes and attribution in the checkout and wheel. Paper reporting/CSV producers, fitted normalization constants, historical configs, administered prompt wrappers, knowledge statements, model panels and datasets are excluded. Included configs/prompts are new illustrative examples. There is no merge of Fabio's branch/history.Engine configuration is validated at invocation, so imports, CLI help and cached work do not require an engine. Provider credentials must be supplied explicitly; no cloud-secret fallback or implicit .env loading. The README names every script and flags that
--dry-run/--cache-onlycan still simulate during corpus construction. The synthetic defaults travel in the installed wheel. Binary evaluation and continuation extraction use an invented snapshot-48/horizon-48 example (97 logged turns, one continuation seed), preserving the yearly-checkpoint validation; continuous commands retain the short example. Guarded actual-main regressions exercise the binary dry-run with invented cached trajectories, stop continuation initialization at the engine boundary, and prove short horizons remain rejected.Validation of corrected head
0722b3bc838ef53908296f0f886917c6e922a3efcombined with current maind3d69213bf5711731b2ed3112f796d40bb81d425(macOS arm64, CPython 3.13.3, fresh offline non-editable wheel environment, existing caches only):uv pip checkpass.uv lock --offline --checkandgit diff --checkpass. All prior lock package records are unchanged; the lock adds Micropolis and json5.CI limitation: the new Micropolis-specific workflow is not included because the current GitHub OAuth authorization cannot publish workflow files. The prepared workflow remains on a separate local branch. Existing hosted core/benchmark checks can run on this PR; they do not execute the Micropolis suite. The 227 Micropolis results above are local only. Live runtime/toolchain validation remains outstanding: source-interface inspection found both engine runners on
fdrocha/MicropolisCoreexploration at4fe3a7238b735f9cdd3ec13c957ccc8a3a3b0481; the fbs branch lacks the continuation producer. README documents Node >=20, pnpm 10.28.0, Emscripten/make and exact path setup. No prepared local WASM engine was available, no engine was vendored/downloaded, and neither a tested Emscripten version nor a historical producer-engine pin is claimed. Synthetic/mock checks do not establish live-engine correctness or reproduce frozen paper results.