LLM-chosen module names collide in flat output directory, silently overwriting or losing documentation files

[StefanZoerner]

Summary

Module and sub-module names are chosen freely by the LLM during clustering and agent delegation, and all resulting .md files are written into a single flat output directory. Nothing enforces uniqueness across hierarchy levels or between sibling sub-modules. When names collide, files are silently overwritten or never written, parent documents end up linking to missing files, and the run still finishes with

✓ Documentation generated successfully!.

Environment

CodeWiki 1.0.1 · Python 3.12 · provider openai-compatible via OpenRouter, model deepseek/deepseek-v4-flash · repo: https://github.com/DokChess/dokchess-en (Java, 43 parsed files) · config: max-token-per-module 20000, max-token-per-leaf-module 16000, max-tokens 16384, max-depth 2

Observed behavior — two collision variants across two consecutive runs on the same repo/config

Variant 1: sub-module names collide with top-level module names.

The repository clustering produced top-level modules including engine, search, and evaluation. The engine module agent then delegated to sub-modules it chose to name search and evaluation — identical to the existing top-level names:

[13:51:03] INFO  LLM module clustering for repository produced 9 top-level modules.
...
[14:15:12] INFO  📄 Processing leaf module: engine
[14:19:28] INFO    └─ Generating documentation for sub-module: search
[14:21:10] INFO    └─ Generating documentation for sub-module: evaluation
[14:25:22] INFO  📄 Processing leaf module: search
[14:25:22] INFO  ✓ Module docs already exists at .../codewiki-docs/search.md
[14:25:22] INFO  📄 Processing leaf module: evaluation
[14:25:22] INFO  ✓ Module docs already exists at .../codewiki-docs/evaluation.md
...
[14:50:39] WARNING  Module docs not found at .../codewiki-docs/engine.md

The top-level search/evaluation modules were skipped because the engine agent's sub-module files already occupied their paths, and engine.md itself was never written. The final summary still reported success.

Variant 2: duplicate sub-module names within one module.

In the next run (same repo, same config, different random clustering), the textui agent generated the same sub-module name twice:

[15:44:48] INFO  📄 Processing leaf module: textui
[15:48:01] INFO    └─ Generating documentation for sub-module: textui_xboard
[15:51:13] INFO    └─ Generating documentation for sub-module: textui_xboard
...
[16:00:17] WARNING  Module docs not found at .../codewiki-docs-v2/main.md

One of the two generations is lost to the collision; in this run main.md additionally went missing. Again: ✓ Documentation generated successfully!.

Notably, in run 2 the engine agent prefixed its sub-modules (engine_search, engine_eval), showing the LLM sometimes avoids collisions on its own — but there is no mechanism guaranteeing it.

Why this matters

Documentation completeness becomes a dice roll: identical repo + config produced 9 modules with one missing file in run 1, 6 modules with a different missing file in run 2.
Parent documents contain links to files that don't exist.
The failure is silent: warnings appear only late in the log, and the process exits successfully, so CI pipelines would not catch it.

Suggested fixes (any of these would help)

Namespace output files by hierarchy, e.g. engine/search.md or engine__search.md, so LLM-chosen names cannot collide across levels.
Enforce uniqueness at write time: refuse/rename on existing path (search_2.md) instead of skipping or overwriting.
Validate at the end of generation that every module in module_tree.json has a corresponding file, and exit non-zero (or clearly report) otherwise.

Full logs of both runs available on request.