feat: Raw reading on zone load and scene tools by aronwk-aaron · Pull Request #1910 · DarkflameUniverse/DarkflameServer

aronwk-aaron · 2025-10-15T03:54:40Z

Copilot

Pull Request Overview

This PR implements raw terrain file reading on zone load to enable scene identification from player positions. Key changes include parsing .raw terrain files, building a scene adjacency graph from scene transitions, and adding developer commands for scene debugging and visualization.

Adds comprehensive RAW terrain file parsing with scene map support
Implements scene ID lookup from world positions and scene adjacency graph construction
Adds developer commands for scene debugging and visualization with OBJ export capability

Reviewed Changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 17 comments.

Show a summary per file

File	Description
resources/worldconfig.ini	Adds config option to export terrain meshes to OBJ files for debugging
dZoneManager/dZoneManager.h	Declares scene position lookup, adjacency list access, and scene graph building methods
dZoneManager/dZoneManager.cpp	Implements scene position queries, scene graph construction from transitions, and global scene connectivity
dZoneManager/Zone.h	Adds Raw terrain data storage and scene lookup methods
dZoneManager/Zone.cpp	Implements zone file version checking, Raw file loading, and optional OBJ export
dZoneManager/Raw.h	Defines complete terrain chunk structures, scene vertex data, and mesh generation/export functions
dZoneManager/Raw.cpp	Implements RAW file parsing, terrain mesh generation with scene IDs, and OBJ export with scene colors
dZoneManager/CMakeLists.txt	Adds Raw.cpp to build and includes SceneColor.h path
dNavigation/dTerrain/*	Removes old unused terrain parsing code
dNavigation/dNavMesh.cpp	Removes unused RawFile.h include
dNavigation/CMakeLists.txt	Removes dTerrain subdirectory from build
dGame/dUtilities/SlashCommands/DEVGMCommands.h	Declares four new scene debugging commands
dGame/dUtilities/SlashCommands/DEVGMCommands.cpp	Implements GetScene, GetAdjacentScenes, SpawnScenePoints, and SpawnAllScenePoints commands
dGame/dUtilities/SlashCommandHandler.cpp	Registers the four new scene debugging commands
dCommon/dClient/SceneColor.h	Defines color palette for visualizing scene IDs in OBJ exports
dCommon/NiColor.h	Adds RGB color structure with grayscale conversion

Comments suppressed due to low confidence (1)

dZoneManager/Raw.cpp:1

Potential division by zero in GetSceneIDFromPosition if chunk dimensions are 1. Similar to the issue in GenerateTerrainMesh, if chunk.width or chunk.height equals 1, this will cause division by zero. Add validation before these calculations.

#include "Raw.h"

Copilot

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (1)

dZoneManager/dZoneManager.cpp:18

This file uses std::clamp, std::min, and std::find but doesn’t include <algorithm> directly. Relying on transitive includes can break builds when include order changes; add the proper standard header and drop unused <cmath> if it’s no longer needed.

#include "WorldConfig.h"
#include "CDZoneTableTable.h"
#include <chrono>
#include <cmath>
#include "eObjectBits.h"
#include "CDZoneTableTable.h"
#include "AssetManager.h"
#include <ranges>

Copilot

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 10 comments.

Comments suppressed due to low confidence (1)

dZoneManager/dZoneManager.cpp:18

This file now uses std::clamp/std::min/std::find and std::set, but the includes only add . Please add the required standard headers (at least for clamp/min/find and for std::set) to avoid relying on transitive includes and potential build breaks on other toolchains.

#include "WorldConfig.h"
#include "CDZoneTableTable.h"
#include <chrono>
#include <cmath>
#include "eObjectBits.h"
#include "CDZoneTableTable.h"
#include "AssetManager.h"
#include <ranges>

Replace old dNavigation/dTerrain raw parser with new Raw module in dZoneManager. Parse heightmaps, color maps, and scene maps from .raw files to determine which scene a position belongs to. Build scene adjacency graph from terrain data and scene transitions. Adds NiColor type, SceneColor lookup table, eSceneType enum, terrain mesh generation with OBJ export, and debug slash commands for scene visualization. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add IsValidForSceneLookup(), GetSceneIDAtGrid(), and GridToWorldPos() to Raw::Chunk. Use them in GenerateTerrainMesh, GetSceneIDFromPosition, SpawnScenePoints, and SpawnAllScenePoints instead of duplicated inline math. Remove unnecessary static_cast in LoadSceneTransitionInfo. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The parser was only correct for v32 files. For v<32: - colorMapResolution is width-1, not width - Color map reads width*width*4 bytes (BGRA per-pixel), not colorMapRes^2*4 - No DDS lightmap - No texture settings byte or blend DDS - Scene map: v31 reads (colorMapRes+1)^2 cells with border skip, v<31 skips 1 byte - No mesh data Verified against all 309 .raw files across multiple client versions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Log a clear message and throw if the zone uses a legacy raw format (LUZ version < 30) since no known maps use it. Remove ReadLegacyRaw and the mapVersion parameter from ReadRaw. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

These fields are skipped by the client in every known build (ptr += 12 and ptr += 4). No client ever reads them into a struct. Keep the original unknown1/unknown2 names from main. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ing or unreadable Log at debug when it parses; build the terrain mesh only for the OBJ export; read the zone name and description only for version 31 and later, as the client does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…time Chunk bounds span (width - 1) * scale; the nearest scene map cell is used and the far edge is kept one cell inside. Chunks are indexed by grid cell when they form a grid. Skipping reads instead of seeking, so a cut-off file can't seek past the end. Terrain mesh indices follow the vertex order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ickIDTable Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…dows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

aronwk-aaron · 2026-10-16T17:54:14Z

got opus 5.5 agent to look oer this again, it found several things that could cause crashes
I manually reviewed the code and it looks good to me, as well as tested it.

EmosewaMC · 2026-10-25T00:03:07Z

+	// A different LEGO brick per scene so the scenes can be told apart, from the BrickIDTable (every brick's LOT)
+	static LOT GetScenePointLot(uint8_t sceneID) {
+		static const std::vector<LOT> brickLots = [] {
+			std::vector<LOT> lots;
+			for (const auto& brick : CDClientManager::GetTable<CDBrickIDTableTable>()->Query([](const CDBrickIDTable&) { return true; })) {
+				lots.push_back(brick.NDObjectID);
+			}
+			std::ranges::sort(lots);
+			return lots;
+		}();
+		return brickLots.empty() ? LOT_NULL : brickLots[sceneID % brickLots.size()];
+	}


EmosewaMC

remove all the em dashes

EmosewaMC · 2026-10-25T00:05:55Z

+			ChatPackets::SendSystemMessage(sysAddr, GeneralUtils::ASCIIToUTF16(feedback.str()));
+			return;
+		}
+
+		// Build the feedback message with current scene
+		std::ostringstream feedback;
+		feedback << "Current Scene: " << sceneID.GetSceneID() << " (" << currentSceneName << ")";
+		ChatPackets::SendSystemMessage(sysAddr, GeneralUtils::ASCIIToUTF16(feedback.str()));
+
+		// List all adjacent scenes
+		feedback.str("");
+		feedback << "Adjacent Scenes (" << adjacentSceneIDs.size() << "):";
+		ChatPackets::SendSystemMessage(sysAddr, GeneralUtils::ASCIIToUTF16(feedback.str()));
+
+		for (const auto& adjSceneID : adjacentSceneIDs) {
+			feedback.str("");
+			feedback << "  - Scene ID: " << adjSceneID.GetSceneID();
+			feedback << " (Layer: " << adjSceneID.GetLayerID() << ")";
+
+			// Get the scene name if available
+			const auto* sceneRef = zone->GetScene(adjSceneID);
+			if (sceneRef && !sceneRef->name.empty()) {
+				feedback << " - " << sceneRef->name;
+			}
+
+			ChatPackets::SendSystemMessage(sysAddr, GeneralUtils::ASCIIToUTF16(feedback.str()));
+		}


why are we unable to send this as one packet?

EmosewaMC · 2026-10-25T00:26:57Z

+			BinaryIO::BinaryRead(stream, flair.colorR);
+			BinaryIO::BinaryRead(stream, flair.colorG);
+			BinaryIO::BinaryRead(stream, flair.colorB);
+			BinaryIO::BinaryRead(stream, flair.colorA);


https://docs.lu-dev.net/en/latest/file-structures/raw.html#raw-types-flair_attributes-seq
i dont see an alpha documented here, is this correct?

EmosewaMC · 2026-10-25T00:31:59Z

+			}
+
+			const size_t heightMapSize = width * height;
+			const size_t elementSize = sizeof(chunk.heightMap[0]);


uh, sizeof is a compile time operation, not a runtime one, but it seems to be used here as a runtime one? Replace this with a constant if so or use the sizeof the actual struct being queried rather than the result of a operator[].

EmosewaMC · 2026-10-25T01:19:50Z

+		}
+	}
+
+	bool WriteTerrainMeshToOBJ(const TerrainMesh& mesh, const std::string& path) {


was this tested? it wasnt called in the original code and i want to make sure this still works

EmosewaMC · 2026-10-25T01:23:45Z

 		}

-		//Read generic zone info:
-		BinaryIO::ReadString<uint8_t>(file, m_ZonePath, BinaryIO::ReadType::String);


Where did this go? why is this not necessary anymore?

aronwk-aaron changed the title ~~Feat: Raw reading on zone load and scene tools~~ feat: Raw reading on zone load and scene tools Oct 15, 2025

aronwk-aaron marked this pull request as ready for review October 16, 2025 01:12

aronwk-aaron requested review from EmosewaMC and Wincent01 October 16, 2025 01:12

aronwk-aaron requested a review from Copilot November 16, 2025 20:10

Copilot started reviewing on behalf of aronwk-aaron November 16, 2025 20:10

View session

Copilot finished reviewing on behalf of aronwk-aaron November 16, 2025 20:12

Copilot AI reviewed Nov 16, 2025

View reviewed changes

aronwk-aaron requested a review from Copilot March 20, 2026 18:09

Copilot started reviewing on behalf of aronwk-aaron March 20, 2026 18:10

View session

Copilot AI reviewed Mar 20, 2026

View reviewed changes

Copilot started work on behalf of aronwk-aaron March 31, 2026 08:35

View session

Copilot finished work on behalf of aronwk-aaron March 31, 2026 08:37

Copilot started work on behalf of aronwk-aaron March 31, 2026 08:42

View session

Copilot finished work on behalf of aronwk-aaron March 31, 2026 08:50

Copilot started work on behalf of aronwk-aaron March 31, 2026 09:19

View session

Copilot finished work on behalf of aronwk-aaron March 31, 2026 09:23

aronwk-aaron requested a review from Copilot March 31, 2026 09:28

Copilot started reviewing on behalf of aronwk-aaron March 31, 2026 09:28

View session

Copilot AI reviewed Mar 31, 2026

View reviewed changes

Copilot started work on behalf of aronwk-aaron March 31, 2026 09:45

View session

Copilot finished work on behalf of aronwk-aaron March 31, 2026 09:50

Copilot started work on behalf of aronwk-aaron March 31, 2026 16:11

View session

Copilot finished work on behalf of aronwk-aaron March 31, 2026 16:14

Copilot started work on behalf of aronwk-aaron March 31, 2026 16:16

View session

Copilot finished work on behalf of aronwk-aaron March 31, 2026 16:20

Copilot started work on behalf of aronwk-aaron March 31, 2026 23:37

View session

Copilot finished work on behalf of aronwk-aaron March 31, 2026 23:39

EmosewaMC reviewed Jun 24, 2026

View reviewed changes

Comment thread dZoneManager/Zone.cpp Outdated

aronwk-aaron force-pushed the raw-parsing-for-scene-data branch from e1b8924 to 7dd918d Compare June 24, 2026 23:22

aronwk-aaron force-pushed the raw-parsing-for-scene-data branch 3 times, most recently from 729b2ae to 0eeb5b0 Compare June 25, 2026 00:02

aronwk-aaron marked this pull request as draft June 25, 2026 00:02

aronwk-aaron force-pushed the raw-parsing-for-scene-data branch from e07cc1b to 9c20622 Compare June 25, 2026 02:26

aronwk-aaron commented Jun 25, 2026

View reviewed changes

Comment thread dZoneManager/Zone.cpp

aronwk-aaron and others added 10 commits October 16, 2026 11:13

feat: scene point commands take a step and use brick LOTs from the Br…

fc70efa

…ickIDTable Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

chore: rename NiColor::ToXYZ to Luminance

db0bae7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

test: raw terrain reading, scene lookup, scene graph and scene commands

36d6245

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

aronwk-aaron force-pushed the raw-parsing-for-scene-data branch from 6df22db to 36d6245 Compare October 16, 2026 16:17

test: close the OBJ file before removing it so the test passes on Win…

b98be36

…dows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

aronwk-aaron marked this pull request as ready for review October 16, 2026 17:53

aronwk-aaron requested a review from EmosewaMC October 16, 2026 17:54

EmosewaMC reviewed Oct 25, 2026

View reviewed changes

EmosewaMC requested changes Oct 25, 2026

View reviewed changes

Conversation

aronwk-aaron commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aronwk-aaron commented Oct 16, 2026

Uh oh!

EmosewaMC Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

EmosewaMC left a comment

Choose a reason for hiding this comment

Uh oh!

EmosewaMC Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

EmosewaMC Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

EmosewaMC Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

EmosewaMC Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

EmosewaMC Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aronwk-aaron commented Oct 15, 2025 •

edited

Loading