Repository navigation
Prepare v0.8.1 performance and internal architecture release - #29
Merged
Merged
Conversation
Borrow decoded COPY bytes while staging patch views, eliminating a result-sized allocation and copy without changing the owning output contract of libmpq__patch_apply(). Add a non-installed benchmark for COPY, BSD0, and mixed patch chains. Verify each generated transform before timing, use a monotonic POSIX clock, and document how to run the benchmark.
Reuse computed MD5 values for patch metadata and attributes, and skip BSD0 candidates that cannot beat the current selection while retaining full validation for eligible candidates. Add a dedicated writer-staging benchmark, keep the reader benchmark focused on materialization, register both as non-installed tools, and document their separate usage.
Decode MPQE input directly into the private working archive while checking source identity and retaining archive validation. Add a non-installed benchmark for ordinary, embedded, and MPQE updates; test the plaintext working copy and document benchmark usage.
- Align run-count validation and best-of-N output across all three tools. - Add aggregate build and run targets with Windows executable suffix support. - Consolidate usage guidance in the developer guide.
Add a non-installed benchmark for archive opening, lookup, and extraction. Strengthen offset-lifetime regression checks and poison benchmark buffers before timed reads.
Benchmark stream opening, incremental reads, and seek workloads across stored, compressed, and encrypted members. Add focused regression tests, wire the targets into Automake, and document benchmark usage.
Avoid rehashing every previously written filename during archive creation. Add an archive-writer benchmark and regression tests for duplicate identity, locale/platform variants, and displaced hash entries.
Add benchmarks for Adler-32, CRC32/MD5, and signature verification, with build targets and developer documentation.
Compare file-backed and memory-backed MPQ/MPQE reads, and cover callback failures, range checks, and context ownership.
Measure encode and decode paths with deterministic inputs, and add the non-installed benchmark to aggregate targets and developer guidance.
Make comments pass the source formatting checks and label the MPQE source workload as random read_at rather than seek/read.
Move private header definitions into mpq-header.[ch] and use shared encode/decode helpers in reader and rebuild paths. Add exact-wire tests and build integration.
Move hash-entry definitions into mpq-hash.[ch] and use shared encode/decode helpers in reader, writer, update, and patch paths. Add exact-wire and slot-marker tests.
Move private block definitions into mpq-block.[ch] and use shared base and extended table codecs across read and rebuild paths. Add exact-wire, bounds, and overflow tests.
Move mpq_file_s and mpq_map_s from mpq-internal.h to mpq-entry.h without changing their layouts. Include the private header in source distributions.
Move archive state and pseudo-member names to mpq-archive.h. Include it directly in archive consumers and add it to source distributions. Leave writer internals in mpq-internal.h.
Move the unchanged mpq_writer structure into mpq-writer.h and include its MD5 dependency there. Keep mpq-internal.h as a transitional consumer of the focused writer header.
Move block and compression flags to their owning headers, replace the private success macro with 0, and use direct includes across source, tests, benchmarks, and fuzz code.
Delete the empty transitional header and remove it from the build manifest.
Move the unchanged static table into mpq-crypto.c and remove the generated header and obsolete generator. Drop their build, CI, source-check, and documentation references.
Treat zero MD5 rows as unavailable so reads and explicit verification do not report false mismatches. Add regression coverage and document the per-file behavior.
Coverage reportBase:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
5:0:1libtool ABI.src/mpq-internal.hcatch-all header and assigns private structures and wire-format responsibilities to focused modules.mpq-header,mpq-hash, andmpq-blockmodules with dedicated regression tests.mpq-crypto.c.src/mpq-crypt-buf.h, the historicaltools/crypt_buf_gen.cgenerator, and the now-unusedtools/build integration while preserving the crypt table contents and behavior.(attributes)handling so all-zero MD5 rows are treated as unavailable placeholders rather than valid digests, preventing false verification failures.Behavioral Change: MD5 Attributes
There is one intentional verification behavior correction. Some archives contain an MD5 array in
(attributes)but use an all-zero 16-byte row as a placeholder for files without an MD5 value.v0.8.1now treats such rows as unavailable MD5 metadata rather than comparing the all-zero value as an actual digest.Consequently:
libmpq__file_attributes()no longer advertises MD5 availability for an all-zero row;libmpq__file_verify()likewise skips unavailable all-zero MD5 entries;Private Internal Interfaces
The internal source layout changes substantially, but these interfaces are private and are not part of the installed SDK.
src/mpq-internal.his removed and its responsibilities are distributed among focused internal modules:mpq-header.[ch]owns MPQ header wire-format structures and serialization.mpq-hash.[ch]owns hash-table representation and wire handling.mpq-block.[ch]owns block-table and extended block-table representation and wire handling.mpq-entry.howns logical file/map entry state.mpq-archive.howns private archive state.mpq-writer.howns private writer state.Because none of these private declarations are installed public interfaces, the refactoring does not constitute an API or ABI break.
Crypto Table Cleanup
The immutable MPQ crypt table is now defined directly in
mpq-crypto.c. The following implementation artifacts are removed:src/mpq-crypt-buf.htools/crypt_buf_gen.ctools/Makefile.amtools/build integrationThe crypt-table values and resulting MPQ hashing, encryption, and decryption behavior remain unchanged.
Performance and Benchmark Improvements
v0.8.1adds a significantly broader benchmark suite and uses it to address measurable internal overhead without changing public behavior.Patch Processing
Archive Updates
Archive Reads
Archive Writing
Integrity Verification
Benchmark Coverage
Dedicated benchmark programs now cover:
Developer benchmark tooling is standardized so these paths can be profiled consistently.
Internal Architecture Changes
Header Wire Format
MPQ header wire-format handling is centralized in
mpq-header.[ch]. Dedicated tests verify header parsing and serialization independently from the rest of the reader/writer implementation.Hash Tables
Hash-table wire-format logic moves into
mpq-hash.[ch]. This gives hash-table representation, serialization, and related validation a clear module owner and corresponding regression coverage.Block Tables
Classic and extended block-table wire-format handling moves into
mpq-block.[ch]. The change removes duplicated representation knowledge from unrelated reader and writer code.Private State Ownership
Logical entry, archive, and writer state are moved out of the global internal header into focused private headers. Remaining dependencies on
mpq-internal.hare removed before deleting the header entirely.Crypto Ownership
The static MPQ crypt table becomes an implementation detail of
mpq-crypto.c, eliminating a generated internal header and the build-time generator used to create it.Commit Overview