Our buildsystem is based on cmake.
We use a bunch of custom cmake modules and python scripts
to detect libraries and generate code.
The entry point of the build system is /CMakeLists.txt.
From there, all buildsystem code of buildsystem/ is then used.
There is a /configure script that wraps the cmake invocation for convenience.
The /justfile forwards recipes like just build and just tests to the build directory.
The buildsystem is pretty sophisticated because openage consists of C++
code, generated C++ code, generated Cython code and Python code. All C++
parts are packed in libopenage, the Python stuff is in the openage
python package. We generate code with the openage.codegen Python
package.
Steps in building openage:
- generate
openage/config.pyandlibopenage/config.{h, cpp}from their*.intemplate files, which contain install prefix and version info - run
openage.codegen(Python module) to generate C++ source files (recipe:codegen) - generate
.pxdCython extension declaration files from annotated.hfiles (recipe:pxdgen) - build and link
libopenage.so(recipe:openage) - build Cython extension modules (generate cpp files and compile them, via
buildsystem.cythonize) (recipe:cython); those link against libopenage.
Additional recipes:
- see
just --list installdoc(generate docs via Doxygen)test(runs the various tests)- various cleaning recipes:
cleanelf,cleancodegen,cleancython,cleaninsourcebuild,cleanpxdgen,cleanbuilddirs,mrproper,mrproperer - various compliance checkers:
checkfast,checkmerge,checkall, ...
cmake reads and interprets all cmake modules and CMakeLists.txt files, populating the build directory with build files and generating config.py, config.h and config.cpp. In addition, the codegen script is invoked to determine the list of files it will generate, and the list of (python) files it depends on.
The ./configure cmake-wrapper script may be used to achieve a pleasant build experience. ./configure automatically creates the build directory and symlinks it to bin, which makes the root justfile work.
For each compiler invocation (gcc -Og, clang -O3, etc...), ./configure creates its own build directory. This allows you to quickly switch compilers and flags (e.g. via ./configure -c clang -O2) without having to re-build all object files when switching back.
cmake supports many backends: Project files for various IDEs and more.
By default, ninja interprets the generated build.ninja at build time; C++ modules need its dependency scanning. It will detect changes in the relevant cmake source files, and re-trigger a cmake execution if necessary (just your standard CMake features; pretty cool stuff).
The recipes codegen, libopenage, pxdgen, cythonize, compilepy and inplacemodules are then run as needed.
The justfile in the project root directory may be used to invoke ninja in the build directory (ninja -C bin/ would be the manual way of doing this).
At install time, the openage library, python modules, Cython extension
modules and game assets are installed to the prefix that was set at cmake
time (default: /usr/local), or, if the DESTDIR variable is present,
to $DESTDIR/$INSTALLPREFIX. Note: The Python/Cython modules have their
separate installation prefix, dictated by the Python environment, and
manually settable via the --py-prefix option to ./configure.
Some more CMake magic: the -rpath is automagically removed from the
installed binaries (if you don't know what that is: Lucky you. CMake
makes sure you don't need to).
The cpp cmake module, apart from verifying the compiler version and setting flags, provides functions for assembling binaries step-by-step.
declare_binarybegins the assembly of a new binary.add_sourcesadds translation units (cpp files) to the binary). TheGENERATEDkeyword is required for addingcodegen-generated translation units, theMODULEkeyword for C++ module interface units.finalize_binaryprints information about the binary, and calls theadd_executableoradd_librarycmake function, depending on its second argument. No more sources may be added after this point. If any generated sources were added to the executable, thecodegenrecipe is added as amakedependency.
At build time, this will cause all relevant object files and binaries to be built/linked. At install time, it will install the libs and binaries to /bin and /lib, respectively.
Plain invocations of add_executable require all translation units to be specified at once; this system is designed to allow each subfolder to have its own CMakeLists.txt file, adding all sources specified in that folder to the main binary.
New C++ code is written as modules, see the code style example.
The module name follows the file path: libopenage/util/color.cpp is openage.util.color.
Usually a module is a single .cpp file that holds both interface and implementation:
module;
#include <string>
export module openage.dir.name;
import openage.util.color;
export namespace openage::dir {
std::string greet() {
return "tachauch";
}
} // namespace openage::dirRegister it in the local CMakeLists.txt after the MODULE keyword:
add_sources(libopenage MODULE
name.cpp
)Several modules in one directory are listed together, one file per module:
add_sources(libopenage MODULE
color.cpp
path.cpp
)Every change to an interface unit recompiles all of its importers. To avoid that for big implementations, or to keep heavy includes away from importers, split the module into units. Where a unit is registered depends on its module declaration:
| Unit | Declaration | Registered |
|---|---|---|
| primary interface | export module openage.dir.name; |
after MODULE |
| interface partition | export module openage.dir.name:part; |
after MODULE |
| implementation partition | module openage.dir.name:part; |
after MODULE |
| implementation unit | module openage.dir.name; |
before MODULE, CMake rejects it in the module file set |
A module has exactly one primary interface, its .cppm file.
The primary interface must re-export every interface partition with export import :part;.
Partitions are only visible inside their module, import :part; there; importers only see openage.dir.name.
Example of a module whose interface is spread over two files, with the implementation in a third:
add_sources(libopenage
# implementation unit: module openage.dir.name;
name.cpp
MODULE
# primary interface: export module openage.dir.name; export import :part;
name.cppm
# interface partition: export module openage.dir.name:part;
name-part.cppm
)Rules for module units:
#includes go only in the global module fragment betweenmodule;and the module declaration; an include after it would attach the header to the moduleimports follow the module declarationmain()can't be part of a named module
ninja scans the sources for module dependencies, so the build order between modules needs no manual setup.
Headers and .cpp files may import modules too, which lets legacy code use converted modules and nyan.
GCC, unlike clang, rejects a textual definition after an import that brought the same entity ("redefinition of ..."), hence:
imports go after the#includes of each file (thecppstylecheck enforces it)- never include headers of a library after importing a module that exposes them
Use <nyan/config.h> for nyan's type aliases, e.g. nyan::fqon_t, and forward declarations when complete classes aren't needed.
Headers that import nyan; also include <nyan/config.h> before the import so later includes of the config header cannot redefine its constants on GCC.
Use the module for the remaining nyan API, not its other headers.
Standard headers are exempt:
GCC builds compile <bits/stdc++.h> into a header unit (StdHeaderUnit.cmake), and GCC translates every standard #include into an import of it.
CMake can't build header units, so a custom command puts it where GCC looks for them, below the build directory: bin/usr/.../bits/stdc++.h.gcm.
With -flang-info-include-translate, GCC reports each translated include.
- importers get the header unit's macros: set macros that change the standard library (
NDEBUG,_GLIBCXX_ASSERTIONS, ...) in the global flags, not per target - C++ targets must depend on the header unit with
use_std_header_unit();declare_binaryandadd_cython_modulesdo that
The python module is similar to the cpp module. It checks the Python interpreter and provides functions to declare Cython and Python modules.
add_cython_modulesadds Cython modules (i.e..pyxor.pyfiles) that will be compiled. See the code for option docs.pxdgenadds C++ headers for.pxdfile generation (see pyinterface).add_pxdsadds any additional.pxdor.pxifiles.add_py_modulesadds pure-Python modules; note that you also must declare.pyfiles that were declared inadd_cython_modules.
Provides the function codegen_run, which
- at cmake-time, runs the openage.codegen python module to obtain a list of sources it will generate
- at build time, actually generates the sources and writes them to the source directory where necessary
It sets the variable CODEGEN_TARGET_TUS, containing a list of created cpp files. That variable is designed to be used in a call to add_sources(openage GENERATED ${CODEGEN_TARGET_TUS}).
The python script automatically determines the python dependencies of the codegen script (via sys.modules) and adds them as dependencies of the codegen recipe. When the list of python dependencies, or output TUS, have changed, it automatically triggers a new cmake run.
The generated .cpp files are placed in the libopenage folder, and all end in .gen.cpp (or .gen.h).
read the source, or nag somebody to write more docs on them! wheeee!