diff options
Diffstat (limited to 'research/tech-stack.org')
| -rw-r--r-- | research/tech-stack.org | 487 |
1 files changed, 487 insertions, 0 deletions
diff --git a/research/tech-stack.org b/research/tech-stack.org new file mode 100644 index 0000000..ce0a9c1 --- /dev/null +++ b/research/tech-stack.org @@ -0,0 +1,487 @@ +#+TITLE: Tech Stack — Locked Choices for Forts Clone +#+AUTHOR: Forts Clone Project +#+DATE: 2025-06-19 +#+OPTIONS: toc:3 num:t + +* Overview + +We are building a 2D physics-based RTS game (Forts-inspired). We explicitly +reject Godot, Unity, and Unreal. We are building a custom engine. The language +is C++20/23. This document locks the stack and explains each choice. + +* The Stack at a Glance + +| Layer | Choice | Link / Reference | +|------------------+---------------+--------------------------------------| +| Language | C++20/23 | ISO C++ | +| Build system | Meson | https://mesonbuild.com | +| Windowing/Input | SDL3 | https://libsdl.org | +| Rendering | bgfx | https://github.com/bkaradzic/bgfx | +| Physics | Box2D v3 | https://box2d.org | +| ECS | EnTT | https://github.com/skypjack/entt | +| Networking | ENet | http://enet.bespin.org | +| | + custom lockstep | (protocol built on top) | +| Audio | SoLoud | https://solhsa.com/soloud/ | +| Scripting/Modding| Lua + sol2 | https://github.com/ThePhD/sol2 | +| Debug UI | Dear ImGui | https://github.com/ocornut/imgui | +| Math | glm | https://github.com/g-truc/glm | +| Profiling | Tracy | https://github.com/wolfpld/tracy | + +* Why C++20/23 + +- Maximum library ecosystem — every library we need has first-class C or C++ + bindings +- Operator overloading keeps math readable (=pos += vel * dt=) +- Stable ISO standard — no breaking changes to chase every 6 months +- Mature debugging and profiling tooling (Visual Studio, GDB, Tracy, perf, + VTune, RenderDoc) +- sol2 (Lua binding) is C++ only — massive productivity win at the engine↔mod + boundary +- EnTT ECS is idiomatic C++ — ergonomic, header-only, battle-tested +- Zig was considered for its build system and cross-compilation, but the + C++ library advantage wins for a multi-year project. Zig's breaking changes + every 6 months are a real cost. If Zig reaches 1.0 and the project is mature, + migration is feasible since both languages consume the same C libraries. + +* Build System: Meson + +** Why Meson + +- Clean, readable syntax — not CMake's turing-tarpit DSL +- Fast — Ninja backend by default +- First-class dependency management via WrapDB and =meson wrap= +- Excellent subproject support for vendoring libraries +- Cross-compilation support via cross-files +- Built-in support for precompiled headers, unity builds, sanitizers +- Growing adoption in game industry (SDL, Mesa, systemd, GNOME) +- Meson wraps for our dependencies (simplified — some may need manual setup): + +| Dependency | Wrap / Integration | +|-------------+---------------------------------------------| +| SDL3 | =dependency('sdl3')= — pkg-config or wrap | +| bgfx | =subproject('bgfx')= — build from source | +| Box2D v3 | =subproject('box2d')= — build from source | +| EnTT | Header-only — =include_directories= | +| ENet | =subproject('enet')= or =dependency('libenet')=| +| SoLoud | =subproject('soloud')= — build from source | +| Lua | =subproject('lua')= or system package | +| sol2 | Header-only — =include_directories= | +| Dear ImGui | =subproject('imgui')= — build from source | +| glm | Header-only — =include_directories= | +| Tracy | =subproject('tracy')= if desired | + +** meson.build sketch +#+begin_src meson +project('forts-clone', 'cpp', + version: '0.1.0', + default_options: [ + 'cpp_std=c++20', + 'warning_level=3', + 'buildtype=debugoptimized', + ] +) + +# Core dependencies +sdl3_dep = dependency('sdl3') +bgfx_dep = dependency('bgfx') +box2d_dep = dependency('box2d') +enet_dep = dependency('libenet') +soloud_dep = dependency('soloud') +lua_dep = dependency('lua5.4') +entt_dep = declare_dependency(include_directories: 'subprojects/entt/include') +sol2_dep = declare_dependency(include_directories: 'subprojects/sol2/include') +imgui_dep = dependency('imgui') +glm_dep = declare_dependency(include_directories: 'subprojects/glm') +tracy_dep = dependency('tracy', required: get_option('tracy')) + +engine_lib = static_library('engine', ...) +executable('forts-clone', 'src/main.cpp', link_with: engine_lib, ...) +#+end_src + +* Rendering: bgfx + +** Why bgfx + +- Abstracts D3D11, D3D12, Vulkan, Metal, OpenGL, OpenGL ES, WebGPU behind a + single C++ API +- Used in production: AAA game tools, editor tooling, shipped games +- Shader compilation toolchain (=shaderc=) — write shaders once, compile to + all backends +- Multi-threaded rendering support (submit from any thread) +- Maintained by Branimir Karadžić (industry veteran, active on GitHub) +- We will build our own 2D rendering layer on top: sprite batcher, debug + drawing, particle renderer, UI rendering +- Debug rendering: integrate bgfx with Dear ImGui via bgfx's ImGui backend +- Higher initial cost than raylib/SDL, but maximum portability and control — + we can target Vulkan/Metal/D3D12 for performance, or OpenGL for simplicity, + without changing render code + +** 2D Renderer Design (built on bgfx) +#+begin_src text +bgfx layer (GPU API abstraction) + ├── SpriteBatcher — batching 2D quads with texture atlases + ├── DebugDraw — lines, circles, boxes for physics debug + ├── ParticleRenderer — GPU particle system + ├── TextRenderer — bitmap font or SDF text + └── ImGuiRenderer — Dear ImGui integration (bgfx backend exists) +#+end_src + +** bgfx Integration Notes +- bgfx does not handle window creation — we use SDL3 for that +- SDL3 creates the window, we pass the native window handle to bgfx +- bgfx + SDL3 is a well-trodden path (bgfx ships SDL examples) +- Shader workflow: write =.sc= shaders → =shaderc= compiles to all backends → + embed in binary or load from disk + +* Windowing & Input: SDL3 + +** Why SDL3 + +- SDL3 stable released January 2025 — modern API refresh +- Provides: window creation, input (keyboard, mouse, gamepad), OpenGL/Vulkan + context creation, audio (basic), threading, file I/O +- We use SDL3 for windowing and input; rendering goes through bgfx (SDL3 creates + the window, bgfx renders into it) +- =SDL_GameController= API for gamepad support +- Cross-platform: Windows, macOS, Linux, and beyond +- Mature, battle-tested in thousands of games (Valheim, Factorio, etc.) +- Can use SDL3's audio subsystem as fallback if SoLoud has issues on a + specific platform + +** Input Architecture +#+begin_src text +SDL3 event loop + ├── SDL_Event → InputSystem (buffered action map) + │ ├── Keyboard → action map (build, fire, select, etc.) + │ ├── Mouse → aim cursor, click-to-fire, drag-to-build + │ └── Gamepad → mapped to same actions + └── Raw SDL events also passed to ImGui for debug UI +#+end_src + +* Physics: Box2D v3 + +** Why Box2D v3 + +- Author: Erin Catto (Blizzard physics lead, industry standard) +- 2D rigid body physics — exactly our domain +- Joint types map directly to building mechanics: + - =b2DistanceJoint= → ropes/cables (maintain fixed distance between two points) + - =b2WeldJoint= → rigid connections (planks attached to beams) + - =b2RevoluteJoint= → hinges, pivoting structures + - =b2PrismaticJoint= → sliding connections +- Cross-platform deterministic (official, August 2024): + - Algorithmic determinism (no random numbers in library) + - Multi-threaded determinism (bit-array merging of worker results) + - Cross-platform determinism (no fast-math, no FMA, custom trig functions) +- =b2DestructionListener= callback when joints/bodies destroyed → cascading + collapse mechanics +- Bodies auto-destroy attached joints when destroyed +- C API with C++ headers — clean integration + +** Structural Physics Mapping (Box2D → Game Concepts) +| Box2D Feature | Game Mechanic | +|----------------------------+--------------------------------------------| +| =b2Body= (dynamic) | Individual building piece (beam, plank) | +| =b2Body= (static) | Terrain, anchored foundations | +| =b2DistanceJoint= | Rope/cable between two points | +| =b2WeldJoint= | Rigidly connected planks/beams | +| =b2RevoluteJoint= | Pivot/hinge for swinging structures | +| =b2DestructionListener= | Detect when a support breaks → cascade | +| =b2ContactListener= | Collision damage, projectile impacts | +| =b2Body::ApplyForce()= | Explosion knockback, wind, impacts | + +** Determinism & Lockstep +- Box2D v3's cross-platform determinism is the foundation for our lockstep + multiplayer +- Use default Box2D settings: no fast-math compiler flags, no FMA + (-ffp-contract=off on Clang/GCC), Box2D's custom =sinf=, =cosf=, =atan2f= +- NOTE: Box2D v3 does NOT yet support rollback determinism (for client-side + prediction). This is on Erin's roadmap. For v1, standard lockstep is sufficient. + +** Determinism Requirements (Box2D) +- Fixed timestep: always step with constant =deltaTime= (e.g., 1/60) +- No random forces in simulation +- Bodies and joints processed in deterministic order (by creation ID) +- All peers use identical Box2D compilation flags + +* ECS: EnTT + +** Why EnTT + +- Header-only, C++17/20 — drop it in, no build step +- Sparse-set based storage — efficient for our entity counts (~1K-10K) +- Excellent API: =registry.view<T>()=, =registry.group<T, U>()=, signals, + observers +- Used in production: Minecraft Bedrock Edition uses EnTT +- Good for dynamic entity creation/destruction (building pieces are created + and destroyed constantly) +- simdjson's author also maintains EnTT — quality codebase +- Type-safe, compile-time validated queries + +** Entity Plan for v0.1 +| Entity Type | Components | ~Count | +|-----------------+--------------------------------------------+-----------| +| BuildingPiece | Transform, BodyRef, Material, Health, Fire | ~500-2000 | +| Projectile | Transform, BodyRef, Damage, Owner | ~10-50 | +| Weapon | Transform, WeaponState, Owner, Cooldown | ~5-20 | +| Reactor | Transform, BodyRef, Health, Owner | ~2 | +| ResourceNode | Transform, ResourceType, Rate | ~5-10 | +| Effect | Transform, EffectType, Lifetime | ~varies | + +** EnTT Integration Notes +- Each entity in EnTT gets a =b2BodyId= component pointing to its Box2D body +- Physics system iterates Box2D bodies, reads/writes transform components +- EnTT is NOT the physics world — Box2D owns physics state, EnTT owns game + state. They reference each other by ID. +- =entt::registry= is the central game state container +- Line-of-sight queries, area queries, and other non-physics spatial lookups + can use a separate spatial index (simple grid or quadtree) keyed by entity ID + +* Networking: ENet + Custom Lockstep + +** Why ENet + +- Lightweight, battle-tested UDP library (Minecraft classic, many indie games) +- C library, trivially linked +- Provides reliable and unreliable channels over UDP +- Simple API: ~10 functions to learn +- Peer-to-peer and client-server modes +- We use it as the transport layer; lockstep protocol is built on top + +** Lockstep Protocol (Built on ENet) + +#+begin_src text +Tick N (at fixed rate, e.g., 60Hz): + + 1. Collect local player input for tick N+K (K = buffer depth) + → Serialize input into command packet + → Send via ENet unreliable channel to all peers + + 2. Input buffer: store incoming commands from peers + → Commands are tagged with the tick they apply to + → Commands for tick N were sent at tick N-K + + 3. At tick N, extract all peer commands for tick N from buffer + → If any peer's command is missing: stall (show "Waiting for player X") + → With K=2 at 60Hz, you have ~33ms before stalling + + 4. Execute all commands: apply to deterministic simulation + → Advance Box2D world by dt = 1/60 + → Run all game systems in deterministic order + + 5. (Every ~20 ticks) Compute state checksum, compare with host + → Mismatch → desync detected → resync or disconnect + + 6. Render the current state (interpolated for smooth display) +#+end_src + +** Input Buffer Design +- Buffer depth K = 2 ticks at 60Hz = 33ms input latency +- Acceptable for an RTS (not a fighting game) +- Can increase K for laggier connections (configurable) +- ENet unreliable channel for input (latest input supersedes lost packets, + no point retransmitting stale input) +- ENet reliable channel for: join/leave, chat, checksum verification, + game setup + +** Deterministic Lockstep Requirements +- Fixed tick rate (60Hz) +- Box2D v3 with deterministic settings (see Physics section) +- Deterministic entity spawn/despawn ordering (sorted by entity ID) +- PCG family RNG with shared seed (not =rand()=) +- No system calls in simulation path (no =malloc=, no =gettimeofday=, + no filesystem access during tick) +- All peers must have identical game data (same Lua files, same mods loaded + in same order) + +** GGPO-Style Rollback (Future Enhancement)** +- Requires: fast state serialization/deserialization + ability to simulate + many frames per tick +- Benefit: no freezing when packets delayed — game stays responsive +- Only pursue after v1.0 lockstep is stable + +* Audio: SoLoud + +** Why SoLoud + +- Designed specifically for games by a game developer (Jari Komppa) +- Extremely simple API — play a sound in ~3 lines +- Built-in DSP effects chain: filters, reverb, echo, chorus, distortion +- 3D spatial audio available (though we mostly use 2D panning) +- Bus system for sub-mixing (SFX bus, Music bus, Ambience bus) +- No memory allocation in the audio thread — predictable performance +- Multi-backend: WASAPI, XAudio2, ALSA, PulseAudio, Core Audio, and more +- Zlib/libPNG license — completely free, no revenue limits +- Format support: WAV, MP3, OGG (via bundled minimp3 and stb_vorbis) + +** Audio Architecture +#+begin_src text +SoLoud engine + ├── SFX bus + │ ├── Weapon fire sounds (cannon, laser, flak) + │ ├── Impact/hit sounds (projectile hits building) + │ ├── Building sounds (place, break, collapse) + │ └── UI sounds (click, hover, alert) + ├── Music bus + │ └── Background music (streamed OGG) + └── Ambience bus + └── Wind, fire crackle, environmental +#+end_src + +* Scripting & Modding: Lua + sol2 + +** Why Lua + sol2 + +- Forts uses Lua for modding — modders already know the language +- sol2: header-only C++ binding library — expose C++ classes/functions/tables + to Lua with minimal boilerplate +- Used in production modding frameworks (ModEngine2 for Souls games, Leadwerks + Engine, many custom engines) +- Standard Lua (PUC-Rio 5.4) is simple, well-documented, and fast enough +- Option to use LuaJIT later for 10-50x performance if needed +- Layered loading with priority system (Forts model) — documented below + +** Mod File Structure (Forts-Inspired) +#+begin_src text +my_mod/ +├── mod.lua # name, priority, category, selectable flag +├── db/ +│ └── constants.lua # physics overrides (gravity, drag, etc.) +├── devices/ +│ ├── device_list.lua # device definitions (costs, prereqs) +│ └── my_device.lua # individual device config +├── weapons/ +│ ├── weapons_list.lua # weapon definitions (fire rate, projectile) +│ └── projectile_list.lua # projectile data (damage, splash, speed) +├── materials/ +│ └── building_materials.lua # material properties (HP, cost, flammability) +└── assets/ + ├── textures/ # custom sprites, HUD icons + └── sounds/ # custom sound effects +#+end_src + +** Layered Loading Protocol +1. Base game Lua files load first +2. Mods sorted by priority (1-10, default 5) then alphabetically +3. Each mod's files load on top — can override values or =table.insert()= to + extend tables +4. Higher priority loads later, has "final say" +5. =RegisterApplyMod(callback)= — schedule a function to run after ALL mods + have loaded, letting mods modify content added by later-loading mods +6. Virtual filesystem: check highest-priority mod first for assets, fall back + to base game + +** Engine ↔ Lua Boundary +- Engine exposes C++ functions to Lua via sol2 (spawn entity, get resource, + fire weapon, etc.) +- Lua defines DATA tables (weapon stats, material properties, device configs) + → engine reads these at startup +- Lua can register game event callbacks (on_build, on_damage, on_projectile_hit) +- Hot-reload: watch mod directories, reload Lua when files change during + development + +* Debug UI: Dear ImGui + +** Why Dear ImGui + +- Industry standard for game debug/editor UI +- Immediate-mode: simple to add debug controls, inspectors, and data views +- bgfx integration exists (=bgfx/examples/common/imgui=) +- Used for: physics debug overlay (draw Box2D bodies/joints), ECS inspector, + network stats overlay, resource graph, weapon/material property editor, + console/command input, mod reload button + +* Math: glm + +** Why glm + +- Header-only C++ math library, OpenGL convention +- =glm::vec2=, =glm::mat4=, =glm::rotate()=, etc. +- Operator overloads: =pos += vel * dt= reads naturally +- Box2D v3 uses its own =b2Vec2= — we bridge: =glm::vec2 ↔ b2Vec2= at the + boundary layer (ECS↔physics) +- glm for rendering transforms, Box2D types for simulation + +* Profiling: Tracy + +** Why Tracy + +- Real-time C++ profiler with GUI client +- Instrumentation macros: =ZoneScoped=, =FrameMark=, =TracyPlot= +- View per-frame CPU usage, find bottlenecks in physics/render/simulation ticks +- Works with bgfx GPU profiling +- Lightweight when not connected (zones become no-ops) +- Essential for maintaining 60fps with growing entity counts + +* Asset Pipeline + +** Source Assets +- Sprite sheets / individual PNGs for building pieces, weapons, projectiles +- Lua files for game data (weapons, materials, devices) +- GLSL shaders → compiled via =shaderc= to bgfx format +- Sound effects (WAV/OGG) and music (OGG) +- Map data (custom binary or text format — TBD) + +** Build-Time Processing +- Texture atlasing: pack individual sprites into atlas textures (tool TBD — + possibly =texturepacker= or custom) +- Shader compilation: =shaderc= compiles =.sc= shaders to all GPU backends +- Asset cooking: convert source assets to optimized runtime formats +- Assets embedded or packed alongside binary + +** Runtime: Virtual Filesystem +- Mount points: base game assets + active mods (priority-ordered) +- File resolution: highest-priority mod first → fall back to base game +- Mods override individual assets without copying entire base game + +* Deterministic Simulation Design + +For lockstep multiplayer, the simulation must be exactly identical on all +machines. + +** Required Properties +1. Same initial state + same inputs → same state after N ticks +2. Box2D v3 cross-platform determinism (no fast-math, no FMA, custom trig) +3. Entity update order must be deterministic (sort by ID) +4. RNG must be synchronized (shared PCG seed sent at game start) +5. No system calls or external state in simulation path (no =malloc=, no + =gettimeofday=, no filesystem access during tick) + +** Box2D v3 Determinism Settings +- Compiler flags: = -ffp-contract=off= (disable fused multiply-add) +- No fast-math optimizations (= -ffast-math= breaks determinism) +- Use Box2D's built-in =sinf=, =cosf=, =atan2f= (cross-platform approximations) +- Fixed timestep: always step with =dt = 1/60.0f= +- Bodies and joints processed in deterministic order + +** State Checksums +- Every ~20 ticks, hash the full game state (entity positions, velocities, + health, fire state, resource counts) +- Host broadcasts its checksum → all peers compare +- Mismatch → desync detected → log state for debugging, resync or disconnect + +* Reference Projects + +Projects to study during development: + +| Project | Why Relevant | +|----------------+--------------------------------------------------------| +| Spring RTS | C++ physics RTS, Lua scripting, lockstep netcode | +| Factorio | Deterministic MP at scale, Lua modding, 2D | +| Cortex Command | 2D physics combat, destructible structures | +| Starbound | 2D, Lua modding ecosystem | +| bgfx examples | Reference for bgfx + SDL3 + ImGui integration | + +* Open Questions + +- [ ] Fixed-point math fallback vs trusting Box2D v3 determinism +- [ ] Exact rope simulation approach within Box2D (distance joints? soft + constraints? custom spring?) +- [ ] Fire propagation model (graph traversal along connected flammable + bodies) +- [ ] Map format design (heightmap? polygon soup? grid-based or freeform?) +- [ ] Exact tick rate: 60Hz simulation? 30Hz with interpolation? +- [ ] Replay system (trivial with lockstep — just record inputs) +- [ ] Observer/spectator mode (passive peer, receives inputs only) +- [ ] Package structure: =libengine= (static lib) + =forts-clone= (executable) + or header-only engine + thin =main.cpp=? |