mirror of
https://github.com/MaSzyna-EU07/maszyna.git
synced 2026-10-18 10:19:20 +02:00
Today's work on the binary scene format plan (binary-scene-format-plan.md), from the first measurement to a terrain that bakes itself on first run and streams from a thread. The step-by-step history is kept on backup/scene-streaming-format-presquash-0916. Measuring first. -loadprofile times a scenery load by stage and by node kind and reports node counts and resident memory. On l204 terrain is 95% of the bytes and 99.8% of the nodes; on quarkmce2007 vehicles, not models, dominate what is left. The terrain turned out to be a heightfield in all but name, a ribbon covering 12% of its bounding box. The heightfield. scene/heightfieldformat.h is the .ehf layout: Morton-ordered tiles of 128 samples with one shared edge sample, a point-sampled mip chain, zstd per tile, a nodata sentinel, a material palette with a measured tiling per material, and a stamp of the rules that decided what counted as terrain. scene/heightfieldreader.h reads it for the tool, the renderer and the loader alike, and scene/terraincooker.h writes it, shared by the terraincook tool and the engine so a first-run bake is the file the tool would have made. The cooker rasterises every triangle where it is drawn (through scene::place_in_world, the placement the region applies too), keeps an upper surface laid within half a metre over another and the lower of two further apart, and then asks each triangle whether the heightfield shows it: laid, reaching a sample, keeping nearly all it claimed, and with the ground around it drawn at every level out to 2 km. Baking on first run. simulation/terrainbake bakes <scenario>.ehf the first time a scenery is opened, choosing the grid step from the first 200,000 vertices, and on later loads leaves out the triangles its node table says the heightfield shows. Everything else - trees, grass tufts, gantries, bridge decks - is drawn as written, so a bad bake can at worst draw something twice, and the heightfield sits 2 cm under the triangles so the original wins there. Nodes are keyed by file, line, include parameters and placement; an edited file changes its keys and those nodes are drawn again. A bake under older rules, or without its table, is thrown away and redone. A launcher copy named $scenario shares the scenario's files. -nobaketerrain shows the triangles as written. A scenery can still name hand-cooked fields, several at different steps. Drawing and streaming. rendering/terrainclipmap draws resident tiles at the mip level their distance earns, deriving the grid from gl_VertexID and texturing in world space with the scenery's own materials. rendering/terraintileloader reads and decodes tiles on a thread of its own; the renderer rescans what is in range only after moving a quarter of a tile, asks for what is missing nearest first, uploads at most 32 tiles a frame, drops arrivals the camera has moved past, and keeps drawing a tile at the level it has until the right one arrives. The terrain has its own range, terrainrange in the ini and a slider in the debug panel, 20 km by default and independent of basedrawrange; the far plane of an ordinary view is pushed out to it. The debug panel shows the range, tiles in view, on the gpu, wanted and queued, and how many sit at each level. Events. The event queue moved out of the events into a binary heap with a deterministic tie-break, and an event is handed its activator instead of storing it, so basic_event is a definition and nothing else. The logic container. scene/logicformat.h and scene/logiccontainer write a scenery's events, their targets and sibling chains as flat, deterministic arrays with a schema hash that refuses a mismatched build. simulation/logiccook cooks it from the loaded state whenever it is missing, stale or unreadable, reads it straight back to check it, and resolves each target to a position in the registry its event kind searches. -cooklogic forces a cook, -verifylogic only checks. Nothing reads it at load yet. Prefabs. scene/prefab describes a signal by the aspects it can show - lights, speeds, animation channels, a sound - instead of pasting an .inc per variant, and expands an instance to ordinary scenery text. Tried on a light signal and a shape signal. Screenshots without a person present: -shot x,y,z[,yaw[,pitch]], -shotsettle, -shotstay.
168 lines
7.1 KiB
C++
168 lines
7.1 KiB
C++
/*
|
|
This Source Code Form is subject to the
|
|
terms of the Mozilla Public License, v.
|
|
2.0. If a copy of the MPL was not
|
|
distributed with this file, You can
|
|
obtain one at
|
|
http://mozilla.org/MPL/2.0/.
|
|
*/
|
|
|
|
#pragma once
|
|
|
|
// On-disk layout of the cooked terrain heightfield. Single source of truth: the cooker
|
|
// (tools/terraincook) writes it, the engine reads it, and neither owns a private copy.
|
|
// Plain structs and free functions only - no engine types, no allocation, no i/o - so
|
|
// that the tool keeps building without the engine and the format stays readable on its own.
|
|
//
|
|
// File layout:
|
|
// header
|
|
// tile directory, one entry per tile, ordered by morton code
|
|
// material table, each entry a uint16 length, that many name bytes, and a float
|
|
// giving how many metres of ground one repeat of the texture covers
|
|
// tile payloads, at the offsets the directory gives
|
|
//
|
|
// A tile payload holds a mip chain, finest level first. Level l has
|
|
// samples_at_level( tilesamples, l ) values along a side, laid out as that many rows of
|
|
// uint16 heights, followed by the same grid of uint8 material indices.
|
|
|
|
#include <cstdint>
|
|
|
|
namespace heightfield {
|
|
|
|
// "EU07HFLD", spelled out so the header needs no string functions
|
|
inline constexpr char magic[ 8 ] { 'E', 'U', '0', '7', 'H', 'F', 'L', 'D' };
|
|
inline constexpr std::uint32_t version { 5 };
|
|
// terrain selection rules, stamped into every cooked file.
|
|
// 0 - every triangle node the scenery holds
|
|
// 1 - world space triangle nodes only: anything under origin, rotate or scale is an object
|
|
// 2 - every triangle as drawn; a triangle stays drawn unless the heightfield shows it, and a
|
|
// triangle bordering a hole always stays drawn
|
|
// 3 - as 2, with a hole checked for diagonally and at the coarser levels too
|
|
inline constexpr std::uint32_t selection_rules { 3 };
|
|
|
|
// height value reserved to mean "this sample carries no terrain"; a cooked map is a
|
|
// ribbon inside a much larger rectangle, so most edge tiles are partly empty
|
|
inline constexpr std::uint16_t nodata { 0xffff };
|
|
// the range the remaining values span, shared by every tile through the header's bias
|
|
// and scale. over l204's 107 m of relief this puts a sample within 2 mm of the source
|
|
inline constexpr std::uint32_t heightrange { 0xfffe };
|
|
|
|
enum class codec : std::uint16_t {
|
|
raw = 0,
|
|
zstd = 1
|
|
};
|
|
|
|
#pragma pack( push, 1 )
|
|
|
|
struct file_header {
|
|
char magic[ 8 ];
|
|
std::uint32_t version;
|
|
// samples along a tile side, not counting the shared edge. a tile therefore stores
|
|
// one more sample in each direction than it owns
|
|
std::uint32_t tilesamples;
|
|
float gridstep; // metres between samples at the finest level
|
|
float heightbias; // world height of stored value 0
|
|
float heightscale; // metres per stored unit
|
|
std::int32_t minx, minz, maxx, maxz; // tile coordinates present in the file, inclusive
|
|
std::uint32_t tilecount;
|
|
std::uint32_t materialcount;
|
|
std::uint32_t miplevels; // levels stored per tile, level 0 being the finest
|
|
// which rules decided what counted as terrain when this was cooked. the layout does
|
|
// not depend on it, so a file with an older value still reads; a heightfield the
|
|
// engine baked by itself is rebaked when it disagrees, one named by the scenery is not
|
|
std::uint32_t selection;
|
|
std::uint32_t reserved;
|
|
};
|
|
|
|
struct tile_entry {
|
|
std::int32_t x, z;
|
|
std::uint64_t offset; // payload position in the file
|
|
std::uint32_t storedsize; // bytes occupied in the file
|
|
std::uint32_t plainsize; // bytes once decoded; equals storedsize for codec::raw
|
|
float minheight, maxheight;
|
|
std::uint32_t coverage; // samples carrying terrain at the finest level
|
|
std::uint16_t tilecodec; // codec enumerator
|
|
std::uint16_t reserved;
|
|
};
|
|
|
|
#pragma pack( pop )
|
|
|
|
static_assert( sizeof( file_header ) == 64 );
|
|
static_assert( sizeof( tile_entry ) == 40 );
|
|
|
|
// identity of a tile in any map keyed by tile coordinates. the cooker, the reader and the
|
|
// renderer all key tiles this way, so it is spelled out once
|
|
constexpr std::int64_t
|
|
tile_key( std::int32_t const X, std::int32_t const Z ) {
|
|
return ( static_cast<std::int64_t>( X ) << 32 ) ^ static_cast<std::uint32_t>( Z );
|
|
}
|
|
|
|
// integer division rounding towards negative infinity. tile and sample coordinates go
|
|
// negative, and every place that maps one onto the other has to round the same way for
|
|
// neighbouring tiles to share an edge exactly
|
|
constexpr std::int64_t
|
|
floor_div( std::int64_t const Value, std::int64_t const Divisor ) {
|
|
auto const quotient { Value / Divisor };
|
|
return ( ( Value % Divisor != 0 ) && ( ( Value < 0 ) != ( Divisor < 0 ) ) ) ? quotient - 1 : quotient;
|
|
}
|
|
|
|
// samples along a tile side at the given mip level, shared edge included
|
|
constexpr std::uint32_t
|
|
samples_at_level( std::uint32_t const Tilesamples, std::uint32_t const Level ) {
|
|
return ( Tilesamples >> Level ) + 1;
|
|
}
|
|
|
|
// bytes one mip level occupies: a grid of heights followed by a grid of material indices
|
|
constexpr std::uint64_t
|
|
level_bytes( std::uint32_t const Tilesamples, std::uint32_t const Level ) {
|
|
std::uint64_t const side { samples_at_level( Tilesamples, Level ) };
|
|
return side * side * ( sizeof( std::uint16_t ) + sizeof( std::uint8_t ) );
|
|
}
|
|
|
|
// bytes a decoded tile payload occupies, mip chain included
|
|
constexpr std::uint64_t
|
|
tile_bytes( std::uint32_t const Tilesamples, std::uint32_t const Miplevels ) {
|
|
std::uint64_t total { 0 };
|
|
for( std::uint32_t level = 0; level < Miplevels; ++level ) {
|
|
total += level_bytes( Tilesamples, level );
|
|
}
|
|
return total;
|
|
}
|
|
|
|
// byte offset of a mip level inside a decoded tile payload
|
|
constexpr std::uint64_t
|
|
level_offset( std::uint32_t const Tilesamples, std::uint32_t const Level ) {
|
|
return tile_bytes( Tilesamples, Level );
|
|
}
|
|
|
|
// deepest level that still has at least Minimumside samples along a side
|
|
constexpr std::uint32_t
|
|
level_count( std::uint32_t const Tilesamples, std::uint32_t const Minimumside ) {
|
|
std::uint32_t levels { 1 };
|
|
while( ( ( Tilesamples >> levels ) >= 1 )
|
|
&& ( samples_at_level( Tilesamples, levels ) >= Minimumside ) ) {
|
|
++levels;
|
|
}
|
|
return levels;
|
|
}
|
|
|
|
// interleaves tile coordinates so that tiles sorted by this value are read in an order
|
|
// that keeps a moving camera reading the file forwards rather than seeking
|
|
constexpr std::uint64_t
|
|
morton( std::int32_t const X, std::int32_t const Z ) {
|
|
auto const spread = []( std::uint32_t Value ) constexpr -> std::uint64_t {
|
|
std::uint64_t result { Value };
|
|
result = ( result | ( result << 16 ) ) & 0x0000ffff0000ffffull;
|
|
result = ( result | ( result << 8 ) ) & 0x00ff00ff00ff00ffull;
|
|
result = ( result | ( result << 4 ) ) & 0x0f0f0f0f0f0f0f0full;
|
|
result = ( result | ( result << 2 ) ) & 0x3333333333333333ull;
|
|
result = ( result | ( result << 1 ) ) & 0x5555555555555555ull;
|
|
return result; };
|
|
// biased into unsigned so that negative tile coordinates keep their relative order
|
|
std::uint32_t const x { static_cast<std::uint32_t>( X + ( 1 << 20 ) ) };
|
|
std::uint32_t const z { static_cast<std::uint32_t>( Z + ( 1 << 20 ) ) };
|
|
return spread( x ) | ( spread( z ) << 1 );
|
|
}
|
|
|
|
} // namespace heightfield
|