The tiles moved out of the scenery's pak into an archive of their own: a
sorted index of fixed-size entries found by where a tile sits in the
quadtree rather than by a name, each entry compressed on its own so a
tile can be read without touching the rest. A name apiece would have been
seventy megabytes of index to read before the first tile could be drawn.
An entry carries what its tile's ground asks for - how far it departs
from the finest mesh, and how long its triangles are - because which
level to draw has to be settled for tiles that are not on the card yet,
and the index is the only thing about them that is read up front. Until
now the figure came from the tile's level, which is the worst tile in it,
so one steep corner spoke for every tile that shared its level.
The mesh carries a normal per vertex, octahedron-encoded into two bytes,
in place of the face normal the fragment stage worked out from how the
view position changed across a triangle. A face normal makes the ground
read as facets, and a coarser level's facets face differently from the
ones they stand for, so the light over a hillside changed as the level
did. A geometry stage hands the fragment stage the three materials of the
triangle it is in and the barycentric weights, so ground changes over a
triangle rather than along its edge; 3.3 has no gl_BaryCoordEXT.
And the level a tile is drawn at now counts the camera's height. The
distance was measured on the ground plane alone, which puts the tile
underfoot nought metres away from a camera a kilometre above it - and
nought metres asks for the finest level there is, so a view from the air
wanted the whole ground at full detail, more tiles than the buffers hold,
and what would not fit came out as holes.
Archives written under the old rules are refused at the door and baked
again.
The ground in a scenery is already an irregular mesh, so the cook keeps it: it cuts the
triangles along the tile grid, welds what coincides, and builds each coarser level by
collapsing edges. Nothing is resampled, nothing is interpolated, and a hole the scenery left
stays a hole. What was here before threw the triangles away, kept their vertices and built a
fresh Delaunay triangulation over them, which filled every hole, merged the two heights of
every retaining wall into one, and - because it never erased the super-triangle - emitted
indices three too large together with triangles reaching out to the artificial corners.
Tiles form a quadtree, as Cesium Quantized Mesh intends. A distant tile is a large one, so the
horizon costs tens of tiles rather than the thousands a fixed grid needed, and the level to
draw follows from the tile's own geometric error against what the screen can resolve - which is
what the format carries that field for. Every vertex on a tile's border survives into all of
its coarser levels, and so does every edge between two of them, so two neighbours drawn at
different levels share their border exactly: there is nothing to stitch and nothing to morph.
The file layout is the format's, with two deliberate departures, both written down in
quantizedmeshformat.h: the geodetic header fields carry scenery metres, there being no
geographic reference to convert from, and the material a vertex belongs to rides in an
extension of ours. Texture coordinates are not stored at all - ground tiles in world space, so
the coordinate follows from the position and from how many metres one repeat of the material
covers, measured off the scenery once and kept in the terrain's table.
Tiles live in one archive beside the scenery, an entry each, compressed on its own. Baltyk
cooks to 546 kB against 2.2 MB of heightfields, quark to 361 kB against 9.6 MB, tomaszewo to
375 kB against 9.9 MB.
Drawing is one call a frame: every resident tile lives in the same vertex and index buffer,
with its vertices measured from the middle of the terrain so that the whole map fits in floats,
and a frame is one glMultiDrawElementsBaseVertex over the tiles the walk chose and the frustum
kept. On baltyk that settles at 291 tiles resident, 16 MB of mesh, 85 tiles and 6k triangles
drawn.
What counts as ground is what the scenery says, not what its triangles look like. A node under
origin, rotate or scale is an object standing on the ground; a material with an alpha channel is
a tree, a bush, a fence or a catenary mast. Everything else is laid whichever way it faces, so
the side of an embankment or a cutting goes in with the rest instead of leaving a hole along
every bank. Whatever is left out is recorded per node, and the scenery goes on drawing it.
A cook is a cache: it is used again only when the archive, the note of which nodes went into it
and the manifest of what the scenery was made from all agree. A manifest listing a file the
scenery names but does not have used to fail that test for ever, since an absent file never
matches - quark has such an include, and nothing beside it could ever be reused.
tools/terraincook cooks the same way outside the engine, and meshtest beside it checks what the
cook promises: a tile reads back as written, neighbouring tiles agree on their border to the
millimetre, a hole is not filled in, a wall is laid exactly once, and a material's tiling comes
out of the source. seamcheck.py goes: it tested the morphing constants of a heightfield nothing
reads any more.
A baked heightfield was trusted for as long as its file was readable, and the logic
container while the .scn was older than it: an edited include went unnoticed by both,
and node keys leaned on file dates, so merely touching a file changed them. And cooking
only ever happened as a side effect of starting a scenery.
Every cooked artifact now has a manifest beside it - <scenario>.ehf.deps,
<scenario>.logic.deps - listing each file it was made from with its size, modification
time and a hash of its content (simulation/cookdeps). The parser records what it
includes as it reads, so the list is exactly what the load read. An artifact is current
when every listed file still has its content: size and time are the quick check, and
when either differs the content is hashed, so a touched or copied file does not force a
rebuild and an edited one always does. A file added to the scenery cannot be in a
manifest; the files read are compared with it after the load, and a mismatch sends the
terrain to be baked on the next start - the new file's nodes, whose keys are unknown,
are drawn as geometry meanwhile - and the logic, cooked from the loaded state, is cooked
at once.
The terrain depends on what the scenario includes and not on the scenario file itself,
which the launcher rewrites on every start; the logic depends on both, since events can
be written in either, and costs milliseconds to cook again. With the manifest vouching
for file content, a triangle node's key is its file, line, include parameters and
placement, without the date.
-cook loads a scenery, brings whatever is stale up to date, and quits: the step that
prepares a scenery ahead of play. It closes the window directly, as there is no
simulation to wind down; a quit command waited a minute in the queue for one. Baking on
a first start stays as the fallback, and -nobaketerrain still turns it off.
Tried on a copy of baltyk with a terrain include of its own: a fresh cook bakes and
cooks everything, a second finds nothing to do, a touched file changes nothing, an
edited include is caught by its hash and both artifacts are rebuilt, and a new include
has the logic recooked at once and the terrain on the next run. A cooking run of baltyk
takes 5.5 s with nothing stale and 10.7 s with everything to do.
Three changes to what goes into a baked heightfield and how it meets the geometry drawn
beside it.
The heightfield held every triangle laid in it, including those it does not replace and
that are therefore drawn as geometry too - so they were drawn twice, and a roof or a
bridge deck left in it bent the ground. Now the first pass lays everything and decides
what is replaced; the second lays the replaced triangles together with the neighbours
that support them - the owners of the corner samples of the cells around them, without
which those cells are not drawn - and only that is written. A first attempt that
repeated passes, dropping whatever bordered a hole, eroded the whole terrain away one
ring per pass: the edge of the ground is always a hole. The replaced triangles stay
replaced in the second pass, since their supports are all there and they have fewer
rivals. The triangles bake no longer keep in memory; they wait in a spill file beside
the heightfield while the scenery parses.
Triangles were refused for an edge longer than 200 m, meant for a water plane under the
whole map. On baltyk and tomaszewo that refused thousands of ordinary fields of a
hand-modelled terrain, and nothing there comes near a whole-map plane. Only triangles
standing on their edge - walls, trees, fences - are refused now. The bake log says why
the triangles not replaced are not: standing on edge, too small, under other surfaces,
next to a hole.
Hand-modelled terrain has triangles hundreds of metres long, elevation-model terrain
small ones along the line, and one grid step either loses the detail or samples a field
every metre. A bake now writes three heightfields a factor of four apart -
<scenario>.ehf, <scenario>.1.ehf, <scenario>.2.ehf - and each triangle goes to the
coarsest one that still puts sixteen samples along its longest edge, and to every
coarser one too, so the coarsest holds all the ground and a triangle at the edge of a
finer field can be replaced by a coarser field beneath it. All of them are picked up
together, and a missing or stale one sends the whole set to be baked again.
Where fields overlap each other or the geometry, the depth test decides: the terrain is
drawn with a polygon offset, negative for the reversed depth buffer and larger for each
coarser field, instead of being lowered by 2 cm - so geometry lying on the ground wins
at any distance and on any slope, and a finer field wins over a coarser one.
On tomaszewo the heightfields went from 176 MB to 10 MB, the load that bakes them from
5:16 to 3:11 and its peak memory from 15.7 GB to 8.5 GB - the scenery alone takes 8.0 -
while the triangles replaced went from 37,130 to 40,402. On baltyk from 19 MB to 2 MB,
with 6,670 triangles replaced against 4,040.
Today's work on the binary scene format plan (binary-scene-format-plan.md), from the
first measurement to a terrain that bakes itself on first run and streams from a
thread. The step-by-step history is kept on backup/scene-streaming-format-presquash-0916.
Measuring first. -loadprofile times a scenery load by stage and by node kind and reports
node counts and resident memory. On l204 terrain is 95% of the bytes and 99.8% of the
nodes; on quarkmce2007 vehicles, not models, dominate what is left. The terrain turned
out to be a heightfield in all but name, a ribbon covering 12% of its bounding box.
The heightfield. scene/heightfieldformat.h is the .ehf layout: Morton-ordered tiles
of 128 samples with one shared edge sample, a point-sampled mip chain, zstd per tile,
a nodata sentinel, a material palette with a measured tiling per material, and a stamp
of the rules that decided what counted as terrain. scene/heightfieldreader.h reads it
for the tool, the renderer and the loader alike, and scene/terraincooker.h writes it,
shared by the terraincook tool and the engine so a first-run bake is the file the tool
would have made. The cooker rasterises every triangle where it is drawn (through
scene::place_in_world, the placement the region applies too), keeps an upper surface
laid within half a metre over another and the lower of two further apart, and then
asks each triangle whether the heightfield shows it: laid, reaching a sample, keeping
nearly all it claimed, and with the ground around it drawn at every level out to 2 km.
Baking on first run. simulation/terrainbake bakes <scenario>.ehf the first time a
scenery is opened, choosing the grid step from the first 200,000 vertices, and on later
loads leaves out the triangles its node table says the heightfield shows. Everything
else - trees, grass tufts, gantries, bridge decks - is drawn as written, so a bad bake
can at worst draw something twice, and the heightfield sits 2 cm under the triangles so
the original wins there. Nodes are keyed by file, line, include parameters and
placement; an edited file changes its keys and those nodes are drawn again. A bake under
older rules, or without its table, is thrown away and redone. A launcher copy named
$scenario shares the scenario's files. -nobaketerrain shows the triangles as written.
A scenery can still name hand-cooked fields, several at different steps.
Drawing and streaming. rendering/terrainclipmap draws resident tiles at the mip level
their distance earns, deriving the grid from gl_VertexID and texturing in world space
with the scenery's own materials. rendering/terraintileloader reads and decodes tiles on
a thread of its own; the renderer rescans what is in range only after moving a quarter
of a tile, asks for what is missing nearest first, uploads at most 32 tiles a frame,
drops arrivals the camera has moved past, and keeps drawing a tile at the level it has
until the right one arrives. The terrain has its own range, terrainrange in the ini and
a slider in the debug panel, 20 km by default and independent of basedrawrange; the far
plane of an ordinary view is pushed out to it. The debug panel shows the range, tiles in
view, on the gpu, wanted and queued, and how many sit at each level.
Events. The event queue moved out of the events into a binary heap with a deterministic
tie-break, and an event is handed its activator instead of storing it, so basic_event
is a definition and nothing else.
The logic container. scene/logicformat.h and scene/logiccontainer write a scenery's
events, their targets and sibling chains as flat, deterministic arrays with a schema
hash that refuses a mismatched build. simulation/logiccook cooks it from the loaded
state whenever it is missing, stale or unreadable, reads it straight back to check it,
and resolves each target to a position in the registry its event kind searches.
-cooklogic forces a cook, -verifylogic only checks. Nothing reads it at load yet.
Prefabs. scene/prefab describes a signal by the aspects it can show - lights, speeds,
animation channels, a sound - instead of pasting an .inc per variant, and expands an
instance to ordinary scenery text. Tried on a light signal and a shape signal.
Screenshots without a person present: -shot x,y,z[,yaw[,pitch]], -shotsettle, -shotstay.
The editor branch was written against headers and the PCH, both of which this
branch has done away with, so its own files were ported rather than merged: the
five headers it added are now interface units and their .cpp files module
implementation units.
- editor/{puwg1992,polandmap,turnouts,orthophoto} become eu07.editor.* modules,
application/planpanel becomes eu07.application.planpanel. The plan tool keeps
to its own module rather than joining the eu07.simcore partitions: it stands on
the editor's libraries and eu07.application.uilayer, not on the simulation core,
so :editoruilayer can export import it without a cycle.
- glm arrives by import eu07.glm, never by include, as the rest of the tree does.
maj0sted is header-only and glm-free, so its headers stay includes in the global
module fragment - every unit that names those types includes them itself, since
a global fragment is not visible across units.
- the one textual conflict was :editoruilayer's include block against this
branch's imports; resolved this branch's way plus the planpanel import.
Builds with clang 22 (-DWITH_BETTER_RENDERER=OFF) and starts into the editor on
empty ground with -editor alone.
AnimModel.cpp reaches cParser through the simulation core, not a direct
import, so the previous pass missed it; the propagation now follows the
whole import closure. Train.cpp, drivermode.cpp and Globals.cpp call
Console:: inside _WIN32 guards without importing eu07.console - invisible
on Linux, where those blocks are skipped.
A template defined in a module interface is instantiated in the importing
unit, where its body's types must be complete. A declaration reaching the
importer only through the interface's global module fragment is not
enough for MSVC: cParser::operator>> uses std::stringstream, and every
unit that parsed a float got an incomplete type.
526 headers across 161 units, derived from what each interface's template
bodies actually name.
Every module unit now includes the standard headers it uses. Declarations
reaching a unit only through an imported module's global module fragment
are reachable but not visible, so ADL misses the free operators older
libstdc++ declares beside a type - reproduced on ubuntu 24.04 (libstdc++
13), where gl/vao.cpp could not compare vector iterators.
Also drops a stale editor/editorSettings.hpp from the MSVC source list,
which broke the Windows configure step, silences the deliberate
include-in-purview warning in the glm wrapper, and removes GCC support:
clang or MSVC only.
Checkpoint of the editor branch so work isn't carried loose across a branch
switch. Builds.
- maj0sted turnout geometry moved into the domain: domain::lay_turnout lays the
diverging path (pre-blade lead, blade at beta, arc of alfa-beta, frog rail) and
solve_junction is a thin adapter onto it.
- the maj0sted web/ tree renamed to editor/ (namespace and paths) - there is no
WASM here, it's plain native C++ the editor calls.
- click places the switch at its start (początek rozjazdu); the branch's leaving
straight is locked tangent to the frog; switches can be slid along the through
track and flipped L/P.
- Polish turnout templates carry catalogue length (Id-1-consistent lengths), plus
blade-angle and pre-blade fields (left 0 until the catalogue figures are in).
- TURNOUT_SPEC.md: the Bezier-based runtime/model contract.
- also folds in other pending branch changes (scenery, tests) to leave a clean tree.
