18
0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-11-21 17:54:58 +01:00

Stream the scenery: a baked terrain heightfield, a logic container, and prefabs

Today's work on the binary scene format plan (binary-scene-format-plan.md), from the
first measurement to a terrain that bakes itself on first run and streams from a
thread. The step-by-step history is kept on backup/scene-streaming-format-presquash-0916.

Measuring first. -loadprofile times a scenery load by stage and by node kind and reports
node counts and resident memory. On l204 terrain is 95% of the bytes and 99.8% of the
nodes; on quarkmce2007 vehicles, not models, dominate what is left. The terrain turned
out to be a heightfield in all but name, a ribbon covering 12% of its bounding box.

The heightfield. scene/heightfieldformat.h is the .ehf layout: Morton-ordered tiles
of 128 samples with one shared edge sample, a point-sampled mip chain, zstd per tile,
a nodata sentinel, a material palette with a measured tiling per material, and a stamp
of the rules that decided what counted as terrain. scene/heightfieldreader.h reads it
for the tool, the renderer and the loader alike, and scene/terraincooker.h writes it,
shared by the terraincook tool and the engine so a first-run bake is the file the tool
would have made. The cooker rasterises every triangle where it is drawn (through
scene::place_in_world, the placement the region applies too), keeps an upper surface
laid within half a metre over another and the lower of two further apart, and then
asks each triangle whether the heightfield shows it: laid, reaching a sample, keeping
nearly all it claimed, and with the ground around it drawn at every level out to 2 km.

Baking on first run. simulation/terrainbake bakes <scenario>.ehf the first time a
scenery is opened, choosing the grid step from the first 200,000 vertices, and on later
loads leaves out the triangles its node table says the heightfield shows. Everything
else - trees, grass tufts, gantries, bridge decks - is drawn as written, so a bad bake
can at worst draw something twice, and the heightfield sits 2 cm under the triangles so
the original wins there. Nodes are keyed by file, line, include parameters and
placement; an edited file changes its keys and those nodes are drawn again. A bake under
older rules, or without its table, is thrown away and redone. A launcher copy named
$scenario shares the scenario's files. -nobaketerrain shows the triangles as written.
A scenery can still name hand-cooked fields, several at different steps.

Drawing and streaming. rendering/terrainclipmap draws resident tiles at the mip level
their distance earns, deriving the grid from gl_VertexID and texturing in world space
with the scenery's own materials. rendering/terraintileloader reads and decodes tiles on
a thread of its own; the renderer rescans what is in range only after moving a quarter
of a tile, asks for what is missing nearest first, uploads at most 32 tiles a frame,
drops arrivals the camera has moved past, and keeps drawing a tile at the level it has
until the right one arrives. The terrain has its own range, terrainrange in the ini and
a slider in the debug panel, 20 km by default and independent of basedrawrange; the far
plane of an ordinary view is pushed out to it. The debug panel shows the range, tiles in
view, on the gpu, wanted and queued, and how many sit at each level.

Events. The event queue moved out of the events into a binary heap with a deterministic
tie-break, and an event is handed its activator instead of storing it, so basic_event
is a definition and nothing else.

The logic container. scene/logicformat.h and scene/logiccontainer write a scenery's
events, their targets and sibling chains as flat, deterministic arrays with a schema
hash that refuses a mismatched build. simulation/logiccook cooks it from the loaded
state whenever it is missing, stale or unreadable, reads it straight back to check it,
and resolves each target to a position in the registry its event kind searches.
-cooklogic forces a cook, -verifylogic only checks. Nothing reads it at load yet.

Prefabs. scene/prefab describes a signal by the aspects it can show - lights, speeds,
animation channels, a sound - instead of pasting an .inc per variant, and expands an
instance to ordinary scenery text. Tried on a light signal and a shape signal.

Screenshots without a person present: -shot x,y,z[,yaw[,pitch]], -shotsettle, -shotstay.
This commit is contained in:
maj00r
2026-09-16 23:18:09 +02:00
parent 366645ef59
commit 2660d14565
47 changed files with 7117 additions and 240 deletions

View File

@@ -176,6 +176,8 @@ set(SOURCES
"model/material.cpp" "model/material.cpp"
"input/messaging.cpp" "input/messaging.cpp"
"scene/scene.cpp" "scene/scene.cpp"
"scene/logiccontainer.cpp"
"scene/prefab.cpp"
"scene/scenenode.cpp" "scene/scenenode.cpp"
"simulation/simulation.cpp" "simulation/simulation.cpp"
"model/vertex.cpp" "model/vertex.cpp"
@@ -186,6 +188,12 @@ set(SOURCES
"application/uitranscripts.cpp" "application/uitranscripts.cpp"
"world/station.cpp" "world/station.cpp"
"application/application.cpp" "application/application.cpp"
"application/shotrunner.cpp"
"rendering/terrainclipmap.cpp"
"rendering/terraintileloader.cpp"
"simulation/loadprofile.cpp"
"simulation/logiccook.cpp"
"simulation/terrainbake.cpp"
"simulation/simulationtime.cpp" "simulation/simulationtime.cpp"
"scene/sceneeditor.cpp" "scene/sceneeditor.cpp"
"rendering/screenshot.cpp" "rendering/screenshot.cpp"
@@ -598,6 +606,8 @@ if (WITH_CRASHPAD)
endif() endif()
target_link_libraries(${PROJECT_NAME} glfw) target_link_libraries(${PROJECT_NAME} glfw)
# the cooked terrain heightfield stores its tiles compressed
target_link_libraries(${PROJECT_NAME} zstd::libzstd)
if(WITH_PYTHON) if(WITH_PYTHON)
find_package(Python3 3.14 REQUIRED COMPONENTS Development) find_package(Python3 3.14 REQUIRED COMPONENTS Development)
@@ -626,6 +636,13 @@ else()
target_link_libraries(${PROJECT_NAME} ${PNG_LIBRARIES}) target_link_libraries(${PROJECT_NAME} ${PNG_LIBRARIES})
endif() endif()
find_package(zstd CONFIG QUIET)
if(NOT zstd_FOUND)
find_package(PkgConfig REQUIRED)
pkg_check_modules(ZSTD REQUIRED IMPORTED_TARGET libzstd)
add_library(zstd::libzstd ALIAS PkgConfig::ZSTD)
endif()
find_package(Threads REQUIRED) find_package(Threads REQUIRED)
target_link_libraries(${PROJECT_NAME} Threads::Threads) target_link_libraries(${PROJECT_NAME} Threads::Threads)

View File

@@ -66,6 +66,8 @@ import eu07.simcore;
import eu07.utilities.globals; import eu07.utilities.globals;
import eu07.utilities.utilities; import eu07.utilities.utilities;
import eu07.simulation.simulation; import eu07.simulation.simulation;
import eu07.simulation.loadprofile;
import eu07.application.shotrunner;
import eu07.simulation.simulationsounds; import eu07.simulation.simulationsounds;
import eu07.vehicle.train; import eu07.vehicle.train;
import eu07.utilities.dictionary; import eu07.utilities.dictionary;
@@ -1171,6 +1173,9 @@ int eu07_application::init_settings(int Argc, char *Argv[])
// process command line arguments // process command line arguments
auto scenerywasasked = false; auto scenerywasasked = false;
std::vector<shot_runner::shot> shots;
auto shotsettle = 3.0;
auto shotquit = true;
for (int i = 1; i < Argc; ++i) for (int i = 1; i < Argc; ++i)
{ {
@@ -1202,16 +1207,91 @@ int eu07_application::init_settings(int Argc, char *Argv[])
{ {
Global.editor_startup = true; Global.editor_startup = true;
} }
else if (token == "-terrain")
{
if (i + 1 < Argc)
{
// cooked heightfield produced by tools/terraincook; may be repeated
Global.terrain_heightfields.push_back(Argv[++i]);
}
}
else if (token == "-shot")
{
// unattended screenshot: "x,y,z[,yaw[,pitch]]", repeatable for a sequence
if (i + 1 < Argc)
{
shot_runner::shot shot;
if (shot_runner::parse(Argv[++i], shot))
{
shots.push_back(shot);
}
else
{
std::cout << "bad -shot viewpoint: " << Argv[i] << std::endl;
return -1;
}
}
}
else if (token == "-shotsettle")
{
// seconds to wait at each viewpoint before saving, so terrain tiles and
// models have time to stream in
if (i + 1 < Argc)
{
shotsettle = std::atof(Argv[++i]);
}
}
else if (token == "-shotstay")
{
// leave the simulation running once the sequence is done
shotquit = false;
}
else if (token == "-loadprofile")
{
// scenery load profiling: where the load time goes, what the scenery holds,
// what it costs in memory. writes to the log and to loadprofile.txt
loadprofile::enable(true);
}
else if (token == "-cooklogic")
{
// cook the scenery's LOGIC container once it has loaded, then read it back and
// check it against the events in memory
Global.cook_logic = true;
}
else if (token == "-nobaketerrain")
{
// load the scenery's terrain triangles as written, without baking or using a
// heightfield baked beside it. the way to see the original for comparison
Global.bake_terrain = false;
}
else if (token == "-verifylogic")
{
// read the LOGIC container that is already on disk and check it against the
// scenery, writing nothing
Global.verify_logic = true;
}
else else
{ {
std::cout << "usage: " << std::string(Argv[0]) << " [-s sceneryfilepath]" std::cout << "usage: " << std::string(Argv[0]) << " [-s sceneryfilepath]"
<< " [-v vehiclename]" << " [-v vehiclename]"
<< " [-editor]" << " [-editor]"
<< " [-terrain terrain.ehf]"
<< " [-shot x,y,z[,yaw[,pitch]]]"
<< " [-shotsettle seconds] [-shotstay]"
<< " [-loadprofile]"
<< " [-cooklogic]"
<< " [-verifylogic]"
<< " [-nobaketerrain]"
<< " (every switch also takes a double dash: --editor)" << std::endl; << " (every switch also takes a double dash: --editor)" << std::endl;
return -1; return -1;
} }
} }
if (false == shots.empty())
{
Shots.configure(shots, shotsettle, shotquit);
}
if (Global.editor_startup && false == scenerywasasked) if (Global.editor_startup && false == scenerywasasked)
{ {
// the editor opens on empty ground unless a scenery was actually asked for. whatever the ini // the editor opens on empty ground unless a scenery was actually asked for. whatever the ini

View File

@@ -22,6 +22,7 @@ module;
#include "vehicle/DynObj_macros.h" #include "vehicle/DynObj_macros.h"
module eu07.application.drivermode; module eu07.application.drivermode;
import eu07.application.shotrunner;
import eu07.console; // Console:: is used under _WIN32 import eu07.console; // Console:: is used under _WIN32
import eu07.application.uitranscripts; import eu07.application.uitranscripts;
import eu07.application.uilayer; import eu07.application.uilayer;
@@ -194,6 +195,40 @@ bool driver_mode::update()
simulation::State.update_scripting_interface(); simulation::State.update_scripting_interface();
simulation::Environment.update(); simulation::Environment.update();
// unattended screenshot sequence, when one was asked for on the command line
if (true == Shots.active())
{
switch (Shots.update(Timer::GetDeltaTime()))
{
case shot_runner::action::place:
{
Camera.Init(Shots.current().position, Shots.current().angle, nullptr);
Global.pCamera = Camera;
FreeFlyModeFlag = true;
break;
}
case shot_runner::action::capture:
{
// where the lens actually is, which need not be where it was put: anything
// that moves the camera between placing and capturing would otherwise make
// the sequence quietly unrepeatable
WriteLog(
"Shot: camera at " + to_string(Camera.Pos)
+ " looking " + to_string(glm::dvec3{Camera.Angle} * 180.0 / M_PI)
+ (FreeFlyModeFlag ? " (free)" : " (attached)"));
Application.queue_screenshot();
break;
}
case shot_runner::action::finish:
{
Application.queue_quit(false);
break;
}
case shot_runner::action::none:
break;
}
}
if (deltatime != 0.0 || false == simulation::is_ready) if (deltatime != 0.0 || false == simulation::is_ready)
{ {
// jak pauza, to nie ma po co tego przeliczać // jak pauza, to nie ma po co tego przeliczać

View File

@@ -47,6 +47,7 @@ import eu07.utilities.logs;
import eu07.widgets.vehicleparams; import eu07.widgets.vehicleparams;
import eu07.utilities.u8; import eu07.utilities.u8;
import eu07.application.driverhints; import eu07.application.driverhints;
import eu07.rendering.terrainstatus;
#ifdef WITH_UART #ifdef WITH_UART
import eu07.utilities.uart; import eu07.utilities.uart;
using namespace Mtable; using namespace Mtable;
@@ -765,6 +766,14 @@ debug_panel::render_section_scenario() {
Global.BaseDrawRange = std::clamp(std::exp(drawrange), 100.0f, 50000.0f); Global.BaseDrawRange = std::clamp(std::exp(drawrange), 100.0f, 50000.0f);
} }
} }
// terrain streaming range slider, separate from the draw range
{
auto terrainrange = std::log(Global.TerrainRange);
if (ImGui::SliderFloat(
(to_string(std::exp(terrainrange), 0, 5) + " m###terrainrange").c_str(), &terrainrange, std::log(500.0f), std::log(100000.0f), "Terrain range")) {
Global.TerrainRange = std::clamp(std::exp(terrainrange), 500.0f, 100000.0f);
}
}
} }
return true; return true;
@@ -1368,28 +1377,33 @@ debug_panel::update_section_eventqueue( std::vector<text_line> &Output ) {
// current event queue // current event queue
auto const time { Timer::GetTime() }; auto const time { Timer::GetTime() };
auto const *event { simulation::Events.begin() };
auto const searchfilter { std::string( m_eventsearch.data() ) }; auto const searchfilter { std::string( m_eventsearch.data() ) };
// a copy in running order; when and by whom an event was queued belongs to the
// queue entry now, not to the event
auto const queue { simulation::Events.queue_snapshot() };
Output.emplace_back( "Delay: Event:", Global.UITextColor ); Output.emplace_back( "Delay: Event:", Global.UITextColor );
while( event != nullptr for( auto const &entry : queue ) {
&& Output.size() < 30 ) {
if( Output.size() >= 30 ) { break; }
auto const *event { simulation::Events.event_at( entry.index ) };
if( event == nullptr ) { continue; }
if( false == event->m_ignored if( false == event->m_ignored
&& false == event->m_passive && false == event->m_passive
&& ( false == m_eventqueueactivevehicleonly && ( false == m_eventqueueactivevehicleonly
|| event->m_activator == m_input.vehicle ) ) { || entry.activator == m_input.vehicle ) ) {
auto const label { event->m_name + ( event->m_activator ? " (by: " + event->m_activator->asName + ")" : "" ) }; auto const label { event->m_name + ( entry.activator ? " (by: " + entry.activator->asName + ")" : "" ) };
if( false == searchfilter.empty() if( false == searchfilter.empty()
&& false == contains(label, searchfilter) ) { && false == contains(label, searchfilter) ) {
event = event->m_next;
continue; continue;
} }
auto const delay { " " + to_string( std::max( 0.0, event->m_launchtime - time ), 1 ) }; auto const delay { " " + to_string( std::max( 0.0, entry.launchtime - time ), 1 ) };
textline = textline =
delay.substr( delay.length() - 6 ) delay.substr( delay.length() - 6 )
+ " " + " "
@@ -1397,11 +1411,10 @@ debug_panel::update_section_eventqueue( std::vector<text_line> &Output ) {
Output.emplace_back( textline, Global.UITextColor ); Output.emplace_back( textline, Global.UITextColor );
} }
event = event->m_next;
} }
if( Output.size() == 1 ) { if( Output.size() == 1 ) {
// event queue can be empty either because no event got through active filters, or because it is genuinely empty // event queue can be empty either because no event got through active filters, or because it is genuinely empty
Output.front().data = simulation::Events.begin() == nullptr ? "(no queued events)" : "(no matching events)"; Output.front().data = simulation::Events.queue_empty() ? "(no queued events)" : "(no matching events)";
} }
} }
@@ -1490,6 +1503,36 @@ debug_panel::update_section_renderer( std::vector<text_line> &Output ) {
Output.emplace_back( textline, Global.UITextColor ); Output.emplace_back( textline, Global.UITextColor );
// terrain streaming: how far tiles are loaded, how many are held, and how far
// behind the camera the loader is
for( auto const &field : TerrainStatus ) {
auto const &stats { field.stats };
auto const slash { field.path.find_last_of( "/\\" ) };
Output.emplace_back(
"Terrain: " + ( slash == std::string::npos ? field.path : field.path.substr( slash + 1 ) )
+ ", grid " + to_string( field.gridstep, 1 ) + " m, tile " + to_string( field.tilesize, 0 ) + " m"
+ ", loaded out to " + to_string( field.range, 0 ) + " m",
Global.UITextColor );
Output.emplace_back(
" tiles: " + std::to_string( stats.inview ) + " in view, "
+ std::to_string( stats.resident ) + " on gpu (" + std::to_string( stats.texturebytes / 1048576 ) + " MB), "
+ std::to_string( stats.wanted ) + " wanted, " + std::to_string( field.backlog ) + " in loader, "
+ std::to_string( stats.uploaded ) + " uploaded, " + std::to_string( stats.dropped ) + " dropped stale",
Global.UITextColor );
std::string levels { " levels:" };
for( std::uint32_t level = 0; ( level < field.levels ) && ( level < stats.perlevel.size() ); ++level ) {
auto const last { level + 1 == field.levels };
levels +=
" L" + std::to_string( level )
+ ( last ? " beyond " + to_string( terrain_level_distance( level > 0 ? level - 1 : 0 ), 0 )
: " <" + to_string( terrain_level_distance( level ), 0 ) )
+ " m (" + to_string( field.gridstep * static_cast<float>( 1u << level ), 0 ) + " m grid): "
+ std::to_string( stats.perlevel[ level ] );
if( false == last ) { levels += " |"; }
}
Output.emplace_back( levels, Global.UITextColor );
}
textline += "\nRendering mode: "; textline += "\nRendering mode: ";
if (Global.GfxRenderer == "default") if (Global.GfxRenderer == "default")

105
application/shotrunner.cpp Normal file
View File

@@ -0,0 +1,105 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <cstddef>
#include <string>
#include <vector>
module eu07.application.shotrunner;
import eu07.glm;
import eu07.utilities.logs;
import eu07.utilities.utilities;
shot_runner Shots;
bool
shot_runner::parse( std::string const &Text, shot &Result ) {
std::vector<double> fields;
for( auto const &field : Split( Text, ',' ) ) {
try {
fields.push_back( std::stod( field ) );
}
catch( ... ) {
return false;
}
}
if( fields.size() < 3 ) { return false; }
Result.position = { fields[ 0 ], fields[ 1 ], fields[ 2 ] };
// the free camera takes its angles as pitch, yaw, roll in radians; a viewpoint is
// written in degrees, because nobody wants to type radians on a command line
Result.angle = {
static_cast<float>( glm::radians( fields.size() > 4 ? fields[ 4 ] : 0.0 ) ),
static_cast<float>( glm::radians( fields.size() > 3 ? fields[ 3 ] : 0.0 ) ),
0.f };
Result.label = Text;
return true;
}
void
shot_runner::configure( std::vector<shot> Shots, double const Settle, bool const Quit ) {
m_shots = std::move( Shots );
m_settle = Settle;
m_quit = Quit;
m_index = 0;
m_stage = stage::placing;
m_elapsed = 0.0;
m_done = m_shots.empty();
}
shot_runner::action
shot_runner::update( double const Deltatime ) {
if( false == active() ) { return action::none; }
switch( m_stage ) {
case stage::placing: {
m_stage = stage::settling;
m_elapsed = 0.0;
WriteLog( "Shot " + std::to_string( m_index + 1 ) + "/" + std::to_string( m_shots.size() )
+ ": moving to " + current().label );
return action::place;
}
case stage::settling: {
m_elapsed += Deltatime;
if( m_elapsed < m_settle ) { return action::none; }
m_stage = stage::capturing;
return action::none;
}
case stage::capturing: {
m_stage = stage::advancing;
WriteLog( "Shot " + std::to_string( m_index + 1 ) + ": capturing" );
return action::capture;
}
case stage::advancing: {
// a frame's grace after the capture, since the image is read back and written
// on another thread and quitting immediately would cut it short
m_elapsed += Deltatime;
if( m_elapsed < m_settle + 1.0 ) { return action::none; }
++m_index;
if( m_index < m_shots.size() ) {
m_stage = stage::placing;
return action::none;
}
m_done = true;
WriteLog( "Shot sequence finished" );
return m_quit ? action::finish : action::none;
}
}
return action::none;
}

View File

@@ -0,0 +1,76 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <cstddef>
#include <string>
#include <vector>
export module eu07.application.shotrunner;
import eu07.glm;
export {
// Drives an unattended sequence of screenshots: place the camera, let the world settle,
// save a frame, move on, quit. Exists so that a visual change can be checked without a
// person sitting at the machine pressing keys, and checked the same way twice.
//
// The runner owns the sequence and its timing and nothing else. It does not know what a
// camera is or how a frame is saved: it reports what should happen next and the caller,
// which does know, carries it out.
class shot_runner {
public:
struct shot {
glm::dvec3 position { 0.0, 0.0, 0.0 };
glm::vec3 angle { 0.f, 0.f, 0.f }; // pitch, yaw, roll in degrees, as the free camera takes them
std::string label; // for the log, so a saved frame can be tied to a viewpoint
};
enum class action {
none, // nothing to do this frame
place, // move the camera to current()
capture, // save a frame
finish // the sequence is done
};
// parses a "x,y,z[,yaw[,pitch]]" viewpoint. returns false on anything malformed,
// leaving Result untouched
static bool parse( std::string const &Text, shot &Result );
void configure( std::vector<shot> Shots, double const Settle, bool const Quit );
bool active() const { return ( false == m_shots.empty() ) && ( false == m_done ); }
shot const &current() const { return m_shots[ m_index ]; }
// advances the sequence by one frame. Deltatime is wall time, so that the settling
// period covers tiles streaming in rather than a fixed number of frames
action update( double const Deltatime );
private:
enum class stage {
placing,
settling,
capturing,
advancing
};
std::vector<shot> m_shots;
std::size_t m_index { 0 };
stage m_stage { stage::placing };
double m_settle { 3.0 };
double m_elapsed { 0.0 };
bool m_quit { true };
bool m_done { false };
};
// the session's sequence, configured from the command line and driven by the driver mode.
// idle and harmless when no viewpoints were given
extern shot_runner Shots;
} // export

773
binary-scene-format-plan.md Normal file
View File

@@ -0,0 +1,773 @@
# Binarny format scenerii pod streaming — plan działania
Dokument roboczy. Cel docelowy: format scenerii, który mapuje się do pamięci i trafia
do GPU bez transformacji danych, z niezależnym streamingiem terenu i obiektów,
gotowy pod renderer Vulkan.
Plan jest ułożony tak, że **każda faza daje wymierną korzyść samodzielnie**.
Fazy 0–4 nie wymagają żadnej decyzji o formacie binarnym ani o rendererze —
jeśli przedsięwzięcie stanie po drodze, to co zrobione, i tak zostaje na plusie.
## Pomiary wejściowe (2026-09-16)
Zmierzone na zbiorze `../pctga` (124 scenariusze, 3829 plików `.inc`, `scenery/` = 7,0 GB)
oraz na instalce Steam. Liczby są powodem, dla którego kolejność faz poniżej wygląda
tak, a nie inaczej.
Sceneria `l204` — 2,3 GB w 154 plikach `.scm`:
```
teren/ 2,2 GB (95% objętości)
citygml/ 84 MB
deko/ 15 MB
tory/ 3,0 MB
```
Populacja węzłów w `l204`:
```
8 970 715 triangles (99,8% wszystkich węzłów)
11 258 track
4 535 model
664 line_strip
474 dynamic
```
Postać danych terenu — jeden `node` na **jeden trójkąt**, ~220 bajtów ASCII:
```
node -1 0 none triangles grass_new
8812.0 35.55 -25152.0 -0.0280234 0.999585 -0.00665975 -307.251 -3449.75 end
8814.0 35.65 -25152.0 0.0115008 0.999892 -0.00910931 -307.501 -3449.75 end
8812.0 35.52 -25154.0 -0.0118979 0.999913 0.00561081 -307.251 -3450.0
endtri
```
Wnioski, które ustawiają plan:
1. **Teren dominuje wszystko.** 95% objętości i 99,8% węzłów sceny. Każdy trójkąt jest
osobnym `shape_node` przechodzącym przez `basic_region::insert()` i `RaTriangleDivider`.
Żadna inna optymalizacja nie zbliża się rzędem wielkości.
2. **Dane nie mają lokalności przestrzennej na dysku.** Kolejne trójkąty skaczą
(x = 8812 → 17898 → 6398); w pierwszych 30 000 wierzchołków zakres x to 18 km.
Streaming z takiego układu jest niemożliwy z konstrukcji — sortowanie przestrzenne
jest warunkiem wstępnym, nie optymalizacją.
3. **Hybryda heightfield + overlay potwierdzona empirycznie.** Około połowy wierzchołków
leży na regularnej siatce 2 m, druga połowa nie — baza jest siatką, nieregularne są
wcięcia torowiska i dróg.
4. **Obiektów jest mało.** 4535 modeli i 11 258 torów na ok. 40 km linii. Prefaby i
rezydencja assetów mają sens jako porządek w danych i podstawa batchowania, ale nie
jako oszczędność pamięci — populacja jest o cztery rzędy wielkości mniejsza od terenu.
5. **Sekcja LOGIC jest darmowa.** Cała topologia to ułamek procenta danych, więc trzymanie
jej rezydentnie w całości nie wymaga żadnego kompromisu.
Szacunek po zamianie bazy terenu na heightfield: 8,8 mln trójkątów na siatce 2 m to
ok. 4,4 mln próbek, czyli `R16` + splat `RGBA8` ≈ **26 MB zamiast 2,2 GB**. Nawet
z overlayem zostajemy trzy rzędy wielkości niżej.
## Pomiar fazy 0 (2026-09-16)
`braniewo_szeroki.scn` (wciąga `l204`), format tekstowy, RelWithDebInfo, clang 22.
Zebrane przełącznikiem `-loadprofile`, surowe dane w `loadprofile.txt`.
```
token loop (rest) 14.42 s 6.6 %
triangle import 125.36 s 57.2 %
triangle insert 3.63 s 1.7 %
of which divider 0.23 s 0.1 %
other node types 75.61 s 34.5 %
total 219.01 s
triangles 9 983 053
track 12 439
model 11 372
dynamic 510
shape vertices 36 799 623
shapes divided 269 (extra pieces: 92 328)
pamięć rezydentna: 403 MB -> 9 547 MB (delta 9 144 MB)
```
Co z tego wynika:
1. **Parsowanie trójkątów to 57% czasu ładowania.** Format binarny kasuje tę pozycję,
nie zmniejsza.
2. **Pamięć boli bardziej niż czas.** 9,1 GB na 36,8 mln wierzchołków to 248 B na
wierzchołek, przy danych wierzchołka rzędu 44 B (~1,6 GB łącznie). Pozostałe ~7,5 GB
to narzut na 9,98 mln osobnych `shape_node` — ok. 960 B na węzeł (`std::string` nazwy,
wektor, `bounding_area`, `lighting_data` = cztery `vec4` na trójkąt, origin, uchwyty).
Struktura zarządzająca waży prawie pięć razy tyle, co dane, którymi zarządza.
3. **`RaTriangleDivider` był w tym planie przeszacowany.** 0,23 s, czyli 0,1%; całe
wstawianie do regionu to 1,7%. Argumentem za rzadkim quadtree zostaje pamięć
i lokalność przestrzenna, nie koszt dzielenia kształtów.
4. **Drugi cel ujawniony przez pomiar:** `other node types` to 75,6 s (34,5%) przy
zaledwie 11 372 modelach — ok. 6,6 ms na model, czyli ładowanie `.e3d` i tekstur.
Wzmacnia to fazę 6 (rejestr assetów), ale dopiero po terenie.
### Porównanie: istniejąca binarka `.sbt`, ta sama sceneria
W `pctga/scenery/` leżał gotowy `braniewo_szeroki.sbt` (1,2 GB); wystarczyło włączyć
`file.binary.terrain`. To jest bezpośredni pomiar tego, ile daje **samo zbinaryzowanie
obecnej struktury**, bez jej zmiany.
```
tekst .sbt
token loop (rest) 14.42 s 7.18 s (zawiera odczyt 1,2 GB i przebiegi Init*)
triangle import 125.36 s 0.00 s
triangle insert 3.63 s 0.00 s
other node types 75.61 s 83.81 s
total 219.01 s 90.99 s
pamięć (delta) 9 144 MB 8 913 MB
```
Dwa wnioski, oba istotne dla kolejności faz:
1. **Czas: 2,4x szybciej.** Parsowanie znika co do zera, dokładnie jak zakładano.
2. **Pamięć: bez zmian (-2,5%).** I to jest najważniejsza liczba całego pomiaru.
`.sbt` deserializuje się do **tych samych 10 mln obiektów `shape_node`**, więc 9 GB
zostaje 9 GB. Zamiana tekstu na bajty nie rusza narzutu strukturalnego, bo narzut
nie brał się z kodowania. Wygrana pamięciowa może przyjść **wyłącznie** ze zmiany
struktury danych — czyli z heightfieldu (faza 2), nie z formatu jako takiego.
Dla porządku wielkości: `.sbt` to 1,2 GB na dysku, szacowany heightfield ~26 MB.
3. **Po zdjęciu terenu dominuje ładowanie assetów.** W przebiegu `.sbt`
`other node types` to 83,8 s, czyli 92% całości. Następnym wąskim gardłem po fazie 2
będzie `.e3d` i tekstury — czyli faza 6, i to z mocniejszym uzasadnieniem, niż plan
pierwotnie zakładał.
Wniosek dla całego przedsięwzięcia: **format binarny sam z siebie kupuje czas ładowania,
ale nie kupuje pamięci.** Kolejność faz (najpierw struktura terenu, potem kontener) jest
tym potwierdzona.
## Pomiar fazy 1: rozkład terenu (2026-09-16)
`tools/terraincook` na `scenery/l204/teren` (8 799 853 trójkąty, 2,2 GB tekstu).
```
-- rozkład, krok siatki 1,00 m
na siatce (heightfield) 8 781 214 99,79 %
poza siatką (overlay) 18 639 0,21 %
spiętrzone próbki 595 (maks. różnica 2,80 m)
-- realna gęstość próbkowania: rozkład długości krawędzi
< 4 m 52,58 %
< 8 m 14,94 %
< 16 m 18,27 %
< 32 m 13,30 %
>= 64 m 0,34 %
-- pokrycie wg rozmiaru komórki
1 m 4 402 162 komórek 4,40 km2
2 m 4 396 690 komórek 17,59 km2
4 m 2 700 291 komórek 43,20 km2
8 m 1 822 254 komórek 116,62 km2
16 m 1 026 922 komórek 262,89 km2
32 m 334 958 komórek 343,00 km2
64 m 95 082 komórek 389,46 km2
```
### Rozstrzygnięcia
1. **Heightfield się broni.** 99,79% trójkątów leży na regularnej siatce, overlay to 0,21%
i jest skupiony lokalnie (podgląd `overlay.pgm` jest praktycznie czarny, kilka plamek
w jednym rejonie). Główne ryzyko fazy 1 — overlay rozlany po całej mapie — nie istnieje.
2. **Mosty i wiadukty to błąd zaokrąglenia.** 595 spiętrzonych próbek na 4,4 mln, czyli
0,014%. Jedyne miejsca, których heightfield nie wyrazi, mieszczą się w garści trójkątów.
3. **Siatka bazowa to 1 m, nie 2 m.** Krok 2 m obejmuje tylko 25% wierzchołków.
4. **Ale teren NIE jest próbkowany co 1 m.** Wierzchołki mają całkowite współrzędne
metrowe, natomiast odległości między sąsiadami to 2–4 m w połowie przypadków, z ogonem
do 32 m. Teren jest zbiorem bloków o różnej gęstości, co widać wprost na `height.pgm`.
5. **Realna powierzchnia terenu to ~390 km², nie 4,4 km².** Liczba „pokrycie 4,40 km2"
z pierwszego przebiegu liczyła unikalne próbki metrowe i dla rzadziej próbkowanych
bloków liczyła ułamek ich powierzchni. Wartość zbiega dopiero przy komórce 32–64 m.
### Uczciwy koszt heightfieldu
Dla pokrycia ~390 km², sama mapa wysokości `R16`:
```
2 m 97,5 mln próbek ~186 MB
4 m 24,4 mln próbek ~47 MB
8 m 6,1 mln próbek ~12 MB
```
Z mipmapami i splat mapą w połowie rozdzielczości, przy bazie 4 m, wychodzi rząd
**70–90 MB** — wobec 2,2 GB tekstu, 1,2 GB `.sbt` i 9 GB w pamięci. Wcześniejsze
oszacowanie „~26 MB" było zbyt optymistyczne, bo opierało się na zaniżonej powierzchni.
Rząd wielkości zysku się utrzymuje, ale to jest ~30x, a nie ~100x.
Otwarte: czy drobne próbkowanie (2–4 m) skupia się przy torze, czy rozkłada się losowo
po blokach. Jeśli przy torze — baza 8 m plus overlay korytarza wystarczy. Jeśli losowo —
trzeba bazy 4 m. Do zmierzenia w fazie 2, gdy narzędzie dostanie dostęp do osi toru.
### Gęstość terenu względem toru
Oś toru wyciągnięta z `tory/*.scm` (11 258 torów, 480,1 km cięciw), próbkowana co 8 m,
odległość liczona transformatą chamfer na siatce 16 m.
```
pasmo trójkąty śr. krawędź drobne <4 m
< 25 m 3 389 721 2,75 m 91,8 %
< 50 m 621 774 5,13 m 51,5 %
< 100 m 638 680 6,80 m 32,3 %
< 200 m 823 895 8,03 m 30,4 %
< 400 m 965 577 9,59 m 30,0 %
< 800 m 1 056 106 12,23 m 20,8 %
< 1600 m 781 125 17,03 m 9,7 %
>= 1600 m 430 853 15,83 m 25,7 %
```
**Drobne próbkowanie jest przy torze i degraduje się monotonicznie.** 38,5% wszystkich
trójkątów terenu leży w pasie 25 m od osi — prawie cztery dziesiąte budżetu geometrii
idzie na wstęgę szerokości 50 m, a 390 km2 pola dostaje resztę.
Dwa ostatnie pasma odbijają lekko w górę; najpewniej to osobne dopracowane rejony daleko
od osi `l204` (stacje sąsiednich linii, obszar `citygml`). To 6% trójkątów, nie zmienia
obrazu, ale nie jest zweryfikowane.
Wniosek dla docelowego „drobno wszędzie": jest osiągalne, ale **dzisiejsze dane nie mają
szczegółu poza korytarzem**. Jednolita baza 2 m niczego nie pogorszy, natomiast poza
pasem 25 m będzie interpolacją, nie detalem. Realne zagęszczenie pola wymaga nowego
źródła wysokości — dla Polski dostępny jest publiczny NMT z ALS w siatce 1 m. I to jest
argument za heightfieldem niezależny od wydajności: **do heightfieldu można podmienić
źródło**, do zupy trójkątów nie.
Koszt jednolitej bazy 2 m dla pokrycia 390 km2: ~186 MB samych wysokości, z mipmapami
i splatem rzędu 250-300 MB. Jedna piąta dzisiejszego `.sbt`, a w pamięci bez znaczenia,
bo clipmapa trzyma tylko pierścienie.
### Ustalenia dla fazy 2 wynikające z pomiaru
1. **Jednolita baza 2 m**, bez wielorozdzielczości. Prostsze, zgodne z celem, mieści się
w budżecie.
2. **Overlay korytarza generowany z geometrii toru**, nie konwertowany ze starego terenu.
Siatka 2 m nie odda ostro skarpy 1:1,5, a na to gracz patrzy przez cały czas.
3. **Woda jako osobna warstwa** (patrz niżej).
4. **Rzadki katalog kafli**, nie gęsta tablica.
### Korekta do fazy 2
Faza 2 zakładała „jeden plik, stały `chunk_stride`, adres liczony arytmetycznie
z pozycji, bez katalogu". **To nie przejdzie na tych danych.** Pokryte ~390 km² leży
w zasięgu roboczym 51 x 46 km, czyli zajętość jest rzędu 16%, a przy pełnym bounding
boksie scenerii dużo gorsza. Gęsta tablica marnuje sześciokrotnie. Kafle terenu muszą
mieć **rzadki katalog** (kafel 256 m, tylko zajęte zapisane) — arytmetyczny offset
zostaje wewnątrz kafla, nie na poziomie całego pliku.
### Warstwa wody
Pełny bounding box danych to 206 km x 161 km, przy zasięgu roboczym 51 x 46 km, a sumaryczny
footprint trójkątów wychodzi na bezsensowne 19 600 km2. Sprawdzone: odpowiada za to
`woda_ter.scm` — 308 trójkątów, w tym kilka olbrzymich, rozciągniętych na całą mapę.
To nie są śmieci, tylko tafla wody, i nie jest to teren w sensie heightfieldu.
Wniosek dla fazy 2: **woda to osobna warstwa**, płaszczyzna z poziomem i materiałem,
nie próbki wysokości. Konwerter musi ją rozpoznawać i wydzielać przed liczeniem zasięgu,
bo inaczej jedna tafla rozciąga bounding box dziesięciokrotnie i psuje każdy szacunek
zajętości.
## Faza 2, krok 1: upieczony heightfield (2026-09-16)
`terraincook -cook` na `l204/teren`, krok siatki 2 m, kafel 128 próbek (256 m).
```
kafle 7 531 (129 x 129 próbek, wspólna krawędź)
zajętość bounding boksu 12,3 %
próbki z danymi 116 855 308 93,24 % zapisanych
zakres wysokości -3,96 .. 103,35 m (0,002 m na jednostkę)
odrzucone zbyt duże 376 (krawędź > 200 m, tafla wody)
zdegenerowane 880
spiętrzone 32 882 (zachowana niższa powierzchnia)
plik surowy 358,8 MB
plik po zstd -9 126,2 MB (kompresja 1,5 s)
czas pieczenia 20 s
szczyt pamięci 1,7 GB
```
Porównanie do dzisiejszych formatów tej samej scenerii: 2,2 GB tekstu, 1,2 GB `.sbt`.
Surowy `.ehf` to 1/6 tekstu i 1/3,4 `.sbt`; po kompresji 1/17 i 1/10.
Korekta wcześniejszego szacunku: zapowiadałem „250-300 MB z mipmapami". Surowy plik
**bez** mipmap wyszedł 358,8 MB, bo doszła płaszczyzna materiału — bajt na próbkę, czyli
jedna trzecia pliku. Kompresuje się niemal do zera (92% terenu to jeden materiał), stąd
tak dobry wynik zstd, ale w postaci surowej trzeba ją liczyć.
Wizualna kontrola (`cooked.pgm`): ciągła powierzchnia z czytelnymi dolinami rzecznymi,
bez dziur i bez szwów między kaflami, zgodna kształtem z oryginałem.
Uwaga o spiętrzeniach: 32 882 przy pieczeniu, wobec 595 znalezionych wcześniej. To nie
sprzeczność - tamto liczyło tylko wspólne wierzchołki na siatce 1 m, a rasteryzacja
porównuje interpolowane powierzchnie, więc łapie każde nałożenie trójkątów, nie tylko
dzielony wierzchołek. 0,03% próbek.
### Następne kroki fazy 2
1. Kompresja per kafel w kontenerze (katalog ma już rozmiar, dochodzi rozmiar
skompresowany i kod kodeka) - z 359 MB robi się ~126 MB bez zmiany logiki odczytu.
2. Mipmapy per kafel, pod poziomy clipmapy.
3. Strona silnika: loader `.ehf` i clipmapa pod OpenGL.
4. Overlay korytarza generowany z geometrii toru.
## Ślepa uliczka (sprawdzone, nie wracać)
**Launchery eventów jako listy 1D po torach.** Pierwotny plan stawiał to jako fazę
o najlepszym stosunku zysku do pracy, bo `basic_cell::update_events()` odpytuje je
promieniowo co klatkę i *wygląda* to na hot path. W danych hot patha nie ma:
```
65 eventlauncherów w całym zbiorze pctga (7 GB scenerii)
0 z traintriggered
```
Większość z nich ma `radius = -1` i godzinę, czyli `IsGlobal()` — nie powinny w ogóle
trafiać do przeglądu przestrzennego. Przeniesienie ich na listy 1D byłoby optymalizacją
czegoś, co nie kosztuje. Pozostaje drobiazg do sprzątnięcia przy okazji: w pętli
`train_triggered` w `scene/scene.cpp` zmienna `radius` jest skalowana prędkością, ale
porównanie i tak idzie do `launcher->dRadius`, więc skalowanie jest martwe.
## Zasady przekrojowe
Obowiązują w każdej fazie, nie są osobnym etapem.
1. **Definicja jest niemutowalna, stan jest osobno.** Plik cooked jest read-only i
mmapowany. Wszystko, co zmienia się w runtime, żyje w równoległych tablicach
alokowanych przy ładowaniu. Bez tego konsekwentnie przestrzeganego nie da się
mmapować niczego.
2. **Logika jest rezydentna, wizualia są streamowane.** Symulacja musi działać
identycznie niezależnie od tego, co jest wczytane. Żaden element symulacji nie
sprawdza rezydencji i żaden nie może się nie wykonać z powodu wyładowanego kafla.
3. **Przepływ jednokierunkowy.** Warstwa logiki zapisuje stan; warstwa wizualna
czyta go przy ładowaniu. Nigdy odwrotnie.
4. **Nazwy rozwiązywane przy kompilacji.** W runtime są indeksy. Stringi lądują
w opcjonalnej sekcji dla edytora, skryptów i debugowania.
5. **Determinizm.** Numeracja kanoniczna (sortowanie po hashu nazwy, nie po
kolejności w plikach), kolejność aktualizacji po indeksie, losowość ze strumieni
indeksowanych ID obiektu — nigdy z globalnego RNG.
6. **Brak pointerów w formacie.** Offsety względem początku chunka i indeksy tablic.
## Poza zakresem
**LOD symulacji fizyki.** Rozważany i odrzucony. Przejścia między poziomami muszą
być bezstratne w obie strony, a syntezowanie stanu pneumatyki przy awansie poziomu
jest źródłem błędów, których nie da się tanio wykryć (skład awansuje i wpada
w hamowanie nagłe). Koszt CPU po stronie AI leży w skanowaniu trasy, nie
w całkowaniu ruchu — adresuje to faza 4. Do tematu wracać tylko wtedy, gdy pomiar
z fazy 0 pokaże, że po fazie 4 fizyka nadal dominuje.
Podział pojazdu na rekord logiczny (rezydentny) i instancję wizualną (streamowaną)
zostaje — to nie jest LOD fizyki, tylko zasada 2.
---
## Faza 0 — pomiar terenu
**Cel.** Zmierzyć to, co pomiary wejściowe wskazały jako dominujące, zanim cokolwiek
zostanie przepisane. Zawężone względem pierwotnego zamysłu: nie cała ramka, tylko
ścieżka terenu.
**Zakres.**
- Czas ładowania `l204` rozbity na: parsowanie `.scm`, tworzenie `shape_node`,
`RaTriangleDivider`, wstawianie do regionu, budowa banków geometrii.
- Zużycie pamięci: dane wierzchołków terenu osobno od struktur zarządzających
(`shape_node`, komórki, sekcje).
- Liczniki populacji przy starcie: węzły wg typu, komórki niepuste, banki geometrii.
- Koszt ramki z grubsza, żeby wiedzieć, czy render terenu w ogóle jest problemem,
czy problemem jest tylko ładowanie i pamięć.
**Gotowe, gdy.** Liczby są zapisane w repo i powtarzalne jedną komendą na `l204`.
**Ryzyko.** Żadne. Po poprzedniej pomyłce z launcherami — nie pomijać.
---
## Faza 1 — konwerter terenu (offline)
**Cel.** Rozstrzygnąć najwyższe ryzyko całego planu, zanim cokolwiek w silniku się
zmieni: czy zupę trójkątów da się rozłożyć na regularny heightfield plus wąski overlay.
Narzędzie samodzielne, silnik nietknięty.
**Zakres.**
- Wejście: pliki `teren/*.scm`. Wyjście: heightfield (`R16`) + splat + lista trójkątów
nieprzystających do siatki.
- Wykrycie siatki bazowej (w `l204` jest to 2 m) i klasyfikacja każdego trójkąta:
na siatce → heightfield, poza siatką → overlay.
- Raport: jaki procent trafia do overlaya, jak overlay jest rozłożony przestrzennie,
ile waży po obu stronach.
- Wizualizacja albo eksport do czegoś oglądalnego — trzeba **zobaczyć**, czy overlay
pokrywa się z torowiskiem i drogami, czy rozsypuje się po całej mapie.
**Gotowe, gdy.** Dla jednego odcinka `l204` znamy proporcję heightfield/overlay i mamy
pewność, że overlay jest wąskim pasem wzdłuż infrastruktury, a nie połową mapy.
**Ryzyko.** **Najwyższe w całym planie i celowo wyciągnięte na początek.** Jeśli overlay
okaże się rozlany po całej powierzchni, model heightfieldu upada i trzeba wrócić do
streamowanego meshu — ale wtedy wiemy to po tygodniu pracy nad narzędziem, a nie po
przepisaniu renderera.
---
## Faza 2 — teren w silniku: clipmapa pod OpenGL
**Cel.** Zastąpić 8,97 mln węzłów `triangles` heightfieldem o stałym budżecie pamięci.
Na razie pod OpenGL — format jest niezależny od backendu.
**Zakres.**
- Teren przestaje być meshem: chunk to `height R16` + `splat RGBA8` + maska materiałów.
Mesh to stały grid albo pozycja pobierana z tekstury wysokości.
- Clipmapa: pierścienie o stałym rozmiarze wokół kamery, toroidalne adresowanie,
aktualizacja tylko wchodzącego pasa. Geomorphing między poziomami w shaderze.
- Jeden plik, stały `chunk_stride`, **adres liczony arytmetycznie z pozycji**, bez
katalogu. Konsolidacja dzisiejszego `terrain_streamer`
(`editor/editorTerrainStreamer.cpp`, tysiące plików `chunk_X_Z.etc`) do tego jednego pliku.
- Warstwa **overlay** z fazy 1 jako osobna geometria wzdłuż infrastruktury; heightfield
pod nią wygładzany przez konwerter.
- **Zapiekanie absolutnego Y instancji** z terenu w pełnej rozdzielczości. Runtime nigdy
nie odpytuje terenu o wysokość — to warunek niezależności strumieni terenu i obiektów.
- Zgodność wysokości na krawędziach poziomów bitowo identyczna z konstrukcji (brak szwów
z definicji, nie przez łatanie).
**Gotowe, gdy.** `l204` ładuje się z heightfieldu, zużycie pamięci terenu jest stałe
niezależnie od położenia, horyzont 20–30 km bez spadku wydajności, brak szwów.
**Ryzyko.** Średnie — po fazie 1 największa niewiadoma jest już zdjęta.
---
## Faza 3 — rozdzielenie definicji od stanu
**Cel.** Warunek wstępny mmapowania czegokolwiek. Efekt uboczny: ciągłe tablice
zamiast rozproszonych alokacji z wirtualnymi wywołaniami.
**Zakres.**
- `basic_event` (`world/Event.h`): `m_launchtime`, `m_inqueue`, `m_activator`,
`m_ignored` wychodzą do równoległych tablic. `m_delay`, `m_delayrandom`, `m_name`,
topologia zostają jako definicja.
- Memcelle (`world/MemCell.h`): wartości początkowe to definicja, bieżące to stan.
- Stan animacji obiektów (`TAnimContainer`) — faza animacji jako tablica, nie pole
w obiekcie.
- Stan zwrotnic i torów.
**Gotowe, gdy.** Kolejka eventów przemiata ciągłą tablicę `launchtime[]`; żadna
struktura definicji nie jest zapisywana w runtime.
**Ryzyko.** Średnie — dotyka dużo miejsc, ale mechanicznie. Robić w krokach po
jednym podsystemie, z testem scenariusza T1/T3/T5 po każdym.
---
---
## Faza 4 — profil trasy i przepisany `scan_route`
**Cel.** Zlikwidować chodzenie po grafie obiektów sceny przy każdym skanowaniu
trasy. To jest właściwy koszt AI.
**Zakres.**
- Struktura per tor, w sekcji logiki: `vmax`, `gradient`, `curvature`, zakresy
sygnałów i zmian limitu wzdłuż toru.
- `TController::scan_route` / `sSpeedTable` (`vehicle/Driver.h`) przepisane na
przemiatanie ciągłej tablicy. W runtime dochodzi tylko to, co dynamiczne:
wskazania sygnałów, zajętości, inne pojazdy.
- Zapieczony profil hamowania dla kategorii pociągu na odcinku — punkt rozpoczęcia
hamowania odczytywany, nie całkowany co klatkę.
- Na tym etapie profil liczony przy ładowaniu; przeniesienie do cookera w fazie 7.
**Gotowe, gdy.** Pomiar z fazy 0 pokazuje spadek kosztu skanowania proporcjonalny
do liczby składów, przy niezmienionym zachowaniu AI na scenariuszach testowych.
**Ryzyko.** Średnie — dotyka zachowania AI, które jest trudne do testowania
regresyjnego. Wymaga zestawu scenariuszy referencyjnych z fazy 0.
---
---
## Faza 5 — walidator jako osobne narzędzie
**Cel.** Wychwycić błędy scenerii przy kompilacji zamiast w postaci cichego
milczenia w runtime. Działa jeszcze na formacie tekstowym, chodzi po tym samym
grafie, co przyszły cooker.
**Zakres.**
- Eventy: wiszące referencje do targetów i memcelli, launchery wskazujące na
nieistniejące eventy, **eventy zadeklarowane za `FirstInit`**, niezgodność typów
slotów memcella, eventy nieosiągalne, cykle `multi` bez opóźnienia.
- Rozkłady: nieistniejąca stacja, brak wskazanego peronu, peron krótszy niż skład,
relacja prowadząca przez tor bez połączenia.
- Po fazie 4, na profilu trasy: **rozkład niewykonalny** — czas przejazdu A→B
mniejszy niż fizycznie możliwy przy tych limitach, spadkach i tej masie.
**Gotowe, gdy.** Narzędzie przechodzi po scenariuszach referencyjnych i raportuje
z numerem pliku i wiersza; znane błędy w istniejących sceneriach są wychwycone.
**Ryzyko.** Niskie. Wartość dla twórców scenerii natychmiastowa i niezależna od
reszty planu.
---
---
## Faza 6 — rejestr assetów i prefaby
**Cel.** Wprowadzić pojęcie współdzielonego archetypu, zanim zabetonuje się je
w formacie binarnym. Weryfikacja na semaforach, bo są najliczniejsze i najbardziej
powtarzalne.
**Zakres.**
- `MdlMngr`: lookup po `asset_id` (hash logicznej ścieżki) zamiast po ścieżce.
- Tabela zależności: `{ pack_id, asset_id, version, content_hash }`.
- Prefab = asset + kanały animacji + definicja stanów + schemat memcella + światła
+ dźwięki. Kanały animacji kompilowane po **indeksach submodeli**, nie po nazwach
(dziś `TAnimContainer` dowiązuje się po nazwie w runtime).
- Sceneria tekstowa zyskuje możliwość zapisu instancji prefabu zamiast modelu plus
pięciu eventów.
**Gotowe, gdy.** Semafory S1/S5/S13, tarcze i semafory kształtowe dają się opisać
parametrami jednego prefabu. **Jeśli się nie dają — warstwa prefabu jest za cienka
i trzeba ją przeprojektować przed fazą 8.** To jest właściwy punkt próby.
**Ryzyko.** Średnie, ale to ryzyko projektowe, a nie wdrożeniowe — i lepiej je
ponieść tu niż po przejściu na binarkę.
### Wynik punktu próby
**Jeden prefab wystarcza — pod warunkiem, że jego parametrem jest lista wskazań,
a nie zestaw liczb.**
Szablony semaforowe różnią się liczbą zdarzeń od 5 do 44, ale liczba parametrów
jest w nich stała (sześć, jeden wyjątek ma siedem). Ta rozbieżność to cała
diagnoza: `(pN)` nie potrafi wyrazić „semafor z wskazaniami {S1, S2, S5, S13}",
więc każda kombinacja komór, drabinki, daszka i tekstury dostaje własny plik.
Stąd ponad tysiąc `.inc` w katalogu scenerii.
Zdarzenia nie są dowolne — wynikają ze wskazań mechanicznie:
```
dwukomorowy (9): sem_info s1 s2 sem_info_stop sem_info_vmax
sem_ligh1 sem_ligh2 uszk sem_ligh0
pięciokomorowy (37): sem_info s1..s5 s10..s13 ms2 m40 sz1
sem_info_stop/vmax/v40/v20/Shunt25/Shunt40
sem_distinfo_stop/vmax/v100/v40
sem_ligh1..13 lighs2 lighz1 uszk sem_ligh0
```
Na każde wskazanie przypada jeden event stanu, jeden event świateł i jedna
prędkość wpisywana do memcella; do tego stały rdzeń (`_sem_info`, `_uszk`,
`_sem_ligh0`) i opcjonalny blok `_distinfo`, gdy semafor powtarza tarczę.
**Co z tego zbudowano.** `scene/prefab.{cppm,cpp}` — rejestr definicji `.pfb`
i rozwijanie instancji. Sceneria zyskała token:
```
prefab ss2czy sem_probny 100 0 -100 90 tabl/a1 endprefab
```
Definicja opisuje wskazania, nie tekst:
```
prefab
model sem/ps_2_l_cz_ks_ds_03.t3d
chambers 2
initial s1
aspect s1 lights 1 0 velocity 0.0 0.0 endaspect
aspect s2 lights 0 1 velocity -1 -1 endaspect
endprefab
```
Instancja rozwija się w tekst scenerii i wchodzi przez `injectString`, czyli tą
samą drogą co include — węzły i eventy powstają istniejącym parserem. Format
binarny skompiluje ją wprost, ale interfejs jest już ustalony: **publiczne są
wyłącznie eventy wskazań `<nazwa>_<wskazanie>`**, a światła, prędkości i memcell
należą do prefabu i wolno je zmienić.
Sprawdzone na `prefabtest.scn`: dziesięć eventów (rdzeń plus trzy na wskazanie),
model wczytany, semafor stoi na scenie. Odpowiednik `ss2czy.inc` ma dziewięć —
różnica bierze się z tego, że `.inc` współdzieli `_sem_info_stop` między
wskazaniem S1 a uszkodzeniem, a prefab nazywa prędkości po wskazaniach.
### Kształtowy — drugi test tej samej warstwy
Semafor kształtowy to przypadek, który miał ją rozerwać: `sk1.inc` ma sto wierszy,
dziesięć kanałów animacji na wskazanie, dźwięk przestawienia i trzy kanały światła.
**Nie rozerwał.** To wciąż ten sam kształt — na wskazanie jeden stan, jedne światła,
jedna prędkość — tylko z listą kanałów animacji doczepioną do wskazania:
```
aspect sr2
lights -1 0 1
lightdelay 0.5
velocity -1 -1
animate rotate ramie_gorne 0 -45 0 45
...
endaspect
```
Definicja urosła o `modelangle`, `attach` (kratownica i jej lod), `sound`,
`initiallights` i `lightmodes`. Wszystko to są cechy modelu, nie nowe pojęcia.
Przy okazji wyszło, że `.inc` dzielą animacje na grupy po osiem
(`_sem_anim_sr1` plus `_sem_anim_sr1cd`). `multi_event` trzyma `std::vector`, więc
ten limit dawno nie istnieje — prefab wystawia jedną listę. Dokładnie takiego
śmiecia warstwa prefabu ma nie przepuszczać dalej.
Sprawdzone na `prefabtest2.scn`: dwa semafory z jednej definicji, 72 eventy,
`sem_a` zostaje na „stój", `sem_b` po odpaleniu `sem_b_sr2` podnosi ramię.
**Czego to jeszcze nie pokrywa.** Kanały animacji idą po nazwach submodeli, tak jak
dziś — kompilacja po indeksach należy do formatu binarnego, bo dopiero tam jest
z czym indeks związać. Zostaje też `asset_id` zamiast ścieżki i tabela zależności
paczek.
---
---
## Faza 7 — cooker i kontener; sekcja LOGIC binarnie
**Cel.** Pierwsze realne wyjście binarne. Renderer nietknięty.
**Zakres.**
- Osobny target `tools/scn-cook`, wejście `.scn` (parser z `utilities/parser.h`),
wyjście kontenera. Deterministyczny bit w bit.
- Kontener: `[header][directory][LOGIC][STRINGS][chunks…]`, wyrównanie 16 B,
`static_assert` na rozmiarze każdej struktury, little-endian z markerem,
`magic + version + hash schematu`; niezgodność = odmowa ładowania, nie migracja.
- Sekcja LOGIC: tory, trakcja, eventy, memcelle, wyzwalacze, profil trasy — płaskie
tablice z indeksami zamiast pointerów. Znika `init_targets()` i `TrackJoin()`
z runtime'u.
- Rozkłady w **osobnym** kontenerze (edytowane niezależnie, przeładowywalne na
gorąco, ten sam teren obsługuje wiele scenariuszy).
**Gotowe, gdy.** Scenariusz ładuje się z binarnej sekcji LOGIC, zachowanie
identyczne, czas ładowania mierzalnie niższy.
**Ryzyko.** Średnie. Do rozstrzygnięcia przed startem: **relacja z VCS** — patrz
sekcja na końcu.
### Uwaga: na branchu `scenery-streaming` leży inny format binarny
`scene/scenerybinary_format.md` (wersja 10) opisuje **twin strumienia tokenów** —
każdy plik tekstowy kompiluje się do binarnego bliźniaka konsumowanego na poziomie
`cParser`, więc deserializer scenerii nigdy nie widzi formatu. Kod siedzi na
`scenery-streaming` (commit `6ba19dd6` i późniejsze), a sama specyfikacja przyjechała
tutaj z „Editor checkpoint" — dokument bez kodu, co jest mylące.
**To nie jest to samo co sekcja LOGIC i nie konkuruje z nią.** Twin skraca
tokenizowanie, zostawiając całą ścieżkę parsowania i budowania obiektów bez zmian.
LOGIC likwiduje samą tę ścieżkę: płaskie rekordy, indeksy zamiast nazw, brak
`init_targets()`. Można mieć oba naraz — twin dla tego, co zostaje tekstem (sceneria
w edycji), LOGIC dla tego, co jest upieczone.
Czego **nie** wolno zrobić: napisać drugiego kontenera na tokeny. Jeśli kiedyś
okaże się, że twin wystarcza, to LOGIC jest zbędny, a nie odwrotnie.
### Odstępstwo: kucharz siedzi w silniku, nie w osobnym targecie
Plan mówił `tools/scn-cook` czytający `.scn` parserem z `utilities/parser.h`.
Zrobione inaczej — `-cooklogic` piecze kontener z **załadowanego stanu**, bo cele
eventu, rodzeństwo i indeksy są produktem całego deserializera scenerii, a nie
parsera; osobne narzędzie musiałoby wyhodować jego drugą kopię. Pieczenie
z załadowanego stanu daje to, o co naprawdę chodzi: zapisane jest dokładnie to, co
produkuje ścieżka tekstowa. Determinizm bierze się z writera, nie z miejsca wywołania
— stringi internowane w kolejności pierwszego wystąpienia, rekordy w kolejności
menedżera, wyrównanie zerowane.
### Zrobione w tym kroku
- `scene/logicformat.h` — układ bajtów, bez typów silnika: `file_header`,
`section_entry`, `string_entry`, `event_record`, `target_entry`, `static_assert`
na każdym rozmiarze, `schema_hash` w nagłówku.
- `scene/logiccontainer.{cppm,cpp}` — writer i reader. Reader odrzuca wszystko, co
nie zgadza się co do magic, wersji, hasha schematu, endianu, liczby sekcji lub
rozmiaru pliku. Odmowa, nie migracja — to cache, zawsze można upiec od nowa.
- `simulation/logiccook.{cppm,cpp}` — `-cooklogic` piecze i **natychmiast czyta
z powrotem**, porównując z eventami w pamięci: nazwa, rodzaj, opóźnienia, liczba
i kolejność celów.
**Czego ten krok jeszcze nie robi.** Kolumna `node` w `target_entry` to `no_index`,
bo nie ma jeszcze sekcji węzłów, w której indeks miałby na co wskazywać —
więc `init_targets()` **nadal działa**. Ten krok stawia kontener, dyscyplinę schematu
i uprząż cook/verify, z których korzysta każda następna sekcja. Pola `reserved0`
w rekordzie nie udaję: flagi per-rodzaj leżą w podklasach i pojadą razem z resztą
ładunku, gdy będzie dla nich sekcja.
---
---
## Faza 8 — kafle wizualne i streamer obiektów
**Cel.** Streaming geometrii i instancji, niezależny od terenu.
**Zakres.**
- Rzadki quadtree po kodzie Mortona zamiast `EU07_CELLSIZE` 250 m /
`EU07_SECTIONSIZE` 1000 m / `500×500` sekcji (95% dzisiejszych sekcji jest pustych,
a stacja węzłowa dostaje ten sam budżet co pole).
- Chunki **posortowane po Mortonie** w pliku — ruch wzdłuż linii to odczyt
sekwencyjny, prefetch to jeden `pread` na kilka kafli.
- Geometria GPU-ready: pozycje `int16` względem origin kafla (rozdzielczość ~4 mm
przy kaflu 256 m, przy okazji likwiduje problem precyzji `dvec3`), normalne
i tangenty oktahedralne, UV `unorm16`, indeksy 16-bit lokalne. Przepuszczone
przez `meshoptimizer` + meshlety.
- LOD-y jako osobne chunki pod tym samym `tile_id` — pobranie lod3 bez czytania lod0.
- Instancje prefabów SoA, posortowane po `model_id`.
- Nagłówek kafla deklaruje zależności assetowe; kafel staje się widoczny dopiero
gdy assety są rezydentne.
- Rezydencja assetów: refcount po kaflach + LRU, z flagą `pinned` dla małych
i wszechobecnych (semafory, wskaźniki, słupki hektometrowe).
- Kompresja per chunk (zstd), nagłówek chunka nieskompresowany.
- **Dwie niezależne kolejki I/O i dwa budżety pamięci.** Teren ma twarde
pierwszeństwo z wywłaszczeniem; pod presją pamięci dociskane są wyłącznie obiekty.
- Streamer najpierw na GL33 (`glBufferSubData`) — format jest już GPU-ready.
**Gotowe, gdy.** Przejazd całą scenerią bez hitchy, przy stałym suficie pamięci.
**Ryzyko.** Średnie. Najwięcej pracy, ale na tym etapie wszystkie decyzje
projektowe są już zweryfikowane wcześniejszymi fazami.
---
---
## Faza 9 — backend Vulkan
**Cel.** Konsumpcja tego samego formatu bez zmian w plikach.
**Zakres.**
- Dedykowana kolejka transferu, persistent-mapped ring staging 64–128 MB.
Rozmiar chunka 0,5–4 MB (mniejsze topią się w barierach, większe dają hitch).
- Arenas przez VMA, `buffer_device_address` = baza areny + offset z katalogu, żeby
deskryptory nie zmieniały się przy load/unload.
- Bindless (`VK_EXT_descriptor_indexing`), materiał jako indeks w SSBO.
- `vkCmdDrawIndexedIndirectCount`, culling meshletów w compute.
- Tekstury poza formatem sceny: KTX2 + BC7/BC5, streamowane niezależnie, mip-tail
rezydentny. Sceneria trzyma tylko `texture_id`.
- Sparse binding rozważyć dopiero po pomiarze — komplikuje i bywa wolniejszy.
**Gotowe, gdy.** Ten sam plik scenerii działa na obu backendach.
---
---
## Do rozstrzygnięcia przed fazą 7: relacja z VCS
Dzisiejszy `.scn` jest diffowalny i cała społeczność na tym pracuje. Binarka to
zabija. Dwie drogi, trzeba wybrać świadomie:
- **Tekst jako źródło, binarka jako artefakt cookera** (jak shader → SPIR-V).
Cooker deterministyczny, uruchamiany w CI. Rekomendowane.
- **Binarka jako jedyne źródło prawdy.** Wtedy obowiązkowo `scn-diff` / `scn-merge`
i git driver — inaczej pierwszy merge dwóch osób na tej samej stacji kończy się
utratą pracy.
## Kolejność w skrócie
Faza 0 mierzy. Fazy 1–2 zdejmują teren, czyli 95% objętości i 99,8% węzłów — i to one
decydują, czy całe przedsięwzięcie ma sens.
Fazy 3–5 są opłacalne niezależnie od formatu i od renderera.
Faza 6 to punkt próby dla modelu prefabu.
Fazy 7–9 wymagają podjętej decyzji o cookerze i o VCS.
Gdyby robić tylko jedną rzecz z tej listy: **faza 1**. Gdyby dwie: **1 i 2**.

View File

@@ -1627,7 +1627,15 @@ void opengl33_renderer::setup_pass(viewport_config &Viewport, renderpass_config
Config.draw_range = std::max( 2000.f, Config.draw_range ); Config.draw_range = std::max( 2000.f, Config.draw_range );
// projection // projection
auto const zfar = ( Zfar > 1.f ? Zfar : Config.draw_range * Zfar ); auto zfar = ( Zfar > 1.f ? Zfar : Config.draw_range * Zfar );
if ((false == Ignoredebug) && (Zfar == 1.f))
{
// the far plane of an ordinary view reaches as far as the terrain is streamed, which
// is its own setting rather than the draw range. only the projection moves: what
// else gets drawn is still decided by draw_range. reverse z on a float depth
// buffer keeps the precision this costs negligible
zfar = std::max(zfar, Global.TerrainRange);
}
auto const znear = ( Znear > 1.f ? Znear : Znear > 0.f ? Znear * zfar : 0.1f * Global.ZoomFactor); auto const znear = ( Znear > 1.f ? Znear : Znear > 0.f ? Znear * zfar : 0.1f * Global.ZoomFactor);
// the editor's top-down plan view drops the perspective, so distances read the same wherever // the editor's top-down plan view drops the perspective, so distances read the same wherever
@@ -2467,6 +2475,7 @@ void opengl33_renderer::Render(scene::basic_region *Region)
{ {
case rendermode::color: case rendermode::color:
{ {
Render_terrain();
Render(std::begin(m_sectionqueue), std::end(m_sectionqueue)); Render(std::begin(m_sectionqueue), std::end(m_sectionqueue));
// draw queue is filled while rendering sections // draw queue is filled while rendering sections
if (EditorModeFlag && m_current_viewport->main) if (EditorModeFlag && m_current_viewport->main)
@@ -2851,6 +2860,84 @@ void opengl33_renderer::Render(cell_sequence::iterator First, cell_sequence::ite
m_frame_instance_buckets.clear(); m_frame_instance_buckets.clear();
} }
// The heightfield draws itself; what it needs from here is the state every other draw in
// this pass gets - the camera-relative modelview in the model block - and the viewpoint
// the world is being rendered around, so it can place its tiles relative to it.
void opengl33_renderer::Render_terrain()
{
// opened on first use rather than at init: a scenery names its heightfields in
// terrain directives, which are not read until the scenario is parsed, long after the
// renderer starts. attempted once either way, so a bad path does not retry per frame
if (false == m_terrainattempted)
{
m_terrainattempted = true;
for (auto const &path : Global.terrain_heightfields)
{
auto field{std::make_unique<terrain_clipmap>()};
if (true == field->open(path))
{
m_terrain.emplace_back(std::move(field));
}
}
// finest first: drawn in this order, the depth test keeps the better surface
// where two fields describe the same ground
std::sort(std::begin(m_terrain), std::end(m_terrain),
[](auto const &Left, auto const &Right) { return Left->gridstep() < Right->gridstep(); });
for (std::size_t index = 0; index < m_terrain.size(); ++index)
{
// every field sits a little under the triangles, so where a baked heightfield and
// the geometry it was baked from are both drawn, the geometry is what shows. a
// coarser field is sunk further still, far enough that rounding cannot let it
// through, and little enough to be invisible at the spacing it is drawn at
m_terrain[index]->depthbias(index == 0 ? 0.02f : 0.1f);
}
}
if (true == m_terrain.empty())
{
return;
}
glm::dvec3 const viewpoint{m_renderpass.pass_camera.position()};
model_ubs.set_modelview(OpenGLMatrices.data(GL_MODELVIEW));
model_ubo->update(model_ubs);
for (auto const &field : m_terrain)
{
// what stays on the gpu is decided by the main view alone. any other colour pass
// draws what is there: letting a second viewport with its own camera and range
// steer residency would evict what the main view is looking at
if (m_current_viewport->main)
{
// the terrain has a range of its own, not the pass draw range
field->range(Global.TerrainRange);
field->update(viewpoint);
}
field->render(viewpoint);
}
if (m_current_viewport->main)
{
TerrainStatus.resize(m_terrain.size());
for (std::size_t index = 0; index < m_terrain.size(); ++index)
{
auto const &field{*m_terrain[index]};
auto &status{TerrainStatus[index]};
if (status.path.empty())
{
status.path = field.path();
status.gridstep = field.gridstep();
status.tilesize = field.tilesize();
status.levels = field.levels();
}
status.range = field.range();
status.backlog = field.backlog();
status.stats = field.stats();
}
}
}
void opengl33_renderer::Draw_Geometry(std::vector<gfx::geometrybank_handle>::iterator begin, std::vector<gfx::geometrybank_handle>::iterator end) void opengl33_renderer::Draw_Geometry(std::vector<gfx::geometrybank_handle>::iterator begin, std::vector<gfx::geometrybank_handle>::iterator end)
{ {
m_geometry.draw(begin, end); m_geometry.draw(begin, end);

View File

@@ -36,6 +36,8 @@ import eu07.rendering.openglcamera;
import eu07.rendering.opengl33light; import eu07.rendering.opengl33light;
import eu07.rendering.opengl33particles; import eu07.rendering.opengl33particles;
import eu07.rendering.opengl33skydome; import eu07.rendering.opengl33skydome;
import eu07.rendering.terrainclipmap;
import eu07.rendering.terrainstatus;
import eu07.rendering.opengl33precipitation; import eu07.rendering.opengl33precipitation;
import eu07.simulation.simulationenvironment; import eu07.simulation.simulationenvironment;
import eu07.simcore; import eu07.simcore;
@@ -46,6 +48,7 @@ import eu07.gl.framebuffer;
import eu07.gl.renderbuffer; import eu07.gl.renderbuffer;
import eu07.gl.postfx; import eu07.gl.postfx;
import eu07.gl.shader; import eu07.gl.shader;
import eu07.rendering.terrainclipmap;
import eu07.gl.cubemap; import eu07.gl.cubemap;
import eu07.gl.glsl_common; import eu07.gl.glsl_common;
import eu07.gl.pbo; import eu07.gl.pbo;
@@ -301,6 +304,8 @@ class opengl33_renderer : public gfx_renderer {
void setup_sunlight_intensity( float const Factor = 1.f); void setup_sunlight_intensity( float const Factor = 1.f);
// runs jobs needed to generate graphics for specified render pass // runs jobs needed to generate graphics for specified render pass
void Render_pass(viewport_config &vp, rendermode const Mode); void Render_pass(viewport_config &vp, rendermode const Mode);
// draws the cooked terrain heightfield, if one is open
void Render_terrain();
// creates dynamic environment cubemap // creates dynamic environment cubemap
bool Render_reflections(viewport_config &vp); bool Render_reflections(viewport_config &vp);
bool Render(world_environment *Environment); bool Render(world_environment *Environment);
@@ -379,6 +384,10 @@ class opengl33_renderer : public gfx_renderer {
double m_environmentupdatetime{0}; // time of the most recent environment map update double m_environmentupdatetime{0}; // time of the most recent environment map update
glm::dvec3 m_environmentupdatelocation; // coordinates of most recent environment map update glm::dvec3 m_environmentupdatelocation; // coordinates of most recent environment map update
opengl33_skydome m_skydomerenderer; opengl33_skydome m_skydomerenderer;
// cooked terrain heightfields, ordered finest first. a scenery carries a coarse
// regional model and a fine corridor per line, and they overlap
std::vector<std::unique_ptr<terrain_clipmap>> m_terrain;
bool m_terrainattempted { false };
opengl33_precipitation m_precipitationrenderer; opengl33_precipitation m_precipitationrenderer;
opengl33_particles m_particlerenderer; // particle visualization subsystem opengl33_particles m_particlerenderer; // particle visualization subsystem

View File

@@ -0,0 +1,516 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <algorithm>
#include <cmath>
#include <array>
#include <cstdint>
#include <functional>
#include <memory>
#include <optional>
#include <string>
#include <unordered_map>
#include <vector>
#include "glad/glad.h"
#include "scene/heightfieldformat.h"
#include "global_include/interfaces/ITexture_macros.h"
#include "scene/heightfieldreader.h"
module eu07.rendering.terrainclipmap;
import eu07.glm;
import eu07.utilities.logs;
import eu07.rendering.renderer;
import eu07.model.material;
namespace {
// NOTE: glTexImage2D rather than glTexStorage2D. Immutable storage arrives with OpenGL
// 4.2 and this renderer holds a 3.3 context, where the entry point is simply not there -
// calling it jumps through a null pointer instead of failing in any visible way
GLuint
create_texture( GLenum const Internalformat, GLenum const Type, GLsizei const Side, void const *Data ) {
GLuint texture { 0 };
::glGenTextures( 1, &texture );
::glBindTexture( GL_TEXTURE_2D, texture );
// integer samples carry the cooker's values, no-data included, so nothing may filter them
::glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST );
::glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST );
::glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE );
::glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE );
::glTexImage2D( GL_TEXTURE_2D, 0, Internalformat, Side, Side, 0, GL_RED_INTEGER, Type, Data );
return texture;
}
} // namespace
terrain_clipmap::~terrain_clipmap() {
close();
}
bool
terrain_clipmap::open( std::string const &Path ) {
close();
if( ( false == m_reader.open( Path ) )
|| ( false == m_loader.open( Path ) ) ) {
ErrorLog( "Terrain: cannot open heightfield \"" + Path + "\"" );
close();
return false;
}
m_path = Path;
build_indices();
build_palette();
resolve_materials();
try {
gl::shader vertex( "terrainclipmap.vert" );
gl::shader fragment( "terrainclipmap.frag" );
m_shader.emplace( std::vector<std::reference_wrapper<gl::shader const>>( { vertex, fragment } ) );
}
catch( gl::shader_exception const &error ) {
ErrorLog( std::string( "Terrain: shader failed, " ) + error.what() );
close();
return false;
}
m_vao.emplace();
resolve_uniforms();
m_ready = true;
WriteLog(
"Terrain: " + Path + ", " + std::to_string( m_reader.tiles().size() ) + " tiles, "
+ std::to_string( m_reader.levels() ) + " levels, grid "
+ std::to_string( m_reader.gridstep() ) + " m, tile " + std::to_string( m_reader.tilesize() ) + " m" );
return true;
}
// uniform locations do not change once the program is linked, so they are looked up
// here rather than per tile per frame
void
terrain_clipmap::resolve_uniforms() {
auto const program { static_cast<GLuint>( *m_shader ) };
m_uniforms.tileorigin = ::glGetUniformLocation( program, "tileorigin" );
m_uniforms.samplestep = ::glGetUniformLocation( program, "samplestep" );
m_uniforms.heightbias = ::glGetUniformLocation( program, "heightbias" );
m_uniforms.heightscale = ::glGetUniformLocation( program, "heightscale" );
m_uniforms.nodata = ::glGetUniformLocation( program, "nodata" );
m_uniforms.side = ::glGetUniformLocation( program, "side" );
m_uniforms.heights = ::glGetUniformLocation( program, "heights" );
m_uniforms.materials = ::glGetUniformLocation( program, "materials" );
m_uniforms.palette = ::glGetUniformLocation( program, "palette" );
m_uniforms.groundtexture = ::glGetUniformLocation( program, "ground" );
m_uniforms.groundscale = ::glGetUniformLocation( program, "groundscale" );
m_uniforms.drawnmaterial = ::glGetUniformLocation( program, "drawnmaterial" );
m_uniforms.tileworld = ::glGetUniformLocation( program, "tileworld" );
m_uniforms.hastexture = ::glGetUniformLocation( program, "hastexture" );
m_uniforms.fallback = ::glGetUniformLocation( program, "fallback" );
}
void
terrain_clipmap::close() {
// the loader first, so nothing arrives for tiles that are being torn down
m_loader.close();
m_wanted.clear();
m_arrived.clear();
m_inrange.clear();
m_wantedlevel.clear();
m_scanrange = -1.f;
for( auto & [ tilekey, tile ] : m_tiles ) { retire( tile ); }
m_tiles.clear();
m_indices.clear();
m_indexcounts.clear();
m_vao.reset();
m_shader.reset();
m_palette.clear();
m_reader.close();
m_stats = {};
m_ready = false;
}
// index buffers, one per mip level. the grid they describe is the same shape at every
// level, only smaller, and the vertices themselves never exist in memory: the shader
// derives a sample's row and column from gl_VertexID
void
terrain_clipmap::build_indices() {
m_indices.resize( m_reader.levels() );
m_indexcounts.resize( m_reader.levels() );
std::vector<std::uint32_t> indices;
for( std::uint32_t level = 0; level < m_reader.levels(); ++level ) {
auto const side { m_reader.samples_per_side( level ) };
indices.clear();
indices.reserve( static_cast<std::size_t>( side - 1 ) * ( side - 1 ) * 6 );
for( std::uint32_t row = 0; row + 1 < side; ++row ) {
for( std::uint32_t column = 0; column + 1 < side; ++column ) {
auto const topleft { row * side + column };
auto const topright { topleft + 1 };
auto const bottomleft { topleft + side };
auto const bottomright { bottomleft + 1 };
indices.push_back( topleft ); indices.push_back( bottomleft ); indices.push_back( topright );
indices.push_back( topright ); indices.push_back( bottomleft ); indices.push_back( bottomright );
}
}
m_indexcounts[ level ] = static_cast<std::uint32_t>( indices.size() );
m_indices[ level ] = std::make_unique<gl::buffer>();
m_indices[ level ]->allocate(
gl::buffer::ELEMENT_ARRAY_BUFFER, indices.size() * sizeof( std::uint32_t ), GL_STATIC_DRAW );
m_indices[ level ]->upload(
gl::buffer::ELEMENT_ARRAY_BUFFER, indices.data(), 0, indices.size() * sizeof( std::uint32_t ) );
}
}
// placeholder colours, one per cooked material, so that the shape of the terrain and the
// division into surfaces can be judged before the real materials are wired in
void
terrain_clipmap::build_palette() {
m_palette.clear();
m_palette.reserve( std::max<std::size_t>( 1, m_reader.materials().size() ) );
for( std::size_t index = 0; index < m_reader.materials().size(); ++index ) {
auto const hue { static_cast<float>( index ) * 0.61803399f };
auto const phase { ( hue - std::floor( hue ) ) * 6.f };
auto const rising { phase - std::floor( phase ) };
glm::vec3 colour { 0.45f, 0.5f, 0.4f };
switch( static_cast<int>( phase ) ) {
case 0: colour = { 0.55f, 0.45f + 0.2f * rising, 0.3f }; break;
case 1: colour = { 0.55f - 0.2f * rising, 0.6f, 0.3f }; break;
case 2: colour = { 0.35f, 0.6f, 0.3f + 0.2f * rising }; break;
case 3: colour = { 0.35f, 0.6f - 0.15f * rising, 0.5f }; break;
case 4: colour = { 0.35f + 0.2f * rising, 0.45f, 0.5f }; break;
default: colour = { 0.55f, 0.45f, 0.5f - 0.15f * rising }; break;
}
m_palette.push_back( colour );
}
if( true == m_palette.empty() ) { m_palette.push_back( { 0.45f, 0.5f, 0.4f } ); }
}
std::uint32_t
terrain_clipmap::level_for( double const Distance ) const {
if( Distance <= terrain_finest_range ) { return 0; }
auto const steps { static_cast<std::uint32_t>( std::log2( Distance / terrain_finest_range ) ) + 1 };
return std::min( steps, m_reader.levels() - 1 );
}
void
terrain_clipmap::upload( terrain_tile_loader::payload const &Tile ) {
auto const tilekey { key( Tile.tile.x, Tile.tile.z ) };
auto const side { m_reader.samples_per_side( Tile.tile.level ) };
auto const existing { m_tiles.find( tilekey ) };
if( existing != m_tiles.end() ) {
// a level change means new textures: the sample count differs
retire( existing->second );
m_tiles.erase( existing );
}
resident_tile tile;
tile.x = Tile.tile.x;
tile.z = Tile.tile.z;
tile.level = Tile.tile.level;
tile.side = side;
tile.lastused = m_frame;
tile.heighttexture = create_texture(
GL_R16UI, GL_UNSIGNED_SHORT, static_cast<GLsizei>( side ), Tile.heights.data() );
tile.materialtexture = create_texture(
GL_R8UI, GL_UNSIGNED_BYTE, static_cast<GLsizei>( side ), Tile.materials.data() );
// the tile is drawn once per material it contains, so the set is worked out here
// rather than rescanned every frame
std::array<bool, 256> present {};
for( auto const material : Tile.materials ) { present[ material ] = true; }
for( std::size_t material = 0; material < present.size(); ++material ) {
if( true == present[ material ] ) { tile.materials.push_back( static_cast<std::uint8_t>( material ) ); }
}
if( auto const *description { m_reader.find( Tile.tile.x, Tile.tile.z ) } ) {
tile.minheight = description->minheight;
tile.maxheight = description->maxheight;
}
m_stats.texturebytes += static_cast<std::size_t>( side ) * side * 3u;
++m_stats.uploaded;
m_tiles.emplace( tilekey, std::move( tile ) );
}
void
terrain_clipmap::retire( resident_tile &Tile ) {
if( Tile.heighttexture != 0 ) { ::glDeleteTextures( 1, &Tile.heighttexture ); Tile.heighttexture = 0; }
if( Tile.materialtexture != 0 ) { ::glDeleteTextures( 1, &Tile.materialtexture ); Tile.materialtexture = 0; }
auto const bytes { static_cast<std::size_t>( Tile.side ) * Tile.side * 3u };
m_stats.texturebytes -= std::min( m_stats.texturebytes, bytes );
}
void
terrain_clipmap::scan( glm::dvec3 const &Viewpoint ) {
m_scanpoint = Viewpoint;
m_scanrange = m_range;
m_inrange.clear();
m_wantedlevel.clear();
auto const tilesize { static_cast<double>( m_reader.tilesize() ) };
auto const reach { std::max( 1, static_cast<std::int32_t>( m_range / tilesize ) + 1 ) };
auto const centrex { m_reader.tile_x( Viewpoint.x ) };
auto const centrez { m_reader.tile_z( Viewpoint.z ) };
std::vector<std::pair<double, terrain_tile_loader::request>> wishes;
for( std::int32_t z = centrez - reach; z <= centrez + reach; ++z ) {
for( std::int32_t x = centrex - reach; x <= centrex + reach; ++x ) {
if( false == m_reader.contains( x, z ) ) { continue; }
// distance to the nearest point of the tile, not to its centre. a regional
// field has tiles kilometres across, and a tile the camera is standing on
// can have its centre far outside the draw range
auto const west { x * tilesize };
auto const north { z * tilesize };
auto const nearestx { std::clamp( Viewpoint.x, west, west + tilesize ) };
auto const nearestz { std::clamp( Viewpoint.z, north, north + tilesize ) };
auto const distance {
glm::length( glm::dvec2 { nearestx - Viewpoint.x, nearestz - Viewpoint.z } ) };
if( distance > m_range ) { continue; }
auto const tilekey { key( x, z ) };
auto const level { level_for( distance ) };
m_inrange.push_back( { tilekey, level } );
m_wantedlevel.emplace( tilekey, level );
auto const resident { m_tiles.find( tilekey ) };
if( ( resident == m_tiles.end() ) || ( resident->second.level != level ) ) {
wishes.push_back( { distance, { x, z, level } } );
}
}
}
// nearest first, so the ground under the camera is never waiting behind the horizon
std::sort( wishes.begin(), wishes.end(),
[]( auto const &Left, auto const &Right ) { return Left.first < Right.first; } );
m_wanted.clear();
for( auto const &wish : wishes ) { m_wanted.push_back( wish.second ); }
m_loader.want( m_wanted );
}
void
terrain_clipmap::update( glm::dvec3 const &Viewpoint ) {
if( false == m_ready ) { return; }
++m_frame;
auto const moved {
glm::length( glm::dvec2 { Viewpoint.x - m_scanpoint.x, Viewpoint.z - m_scanpoint.z } ) };
if( ( m_range != m_scanrange )
|| ( moved > m_reader.tilesize() * 0.25 ) ) {
scan( Viewpoint );
}
// the tiles in range are touched every frame, which is what render() draws and what
// eviction spares
m_stats.inview = 0;
m_stats.wanted = 0;
m_stats.perlevel.fill( 0 );
for( auto const &tile : m_inrange ) {
auto const resident { m_tiles.find( tile.key ) };
if( resident == m_tiles.end() ) {
++m_stats.wanted;
continue;
}
// still wanted, at whatever level it has until the right one arrives
resident->second.lastused = m_frame;
++m_stats.inview;
++m_stats.perlevel[ std::min<std::size_t>( resident->second.level, m_stats.perlevel.size() - 1 ) ];
if( resident->second.level != tile.level ) { ++m_stats.wanted; }
}
m_arrived.clear();
m_loader.collect( m_arrived, m_uploadsperframe );
for( auto const &tile : m_arrived ) {
if( false == tile.valid ) { continue; }
// a tile read for where the camera was is put up only if nothing of it is on the gpu
// yet - some level is better than a hole. otherwise it is dropped, and the level
// wanted now is already queued
auto const tilekey { key( tile.tile.x, tile.tile.z ) };
auto const wanted { m_wantedlevel.find( tilekey ) };
auto const stillwanted { ( wanted != m_wantedlevel.end() ) && ( wanted->second == tile.tile.level ) };
if( stillwanted || ( false == m_tiles.contains( tilekey ) ) ) {
upload( tile );
// put up this frame, so drawn this frame
m_tiles.find( tilekey )->second.lastused = m_frame;
}
else {
++m_stats.dropped;
}
}
if( ( m_tiles.size() > m_budget ) && ( m_tiles.size() > m_stats.inview ) ) {
// over budget, and not everything is in view: drop whatever has gone longest without
// being asked for
std::vector<std::pair<std::uint64_t, std::int64_t>> aged;
for( auto const & [ tilekey, tile ] : m_tiles ) {
if( tile.lastused != m_frame ) { aged.emplace_back( tile.lastused, tilekey ); }
}
auto const excess { std::min( aged.size(), m_tiles.size() - m_budget ) };
std::partial_sort( aged.begin(), aged.begin() + excess, aged.end() );
for( std::size_t index = 0; index < excess; ++index ) {
auto const victim { m_tiles.find( aged[ index ].second ) };
retire( victim->second );
m_tiles.erase( victim );
}
}
m_stats.resident = m_tiles.size();
report_residency();
}
// the cooked material names are the ones the scenery used, so the engine resolves them
// exactly as it does for any other geometry. only the diffuse texture is taken: the
// terrain has its own shader and none of the rest of a material applies to it
void
terrain_clipmap::resolve_materials() {
m_materials.clear();
auto const &names { m_reader.materials() };
auto const &scales { m_reader.material_scales() };
m_materials.reserve( names.size() );
std::size_t resolved { 0 };
std::string unresolved;
for( std::size_t index = 0; index < names.size(); ++index ) {
material_binding binding;
binding.scale = ( index < scales.size() ? scales[ index ] : 4.f );
if( binding.scale < 0.01f ) { binding.scale = 4.f; }
auto const material { GfxRenderer->Fetch_Material( names[ index ] ) };
if( material != null_handle ) {
binding.texture = GfxRenderer->Material( material )->GetTexture( 0 );
}
if( binding.texture != null_handle ) {
++resolved;
}
else {
// drawn in its placeholder colour instead, so a missing texture shows up as
// an obvious flat patch rather than as black ground
unresolved += ( unresolved.empty() ? "" : ", " ) + names[ index ];
}
m_materials.push_back( binding );
}
WriteLog(
"Terrain: " + std::to_string( resolved ) + " of " + std::to_string( names.size() )
+ " materials resolved to textures" );
if( false == unresolved.empty() ) {
WriteLog( "Terrain: no texture for " + unresolved );
}
}
// residency settles within a second of the camera stopping and then barely moves, so a
// line is written when it shifts by a noticeable amount rather than every frame. that
// makes the warm figure - what the terrain actually costs - readable from the log
void
terrain_clipmap::report_residency() {
auto const difference {
m_stats.resident > m_reportedresidency ?
m_stats.resident - m_reportedresidency :
m_reportedresidency - m_stats.resident };
if( ( m_reported == true ) && ( difference < 16 ) ) { return; }
m_reported = true;
m_reportedresidency = m_stats.resident;
WriteLog(
"Terrain: " + std::to_string( m_stats.resident ) + " tiles resident, "
+ std::to_string( m_stats.texturebytes / 1048576 ) + " MB of samples, "
+ std::to_string( m_stats.uploaded ) + " uploads so far" );
}
void
terrain_clipmap::render( glm::dvec3 const &Viewpoint ) {
if( ( false == m_ready ) || ( true == m_tiles.empty() ) ) { return; }
m_shader->bind();
m_vao->bind();
::glUniform1f( m_uniforms.heightbias, m_reader.height( 0 ) );
::glUniform1f( m_uniforms.heightscale, m_reader.height( 1 ) - m_reader.height( 0 ) );
::glUniform1ui( m_uniforms.nodata, heightfield::nodata );
::glUniform1i( m_uniforms.heights, 0 );
::glUniform1i( m_uniforms.materials, 1 );
::glUniform1i( m_uniforms.groundtexture, 2 );
::glUniform3fv(
m_uniforms.palette, static_cast<GLsizei>( m_palette.size() ), &m_palette.front().x );
auto const tilesize { static_cast<double>( m_reader.tilesize() ) };
auto const gridstep { m_reader.gridstep() };
for( auto const & [ tilekey, tile ] : m_tiles ) {
if( tile.lastused != m_frame ) { continue; }
// tile position relative to the viewpoint, so that the shader never sees a
// coordinate large enough for single precision to matter
glm::vec3 const origin {
static_cast<float>( tile.x * tilesize - Viewpoint.x ),
static_cast<float>( -Viewpoint.y ) - m_depthbias,
static_cast<float>( tile.z * tilesize - Viewpoint.z ) };
::glUniform3fv( m_uniforms.tileorigin, 1, &origin.x );
::glUniform1f( m_uniforms.samplestep, gridstep * static_cast<float>( 1u << tile.level ) );
::glUniform1i( m_uniforms.side, static_cast<GLint>( tile.side ) );
::glUniform2f(
m_uniforms.tileworld,
static_cast<float>( tile.x * tilesize ), static_cast<float>( tile.z * tilesize ) );
::glActiveTexture( GL_TEXTURE0 );
::glBindTexture( GL_TEXTURE_2D, tile.heighttexture );
::glActiveTexture( GL_TEXTURE1 );
::glBindTexture( GL_TEXTURE_2D, tile.materialtexture );
m_indices[ tile.level ]->bind( gl::buffer::ELEMENT_ARRAY_BUFFER );
// one pass per material in the tile: each wants a different texture bound, and
// a fragment belonging to any other material is dropped
for( auto const material : tile.materials ) {
auto const &binding {
material < m_materials.size() ? m_materials[ material ] : material_binding {} };
GfxRenderer->Bind_Texture( 2, binding.texture );
::glUniform1f( m_uniforms.groundscale, binding.scale );
::glUniform1ui( m_uniforms.drawnmaterial, material );
::glUniform1i( m_uniforms.hastexture, binding.texture != null_handle ? 1 : 0 );
auto const &fallback {
m_palette[ std::min<std::size_t>( material, m_palette.size() - 1 ) ] };
::glUniform3fv( m_uniforms.fallback, 1, &fallback.x );
::glDrawElements(
GL_TRIANGLES, static_cast<GLsizei>( m_indexcounts[ tile.level ] ), GL_UNSIGNED_INT, nullptr );
}
}
::glActiveTexture( GL_TEXTURE0 );
m_vao->unbind();
gl::program::bind( 0 );
}

View File

@@ -0,0 +1,179 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <array>
#include <cstdint>
#include <memory>
#include <optional>
#include <string>
#include <unordered_map>
#include <vector>
#include "global_include/interfaces/ITexture_macros.h"
#include "scene/heightfieldreader.h"
export module eu07.rendering.terrainclipmap;
import eu07.gl.shader;
import eu07.gl.buffer;
import eu07.gl.vao;
import eu07.glm;
import eu07.rendering.terraintileloader;
import eu07.rendering.terrainstatus;
import eu07.model.material;
import eu07.global_include.interfaces.itexture;
export {
// Draws the cooked terrain heightfield.
//
// Not a toroidal clipmap, despite what the plan called it. The measurements say the
// terrain is a ribbon: 390 km2 of surface inside a 51 x 46 km rectangle, twelve percent
// of the tile bounding box occupied. Clipmap rings sized for that rectangle would be
// mostly empty, so this draws the cooked tiles themselves, each at the mip level its
// distance earns. The tiles were laid out for exactly this - sorted by morton code, one
// shared sample of overlap so neighbouring levels meet without skirts.
//
// The class owns its gl resources and the file handle, and knows nothing about the
// scenery, the camera class or the rest of the renderer: it is given a viewpoint and a
// pair of matrices, and draws.
class terrain_clipmap {
public:
terrain_clipmap() = default;
~terrain_clipmap();
terrain_clipmap( terrain_clipmap const & ) = delete;
terrain_clipmap &operator=( terrain_clipmap const & ) = delete;
// opens a cooked heightfield and builds the gl resources. returns false when the
// file cannot be read, leaving the object unusable but safe to destroy
bool open( std::string const &Path );
bool ready() const { return m_ready; }
void close();
// asks for the tiles around the viewpoint, uploads a bounded number of those that have
// arrived, and retires the ones that left range. safe to call every frame: the reading
// and decoding happen on the loader's thread, and a tile waiting for a finer level keeps
// being drawn at the level it has
void update( glm::dvec3 const &Viewpoint );
// draws the resident tiles. the caller is expected to have the scene and model
// uniform blocks already set for a camera-relative pass, the way the rest of the
// renderer does: tile positions reach the shader relative to the viewpoint, so
// nothing here needs a matrix of its own
void render( glm::dvec3 const &Viewpoint );
// how far from the viewpoint terrain is drawn, in metres
void range( float const Range ) { m_range = Range; }
float range() const { return m_range; }
// metres between samples at the finest level; a scenery's fields are drawn from the
// finest to the coarsest so that the depth test keeps the better surface
float gridstep() const { return m_reader.gridstep(); }
// sinks this field by the given amount, so a coarse field cannot win fragments from
// a finer one describing the same ground
void depthbias( float const Metres ) { m_depthbias = Metres; }
terrain_statistics const &stats() const { return m_stats; }
std::string const &path() const { return m_path; }
float tilesize() const { return static_cast<float>( m_reader.tilesize() ); }
std::uint32_t levels() const { return m_reader.levels(); }
std::size_t backlog() const { return m_loader.backlog(); }
private:
// one tile resident on the gpu. heights are unsigned so that the no-data value stays
// exactly what the cooker wrote instead of being normalised into a float
struct resident_tile {
std::int32_t x { 0 }, z { 0 };
std::uint32_t level { 0 }; // mip level currently uploaded
std::uint32_t side { 0 }; // samples per side at that level
std::uint32_t heighttexture { 0 };
std::uint32_t materialtexture { 0 };
float minheight { 0.f }, maxheight { 0.f };
std::uint64_t lastused { 0 };
// materials appearing in this tile. a tile is drawn once per entry, since each
// needs its own texture bound, and most tiles carry only one or two
std::vector<std::uint8_t> materials;
};
// mip level a tile at the given distance is drawn at
std::uint32_t level_for( double const Distance ) const;
// works out which tiles are in range and at what level, and tells the loader what is
// missing. the answer only changes once the camera has moved a fair part of a tile
void scan( glm::dvec3 const &Viewpoint );
// puts a tile the loader has read on the gpu, replacing whatever level it had before
void upload( terrain_tile_loader::payload const &Tile );
void retire( resident_tile &Tile );
void build_indices();
void build_palette();
// resolves the cooked material names to the engine's materials, once
void resolve_materials();
void resolve_uniforms();
void report_residency();
static std::int64_t key( std::int32_t const X, std::int32_t const Z ) { return heightfield::tile_key( X, Z ); }
// for the file's layout only: tile coordinates, levels, materials. tiles themselves are
// read by the loader, on its own handle
heightfield::reader m_reader;
terrain_tile_loader m_loader;
std::vector<terrain_tile_loader::request> m_wanted;
std::vector<terrain_tile_loader::payload> m_arrived;
// one tile within range, and the level its distance earns
struct in_range {
std::int64_t key;
std::uint32_t level;
};
// what the last scan found in range, and the level wanted for each tile, to tell a useful
// arrival from one the camera has already moved past
std::vector<in_range> m_inrange;
std::unordered_map<std::int64_t, std::uint32_t> m_wantedlevel;
glm::dvec3 m_scanpoint { 0.0 };
float m_scanrange { -1.f };
std::unordered_map<std::int64_t, resident_tile> m_tiles;
// one index buffer per mip level, over a grid of that level's sample count. vertices
// are computed in the shader from gl_VertexID, so there is no vertex buffer at all.
// held by pointer because a gl::buffer owns a name it cannot hand to a copy
std::vector<std::unique_ptr<gl::buffer>> m_indices;
std::vector<std::uint32_t> m_indexcounts;
std::optional<gl::vao> m_vao;
std::optional<gl::program> m_shader;
// resolved once at link time; -1 for a uniform the compiler dropped, which gl ignores
struct uniform_locations {
int tileorigin { -1 }, samplestep { -1 };
int heightbias { -1 }, heightscale { -1 }, nodata { -1 }, side { -1 };
int heights { -1 }, materials { -1 }, palette { -1 };
int groundtexture { -1 }, groundscale { -1 }, drawnmaterial { -1 }, tileworld { -1 };
int hastexture { -1 }, fallback { -1 };
} m_uniforms;
std::vector<glm::vec3> m_palette;
// one entry per cooked material: the engine texture to bind and how much ground one
// repeat of it covers
struct material_binding {
texture_handle texture { null_handle };
float scale { 4.f };
};
std::vector<material_binding> m_materials;
terrain_statistics m_stats;
std::uint64_t m_frame { 0 };
std::size_t m_reportedresidency { 0 };
bool m_reported { false };
float m_range { 6000.f };
float m_depthbias { 0.f };
// tiles allowed to stay resident; at 129 samples a tile costs about 50 kB of texture
std::size_t m_budget { 2048 };
// tiles put on the gpu per frame. a tile is about 50 kB of texture, so this is not about
// the upload itself but about never spending a frame on terrain the camera will reach
// later anyway
std::size_t m_uploadsperframe { 32 };
bool m_ready { false };
std::string m_path;
};
} // export

View File

@@ -0,0 +1,57 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <array>
#include <cmath>
#include <cstddef>
#include <cstdint>
#include <string>
#include <vector>
export module eu07.rendering.terrainstatus;
export {
// distance out to which a terrain tile is drawn at its finest level; each further level
// doubles it
inline constexpr double terrain_finest_range { 512.0 };
// the distance up to which a level is used
inline double
terrain_level_distance( std::uint32_t const Level ) {
return terrain_finest_range * std::pow( 2.0, Level );
}
// what one terrain field's streamer is doing, kept by the field and read by the debug panel
struct terrain_statistics {
std::size_t resident { 0 }; // tiles holding gl textures
std::size_t inview { 0 }; // resident tiles within range in the last update
std::size_t wanted { 0 }; // tiles in range missing, or at the wrong level
std::size_t uploaded { 0 }; // uploads since opening
std::size_t dropped { 0 }; // arrivals discarded as stale since opening
std::size_t texturebytes { 0 };
std::array<std::size_t, 8> perlevel {}; // tiles in view at each level
};
// one field as the debug panel shows it. the renderer writes it once a frame from the main
// view, the panel reads it; neither needs to know the other's types
struct terrain_field_status {
std::string path;
float gridstep { 0.f };
float tilesize { 0.f };
float range { 0.f };
std::uint32_t levels { 0 };
std::size_t backlog { 0 }; // tiles the loader has queued or finished and not handed over
terrain_statistics stats;
};
inline std::vector<terrain_field_status> TerrainStatus;
}

View File

@@ -0,0 +1,129 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <algorithm>
#include <cstddef>
#include <mutex>
#include <string>
#include <thread>
#include <utility>
#include <vector>
#include "scene/heightfieldreader.h"
module eu07.rendering.terraintileloader;
terrain_tile_loader::~terrain_tile_loader() {
close();
}
bool
terrain_tile_loader::open( std::string const &Path ) {
close();
if( false == m_reader.open( Path ) ) { return false; }
m_stop = false;
m_thread = std::thread( &terrain_tile_loader::work, this );
return true;
}
void
terrain_tile_loader::close() {
if( true == m_thread.joinable() ) {
{
std::lock_guard<std::mutex> lock( m_mutex );
m_stop = true;
}
m_wake.notify_all();
m_thread.join();
}
std::lock_guard<std::mutex> lock( m_mutex );
m_pending.clear();
m_done.clear();
m_running.reset();
m_reader.close();
}
bool
terrain_tile_loader::underway( request const &Tile ) const {
if( ( m_running.has_value() ) && ( *m_running == Tile ) ) { return true; }
return std::any_of(
m_done.begin(), m_done.end(),
[ &Tile ]( payload const &Done ) { return Done.tile == Tile; } );
}
void
terrain_tile_loader::want( std::vector<request> const &Requests ) {
{
std::lock_guard<std::mutex> lock( m_mutex );
m_pending.clear();
for( auto const &tile : Requests ) {
if( false == underway( tile ) ) {
m_pending.push_back( tile );
}
}
}
m_wake.notify_one();
}
std::size_t
terrain_tile_loader::collect( std::vector<payload> &Out, std::size_t const Count ) {
std::unique_lock<std::mutex> lock( m_mutex );
std::size_t moved { 0 };
while( ( moved < Count ) && ( false == m_done.empty() ) ) {
Out.emplace_back( std::move( m_done.front() ) );
m_done.pop_front();
++moved;
}
lock.unlock();
if( moved > 0 ) {
// room was made; the thread may be waiting for it
m_wake.notify_one();
}
return moved;
}
std::size_t
terrain_tile_loader::backlog() const {
std::lock_guard<std::mutex> lock( m_mutex );
return m_pending.size() + m_done.size() + ( m_running.has_value() ? 1 : 0 );
}
void
terrain_tile_loader::work() {
while( true ) {
request tile;
{
std::unique_lock<std::mutex> lock( m_mutex );
m_wake.wait( lock, [ this ]() {
return m_stop || ( ( false == m_pending.empty() ) && ( m_done.size() < donelimit ) ); } );
if( true == m_stop ) { return; }
tile = m_pending.front();
m_pending.pop_front();
m_running = tile;
}
// the read itself happens outside the lock, so the render thread can replace the
// wish list while a tile is being decoded
payload result;
result.tile = tile;
result.valid = m_reader.read_level( tile.x, tile.z, tile.level, result.heights, result.materials );
std::lock_guard<std::mutex> lock( m_mutex );
m_running.reset();
m_done.emplace_back( std::move( result ) );
}
}

View File

@@ -0,0 +1,90 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <condition_variable>
#include <cstddef>
#include <cstdint>
#include <deque>
#include <mutex>
#include <optional>
#include <string>
#include <thread>
#include <vector>
#include "scene/heightfieldreader.h"
export module eu07.rendering.terraintileloader;
export {
// Reads and decodes heightfield tiles on a thread of its own.
//
// Reading a tile means a seek, a read and a zstd decode, which is too much to do on the render
// thread for the dozens of tiles a moving camera asks for at once. This does that part and
// nothing else: it knows no gl, keeps its own file handle, and hands back plain sample arrays
// for the renderer to upload at a pace it chooses.
//
// The renderer says each frame what it wants, nearest first, and that list replaces whatever
// was asked for before and has not been started. A tile the camera has already left is never
// read, and the nearest tile is always the next one read.
class terrain_tile_loader {
public:
struct request {
std::int32_t x { 0 }, z { 0 };
std::uint32_t level { 0 };
bool operator==( request const & ) const = default;
};
struct payload {
request tile;
bool valid { false };
std::vector<std::uint16_t> heights;
std::vector<std::uint8_t> materials;
};
terrain_tile_loader() = default;
~terrain_tile_loader();
terrain_tile_loader( terrain_tile_loader const & ) = delete;
terrain_tile_loader &operator=( terrain_tile_loader const & ) = delete;
// opens the file on the loader's own handle and starts the thread
bool open( std::string const &Path );
// stops the thread; anything not collected is dropped
void close();
// replaces the list of wanted tiles, nearest first. tiles being read or already read and
// not yet collected are left out of it, so asking again every frame costs nothing
void want( std::vector<request> const &Requests );
// moves up to Count finished tiles into Out, returning how many
std::size_t collect( std::vector<payload> &Out, std::size_t const Count );
// tiles queued or finished and not yet collected
std::size_t backlog() const;
private:
void work();
// whether the tile is being read or waits to be collected. caller holds the lock
bool underway( request const &Tile ) const;
heightfield::reader m_reader;
std::thread m_thread;
mutable std::mutex m_mutex;
std::condition_variable m_wake;
std::deque<request> m_pending;
std::optional<request> m_running;
std::deque<payload> m_done;
bool m_stop { false };
// finished tiles allowed to wait for collection. past this the thread stops reading:
// the renderer uploads at its own pace, and a backlog read for where the camera was a
// second ago is work spent on tiles nobody wants any more
static constexpr std::size_t donelimit { 32 };
};
}

167
scene/heightfieldformat.h Normal file
View File

@@ -0,0 +1,167 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#pragma once
// On-disk layout of the cooked terrain heightfield. Single source of truth: the cooker
// (tools/terraincook) writes it, the engine reads it, and neither owns a private copy.
// Plain structs and free functions only - no engine types, no allocation, no i/o - so
// that the tool keeps building without the engine and the format stays readable on its own.
//
// File layout:
// header
// tile directory, one entry per tile, ordered by morton code
// material table, each entry a uint16 length, that many name bytes, and a float
// giving how many metres of ground one repeat of the texture covers
// tile payloads, at the offsets the directory gives
//
// A tile payload holds a mip chain, finest level first. Level l has
// samples_at_level( tilesamples, l ) values along a side, laid out as that many rows of
// uint16 heights, followed by the same grid of uint8 material indices.
#include <cstdint>
namespace heightfield {
// "EU07HFLD", spelled out so the header needs no string functions
inline constexpr char magic[ 8 ] { 'E', 'U', '0', '7', 'H', 'F', 'L', 'D' };
inline constexpr std::uint32_t version { 5 };
// terrain selection rules, stamped into every cooked file.
// 0 - every triangle node the scenery holds
// 1 - world space triangle nodes only: anything under origin, rotate or scale is an object
// 2 - every triangle as drawn; a triangle stays drawn unless the heightfield shows it, and a
// triangle bordering a hole always stays drawn
// 3 - as 2, with a hole checked for diagonally and at the coarser levels too
inline constexpr std::uint32_t selection_rules { 3 };
// height value reserved to mean "this sample carries no terrain"; a cooked map is a
// ribbon inside a much larger rectangle, so most edge tiles are partly empty
inline constexpr std::uint16_t nodata { 0xffff };
// the range the remaining values span, shared by every tile through the header's bias
// and scale. over l204's 107 m of relief this puts a sample within 2 mm of the source
inline constexpr std::uint32_t heightrange { 0xfffe };
enum class codec : std::uint16_t {
raw = 0,
zstd = 1
};
#pragma pack( push, 1 )
struct file_header {
char magic[ 8 ];
std::uint32_t version;
// samples along a tile side, not counting the shared edge. a tile therefore stores
// one more sample in each direction than it owns
std::uint32_t tilesamples;
float gridstep; // metres between samples at the finest level
float heightbias; // world height of stored value 0
float heightscale; // metres per stored unit
std::int32_t minx, minz, maxx, maxz; // tile coordinates present in the file, inclusive
std::uint32_t tilecount;
std::uint32_t materialcount;
std::uint32_t miplevels; // levels stored per tile, level 0 being the finest
// which rules decided what counted as terrain when this was cooked. the layout does
// not depend on it, so a file with an older value still reads; a heightfield the
// engine baked by itself is rebaked when it disagrees, one named by the scenery is not
std::uint32_t selection;
std::uint32_t reserved;
};
struct tile_entry {
std::int32_t x, z;
std::uint64_t offset; // payload position in the file
std::uint32_t storedsize; // bytes occupied in the file
std::uint32_t plainsize; // bytes once decoded; equals storedsize for codec::raw
float minheight, maxheight;
std::uint32_t coverage; // samples carrying terrain at the finest level
std::uint16_t tilecodec; // codec enumerator
std::uint16_t reserved;
};
#pragma pack( pop )
static_assert( sizeof( file_header ) == 64 );
static_assert( sizeof( tile_entry ) == 40 );
// identity of a tile in any map keyed by tile coordinates. the cooker, the reader and the
// renderer all key tiles this way, so it is spelled out once
constexpr std::int64_t
tile_key( std::int32_t const X, std::int32_t const Z ) {
return ( static_cast<std::int64_t>( X ) << 32 ) ^ static_cast<std::uint32_t>( Z );
}
// integer division rounding towards negative infinity. tile and sample coordinates go
// negative, and every place that maps one onto the other has to round the same way for
// neighbouring tiles to share an edge exactly
constexpr std::int64_t
floor_div( std::int64_t const Value, std::int64_t const Divisor ) {
auto const quotient { Value / Divisor };
return ( ( Value % Divisor != 0 ) && ( ( Value < 0 ) != ( Divisor < 0 ) ) ) ? quotient - 1 : quotient;
}
// samples along a tile side at the given mip level, shared edge included
constexpr std::uint32_t
samples_at_level( std::uint32_t const Tilesamples, std::uint32_t const Level ) {
return ( Tilesamples >> Level ) + 1;
}
// bytes one mip level occupies: a grid of heights followed by a grid of material indices
constexpr std::uint64_t
level_bytes( std::uint32_t const Tilesamples, std::uint32_t const Level ) {
std::uint64_t const side { samples_at_level( Tilesamples, Level ) };
return side * side * ( sizeof( std::uint16_t ) + sizeof( std::uint8_t ) );
}
// bytes a decoded tile payload occupies, mip chain included
constexpr std::uint64_t
tile_bytes( std::uint32_t const Tilesamples, std::uint32_t const Miplevels ) {
std::uint64_t total { 0 };
for( std::uint32_t level = 0; level < Miplevels; ++level ) {
total += level_bytes( Tilesamples, level );
}
return total;
}
// byte offset of a mip level inside a decoded tile payload
constexpr std::uint64_t
level_offset( std::uint32_t const Tilesamples, std::uint32_t const Level ) {
return tile_bytes( Tilesamples, Level );
}
// deepest level that still has at least Minimumside samples along a side
constexpr std::uint32_t
level_count( std::uint32_t const Tilesamples, std::uint32_t const Minimumside ) {
std::uint32_t levels { 1 };
while( ( ( Tilesamples >> levels ) >= 1 )
&& ( samples_at_level( Tilesamples, levels ) >= Minimumside ) ) {
++levels;
}
return levels;
}
// interleaves tile coordinates so that tiles sorted by this value are read in an order
// that keeps a moving camera reading the file forwards rather than seeking
constexpr std::uint64_t
morton( std::int32_t const X, std::int32_t const Z ) {
auto const spread = []( std::uint32_t Value ) constexpr -> std::uint64_t {
std::uint64_t result { Value };
result = ( result | ( result << 16 ) ) & 0x0000ffff0000ffffull;
result = ( result | ( result << 8 ) ) & 0x00ff00ff00ff00ffull;
result = ( result | ( result << 4 ) ) & 0x0f0f0f0f0f0f0f0full;
result = ( result | ( result << 2 ) ) & 0x3333333333333333ull;
result = ( result | ( result << 1 ) ) & 0x5555555555555555ull;
return result; };
// biased into unsigned so that negative tile coordinates keep their relative order
std::uint32_t const x { static_cast<std::uint32_t>( X + ( 1 << 20 ) ) };
std::uint32_t const z { static_cast<std::uint32_t>( Z + ( 1 << 20 ) ) };
return spread( x ) | ( spread( z ) << 1 );
}
} // namespace heightfield

237
scene/heightfieldreader.h Normal file
View File

@@ -0,0 +1,237 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#pragma once
// Reader for the cooked terrain heightfield. Header-only and free of engine types, so
// that the cooker can verify its own output with the same code the engine runs - one
// reader, not two that drift.
//
// The class owns the file handle and the directory; sample data it decodes into buffers
// the caller provides and recycles. Nothing about the on-disk layout escapes past
// read_level(): callers see tile coordinates, mip levels and samples.
#include <cmath>
#include <cstdint>
#include <cstring>
#include <fstream>
#include <string>
#include <unordered_map>
#include <vector>
#include <zstd.h>
#include "scene/heightfieldformat.h"
namespace heightfield {
class reader {
public:
struct tile_description {
std::int32_t x { 0 }, z { 0 };
float minheight { 0.f }, maxheight { 0.f };
std::uint32_t coverage { 0 }; // samples carrying terrain at the finest level
};
reader() = default;
~reader() { ZSTD_freeDCtx( m_context ); }
reader( reader const & ) = delete;
reader &operator=( reader const & ) = delete;
// returns false and leaves the reader closed when the file is missing, truncated,
// or written by a version this build does not know
bool
open( std::string const &Path ) {
close();
m_file.open( Path, std::ios::binary );
if( false == m_file.good() ) { return false; }
if( false == read_at( 0, &m_header, sizeof( m_header ) ) ) { return close_failed(); }
if( 0 != std::memcmp( m_header.magic, magic, sizeof( magic ) ) ) { return close_failed(); }
if( m_header.version != version ) { return close_failed(); }
if( ( m_header.tilesamples == 0 ) || ( m_header.miplevels == 0 ) ) { return close_failed(); }
std::vector<tile_entry> directory( m_header.tilecount );
if( false == read_at(
sizeof( file_header ), directory.data(),
directory.size() * sizeof( tile_entry ) ) ) { return close_failed(); }
m_entries.reserve( directory.size() );
m_index.reserve( directory.size() );
m_descriptions.reserve( directory.size() );
for( auto const &entry : directory ) {
m_index.emplace( tile_key( entry.x, entry.z ), m_entries.size() );
m_entries.push_back( entry );
m_descriptions.push_back( { entry.x, entry.z, entry.minheight, entry.maxheight, entry.coverage } );
}
auto offset { sizeof( file_header ) + directory.size() * sizeof( tile_entry ) };
m_materials.reserve( m_header.materialcount );
for( std::uint32_t index = 0; index < m_header.materialcount; ++index ) {
std::uint16_t length { 0 };
if( false == read_at( offset, &length, sizeof( length ) ) ) { return close_failed(); }
offset += sizeof( length );
std::string name( length, '\0' );
if( ( length > 0 ) && ( false == read_at( offset, name.data(), length ) ) ) { return close_failed(); }
offset += length;
float scale { 4.f };
if( false == read_at( offset, &scale, sizeof( scale ) ) ) { return close_failed(); }
offset += sizeof( scale );
m_materials.push_back( std::move( name ) );
m_materialscales.push_back( scale );
}
m_open = true;
return true;
}
void
close() {
m_file.close();
m_file.clear();
m_entries.clear();
m_descriptions.clear();
m_index.clear();
m_materials.clear();
m_materialscales.clear();
m_header = {};
m_open = false; }
bool ready() const { return m_open; }
// metres between samples at the finest level
float gridstep() const { return m_header.gridstep; }
// samples a tile owns along a side, shared edge excluded
std::uint32_t tilesamples() const { return m_header.tilesamples; }
// side of a tile in metres
float tilesize() const { return m_header.gridstep * static_cast<float>( m_header.tilesamples ); }
std::uint32_t levels() const { return m_header.miplevels; }
std::uint32_t samples_per_side( std::uint32_t const Level ) const {
return samples_at_level( m_header.tilesamples, Level ); }
std::vector<tile_description> const &tiles() const { return m_descriptions; }
std::vector<std::string> const &materials() const { return m_materials; }
// metres of ground covered by one repeat of a material's texture, measured by the
// cooker from the texture coordinates the source geometry carried
std::vector<float> const &material_scales() const { return m_materialscales; }
// tile coordinates covering a world position
std::int32_t tile_x( double const X ) const { return tile_of( X ); }
std::int32_t tile_z( double const Z ) const { return tile_of( Z ); }
// the terrain selection rules the file was cooked under
std::uint32_t
selection() const { return m_header.selection; }
bool
contains( std::int32_t const X, std::int32_t const Z ) const {
return m_index.find( tile_key( X, Z ) ) != m_index.end(); }
// description of one tile, or nullptr when the file does not hold it
tile_description const *
find( std::int32_t const X, std::int32_t const Z ) const {
auto const lookup { m_index.find( tile_key( X, Z ) ) };
return lookup == m_index.end() ? nullptr : &m_descriptions[ lookup->second ]; }
// world height of a stored sample; Value must not be nodata
float
height( std::uint16_t const Value ) const {
return m_header.heightbias + static_cast<float>( Value ) * m_header.heightscale; }
// decodes one mip level of one tile into the caller's buffers, which are resized to
// the level's sample count. returns false when the tile is absent or the payload is
// damaged; the buffers are then left untouched
bool
read_level(
std::int32_t const X, std::int32_t const Z, std::uint32_t const Level,
std::vector<std::uint16_t> &Heights, std::vector<std::uint8_t> &Materials ) {
if( false == m_open ) { return false; }
if( Level >= m_header.miplevels ) { return false; }
auto const lookup { m_index.find( tile_key( X, Z ) ) };
if( lookup == m_index.end() ) { return false; }
auto const &entry { m_entries[ lookup->second ] };
if( false == decode( entry ) ) { return false; }
auto const side { samples_at_level( m_header.tilesamples, Level ) };
auto const samples { static_cast<std::size_t>( side ) * side };
auto const offset { static_cast<std::size_t>( level_offset( m_header.tilesamples, Level ) ) };
if( offset + samples * 3u > m_plain.size() ) { return false; }
Heights.resize( samples );
Materials.resize( samples );
std::memcpy( Heights.data(), m_plain.data() + offset, samples * sizeof( std::uint16_t ) );
std::memcpy( Materials.data(), m_plain.data() + offset + samples * sizeof( std::uint16_t ), samples );
return true;
}
private:
std::int32_t
tile_of( double const World ) const {
return static_cast<std::int32_t>( std::floor( World / static_cast<double>( tilesize() ) ) ); }
bool
read_at( std::uint64_t const Offset, void *Target, std::size_t const Bytes ) {
if( Bytes == 0 ) { return true; }
m_file.seekg( static_cast<std::streamoff>( Offset ), std::ios::beg );
m_file.read( static_cast<char *>( Target ), static_cast<std::streamsize>( Bytes ) );
return m_file.good(); }
bool
close_failed() {
close();
return false; }
// keeps the most recently decoded payload, because a caller walking the mip chain of
// one tile would otherwise decompress it once per level
bool
decode( tile_entry const &Entry ) {
if( ( true == m_decoded ) && ( m_decodedoffset == Entry.offset ) ) { return true; }
m_decoded = false;
m_stored.resize( Entry.storedsize );
if( false == read_at( Entry.offset, m_stored.data(), Entry.storedsize ) ) { return false; }
if( static_cast<codec>( Entry.tilecodec ) == codec::raw ) {
m_plain.swap( m_stored );
}
else {
m_plain.resize( Entry.plainsize );
// one context for the life of the reader: creating one per tile costs more than
// decoding a small tile does
if( m_context == nullptr ) { m_context = ZSTD_createDCtx(); }
auto const written {
ZSTD_decompressDCtx( m_context, m_plain.data(), m_plain.size(), m_stored.data(), m_stored.size() ) };
if( ( 0 != ZSTD_isError( written ) ) || ( written != Entry.plainsize ) ) { return false; }
}
m_decoded = true;
m_decodedoffset = Entry.offset;
return true;
}
std::ifstream m_file;
ZSTD_DCtx *m_context { nullptr };
file_header m_header {};
std::vector<tile_entry> m_entries;
std::vector<tile_description> m_descriptions;
std::unordered_map<std::int64_t, std::size_t> m_index;
std::vector<std::string> m_materials;
std::vector<float> m_materialscales;
std::vector<std::uint8_t> m_stored, m_plain;
std::uint64_t m_decodedoffset { 0 };
bool m_decoded { false };
bool m_open { false };
};
} // namespace heightfield

283
scene/logiccontainer.cpp Normal file
View File

@@ -0,0 +1,283 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <array>
#include <cstdint>
#include <cstring>
#include <fstream>
#include <string>
#include <string_view>
#include <vector>
#include "scene/logicformat.h"
module eu07.scene.logiccontainer;
import eu07.utilities.logs;
import eu07.utilities.utilities;
namespace scene::logic {
namespace {
// pads the stream out to the next section boundary with zeroes, so a cooked file has no
// uninitialised bytes in it and two cooks of the same scenery compare equal
void
pad( std::ofstream &Output ) {
std::array<char, alignment> zeroes {};
auto const position { static_cast<std::size_t>( Output.tellp() ) };
auto const padding { aligned( position ) - position };
if( padding > 0 ) {
Output.write( zeroes.data(), static_cast<std::streamsize>( padding ) );
}
}
template <typename Type_>
void
write_array( std::ofstream &Output, std::vector<Type_> const &Data ) {
if( false == Data.empty() ) {
Output.write( reinterpret_cast<char const *>( Data.data() ), static_cast<std::streamsize>( Data.size() * sizeof( Type_ ) ) );
}
}
template <typename Type_>
bool
read_array( std::ifstream &Input, section_entry const &Section, std::vector<Type_> &Data ) {
if( Section.elementsize != sizeof( Type_ ) ) { return false; }
if( Section.bytes != Section.count * sizeof( Type_ ) ) { return false; }
Data.resize( Section.count );
Input.seekg( static_cast<std::streamoff>( Section.offset ) );
if( Section.count > 0 ) {
Input.read( reinterpret_cast<char *>( Data.data() ), static_cast<std::streamsize>( Section.bytes ) );
}
return static_cast<bool>( Input );
}
} // anonymous namespace
std::uint32_t
container_writer::intern( std::string const &Text ) {
auto const lookup { m_interned.find( Text ) };
if( lookup != m_interned.end() ) {
return lookup->second;
}
auto const index { static_cast<std::uint32_t>( m_strings.size() ) };
m_strings.emplace_back( Text );
m_interned.emplace( Text, index );
return index;
}
void
container_writer::add_event( event_record Record, std::vector<target_entry> const &Targets ) {
Record.targetfirst = static_cast<std::uint32_t>( m_targets.size() );
Record.targetcount = static_cast<std::uint32_t>( Targets.size() );
m_targets.insert( m_targets.end(), Targets.begin(), Targets.end() );
m_events.emplace_back( Record );
}
bool
container_writer::write( std::string const &Path ) const {
std::ofstream output( Path, std::ios::binary | std::ios::trunc );
if( false == output.good() ) {
ErrorLog( "Logic container: cannot write \"" + Path + "\"" );
return false;
}
// the strings blob and its table are built first, since the header has to know how big
// every section is before any of them is written
std::vector<char> blob;
std::vector<string_entry> table;
table.reserve( m_strings.size() );
for( auto const &text : m_strings ) {
table.emplace_back( string_entry{ static_cast<std::uint32_t>( blob.size() ), static_cast<std::uint32_t>( text.size() ) } );
blob.insert( blob.end(), text.begin(), text.end() );
}
std::array<section_entry, static_cast<std::size_t>( section::count )> sections {};
auto offset { aligned( sizeof( file_header ) + sizeof( sections ) ) };
auto const place {
[ &offset, &sections ]( section const Kind, std::size_t const Elementsize, std::size_t const Count ) {
auto &entry { sections[ static_cast<std::size_t>( Kind ) ] };
entry.kind = static_cast<std::uint32_t>( Kind );
entry.elementsize = static_cast<std::uint32_t>( Elementsize );
entry.count = Count;
entry.bytes = Elementsize * Count;
entry.offset = offset;
offset = aligned( offset + entry.bytes );
} };
place( section::strings, 1, blob.size() );
place( section::stringtable, sizeof( string_entry ), table.size() );
place( section::events, sizeof( event_record ), m_events.size() );
place( section::targets, sizeof( target_entry ), m_targets.size() );
file_header header {};
header.magic = magic;
header.version = version;
header.schema_hash = schema_hash;
header.endian = endian_marker;
header.sectioncount = static_cast<std::uint32_t>( section::count );
header.filesize = offset;
output.write( reinterpret_cast<char const *>( &header ), sizeof( header ) );
output.write( reinterpret_cast<char const *>( sections.data() ), sizeof( sections ) );
pad( output );
write_array( output, blob );
pad( output );
write_array( output, table );
pad( output );
write_array( output, m_events );
pad( output );
write_array( output, m_targets );
pad( output );
if( false == output.good() ) {
ErrorLog( "Logic container: failed while writing \"" + Path + "\"" );
return false;
}
WriteLog(
"Logic container: wrote \"" + Path + "\", "
+ std::to_string( m_events.size() ) + " events, "
+ std::to_string( m_targets.size() ) + " targets, "
+ std::to_string( m_strings.size() ) + " strings, "
+ std::to_string( offset ) + " bytes" );
return true;
}
// reads the header and checks it, and the file's size, against what this build writes. a
// cooked container is a cache, so anything unexpected means fall back to the text, not guess
// at what an older writer meant
static bool
accepted( std::ifstream &Input, std::string const &Path ) {
if( false == Input.good() ) { return false; }
file_header header {};
Input.read( reinterpret_cast<char *>( &header ), sizeof( header ) );
if( false == static_cast<bool>( Input ) ) { return false; }
if( header.magic != magic ) {
ErrorLog( "Logic container \"" + Path + "\" is not one" );
return false;
}
if( ( header.version != version ) || ( header.schema_hash != schema_hash ) ) {
WriteLog( "Logic container \"" + Path + "\" was cooked by a different build, ignoring it" );
return false;
}
if( header.endian != endian_marker ) {
ErrorLog( "Logic container \"" + Path + "\" was cooked on the other endianness" );
return false;
}
if( header.sectioncount != static_cast<std::uint32_t>( section::count ) ) {
WriteLog( "Logic container \"" + Path + "\" holds a different set of sections, ignoring it" );
return false;
}
Input.seekg( 0, std::ios::end );
if( static_cast<std::uint64_t>( Input.tellg() ) != header.filesize ) {
ErrorLog( "Logic container \"" + Path + "\" is truncated" );
return false;
}
return true;
}
bool
container_reader::readable( std::string const &Path ) {
std::ifstream input( Path, std::ios::binary );
return accepted( input, Path );
}
bool
container_reader::open( std::string const &Path ) {
close();
std::ifstream input( Path, std::ios::binary );
if( false == accepted( input, Path ) ) { return false; }
input.seekg( sizeof( file_header ) );
std::array<section_entry, static_cast<std::size_t>( section::count )> sections {};
input.read( reinterpret_cast<char *>( sections.data() ), sizeof( sections ) );
if( false == static_cast<bool>( input ) ) { return false; }
auto const &strings { sections[ static_cast<std::size_t>( section::strings ) ] };
m_strings.resize( strings.count );
input.seekg( static_cast<std::streamoff>( strings.offset ) );
if( strings.count > 0 ) {
input.read( m_strings.data(), static_cast<std::streamsize>( strings.bytes ) );
}
if( ( false == read_array( input, sections[ static_cast<std::size_t>( section::stringtable ) ], m_stringtable ) )
|| ( false == read_array( input, sections[ static_cast<std::size_t>( section::events ) ], m_events ) )
|| ( false == read_array( input, sections[ static_cast<std::size_t>( section::targets ) ], m_targets ) ) ) {
ErrorLog( "Logic container \"" + Path + "\" has a section this build cannot read" );
close();
return false;
}
m_ready = true;
return true;
}
void
container_reader::close() {
m_ready = false;
m_strings.clear();
m_stringtable.clear();
m_events.clear();
m_targets.clear();
}
std::string_view
container_reader::string( std::uint32_t const Index ) const {
if( Index >= m_stringtable.size() ) { return {}; }
auto const &entry { m_stringtable[ Index ] };
return { m_strings.data() + entry.offset, entry.length };
}
std::string
container_path( std::string const &Sceneryfile ) {
return scenery_sidecar( Sceneryfile, ".logic" );
}
event_kind
kind_of( std::string const &Type ) {
if( Type == "updatevalues" ) { return event_kind::updatevalues; }
else if( Type == "addvalues" ) { return event_kind::addvalues; }
else if( Type == "copyvalues" ) { return event_kind::copyvalues; }
else if( Type == "getvalues" ) { return event_kind::getvalues; }
else if( Type == "putvalues" ) { return event_kind::putvalues; }
else if( Type == "whois" ) { return event_kind::whois; }
else if( Type == "logvalues" ) { return event_kind::logvalues; }
else if( Type == "multiple" ) { return event_kind::multiple; }
else if( Type == "switch" ) { return event_kind::switchevent; }
else if( Type == "trackvel" ) { return event_kind::trackvel; }
else if( Type == "sound" ) { return event_kind::sound; }
else if( Type == "texture" ) { return event_kind::texture; }
else if( Type == "animation" ) { return event_kind::animation; }
else if( Type == "lights" ) { return event_kind::lights; }
else if( Type == "voltage" ) { return event_kind::voltage; }
else if( Type == "visible" ) { return event_kind::visible; }
else if( Type == "friction" ) { return event_kind::friction; }
else if( Type == "message" ) { return event_kind::message; }
else if( Type == "lua" ) { return event_kind::lua; }
return event_kind::unknown;
}
} // namespace scene::logic

84
scene/logiccontainer.cppm Normal file
View File

@@ -0,0 +1,84 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <cstdint>
#include <map>
#include <string>
#include <string_view>
#include <vector>
#include "scene/logicformat.h"
export module eu07.scene.logiccontainer;
export {
// Writing and reading the LOGIC container. The layout it produces is described entirely by
// logicformat.h; this is the part that knows how to get bytes in and out of it.
//
// The writer is deterministic: the same scenery cooked twice produces the same file, byte for
// byte. Strings are interned in first-seen order rather than by a hash, records are written in
// the order the manager holds them, and the padding between sections is zeroed. That is worth
// the small cost, because it is what makes a cooked file comparable - a diff that is empty
// proves a refactor changed nothing.
namespace scene::logic {
class container_writer {
public:
// interns a string, returning its index. the same text always gets the same index
std::uint32_t intern( std::string const &Text );
// adds one event and its targets, already resolved to the registry that holds them
void add_event( event_record Record, std::vector<target_entry> const &Targets );
// writes the container. returns false and logs on any failure
bool write( std::string const &Path ) const;
std::size_t events() const { return m_events.size(); }
private:
std::vector<std::string> m_strings;
std::map<std::string, std::uint32_t> m_interned;
std::vector<event_record> m_events;
std::vector<target_entry> m_targets;
};
class container_reader {
public:
// reads the whole container into memory. refuses anything whose magic, version, schema
// hash, endianness or size disagrees with what this build expects
bool open( std::string const &Path );
// whether this build can read the container, judged from its header and size alone
static bool readable( std::string const &Path );
void close();
bool ready() const { return m_ready; }
std::size_t events() const { return m_events.size(); }
event_record const & event( std::size_t const Index ) const { return m_events[ Index ]; }
std::size_t targets() const { return m_targets.size(); }
target_entry const & target( std::size_t const Index ) const { return m_targets[ Index ]; }
// a view into the string blob, valid while the reader is open
std::string_view string( std::uint32_t const Index ) const;
private:
bool m_ready { false };
std::vector<char> m_strings;
std::vector<string_entry> m_stringtable;
std::vector<event_record> m_events;
std::vector<target_entry> m_targets;
};
// the container's name for a scenery file: alongside it, with the extension replaced
std::string container_path( std::string const &Sceneryfile );
// maps the word an event writes as its type onto the cooked kind
event_kind kind_of( std::string const &Type );
} // namespace scene::logic
}

172
scene/logicformat.h Normal file
View File

@@ -0,0 +1,172 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#pragma once
#include <cstddef>
#include <cstdint>
// On-disk layout of the scenery's LOGIC container. Single source of truth for both the
// cooker and the engine, so it deliberately knows nothing about either: no engine types, no
// includes beyond the fixed-width integers.
//
// The container holds what the simulation needs before it can run - events, and the tables
// that bind them to each other - as flat arrays indexed by position. Nothing here is a
// pointer and nothing is looked up by name at load: an event's targets are a range in a
// shared array, and a sibling is an index. That is the whole point of the section. The text
// path resolves those names in init_targets() every time a scenery loads, and it is the part
// of the load that cannot be made cheaper while the names are still there.
//
// Refusing to load is the only migration story. A container whose magic, version or schema
// hash disagrees with this header is rejected and the scenery falls back to the text path,
// because a cooked file is a cache and can always be rebuilt.
namespace scene::logic {
// "EU07LOGC", little-endian, so a file opened in a hex editor names itself
inline constexpr std::uint64_t magic { 0x43474f4c37305545ull };
inline constexpr std::uint32_t version { 1 };
// stamped into the header and compared on load. bump it whenever a record below changes
// shape, so a stale container is refused rather than misread
inline constexpr std::uint32_t schema_hash { 0x2f74be09u };
// written as 1; a reader seeing 0x01000000 knows it is looking at the wrong endianness
inline constexpr std::uint32_t endian_marker { 1 };
// every section starts on this boundary, so records can be read in place
inline constexpr std::size_t alignment { 16 };
// index meaning "no such element", used where a record refers to another and need not
inline constexpr std::uint32_t no_index { 0xffffffffu };
enum class section : std::uint32_t {
strings = 0, // the string blob every name indexes into
stringtable = 1, // one string_entry per interned string
events = 2, // one event_record per event, in the order the manager holds them
targets = 3, // target_entry array; an event owns a contiguous run of it
count
};
struct file_header {
std::uint64_t magic;
std::uint32_t version;
std::uint32_t schema_hash;
std::uint32_t endian;
std::uint32_t sectioncount;
// total size of the file as written, so a truncated container is caught before it is read
std::uint64_t filesize;
std::uint64_t reserved[ 3 ];
};
struct section_entry {
std::uint32_t kind;
std::uint32_t elementsize;
std::uint64_t offset;
std::uint64_t bytes;
std::uint64_t count;
};
// one interned string: where it starts in the strings blob and how long it is. strings are
// not terminated, so a reader hands out views straight into the mapped buffer
struct string_entry {
std::uint32_t offset;
std::uint32_t length;
};
// what an event does. the values are written to disk, so they are assigned explicitly and
// never reordered - a new kind goes on the end
enum class event_kind : std::uint16_t {
unknown = 0,
updatevalues = 1,
copyvalues = 2,
getvalues = 3,
putvalues = 4,
whois = 5,
logvalues = 6,
multiple = 7,
switchevent = 8, // written "switch" in the scenery, which is a keyword here
trackvel = 9,
sound = 10,
texture = 11,
animation = 12,
lights = 13,
voltage = 14,
visible = 15,
friction = 16,
message = 17,
lua = 18,
// the same class as updatevalues, told apart by a flag the base does not hold. it gets
// its own value rather than being cooked as updatevalues, because until the flag word is
// cooked too that is the only thing keeping the two apart
addvalues = 19,
};
// the part of an event that every kind has. what the kind itself needs beyond this lives in
// the payload range, which the cooker writes and the matching subclass reads back
struct event_record {
std::uint16_t kind;
// the per-kind flag word lives in the subclasses, not in the base, so it is cooked with
// the rest of the payload once the sections for it exist. zero until then
std::uint16_t reserved0;
std::uint32_t name; // index into the string table
std::uint32_t targetfirst; // first entry in the targets section
std::uint32_t targetcount;
std::uint32_t sibling; // index of the next event sharing this name, or no_index
std::uint32_t group; // scene group handle as cooked, or no_index
double delay;
double delayrandom;
// NaN when the event is not tied to a departure. written as the exact bit pattern the
// text path produces, so a round trip compares equal bit for bit
double delaydeparture;
std::uint8_t ignored;
std::uint8_t passive;
std::uint8_t padding[ 6 ];
};
// one target of one event: the name it was written with, and the index of whatever it
// resolved to. the name is kept because the editor exports text again, and because an
// unresolved target has to be reported by the name the author wrote
// which of the engine's node registries a target was found in. an event resolves its
// targets against one or two of them, decided by what kind of event it is - and for a
// visible event, decided per target, since it looks in the instances first and the tracks
// after. the values are written to disk, so they are assigned explicitly
enum class node_table : std::uint16_t {
none = 0,
memory = 1,
instance = 2,
path = 3,
sound = 4,
powergrid = 5,
};
// one target of one event: which registry held it and where in that registry, plus the
// name it was written with.
//
// The index is the target's position in that registry, which is the order the scenery
// built it in - so it is only meaningful for the parse this container was cooked from.
// The name is kept as the check on that: a loader compares it against the node the index
// lands on and falls back to a lookup when they disagree, which is what stops a container
// that went stale without being noticed from binding events to the wrong nodes.
struct target_entry {
std::uint32_t name;
std::uint16_t table;
std::uint16_t reserved;
std::uint32_t index;
};
static_assert( sizeof( file_header ) == 56, "logic file_header changed shape" );
static_assert( sizeof( section_entry ) == 32, "logic section_entry changed shape" );
static_assert( sizeof( string_entry ) == 8, "logic string_entry changed shape" );
static_assert( sizeof( event_record ) == 56, "logic event_record changed shape" );
static_assert( sizeof( target_entry ) == 12, "logic target_entry changed shape" );
static_assert( alignof( event_record ) <= alignment, "logic records must fit the section alignment" );
// where a section's data starts, given what has been written so far
inline constexpr std::size_t
aligned( std::size_t const Offset ) {
return ( Offset + alignment - 1 ) & ~( alignment - 1 );
}
} // namespace scene::logic

278
scene/prefab.cpp Normal file
View File

@@ -0,0 +1,278 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <sstream>
#include <string>
#include <vector>
#include "utilities/Globals_macros.h"
module eu07.scene.prefab;
import eu07.utilities.parser;
import eu07.utilities.logs;
import eu07.utilities.globals;
namespace scene {
namespace {
// a light pattern as a lights event writes it: one value per channel, each preceded by a
// space, channels the given values do not reach set to Fill
std::string
pattern( std::vector<int> const &Lights, std::size_t const Chambers, int const Fill = 0 ) {
std::string result;
for( std::size_t chamber = 0; chamber < Chambers; ++chamber ) {
result += ' ' + std::to_string( chamber < Lights.size() ? Lights[ chamber ] : Fill );
}
return result;
}
void
parse_aspect( cParser &Input, prefab_definition const &Definition, prefab_aspect &Aspect ) {
auto token { Input.getToken<std::string>() };
while( ( false == token.empty() ) && ( token != "endaspect" ) ) {
if( token == "lights" ) {
for( std::size_t chamber = 0; chamber < Definition.chambers; ++chamber ) {
Aspect.lights.emplace_back( Input.getToken<int>() );
}
}
else if( token == "lightdelay" ) {
Aspect.lightdelay = Input.getToken<std::string>();
}
else if( token == "velocity" ) {
Aspect.velocity = Input.getToken<std::string>();
Aspect.velocitynext = Input.getToken<std::string>();
}
else if( token == "animate" ) {
prefab_animation animation;
animation.kind = Input.getToken<std::string>();
animation.submodel = Input.getToken<std::string>( false );
animation.x = Input.getToken<std::string>();
animation.y = Input.getToken<std::string>();
animation.z = Input.getToken<std::string>();
animation.speed = Input.getToken<std::string>();
Aspect.animations.emplace_back( animation );
}
else {
ErrorLog( "Prefab \"" + Definition.name + "\": unexpected \"" + token + "\" in aspect \"" + Aspect.name + "\"" );
}
token = Input.getToken<std::string>();
}
}
} // anonymous namespace
bool
prefab_registry::load( std::string const &Name ) {
auto const path { Global.asCurrentSceneryPath + Name + ".pfb" };
cParser input( path, cParser::buffer_FILE );
if( false == input.ok() ) {
ErrorLog( "Prefab \"" + Name + "\" not found (" + path + ")" );
return false;
}
prefab_definition definition;
definition.name = Name;
input.expectToken( "prefab" );
auto token { input.getToken<std::string>() };
while( ( false == token.empty() ) && ( token != "endprefab" ) ) {
if( token == "model" ) {
definition.model = input.getToken<std::string>( false );
}
else if( token == "modelangle" ) {
definition.modelangle = input.getToken<std::string>();
}
else if( token == "attach" ) {
prefab_attachment attachment;
attachment.rangemax = input.getToken<std::string>();
attachment.rangemin = input.getToken<std::string>();
attachment.model = input.getToken<std::string>( false );
attachment.texture = input.getToken<std::string>( false );
definition.attachments.emplace_back( attachment );
}
else if( token == "chambers" ) {
definition.chambers = input.getToken<int>();
}
else if( token == "initial" ) {
definition.initial = input.getToken<std::string>();
}
else if( token == "initiallights" ) {
for( std::size_t chamber = 0; chamber < definition.chambers; ++chamber ) {
definition.initiallights.emplace_back( input.getToken<int>() );
}
}
else if( token == "sound" ) {
definition.sound = input.getToken<std::string>( false );
}
else if( token == "lightmodes" ) {
definition.lightmodes = true;
}
else if( token == "aspect" ) {
prefab_aspect aspect;
aspect.name = input.getToken<std::string>();
parse_aspect( input, definition, aspect );
definition.aspects.emplace_back( aspect );
}
else {
ErrorLog( "Prefab \"" + Name + "\": unexpected \"" + token + "\"" );
}
token = input.getToken<std::string>();
}
if( definition.chambers == 0 ) {
// the chamber count has to come before the aspects, since it says how many light
// values each of them reads. an aspect list with no lights in it means it did not
ErrorLog( "Prefab \"" + Name + "\" declares no chambers" );
return false;
}
m_definitions.emplace( Name, definition );
WriteLog( "Prefab \"" + Name + "\": " + std::to_string( definition.aspects.size() ) + " aspects, " + std::to_string( definition.chambers ) + " chambers" );
return true;
}
prefab_definition const *
prefab_registry::definition( std::string const &Name ) {
auto const lookup { m_definitions.find( Name ) };
if( lookup != m_definitions.end() ) {
return &( lookup->second );
}
if( true == m_missing[ Name ] ) {
return nullptr;
}
if( false == load( Name ) ) {
m_missing[ Name ] = true;
return nullptr;
}
return &( m_definitions.find( Name )->second );
}
std::string
prefab_registry::expand( std::string const &Name, prefab_placement const &Placement ) {
auto const *definition { this->definition( Name ) };
if( definition == nullptr ) {
return {};
}
auto const &instance { Placement.name };
auto const position {
std::to_string( Placement.x ) + " "
+ std::to_string( Placement.y ) + " "
+ std::to_string( Placement.z ) };
auto const texture { Placement.texture.empty() ? std::string( "none" ) : Placement.texture };
// what the signal shows until something sets it: the node's own pattern if the
// definition gives one, otherwise the initial aspect's, otherwise dark
auto const *initiallights { &definition->initiallights };
if( true == initiallights->empty() ) {
for( auto const &aspect : definition->aspects ) {
if( aspect.name == definition->initial ) {
initiallights = &aspect.lights;
}
}
}
auto const initial { pattern( *initiallights, definition->chambers ) };
std::ostringstream scenery;
scenery
<< "origin " << position << "\n"
<< "rotate 0 " << Placement.yaw << " 0\n"
<< "node 1000 0 " << instance << " model 0 0 0 " << definition->modelangle << " "
<< definition->model << " " << texture << " lights" << initial << " endmodel\n";
for( auto const &attachment : definition->attachments ) {
scenery
<< "node " << attachment.rangemax << " " << attachment.rangemin
<< " none model 0 0 0 " << definition->modelangle << " " << attachment.model << " "
<< ( attachment.texture == "instance" ? texture : attachment.texture ) << " endmodel\n";
}
scenery
<< "rotate 0 0 0\n"
<< "endorigin\n"
// the speed the driver reads off the signal. the aspect events write it, the track
// event beside the signal reads it
<< "node -1 0 " << instance << "_sem_mem memcell " << position << " SetVelocity 0.0 0.0 none endmemcell\n"
<< "event " << instance << "_sem_info getvalues 0 " << instance << "_sem_mem endevent\n";
if( false == definition->sound.empty() ) {
scenery
<< "node 50 0 " << instance << "_dzwiek sound " << position << " " << definition->sound << " endsound\n"
<< "event " << instance << "_dzwiek sound 0.0 " << instance << "_dzwiek 1 endevent\n";
}
// one state event per aspect, and that is the whole public surface of the instance:
// whatever sets the signal launches <name>_<aspect>, and the lights, speeds, animation
// and sound behind it belong to the prefab
for( auto const &aspect : definition->aspects ) {
scenery
<< "event " << instance << "_" << aspect.name << " multiple 0 none "
<< instance << "_sem_ligh_" << aspect.name << " "
<< instance << "_sem_vel_" << aspect.name;
if( false == aspect.animations.empty() ) {
scenery << " " << instance << "_sem_anim_" << aspect.name;
}
if( false == definition->sound.empty() ) {
scenery << " " << instance << "_dzwiek";
}
scenery
<< " endevent\n"
<< "event " << instance << "_sem_ligh_" << aspect.name << " lights " << aspect.lightdelay << " "
<< instance << pattern( aspect.lights, definition->chambers ) << " endevent\n"
<< "event " << instance << "_sem_vel_" << aspect.name << " updatevalues 0 "
<< instance << "_sem_mem SetVelocity " << aspect.velocity << " " << aspect.velocitynext << " endevent\n";
if( false == aspect.animations.empty() ) {
// the templates split this into groups of eight, which a multi_event has not
// needed for a long time - it holds a vector. one list, one event
scenery << "event " << instance << "_sem_anim_" << aspect.name << " multiple 0 none";
for( std::size_t channel = 0; channel < aspect.animations.size(); ++channel ) {
scenery << " " << instance << "_sem_anim_" << aspect.name << "_" << channel;
}
scenery << " endevent\n";
std::size_t channel { 0 };
for( auto const &animation : aspect.animations ) {
scenery
<< "event " << instance << "_sem_anim_" << aspect.name << "_" << channel
<< " animation 0 " << instance << " " << animation.kind << " " << animation.submodel << " "
<< animation.x << " " << animation.y << " " << animation.z << " " << animation.speed << " endevent\n";
++channel;
}
}
}
// a failed signal is dark and stops what comes at it
scenery
<< "event " << instance << "_uszk multiple 0 none "
<< instance << "_sem_ligh0 " << instance << "_sem_vel_stop endevent\n"
<< "event " << instance << "_sem_ligh0 lights 0.0 " << instance << pattern( {}, definition->chambers ) << " endevent\n"
<< "event " << instance << "_sem_vel_stop updatevalues 0 "
<< instance << "_sem_mem SetVelocity 0.0 0.0 endevent\n";
if( true == definition->lightmodes ) {
// channel zero picks day, night or automatic lighting; the rest is left alone
auto const untouched { pattern( {}, definition->chambers > 0 ? definition->chambers - 1 : 0, -1 ) };
scenery
<< "event " << instance << "_dzienny lights 0.0 " << instance << " 0" << untouched << " endevent\n"
<< "event " << instance << "_nocny lights 0.0 " << instance << " 1" << untouched << " endevent\n"
<< "event " << instance << "_auto lights 0.0 " << instance << " 3" << untouched << " endevent\n";
}
return scenery.str();
}
} // namespace scene

118
scene/prefab.cppm Normal file
View File

@@ -0,0 +1,118 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <string>
#include <unordered_map>
#include <vector>
export module eu07.scene.prefab;
export {
// Shared archetypes. A prefab is a model plus the events, memcell, animation channels and
// sounds that give it behaviour, described once and instanced many times. It exists because
// the .inc templates it replaces are not templates at all: cParser pastes text and
// substitutes (pN), so a signal that differs only by which aspects it can display needs a
// whole new file. The scenery directory holds over twelve hundred of them.
//
// What makes one prefab enough for the whole signal family is that its parameter is a list,
// not a number. Every signal template is the same shape - a state event, a light pattern and
// a speed the driver reads, for each aspect it can display, wrapped in a fixed core of
// _sem_info, _uszk and the dark state. Give the prefab the aspects and the rest follows. A
// shape signal adds animation channels to each aspect and a sound to the transition, which
// is more per aspect but not a different shape.
//
// Instances expand to scenery text and go through the ordinary node and event parsing. The
// binary format will compile them directly; until then this keeps one description of the
// signal family instead of one per variant.
namespace scene {
// one submodel driven by an aspect. kind is the word the animation event takes, translate or
// rotate; target is where the submodel goes, in metres or degrees; speed is how fast
struct prefab_animation {
std::string kind;
std::string submodel;
std::string x{ "0" }, y{ "0" }, z{ "0" };
std::string speed{ "0" };
};
// one thing the signal can show. lights is the pattern the model's light channels take, one
// entry per channel, -1 leaving a channel alone; velocity is what the driver reads from the
// memcell, -1 meaning no limit
struct prefab_aspect {
std::string name;
std::vector<int> lights;
std::string lightdelay{ "0.0" };
std::string velocity{ "-1" };
std::string velocitynext{ "-1" };
std::vector<prefab_animation> animations;
};
// a second model hung on the instance: the lattice mast a shape signal stands on, its
// distant lod, the plate carrying the signal's symbol
struct prefab_attachment {
std::string rangemax{ "50" };
std::string rangemin{ "0" };
std::string model;
// the instance's own texture when "instance", otherwise a literal, "none" included
std::string texture{ "none" };
};
struct prefab_definition {
std::string name;
std::string model;
std::string modelangle{ "0" };
std::vector<prefab_attachment> attachments;
// how many light channels the model has, which is how many values an aspect's pattern holds
std::size_t chambers{ 0 };
std::vector<prefab_aspect> aspects;
// aspect the signal shows before anything sets it, by name; empty lights every channel off
std::string initial;
// the node's own starting pattern, when it is not simply the initial aspect: a shape
// signal starts with channel zero on automatic, which is not a thing any aspect says
std::vector<int> initiallights;
// played when the signal changes aspect. shape signals have one, light signals do not
std::string sound;
// emit the _dzienny / _nocny / _auto events which switch channel zero between day,
// night and automatic lighting. only models with that channel want them
bool lightmodes{ false };
};
// arguments one instance supplies
struct prefab_placement {
std::string name;
double x{ 0.0 }, y{ 0.0 }, z{ 0.0 };
double yaw{ 0.0 };
std::string texture;
};
class prefab_registry {
public:
// looks the definition up, loading <Name>.pfb from the scenery directory on first use.
// returns nullptr and logs if there is no such prefab
prefab_definition const * definition( std::string const &Name );
// scenery text realising the instance: the model node, its attachments, memcell, sound
// and events. empty if the definition is missing
std::string expand( std::string const &Name, prefab_placement const &Placement );
private:
bool load( std::string const &Name );
std::unordered_map<std::string, prefab_definition> m_definitions;
// names which failed to load, so a scenery full of them complains once each
std::unordered_map<std::string, bool> m_missing;
};
} // namespace scene
inline scene::prefab_registry Prefabs;
}

View File

@@ -34,6 +34,7 @@ module;
module eu07.simcore; module eu07.simcore;
import eu07.simulation.simulation; import eu07.simulation.simulation;
import eu07.simulation.loadprofile;
import eu07.rendering.renderer; import eu07.rendering.renderer;
import :mover; import :mover;
import eu07.utilities.globals; import eu07.utilities.globals;
@@ -1306,17 +1307,50 @@ std::vector<std::string> switchtrackbedtextures {
"zwrotl65r1200pods-new", "zwrotl65r1200pods-new",
"zwrotp65r1200pods-new" }; "zwrotp65r1200pods-new" };
bool
is_switch_trackbed( std::string const &Materialname ) {
for( auto const &switchtrackbedtexture : switchtrackbedtextures ) {
if( contains( Materialname, switchtrackbedtexture ) ) {
return true;
}
}
return false;
}
void
place_in_world( std::vector<world_vertex> &Vertices, scratch_data const &Scratchpad ) {
if( Scratchpad.location.rotation != glm::vec3( 0, 0, 0 ) ) {
// rotate...
auto const rotation = glm::radians( Scratchpad.location.rotation );
for( auto &vertex : Vertices ) {
vertex.position = glm::rotateZ<double>( vertex.position, rotation.z );
vertex.position = glm::rotateX<double>( vertex.position, rotation.x );
vertex.position = glm::rotateY<double>( vertex.position, rotation.y );
vertex.normal = glm::rotateZ( vertex.normal, rotation.z );
vertex.normal = glm::rotateX( vertex.normal, rotation.x );
vertex.normal = glm::rotateY( vertex.normal, rotation.y );
}
}
if( false == Scratchpad.location.offset.empty()
&& Scratchpad.location.offset.top() != glm::dvec3( 0, 0, 0 ) ) {
// ...and move
auto const &offset = Scratchpad.location.offset.top();
for( auto &vertex : Vertices ) {
vertex.position += offset;
}
}
}
void void
basic_region::insert( shape_node Shape, scratch_data &Scratchpad, bool const Transform ) { basic_region::insert( shape_node Shape, scratch_data &Scratchpad, bool const Transform ) {
if( Global.CreateSwitchTrackbeds ) { if( Global.CreateSwitchTrackbeds ) {
auto const materialname{GfxRenderer->Material(Shape.data().material)->GetName()}; if( true == is_switch_trackbed( GfxRenderer->Material( Shape.data().material )->GetName() ) ) {
for( auto const &switchtrackbedtexture : switchtrackbedtextures ) { // geometry with blacklisted texture, part of old switch trackbed; ignore it
if( contains( materialname, switchtrackbedtexture ) ) { return;
// geometry with blacklisted texture, part of old switch trackbed; ignore it
return;
}
} }
} }
// shape might need to be split into smaller pieces, so we create list of nodes instead of just single one // shape might need to be split into smaller pieces, so we create list of nodes instead of just single one
@@ -1328,37 +1362,24 @@ basic_region::insert( shape_node Shape, scratch_data &Scratchpad, bool const Tra
// adjust input if necessary: // adjust input if necessary:
if( true == Transform ) { if( true == Transform ) {
// shapes generated from legacy terrain come with world space coordinates and don't need processing // shapes generated from legacy terrain come with world space coordinates and don't need processing
if( Scratchpad.location.rotation != glm::vec3( 0, 0, 0 ) ) { place_in_world( shape.m_data.vertices, Scratchpad );
// rotate...
auto const rotation = glm::radians( Scratchpad.location.rotation );
for( auto &vertex : shape.m_data.vertices ) {
vertex.position = glm::rotateZ<double>( vertex.position, rotation.z );
vertex.position = glm::rotateX<double>( vertex.position, rotation.x );
vertex.position = glm::rotateY<double>( vertex.position, rotation.y );
vertex.normal = glm::rotateZ( vertex.normal, rotation.z );
vertex.normal = glm::rotateX( vertex.normal, rotation.x );
vertex.normal = glm::rotateY( vertex.normal, rotation.y );
}
}
if( false == Scratchpad.location.offset.empty()
&& Scratchpad.location.offset.top() != glm::dvec3(0, 0, 0) ) {
// ...and move
auto const& offset = Scratchpad.location.offset.top();
for( auto &vertex : shape.m_data.vertices ) {
vertex.position += offset;
}
}
// calculate bounding area // calculate bounding area
for( auto const &vertex : shape.m_data.vertices ) { for( auto const &vertex : shape.m_data.vertices ) {
shape.m_data.area.center += vertex.position; shape.m_data.area.center += vertex.position;
} }
shape.m_data.area.center /= shape.m_data.vertices.size(); shape.m_data.area.center /= shape.m_data.vertices.size();
// trim the shape if needed. trimmed parts will be added to list as separate nodes // trim the shape if needed. trimmed parts will be added to list as separate nodes
for( std::size_t index = 0; index < shapes.size(); ++index ) { {
while( true == RaTriangleDivider( shapes[ index ], shapes ) ) { loadprofile::scoped_timer const dividetime { loadprofile::stage::shape_divide };
; // all work is done during expression check for( std::size_t index = 0; index < shapes.size(); ++index ) {
while( true == RaTriangleDivider( shapes[ index ], shapes ) ) {
; // all work is done during expression check
}
} }
} }
if( shapes.size() > 1 ) {
loadprofile::add_split( shapes.size() - 1 );
}
} }
// move the data into appropriate section(s) // move the data into appropriate section(s)
for( auto &shape : shapes ) { for( auto &shape : shapes ) {

View File

@@ -371,6 +371,15 @@ public:
}; };
// top-level of scene spatial structure, holds collection of sections // top-level of scene spatial structure, holds collection of sections
// whether a material belongs to the old hand-made switch trackbeds, which the engine
// replaces with generated ones when CreateSwitchTrackbeds is on
bool is_switch_trackbed( std::string const &Materialname );
// moves shape vertices from the coordinates they were written in to world space, applying the
// rotate and origin in force where the node stood. the one place this happens, so that anything
// else looking at a shape - the terrain bake - sees it where it is drawn
void place_in_world( std::vector<world_vertex> &Vertices, scratch_data const &Scratchpad );
class basic_region { class basic_region {
public: public:
// constructors // constructors

View File

@@ -167,6 +167,19 @@ public:
// data access // data access
shapenode_data const & shapenode_data const &
data() const; data() const;
// keeps only the triangles a predicate accepts, by their position in the vertex list.
// used to leave out what a baked heightfield already shows
template <typename Predicate_>
void
keep_triangles( Predicate_ &&Keep ) {
std::vector<world_vertex> kept;
kept.reserve( m_data.vertices.size() );
for( std::size_t index = 0; index + 2 < m_data.vertices.size(); index += 3 ) {
if( true == Keep( index / 3 ) ) {
kept.insert( kept.end(), m_data.vertices.begin() + index, m_data.vertices.begin() + index + 3 );
}
}
m_data.vertices = std::move( kept ); }
// get bounding radius // get bounding radius
// NOTE: use this method instead of direct access to the data member, due to lazy radius evaluation // NOTE: use this method instead of direct access to the data member, due to lazy radius evaluation
float radius(); float radius();

789
scene/terraincooker.h Normal file
View File

@@ -0,0 +1,789 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#pragma once
#include <algorithm>
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <cstring>
#include <climits>
#include <filesystem>
#include <fstream>
#include <functional>
#include <limits>
#include <memory>
#include <string>
#include <string_view>
#include <unordered_map>
#include <vector>
#include <zstd.h>
#include "scene/heightfieldformat.h"
// Rasterising a scenery's terrain triangles into a tiled heightfield.
//
// Both the standalone terraincook tool and the engine cook with this, which is the point of
// it being here: a heightfield baked on first run has to come out identical to one baked by
// the tool, and the only way to be sure of that is for there to be one implementation.
//
// The cooker knows nothing about where triangles come from. Feed it rasterize() per triangle
// and call finish(); the tool reads them out of scenery text, the engine hands them over as
// it parses.
namespace terrain {
// candidate grid steps tried when the step is not given; the coarsest one that still holds
// for practically every vertex names the grid (see detect_step).
// the range has to reach far past a corridor's metre grid: a regional elevation model
// is sampled every hundred metres, and a candidate list that stops at ten would report
// it as a ten metre grid, because every point of a 100 m grid sits on a 10 m one too
constexpr double gridsteps[] { 0.5, 1.0, 2.0, 2.5, 4.0, 5.0, 10.0, 20.0, 25.0, 50.0, 100.0, 200.0 };
// how far off a grid line a coordinate may sit and still count as on it
constexpr double gridepsilon { 0.01 };
// height difference at one grid point above which the samples are treated as a genuine
// conflict - two surfaces stacked over each other, which a heightfield cannot express
constexpr double conflictepsilon { 0.25 };
// share of sampled vertices a candidate step has to cover to count as the base grid
constexpr double gridthreshold { 99.0 };
struct vertex {
double x, y, z;
double u { 0.0 }, v { 0.0 }; // texture coordinates, kept to measure material scale
};
// length of the segment between two vertices as seen from above
inline double
planar_length( vertex const &From, vertex const &To ) {
return std::sqrt( ( To.x - From.x ) * ( To.x - From.x ) + ( To.z - From.z ) * ( To.z - From.z ) );
}
// whether a coordinate lies on a grid of the given step
inline bool
on_grid( double const Value, double const Step ) {
return std::fabs( Value / Step - std::round( Value / Step ) ) * Step <= gridepsilon;
}
// writes a greyscale preview. Values holds Width * Height samples, absent ones marked
// by the sentinel; those come out black
inline void
writepreview(
std::filesystem::path const &Path,
std::vector<float> const &Values,
std::size_t const Width, std::size_t const Height,
float const Sentinel ) {
auto minimum { 1e30f }, maximum { -1e30f };
for( auto const value : Values ) {
if( value == Sentinel ) { continue; }
minimum = std::min( minimum, value );
maximum = std::max( maximum, value );
}
if( minimum > maximum ) { return; }
auto const range { std::max( 1e-6f, maximum - minimum ) };
std::ofstream file( Path, std::ios::binary );
file << "P5\n" << Width << " " << Height << "\n255\n";
std::vector<unsigned char> row( Width );
for( auto y { 0u }; y < Height; ++y ) {
for( auto x { 0u }; x < Width; ++x ) {
auto const value { Values[ y * Width + x ] };
row[ x ] = (
value == Sentinel ?
0u :
static_cast<unsigned char>( 1.0f + 254.0f * ( value - minimum ) / range ) );
}
file.write( reinterpret_cast<char const *>( row.data() ), static_cast<std::streamsize>( Width ) );
}
}
class cooker {
public:
// samples along a tile side, shared edge excluded; with a 2 m grid this is a 256 m tile
static constexpr std::uint32_t tilesamples { 128 };
static constexpr std::uint32_t tilestride { heightfield::samples_at_level( tilesamples, 0 ) };
// mip chain down to a 5 sample side, which is where a tile stops being worth its entry
static constexpr std::uint32_t miplevels { heightfield::level_count( tilesamples, 5 ) };
// triangles larger than this are not terrain: the water plane spans the whole map
static constexpr double maxedge { 200.0 };
// two surfaces closer than this are the same ground, one laid over the other, rather
// than a deck carried above it. a kerb or a slab stands centimetres proud; a viaduct
// stands metres
static constexpr double surfaceepsilon { 0.5 };
// stop printing the cooking report. the tool wants the tables; the engine, baking on
// a first run, wants one line in its own log instead
void
quiet() { m_verbose = false; }
void
configure( double const Step, bool const Compress ) {
m_step = Step;
m_compress = Compress; }
// triangle identity for representation tracking. callers that do not track it pass
// nothing and get notriangle, which is never reported as represented
static constexpr std::uint32_t notriangle { 0xffffffffu };
void
rasterize( vertex const &A, vertex const &B, vertex const &C, std::string_view const Material,
std::uint32_t const Triangle = notriangle ) {
m_current = Triangle;
if( Triangle != notriangle ) {
if( Triangle >= m_triangleclaimed.size() ) {
auto const size { static_cast<std::size_t>( Triangle ) + 1u };
m_triangleclaimed.resize( size, 0u );
m_trianglekept.resize( size, 0u );
m_trianglerejected.resize( size, 0u );
}
}
vertex const points[ 3 ] { A, B, C };
for( auto index { 0u }; index < 3u; ++index ) {
if( planar_length( points[ index ], points[ ( index + 1 ) % 3 ] ) > maxedge ) {
++m_oversized;
reject();
return;
}
}
// barycentric setup in the horizontal plane; a triangle standing on its edge
// covers no samples and would divide by zero
auto const area { ( B.x - A.x ) * ( C.z - A.z ) - ( C.x - A.x ) * ( B.z - A.z ) };
if( std::abs( area ) < 1e-9 ) { ++m_degenerate; reject(); return; }
auto const inversearea { 1.0 / area };
// a palette slot is taken only by a material that reaches this far, so triangles
// standing on their edge do not fill the palette
auto const material { material_index( Material ) };
measure_scale( material, points );
auto const firstcolumn { static_cast<std::int64_t>( std::ceil( std::min( { A.x, B.x, C.x } ) / m_step ) ) };
auto const lastcolumn { static_cast<std::int64_t>( std::floor( std::max( { A.x, B.x, C.x } ) / m_step ) ) };
auto const firstrow { static_cast<std::int64_t>( std::ceil( std::min( { A.z, B.z, C.z } ) / m_step ) ) };
auto const lastrow { static_cast<std::int64_t>( std::floor( std::max( { A.z, B.z, C.z } ) / m_step ) ) };
std::size_t claimed { 0 };
for( auto row { firstrow }; row <= lastrow; ++row ) {
for( auto column { firstcolumn }; column <= lastcolumn; ++column ) {
auto const x { column * m_step };
auto const z { row * m_step };
auto const weightb { ( ( x - A.x ) * ( C.z - A.z ) - ( C.x - A.x ) * ( z - A.z ) ) * inversearea };
auto const weightc { ( ( B.x - A.x ) * ( z - A.z ) - ( x - A.x ) * ( B.z - A.z ) ) * inversearea };
auto const weighta { 1.0 - weightb - weightc };
constexpr double slack { -1e-9 };
if( ( weighta < slack ) || ( weightb < slack ) || ( weightc < slack ) ) { continue; }
++claimed;
store( column, row, weighta * A.y + weightb * B.y + weightc * C.y, material );
}
}
// every sample this triangle wanted, whether or not it ended up keeping it.
// comparing this with what the finished grid holds says how much of a
// material was lost to triangles stacked over it
if( m_current != notriangle ) { m_triangleclaimed[ m_current ] += static_cast<std::uint32_t>( claimed ); }
tally( m_materialclaimed, material, claimed );
if( material >= m_materialwhere.size() ) { m_materialwhere.resize( material + 1u, { 0.0, 0.0, false } ); }
if( false == m_materialwhere[ material ].seen ) {
// one place the material was laid, so a report line can be looked at
m_materialwhere[ material ] = { ( A.x + B.x + C.x ) / 3.0, ( A.z + B.z + C.z ) / 3.0, true };
}
if( claimed == 0 ) {
// smaller than the gap between samples: it left no trace at all
tally( m_materialmissed, material, 1 );
++m_toosmall;
}
}
// whether the heightfield shows this triangle well enough for it not to be drawn on its
// own: it was laid at all, it reached at least one sample, and it still holds nearly
// every sample it wanted once everything else has been laid. valid after finish()
bool
represented( std::uint32_t const Triangle ) const {
if( Triangle >= m_triangleclaimed.size() ) { return false; }
if( m_trianglerejected[ Triangle ] != 0 ) { return false; }
auto const claimed { m_triangleclaimed[ Triangle ] };
if( claimed == 0 ) { return false; }
return static_cast<double>( m_trianglekept[ Triangle ] ) >= representedshare * static_cast<double>( claimed ); }
void
finish( std::filesystem::path const &Path, std::filesystem::path const &Previewpath ) {
if( true == m_tiles.empty() ) {
report( " nothing to cook\n" );
return; }
// every sample counted once, in the tile that owns it rather than in the
// neighbours carrying a copy of the shared edge
for( auto const & [ key, entry ] : m_tiles ) {
for( std::uint32_t row = 0; row < tilesamples; ++row ) {
for( std::uint32_t column = 0; column < tilesamples; ++column ) {
auto const owner { entry.owner[ row * tilestride + column ] };
if( ( owner == notriangle ) || ( owner >= m_trianglekept.size() ) ) { continue; }
++m_trianglekept[ owner ];
// the heightfield is drawn only between samples that all carry ground,
// so a triangle reaching past the last full cell leaves a strip nothing
// covers. one holding a sample next to a hole keeps being drawn itself
auto const globalcolumn { static_cast<std::int64_t>( entry.x ) * tilesamples + column };
auto const globalrow { static_cast<std::int64_t>( entry.z ) * tilesamples + row };
if( m_trianglerejected[ owner ] != 0 ) { continue; }
if( false == drawn_around( globalcolumn, globalrow ) ) {
m_trianglerejected[ owner ] = 1u;
}
}
}
}
auto const scale { std::max( 1.0, m_maxheight - m_minheight ) / heightfield::heightrange };
std::vector<tile const *> ordered;
ordered.reserve( m_tiles.size() );
for( auto const & [ key, entry ] : m_tiles ) { ordered.push_back( &entry ); }
std::sort( ordered.begin(), ordered.end(),
[]( tile const *Left, tile const *Right ) {
return heightfield::morton( Left->x, Left->z ) < heightfield::morton( Right->x, Right->z ); } );
std::ofstream file( Path, std::ios::binary );
if( false == file.good() ) {
report( " cannot write %s\n", Path.string().c_str() );
return; }
heightfield::file_header header {};
std::memcpy( header.magic, heightfield::magic, sizeof( header.magic ) );
header.version = heightfield::version;
header.tilesamples = tilesamples;
header.gridstep = static_cast<float>( m_step );
header.heightbias = static_cast<float>( m_minheight );
header.heightscale = static_cast<float>( scale );
header.tilecount = static_cast<std::uint32_t>( ordered.size() );
header.materialcount = static_cast<std::uint32_t>( m_materials.size() );
header.miplevels = miplevels;
header.selection = heightfield::selection_rules;
header.minx = header.minz = INT32_MAX;
header.maxx = header.maxz = INT32_MIN;
for( auto const *entry : ordered ) {
header.minx = std::min( header.minx, entry->x ); header.maxx = std::max( header.maxx, entry->x );
header.minz = std::min( header.minz, entry->z ); header.maxz = std::max( header.maxz, entry->z );
}
std::size_t materialbytes { 0 };
for( auto const &name : m_materials ) {
materialbytes += sizeof( std::uint16_t ) + name.size() + sizeof( float );
}
auto const directorybytes { ordered.size() * sizeof( heightfield::tile_entry ) };
auto const payloadstart { sizeof( heightfield::file_header ) + directorybytes + materialbytes };
// the payloads are encoded up front: the directory records where each one
// lands and how large it ended up, so it cannot be written before they exist
std::vector<heightfield::tile_entry> directory;
std::vector<std::vector<std::uint8_t>> payloads;
directory.reserve( ordered.size() );
payloads.reserve( ordered.size() );
auto offset { payloadstart };
auto const plainsize { heightfield::tile_bytes( tilesamples, miplevels ) };
std::vector<std::uint8_t> plain( plainsize );
for( auto const *entry : ordered ) {
encode( *entry, scale, plain );
auto payload { m_compress ? compress( plain ) : plain };
m_storedbytes += payload.size();
m_plainbytes += plain.size();
heightfield::tile_entry record {};
record.x = entry->x;
record.z = entry->z;
record.offset = offset;
record.storedsize = static_cast<std::uint32_t>( payload.size() );
record.plainsize = static_cast<std::uint32_t>( plain.size() );
record.minheight = entry->minheight;
record.maxheight = entry->maxheight;
record.coverage = static_cast<std::uint32_t>( entry->coverage );
record.tilecodec = static_cast<std::uint16_t>(
m_compress ? heightfield::codec::zstd : heightfield::codec::raw );
offset += payload.size();
directory.push_back( record );
payloads.push_back( std::move( payload ) );
}
file.write( reinterpret_cast<char const *>( &header ), sizeof( header ) );
file.write( reinterpret_cast<char const *>( directory.data() ), static_cast<std::streamsize>( directorybytes ) );
for( std::size_t index = 0; index < m_materials.size(); ++index ) {
auto const &name { m_materials[ index ] };
std::uint16_t const length { static_cast<std::uint16_t>( name.size() ) };
file.write( reinterpret_cast<char const *>( &length ), sizeof( length ) );
file.write( name.data(), static_cast<std::streamsize>( name.size() ) );
auto const scale { static_cast<float>( material_scale( index ) ) };
file.write( reinterpret_cast<char const *>( &scale ), sizeof( scale ) );
}
for( auto const &payload : payloads ) {
file.write( reinterpret_cast<char const *>( payload.data() ), static_cast<std::streamsize>( payload.size() ) );
}
file.close();
report( Path, header, offset );
if( false == Previewpath.empty() ) {
writecookedpreview( Previewpath, header );
}
}
private:
struct tile {
std::int32_t x { 0 }, z { 0 };
std::vector<float> height;
std::vector<std::uint8_t> material;
std::vector<std::uint8_t> covered;
// which triangle the sample came from, so that once everything is laid each
// triangle can be asked whether the heightfield still shows it
std::vector<std::uint32_t> owner;
float minheight { 1e30f }, maxheight { -1e30f };
std::size_t coverage { 0 };
};
// the palette slot of a material, allocated on first use. looked up by view, so a
// material already known costs no allocation
std::uint8_t
material_index( std::string_view const Name ) {
auto const lookup { m_materialindex.find( Name ) };
if( lookup != m_materialindex.end() ) { return lookup->second; }
if( m_materials.size() >= 255 ) { ++m_materialoverflow; return 255; }
auto const index { static_cast<std::uint8_t>( m_materials.size() ) };
m_materials.emplace_back( Name );
m_materialindex.emplace( std::string { Name }, index );
return index; }
// the source carries texture coordinates, so the tiling of every material can be
// measured instead of guessed: the ratio of world distance to texture distance along
// a triangle edge is how many metres one repeat covers. edges too short or with no
// texture movement say nothing and are left out
void
measure_scale( std::uint8_t const Material, vertex const ( &Points )[ 3 ] ) {
if( Material >= m_scalesums.size() ) {
m_scalesums.resize( Material + 1, 0.0 );
m_scalesquares.resize( Material + 1, 0.0 );
m_scalecounts.resize( Material + 1, 0 );
}
for( auto index { 0u }; index < 3u; ++index ) {
auto const &from { Points[ index ] };
auto const &to { Points[ ( index + 1 ) % 3 ] };
auto const world { planar_length( from, to ) };
auto const texture {
std::sqrt( ( to.u - from.u ) * ( to.u - from.u ) + ( to.v - from.v ) * ( to.v - from.v ) ) };
if( ( world < 0.5 ) || ( texture < 1e-4 ) ) { continue; }
auto const ratio { world / texture };
m_scalesums[ Material ] += ratio;
m_scalesquares[ Material ] += ratio * ratio;
++m_scalecounts[ Material ];
}
}
void
store(
std::int64_t const Column, std::int64_t const Row, double const Height,
std::uint8_t const Material ) {
// a sample on a tile boundary belongs to both neighbours. writing it to each
// of them is what makes the shared edge identical rather than merely close
auto const basex { static_cast<std::int32_t>( heightfield::floor_div( Column, tilesamples ) ) };
auto const basez { static_cast<std::int32_t>( heightfield::floor_div( Row, tilesamples ) ) };
auto const onxedge { Column == static_cast<std::int64_t>( basex ) * tilesamples };
auto const onzedge { Row == static_cast<std::int64_t>( basez ) * tilesamples };
for( auto duplicate { 0u }; duplicate < 4u; ++duplicate ) {
if( ( ( duplicate & 1u ) != 0 ) && ( false == onxedge ) ) { continue; }
if( ( ( duplicate & 2u ) != 0 ) && ( false == onzedge ) ) { continue; }
auto const tilex { basex - ( ( duplicate & 1u ) != 0 ? 1 : 0 ) };
auto const tilez { basez - ( ( duplicate & 2u ) != 0 ? 1 : 0 ) };
auto const column { static_cast<std::size_t>( Column - static_cast<std::int64_t>( tilex ) * tilesamples ) };
auto const row { static_cast<std::size_t>( Row - static_cast<std::int64_t>( tilez ) * tilesamples ) };
if( ( column >= tilestride ) || ( row >= tilestride ) ) { continue; }
auto &target { tileat( tilex, tilez ) };
auto const index { row * tilestride + column };
if( target.covered[ index ] == 0 ) {
target.covered[ index ] = 1;
target.height[ index ] = static_cast<float>( Height );
target.material[ index ] = Material;
target.owner[ index ] = m_current;
++target.coverage;
++m_covered;
}
else {
auto const difference { Height - static_cast<double>( target.height[ index ] ) };
if( std::abs( difference ) > conflictepsilon ) { ++m_stacked; }
if( std::abs( difference ) <= surfaceepsilon ) {
// near enough to be the same ground: a slab, a path, a patch of
// gravel laid over it. what is laid on top is what one sees, so
// the upper surface takes the sample. comparing heights rather
// than arrival order keeps the result independent of file order
if( difference > 0.0 ) {
target.height[ index ] = static_cast<float>( Height );
target.material[ index ] = Material;
target.owner[ index ] = m_current;
}
else if( ( std::abs( difference ) <= coincident )
&& ( duplicate == 0u )
&& ( m_current != notriangle ) ) {
// two triangles meeting along an edge both claim the samples on
// it, at the same height. the one that arrived second shows there
// just as much as the first, so it is credited with the sample
++m_trianglekept[ m_current ];
}
}
else if( difference < 0.0 ) {
// far apart: one of them is carried above the ground on a
// structure, and the ground is the lower of the two
target.height[ index ] = static_cast<float>( Height );
target.material[ index ] = Material;
target.owner[ index ] = m_current;
}
}
target.minheight = std::min( target.minheight, static_cast<float>( Height ) );
target.maxheight = std::max( target.maxheight, static_cast<float>( Height ) );
}
m_minheight = std::min( m_minheight, Height );
m_maxheight = std::max( m_maxheight, Height );
}
// the tile at the given coordinates, created on first use. the last one returned is
// remembered: consecutive samples nearly always fall in the same tile, and a map's
// elements stay where they are when it grows
tile &
tileat( std::int32_t const X, std::int32_t const Z ) {
auto const key { heightfield::tile_key( X, Z ) };
if( ( m_lasttile != nullptr ) && ( m_lasttilekey == key ) ) { return *m_lasttile; }
m_lasttilekey = key;
auto const lookup { m_tiles.find( key ) };
if( lookup != m_tiles.end() ) { return *( m_lasttile = &lookup->second ); }
tile fresh;
fresh.x = X;
fresh.z = Z;
fresh.height.assign( tilestride * tilestride, 0.0f );
fresh.material.assign( tilestride * tilestride, 0u );
fresh.covered.assign( tilestride * tilestride, 0u );
fresh.owner.assign( tilestride * tilestride, notriangle );
return *( m_lasttile = &m_tiles.emplace( key, std::move( fresh ) ).first->second ); }
// lays the mip chain of one tile into the buffer, finest level first.
// coarse levels are point sampled rather than averaged, deliberately: every level
// keeps the samples that sit on the tile boundary exactly as its neighbour has them,
// so the seam between two tiles stays identical at every level instead of needing
// skirts to hide a mismatch. the cost is aliasing on distant terrain
void
encode( tile const &Tile, double const Scale, std::vector<std::uint8_t> &Buffer ) const {
auto *cursor { Buffer.data() };
for( auto level { 0u }; level < miplevels; ++level ) {
auto const side { heightfield::samples_at_level( tilesamples, level ) };
auto const stepping { std::size_t { 1 } << level };
auto *heights { reinterpret_cast<std::uint16_t *>( cursor ) };
auto *materials { cursor + static_cast<std::size_t>( side ) * side * sizeof( std::uint16_t ) };
for( auto row { 0u }; row < side; ++row ) {
for( auto column { 0u }; column < side; ++column ) {
auto const source { ( row * stepping ) * tilestride + ( column * stepping ) };
auto const target { static_cast<std::size_t>( row ) * side + column };
heights[ target ] = (
Tile.covered[ source ] == 0 ?
heightfield::nodata :
static_cast<std::uint16_t>( ( Tile.height[ source ] - m_minheight ) / Scale ) );
materials[ target ] = Tile.material[ source ];
}
}
cursor += heightfield::level_bytes( tilesamples, level );
}
}
std::vector<std::uint8_t>
compress( std::vector<std::uint8_t> const &Plain ) {
// one context for the whole cook: creating one per tile costs more than a tile
if( m_context == nullptr ) { m_context.reset( ZSTD_createCCtx() ); }
std::vector<std::uint8_t> encoded( ZSTD_compressBound( Plain.size() ) );
auto const written {
ZSTD_compressCCtx( m_context.get(), encoded.data(), encoded.size(), Plain.data(), Plain.size(), compressionlevel ) };
if( 0 != ZSTD_isError( written ) ) {
// a tile that will not compress is still a valid tile; it just stays raw
return Plain;
}
encoded.resize( written );
return encoded;
}
void
report(
std::filesystem::path const &Path,
heightfield::file_header const &Header,
std::size_t const Bytes ) const {
auto const tilearea {
( static_cast<std::int64_t>( Header.maxx ) - Header.minx + 1 )
* ( static_cast<std::int64_t>( Header.maxz ) - Header.minz + 1 ) };
auto const storedsamples {
static_cast<std::size_t>( Header.tilecount ) * tilestride * tilestride };
report( "\n-- cooked heightfield\n" );
report( " grid step %12.2f m\n", m_step );
report( " tiles %12u (%u x %u samples, %.0f m each)\n",
Header.tilecount, tilestride, tilestride, tilesamples * m_step );
report( " mip levels %12u (finest %u, coarsest %u samples)\n",
miplevels, tilestride, heightfield::samples_at_level( tilesamples, miplevels - 1 ) );
report( " tile span %12d .. %d x %d .. %d z\n",
Header.minx, Header.maxx, Header.minz, Header.maxz );
report( " occupancy %12.1f %% of the tile bounding box\n",
100.0 * static_cast<double>( Header.tilecount ) / static_cast<double>( tilearea ) );
report( " samples with data %12zu %6.2f %% of the finest level\n",
m_covered, 100.0 * static_cast<double>( m_covered ) / static_cast<double>( storedsamples ) );
report( " height range %12.2f .. %.2f m (%.4f m per unit)\n",
m_minheight, m_maxheight, Header.heightscale );
report( " materials %12zu%s\n", m_materials.size(),
m_materialoverflow > 0 ? " (palette full, some triangles share the last slot)" : "" );
auto const coverage { material_coverage() };
report( "\n-- what survived, per material\n" );
report( " %-28s %10s %10s %8s %8s %s\n", "material", "claimed", "kept", "kept %", "vanished", "seen at x,z" );
for( std::size_t index = 0; index < m_materials.size(); ++index ) {
auto const claimed { index < m_materialclaimed.size() ? m_materialclaimed[ index ] : 0 };
auto const kept { index < coverage.size() ? coverage[ index ] : 0 };
auto const missed { index < m_materialmissed.size() ? m_materialmissed[ index ] : 0 };
auto const where { index < m_materialwhere.size() ? m_materialwhere[ index ] : place{} };
report( " %-28s %10zu %10zu %7.1f %% %8zu %.0f,%.0f\n",
m_materials[ index ].c_str(), claimed, kept,
claimed == 0 ? 0.0 : 100.0 * static_cast<double>( kept ) / static_cast<double>( claimed ),
missed, where.x, where.z );
}
report( " triangles too small to reach a sample: %zu\n", m_toosmall );
report( "\n-- material tiling\n" );
for( std::size_t index = 0; index < m_materials.size(); ++index ) {
report( " %-28s %6.2f m per repeat, spread %5.1f %%%s\n",
m_materials[ index ].c_str(), material_scale( index ), 100.0 * scale_spread( index ),
( index < m_scalecounts.size() ) && ( m_scalecounts[ index ] > 0 ) ? "" : " (assumed)" );
}
report( " oversized skipped %12zu (edges over %.0f m, the water plane)\n", m_oversized, maxedge );
report( " degenerate skipped %12zu\n", m_degenerate );
report( " stacked samples %12zu (surfaces more than %.2f m apart)\n", m_stacked, conflictepsilon );
if( true == m_compress ) {
report( " payload %12.1f MB from %.1f MB (%.2fx, zstd level %d)\n",
m_storedbytes / 1048576.0, m_plainbytes / 1048576.0,
m_plainbytes > 0 ? static_cast<double>( m_plainbytes ) / static_cast<double>( m_storedbytes ) : 0.0,
compressionlevel );
}
report( " file size %12.1f MB -> %s\n", Bytes / 1048576.0, Path.string().c_str() );
}
void
writecookedpreview( std::filesystem::path const &Path, heightfield::file_header const &Header ) const {
auto const width { static_cast<std::size_t>( Header.maxx - Header.minx + 1 ) };
auto const height { static_cast<std::size_t>( Header.maxz - Header.minz + 1 ) };
// one preview pixel per coarsest mip sample, capped so the image stays viewable
auto pixelspertile { heightfield::samples_at_level( tilesamples, miplevels - 1 ) - 1 };
while( ( pixelspertile > 1 ) && ( std::max( width, height ) * pixelspertile > 2048 ) ) { pixelspertile /= 2; }
auto const previewwidth { width * pixelspertile };
auto const previewheight { height * pixelspertile };
auto const samplesper { tilesamples / pixelspertile };
constexpr float sentinel { -1e30f };
std::vector<float> preview( previewwidth * previewheight, sentinel );
for( auto const & [ key, tile ] : m_tiles ) {
for( auto y { 0u }; y < pixelspertile; ++y ) {
for( auto x { 0u }; x < pixelspertile; ++x ) {
auto sum { 0.0 };
std::size_t count { 0 };
for( auto sy { 0u }; sy < samplesper; ++sy ) {
for( auto sx { 0u }; sx < samplesper; ++sx ) {
auto const index { ( y * samplesper + sy ) * tilestride + ( x * samplesper + sx ) };
if( tile.covered[ index ] == 0 ) { continue; }
sum += tile.height[ index ];
++count;
}
}
if( count == 0 ) { continue; }
auto const column { static_cast<std::size_t>( tile.x - Header.minx ) * pixelspertile + x };
auto const row { static_cast<std::size_t>( tile.z - Header.minz ) * pixelspertile + y };
if( ( column >= previewwidth ) || ( row >= previewheight ) ) { continue; }
preview[ row * previewwidth + column ] = static_cast<float>( sum / static_cast<double>( count ) );
}
}
}
writepreview( Path, preview, previewwidth, previewheight, sentinel );
}
// metres of ground per texture repeat; materials the measurement never caught fall
// back to something plausible rather than to zero, which would divide in the shader
double
material_scale( std::size_t const Index ) const {
if( ( Index >= m_scalecounts.size() ) || ( m_scalecounts[ Index ] == 0 ) ) { return 4.0; }
return m_scalesums[ Index ] / static_cast<double>( m_scalecounts[ Index ] ); }
// how much the measured tiling varies within one material, relative to its mean.
// a single global scale per texture is only faithful where this is small; a material
// the source scaled differently from place to place cannot be served by one number
double
scale_spread( std::size_t const Index ) const {
if( ( Index >= m_scalecounts.size() ) || ( m_scalecounts[ Index ] < 2 ) ) { return 0.0; }
auto const count { static_cast<double>( m_scalecounts[ Index ] ) };
auto const mean { m_scalesums[ Index ] / count };
auto const variance { std::max( 0.0, m_scalesquares[ Index ] / count - mean * mean ) };
return mean > 1e-6 ? std::sqrt( variance ) / mean : 0.0; }
static void
tally( std::vector<std::size_t> &Counters, std::uint8_t const Material, std::size_t const Amount ) {
if( Material >= Counters.size() ) { Counters.resize( Material + 1, 0 ); }
Counters[ Material ] += Amount; }
// what the finished grid actually holds, counted once the last triangle is in
std::vector<std::size_t>
material_coverage() const {
std::vector<std::size_t> counters( m_materials.size(), 0 );
for( auto const & [ key, tile ] : m_tiles ) {
// only the samples a tile owns, so the shared edge is not counted twice
for( std::uint32_t row = 0; row < tilesamples; ++row ) {
for( std::uint32_t column = 0; column < tilesamples; ++column ) {
auto const index { row * tilestride + column };
if( tile.covered[ index ] == 0 ) { continue; }
auto const material { tile.material[ index ] };
if( material < counters.size() ) { ++counters[ material ]; }
}
}
}
return counters; }
template <typename... Arguments_>
void
report( char const *Format, Arguments_... Values ) const {
if( true == m_verbose ) { std::printf( Format, Values... ); } }
static constexpr int compressionlevel { 9 };
bool m_verbose { true };
double m_step { 2.0 };
bool m_compress { true };
std::unordered_map<std::int64_t, tile> m_tiles;
std::vector<std::string> m_materials;
// looked up by string_view, so finding a material does not build a string
struct name_hash {
using is_transparent = void;
std::size_t operator()( std::string_view const Name ) const { return std::hash<std::string_view>{}( Name ); } };
std::unordered_map<std::string, std::uint8_t, name_hash, std::equal_to<>> m_materialindex;
double m_minheight { 1e30 }, m_maxheight { -1e30 };
std::size_t m_covered { 0 }, m_stacked { 0 }, m_oversized { 0 }, m_degenerate { 0 };
std::size_t m_materialoverflow { 0 };
// whether the sample at a grid position carries ground, wherever it is stored. the last
// tile looked in is remembered, since neighbouring samples nearly always share one
bool
covered_at( std::int64_t const Column, std::int64_t const Row ) const {
auto const tilex { static_cast<std::int32_t>( heightfield::floor_div( Column, tilesamples ) ) };
auto const tilez { static_cast<std::int32_t>( heightfield::floor_div( Row, tilesamples ) ) };
auto const tilekey { heightfield::tile_key( tilex, tilez ) };
if( ( m_lastcovered == nullptr ) || ( m_lastcoveredkey != tilekey ) ) {
auto const lookup { m_tiles.find( tilekey ) };
if( lookup == m_tiles.end() ) { return false; }
m_lastcovered = &lookup->second;
m_lastcoveredkey = tilekey;
}
auto const column { static_cast<std::size_t>( Column - static_cast<std::int64_t>( tilex ) * tilesamples ) };
auto const row { static_cast<std::size_t>( Row - static_cast<std::int64_t>( tilez ) * tilesamples ) };
return m_lastcovered->covered[ row * tilestride + column ] != 0; }
// whether the ground around a sample is drawn at every level a triangle node is likely to
// be seen at. the heightfield draws a cell only when all four of its corners carry ground,
// and a coarser level has larger cells, so a sample can be drawn up close and fall into a
// hole further out. corners are checked on every cell touching the sample, diagonals too
bool
drawn_around( std::int64_t const Column, std::int64_t const Row ) const {
for( std::uint32_t level = 0; level < checkedlevels; ++level ) {
auto const stride { static_cast<std::int64_t>( 1 ) << level };
auto const basecolumn { heightfield::floor_div( Column, stride ) * stride };
auto const baserow { heightfield::floor_div( Row, stride ) * stride };
// a sample on a cell line touches the cells either side of it
auto const firstcolumn { basecolumn == Column ? basecolumn - stride : basecolumn };
auto const firstrow { baserow == Row ? baserow - stride : baserow };
for( auto row { firstrow }; row <= baserow + stride; row += stride ) {
for( auto column { firstcolumn }; column <= basecolumn + stride; column += stride ) {
if( false == covered_at( column, row ) ) { return false; }
}
}
}
return true; }
void
reject() {
if( m_current != notriangle ) { m_trianglerejected[ m_current ] = 1u; } }
// levels up to 2048 m, the farthest a triangle node of the terrain is normally drawn
static constexpr std::uint32_t checkedlevels { 3 };
mutable tile const *m_lastcovered { nullptr };
mutable std::int64_t m_lastcoveredkey { 0 };
// heights this close are one surface rather than two
static constexpr double coincident { 0.05 };
// share of its samples a triangle has to keep to count as shown by the heightfield
static constexpr double representedshare { 0.9 };
std::uint32_t m_current { notriangle };
std::vector<std::uint32_t> m_triangleclaimed, m_trianglekept;
std::vector<std::uint8_t> m_trianglerejected;
std::vector<std::size_t> m_materialclaimed, m_materialmissed;
struct place { double x { 0.0 }, z { 0.0 }; bool seen { false }; };
std::vector<place> m_materialwhere;
std::size_t m_toosmall { 0 };
std::vector<double> m_scalesums, m_scalesquares;
std::vector<std::size_t> m_scalecounts;
std::size_t m_storedbytes { 0 }, m_plainbytes { 0 };
tile *m_lasttile { nullptr };
std::int64_t m_lasttilekey { 0 };
struct context_release {
void operator()( ZSTD_CCtx *Context ) const { ZSTD_freeCCtx( Context ); } };
std::unique_ptr<ZSTD_CCtx, context_release> m_context;
};
// the coarsest candidate step that still holds for practically every sampled vertex.
// every vertex sitting on a 2 m grid also sits on a 1 m and a 0.5 m one, so the finest
// candidate always scores best and says nothing - it is the coarsest that names the grid
// Report is called with each candidate and the share of vertices on it, for a caller that
// wants to show the working
template <typename Sample_, typename Report_>
double
detect_step( Sample_ const &Sample, Report_ &&Report ) {
auto best { 0.0 };
for( auto const candidate : gridsteps ) {
std::size_t hits { 0 };
for( auto const &point : Sample ) {
if( on_grid( point.x, candidate ) && on_grid( point.z, candidate ) ) { ++hits; }
}
auto const share { std::empty( Sample ) ? 0.0 : 100.0 * static_cast<double>( hits ) / static_cast<double>( std::size( Sample ) ) };
Report( candidate, share );
if( share >= gridthreshold ) { best = candidate; }
}
return ( best > 0.0 ? best : 1.0 );
}
template <typename Sample_>
double
detect_step( Sample_ const &Sample ) {
return detect_step( Sample, []( double, double ) {} );
}
} // namespace terrain

View File

@@ -0,0 +1,57 @@
// Terrain heightfield, fragment stage.
//
// A tile is drawn once per material it contains, with that material's texture bound, and
// every fragment belonging to a different material is dropped. Texture coordinates are
// world space divided by the tiling the cooker measured from the source geometry, so the
// ground keeps the scale the scenery was authored with.
in vec4 f_pos;
in vec3 f_normal;
in vec2 f_ground;
in float f_valid;
flat in uint f_material;
#include <common>
#include <apply_fog.glsl>
#include <tonemapping.glsl>
uniform sampler2D ground;
uniform float groundscale; // metres of ground covered by one texture repeat
uniform uint drawnmaterial; // the material this pass is drawing
uniform bool hastexture; // false when the material has no texture to sample
uniform vec3 fallback; // colour used in that case
layout(location = 0) out vec4 out_color;
#if MOTIONBLUR_ENABLED
layout(location = 1) out vec4 out_motion;
#endif
void main()
{
// f_valid is 1.0 only where every vertex of this triangle carried terrain, so a
// triangle reaching into a hole disappears. it erodes the boundary by up to one
// sample, which is the price of holes being a property of samples, not of triangles
if( f_valid < 0.999 ) { discard; }
// this pass belongs to one material; everything else is drawn by another pass
if( f_material != drawnmaterial ) { discard; }
vec3 albedo = (
hastexture ?
texture( ground, f_ground / max( groundscale, 0.01 ) ).rgb :
fallback );
// a plain lambert term against the first directional light, so that relief reads
float lambert = 0.35;
if( lights_count >= 1U && lights[ 0 ].type == LIGHT_DIR ) {
lambert = 0.35 + 0.65 * max( dot( normalize( f_normal ), normalize( -lights[ 0 ].dir ) ), 0.0 );
}
vec3 col = albedo * lambert;
#if POSTFX_ENABLED
out_color = vec4( apply_fog( col ), 1.0 );
#else
out_color = tonemap( vec4( apply_fog( col ), 1.0 ) );
#endif
#if MOTIONBLUR_ENABLED
out_motion = vec4( 0.0 );
#endif
}

View File

@@ -0,0 +1,90 @@
// Terrain heightfield, vertex stage.
//
// There is no vertex buffer: the grid exists only as an index buffer, and a sample's
// row and column come from gl_VertexID. Heights are read as unsigned integers so that
// the cooker's no-data value survives unchanged instead of being normalised into a float.
#include <common>
uniform usampler2D heights;
uniform usampler2D materials;
// tile origin relative to the viewpoint, with the viewpoint height already folded into y
uniform vec3 tileorigin;
uniform float samplestep; // metres between samples at the level this tile is drawn at
uniform float heightbias; // world height of stored value 0
uniform float heightscale; // metres per stored unit
uniform uint nodata;
uniform int side; // samples along a tile side at this level
uniform vec2 tileworld; // tile origin in world space, for texture coordinates
out vec4 f_pos;
out vec3 f_normal;
out vec2 f_ground;
out float f_valid;
flat out uint f_material;
// height of a neighbouring sample, for the central difference the normal is built from.
// a neighbour that carries no terrain would otherwise decode as the no-data value and
// read as a surface a hundred metres up, wrecking the normal and leaving the ground
// unlit; Fallback keeps the difference at zero there, so the edge of a hole comes out
// flat rather than black
// height of the first neighbouring sample that carries terrain, or the lowest height on
// the map when the sample is surrounded by emptiness
float borrowed_height( ivec2 Texel )
{
const ivec2 offsets[ 4 ] = ivec2[ 4 ]( ivec2( -1, 0 ), ivec2( 1, 0 ), ivec2( 0, -1 ), ivec2( 0, 1 ) );
for( int index = 0; index < 4; ++index ) {
ivec2 neighbour = clamp( Texel + offsets[ index ], ivec2( 0 ), ivec2( side - 1 ) );
uint raw = texelFetch( heights, neighbour, 0 ).r;
if( raw != nodata ) { return heightbias + float( raw ) * heightscale; }
}
return heightbias;
}
float sample_height( ivec2 Texel, float Fallback )
{
ivec2 clamped = clamp( Texel, ivec2( 0 ), ivec2( side - 1 ) );
uint raw = texelFetch( heights, clamped, 0 ).r;
if( raw == nodata ) { return Fallback; }
return heightbias + float( raw ) * heightscale;
}
void main()
{
int column = gl_VertexID % side;
int row = gl_VertexID / side;
ivec2 texel = ivec2( column, row );
uint raw = texelFetch( heights, texel, 0 ).r;
f_valid = ( raw == nodata ) ? 0.0 : 1.0;
f_material = texelFetch( materials, texel, 0 ).r;
// a sample outside the cooked ground has no height. decoding the no-data value as
// one puts the vertex at the top of the map's range, and since f_valid is
// interpolated, the slivers of triangle nearest the valid corners survive the
// fragment test and stretch up there. so an empty sample borrows a neighbour's
// height: the fragments are still dropped, but the geometry stays on the ground
float height = heightbias + float( raw ) * heightscale;
if( raw == nodata ) {
height = borrowed_height( texel );
}
vec3 local = tileorigin + vec3( float( column ) * samplestep, height, float( row ) * samplestep );
// central differences over the neighbouring samples; at a tile edge the clamp repeats
// the border sample, which is what the neighbouring tile holds there as well
float west = sample_height( texel - ivec2( 1, 0 ), height );
float east = sample_height( texel + ivec2( 1, 0 ), height );
float north = sample_height( texel - ivec2( 0, 1 ), height );
float south = sample_height( texel + ivec2( 0, 1 ), height );
vec3 normal = normalize( vec3( west - east, 2.0 * samplestep, north - south ) );
// world coordinates, so the ground texture stays put as the camera moves and as the
// tile changes mip level
f_ground = tileworld + vec2( float( column ), float( row ) ) * samplestep;
f_pos = modelview * vec4( local, 1.0 );
f_normal = normalize( modelviewnormal * normal );
gl_Position = projection * f_pos;
}

231
simulation/loadprofile.cpp Normal file
View File

@@ -0,0 +1,231 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <array>
#include <atomic>
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <fstream>
#include <iomanip>
#include <sstream>
#include <string>
#include <string_view>
#include <unordered_map>
#ifdef __linux__
#include <unistd.h>
#endif
module eu07.simulation.loadprofile;
import eu07.utilities.logs;
namespace loadprofile {
namespace {
bool profiling { false };
std::array<double, static_cast<std::size_t>( stage::count )> stagetime {};
std::array<std::size_t, static_cast<std::size_t>( kind::count )> nodecount {};
std::array<double, static_cast<std::size_t>( kind::count )> kindtime {};
std::size_t vertexcount { 0 };
std::size_t splitcount { 0 }; // shapes the trimmer had to divide
std::size_t splitpieces { 0 }; // extra pieces the trimmer produced
std::size_t memoryatstart { 0 };
double totaltime { 0.0 };
// resident set size in kilobytes, 0 when it cannot be read. linux only; on other
// platforms the memory lines of the report simply stay at zero
std::size_t
resident_memory() {
#ifdef __linux__
std::ifstream statm( "/proc/self/statm" );
if( false == statm.good() ) { return 0; }
std::size_t total { 0 }, resident { 0 };
statm >> total >> resident;
return resident * ( static_cast<std::size_t>( ::sysconf( _SC_PAGESIZE ) ) / 1024 );
#else
return 0;
#endif
}
std::string const stagenames[] {
"token loop (rest)",
"triangle import",
"triangle insert",
" of which divider",
"other node types" };
std::string const kindnames[] {
"triangles", "lines", "track", "traction", "model",
"dynamic", "memcell", "eventlauncher", "sound", "other" };
std::string
megabytes( std::size_t const Kilobytes ) {
std::ostringstream converted;
converted << std::fixed << std::setprecision( 1 ) << ( Kilobytes / 1024.0 ) << " MB";
return converted.str();
}
std::string
seconds( double const Seconds ) {
std::ostringstream converted;
converted << std::fixed << std::setprecision( 2 ) << Seconds << " s";
return converted.str();
}
} // namespace
bool
enabled() {
return profiling;
}
void
enable( bool const Enable ) {
profiling = Enable;
}
void
reset() {
stagetime.fill( 0.0 );
nodecount.fill( 0 );
kindtime.fill( 0.0 );
totaltime = 0.0;
vertexcount = 0;
splitcount = 0;
splitpieces = 0;
memoryatstart = resident_memory();
}
void
add_time( stage const Stage, double const Seconds ) {
if( false == profiling ) { return; }
stagetime[ static_cast<std::size_t>( Stage ) ] += Seconds;
}
void
add_total( double const Seconds ) {
if( false == profiling ) { return; }
totaltime += Seconds;
}
void
add_time( kind const Kind, double const Seconds ) {
if( false == profiling ) { return; }
kindtime[ static_cast<std::size_t>( Kind ) ] += Seconds;
}
void
add_node( kind const Kind ) {
if( false == profiling ) { return; }
++nodecount[ static_cast<std::size_t>( Kind ) ];
}
void
add_vertices( std::size_t const Vertices ) {
if( false == profiling ) { return; }
vertexcount += Vertices;
}
void
add_split( std::size_t const Pieces ) {
if( false == profiling ) { return; }
++splitcount;
splitpieces += Pieces;
}
kind
kind_of( std::string_view const Type ) {
if( ( Type == "triangles" ) || ( Type == "triangle_strip" ) || ( Type == "triangle_fan" ) ) { return kind::triangles; }
if( ( Type == "lines" ) || ( Type == "line_strip" ) || ( Type == "line_loop" ) ) { return kind::lines; }
if( Type == "track" ) { return kind::track; }
if( Type == "traction" ) { return kind::traction; }
if( Type == "model" ) { return kind::model; }
if( Type == "dynamic" ) { return kind::dynamic; }
if( Type == "memcell" ) { return kind::memcell; }
if( Type == "eventlauncher" ) { return kind::eventlauncher; }
if( Type == "sound" ) { return kind::sound; }
return kind::other;
}
void
report( std::string const &Scenariofile ) {
if( false == profiling ) { return; }
auto const memoryatend { resident_memory() };
std::ostringstream output;
output << "-- scenery load profile: " << Scenariofile << "\n";
// the node stages are measured inside the token loop, so what is left of the loop
// time once they are taken out is the parsing itself. the divider is a subset of
// the insert stage and stays out of both sums
auto accounted { 0.0 };
for( auto stageindex { 0u }; stageindex < static_cast<std::size_t>( stage::count ); ++stageindex ) {
if( ( stageindex != static_cast<std::size_t>( stage::shape_divide ) )
&& ( stageindex != static_cast<std::size_t>( stage::parse ) ) ) {
accounted += stagetime[ stageindex ];
}
}
stagetime[ static_cast<std::size_t>( stage::parse ) ] = ( totaltime > accounted ? totaltime - accounted : 0.0 );
auto const total { totaltime > accounted ? totaltime : accounted };
for( auto stageindex { 0u }; stageindex < static_cast<std::size_t>( stage::count ); ++stageindex ) {
auto const share {
total > 0.0 ?
100.0 * stagetime[ stageindex ] / total :
0.0 };
output
<< " " << std::left << std::setw( 20 ) << stagenames[ stageindex ]
<< std::right << std::setw( 10 ) << seconds( stagetime[ stageindex ] )
<< std::setw( 8 ) << std::fixed << std::setprecision( 1 ) << share << " %\n";
}
output << " " << std::left << std::setw( 20 ) << "total" << std::right << std::setw( 10 ) << seconds( total ) << "\n";
output << "-- nodes\n";
for( auto kindindex { 0u }; kindindex < static_cast<std::size_t>( kind::count ); ++kindindex ) {
if( nodecount[ kindindex ] == 0 ) { continue; }
output
<< " " << std::left << std::setw( 20 ) << kindnames[ kindindex ]
<< std::right << std::setw( 12 ) << nodecount[ kindindex ] << "\n";
}
output << "-- time by node kind\n";
for( auto kindindex { 0u }; kindindex < static_cast<std::size_t>( kind::count ); ++kindindex ) {
if( kindtime[ kindindex ] < 0.005 ) { continue; }
output
<< " " << std::left << std::setw( 20 ) << kindnames[ kindindex ]
<< std::right << std::setw( 10 ) << seconds( kindtime[ kindindex ] )
<< std::setw( 12 ) << nodecount[ kindindex ] << " nodes\n";
}
output << "-- nodes\n";
output << " " << std::left << std::setw( 20 ) << "shape vertices" << std::right << std::setw( 12 ) << vertexcount << "\n";
output << " " << std::left << std::setw( 20 ) << "shapes divided" << std::right << std::setw( 12 ) << splitcount
<< " (extra pieces: " << splitpieces << ")\n";
output << "-- memory (resident)\n";
output << " " << std::left << std::setw( 20 ) << "before load" << std::right << std::setw( 12 ) << megabytes( memoryatstart ) << "\n";
output << " " << std::left << std::setw( 20 ) << "after load" << std::right << std::setw( 12 ) << megabytes( memoryatend ) << "\n";
output << " " << std::left << std::setw( 20 ) << "delta" << std::right << std::setw( 12 )
<< megabytes( memoryatend > memoryatstart ? memoryatend - memoryatstart : 0 ) << "\n";
WriteLog( output.str() );
std::ofstream file( "loadprofile.txt", std::ios::app );
if( true == file.good() ) {
file << output.str() << std::endl;
}
}
} // loadprofile

108
simulation/loadprofile.cppm Normal file
View File

@@ -0,0 +1,108 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <array>
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <string>
#include <string_view>
export module eu07.simulation.loadprofile;
export {
// Scenery load profiler. Answers the questions phase 0 of the binary format plan asks:
// where the load time goes, how many nodes of each kind the scenery holds, and how much
// memory the result occupies. Measurement only - nothing here changes what gets loaded.
namespace loadprofile {
// stages the load time is split into. shape_import covers parsing the vertex data of a
// triangle node, shape_insert the region side of it: trimming to cell boundaries and
// handing the geometry to its bank
enum class stage : std::size_t {
parse, // token loop remainder: total time less every stage below
shape_import,
shape_insert,
shape_divide, // RaTriangleDivider, the subset of shape_insert spent splitting
other_nodes, // every node type other than the triangle family
count
};
// node kinds counted separately; anything else lands in 'other'
enum class kind : std::size_t {
triangles,
lines,
track,
traction,
model,
dynamic,
memcell,
eventlauncher,
sound,
other,
count
};
// discards everything gathered so far and takes the starting memory reading
void reset();
// accumulates elapsed time in the given stage
void add_time( stage Stage, double const Seconds );
// accumulates elapsed time against a node kind, so that "everything else" can be told
// apart into what it actually is
void add_time( kind const Kind, double const Seconds );
// accumulates elapsed time of the whole token loop. the parse stage is reported as
// what is left of it once the node stages are taken out
void add_total( double const Seconds );
// registers a node of the given kind
void add_node( kind const Kind );
// adds to the tally of vertices held by shape nodes
void add_vertices( std::size_t const Vertices );
// registers a shape split into extra pieces by the cell-boundary trimmer
void add_split( std::size_t const Pieces );
// true when profiling is switched on; the instrumentation is skipped otherwise
bool enabled();
// switches profiling on or off. off by default, turned on by the -loadprofile switch
void enable( bool const Enable );
// resolves the node type name used by the scenario format to a counter
kind kind_of( std::string_view const Type );
// writes the gathered numbers to the log and to loadprofile.txt in the working directory
void report( std::string const &Scenariofile );
// RAII helper accumulating the lifetime of the scope against a stage or a node kind. the
// clock is not read at all while profiling is off, since this sits on per-node paths
template <typename Counter_>
class scoped_timer {
public:
explicit scoped_timer( Counter_ const Counter ) :
m_counter( Counter ) {
if( true == enabled() ) {
m_start = std::chrono::steady_clock::now();
m_running = true;
} }
~scoped_timer() {
if( false == m_running ) { return; }
add_time(
m_counter,
std::chrono::duration<double>( std::chrono::steady_clock::now() - m_start ).count() ); }
scoped_timer( scoped_timer const & ) = delete;
scoped_timer &operator=( scoped_timer const & ) = delete;
private:
Counter_ m_counter;
std::chrono::steady_clock::time_point m_start;
bool m_running { false };
};
} // loadprofile
} // export

264
simulation/logiccook.cpp Normal file
View File

@@ -0,0 +1,264 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <cstdint>
#include <string>
#include <algorithm>
#include <set>
#include <utility>
#include <vector>
#include "utilities/Globals_macros.h"
#include "global_include/interfaces/ITexture_macros.h"
#include "scene/logicformat.h"
module eu07.simulation.logiccook;
import eu07.scene.logiccontainer;
import eu07.simcore;
import eu07.simulation.simulation;
import eu07.simulation.simulationsounds;
import eu07.utilities.logs;
import eu07.utilities.globals;
import eu07.utilities.utilities;
namespace simulation {
namespace {
// the flat record an event becomes. everything here is read off the event itself, so the
// container cannot drift from what the text path built
scene::logic::event_record
record_of( basic_event const &Event, scene::logic::event_kind const Kind, std::uint32_t const Sibling, std::uint32_t const Name ) {
scene::logic::event_record record {};
record.kind = static_cast<std::uint16_t>( Kind );
record.name = Name;
record.sibling = ( Sibling < simulation::Events.event_count() ? Sibling : scene::logic::no_index );
record.group = ( Event.group() == null_handle ? scene::logic::no_index : static_cast<std::uint32_t>( Event.group() ) );
record.delay = Event.m_delay;
record.delayrandom = Event.m_delayrandom;
record.delaydeparture = Event.m_delaydeparture;
record.ignored = ( Event.m_ignored ? 1 : 0 );
record.passive = ( Event.m_passive ? 1 : 0 );
return record;
}
// where an event of this kind looks for its targets, and where it looks next. this mirrors
// each subclass's init() rather than guessing: an event that resolves against the memory
// cells will never find its target among the tracks, and saying otherwise here would bind
// it to whatever happened to share the name
std::pair<scene::logic::node_table, scene::logic::node_table>
tables_for( scene::logic::event_kind const Kind ) {
using kind = scene::logic::event_kind;
using table = scene::logic::node_table;
switch( Kind ) {
case kind::updatevalues:
case kind::addvalues:
case kind::copyvalues:
case kind::getvalues:
case kind::putvalues:
case kind::whois:
case kind::logvalues:
case kind::multiple: return { table::memory, table::none };
case kind::texture:
case kind::animation:
case kind::lights: return { table::instance, table::none };
case kind::switchevent:
case kind::trackvel: return { table::path, table::none };
case kind::voltage: return { table::powergrid, table::none };
case kind::sound: return { table::sound, table::none };
// a visible event takes model instances and tracks alike, and looks in that order
case kind::visible: return { table::instance, table::path };
default: return { table::none, table::none };
}
}
std::uint32_t
index_in( scene::logic::node_table const Table, std::string const &Name ) {
using table = scene::logic::node_table;
switch( Table ) {
case table::memory: return simulation::Memory.find_id( Name );
case table::instance: return simulation::Instances.find_id( Name );
case table::path: return simulation::Paths.find_id( Name );
case table::sound: return simulation::Sounds.find_id( Name );
case table::powergrid: return simulation::Powergrid.find_id( Name );
default: return static_cast<std::uint32_t>( -1 );
}
}
// the entries an event's targets become, each one carrying the registry it was found in
std::vector<scene::logic::target_entry>
targets_of( basic_event const &Event, scene::logic::event_kind const Kind, scene::logic::container_writer &Writer ) {
auto const kind { Kind };
auto const [ first, second ] { tables_for( kind ) };
std::vector<scene::logic::target_entry> entries;
for( auto const &name : Event.target_names() ) {
scene::logic::target_entry entry {};
entry.name = Writer.intern( name );
entry.table = static_cast<std::uint16_t>( scene::logic::node_table::none );
entry.index = scene::logic::no_index;
for( auto const table : { first, second } ) {
if( table == scene::logic::node_table::none ) { continue; }
auto const index { index_in( table, name ) };
if( index != static_cast<std::uint32_t>( -1 ) ) {
entry.table = static_cast<std::uint16_t>( table );
entry.index = index;
break;
}
}
entries.emplace_back( entry );
}
return entries;
}
} // anonymous namespace
bool
cook_logic( std::string const &Sceneryfile ) {
scene::logic::container_writer writer;
auto const count { simulation::Events.event_count() };
std::uint32_t unknown { 0 };
std::set<std::string> unknownkinds;
for( std::uint32_t index = 0; index < count; ++index ) {
auto const *cooked { simulation::Events.event_at( index ) };
if( cooked == nullptr ) { continue; }
auto const name { writer.intern( cooked->name() ) };
auto const kind { scene::logic::kind_of( cooked->type_name() ) };
auto const record { record_of( *cooked, kind, simulation::Events.sibling_of( index ), name ) };
if( record.kind == static_cast<std::uint16_t>( scene::logic::event_kind::unknown ) ) {
++unknown;
if( unknownkinds.insert( cooked->type_name() ).second ) {
ErrorLog( "Logic container: no cooked kind for event type \"" + cooked->type_name() + "\" (first seen on \"" + cooked->name() + "\")" );
}
}
writer.add_event( record, targets_of( *cooked, kind, writer ) );
}
if( unknown > 0 ) {
// a kind the container has no value for would come back as something else entirely,
// so say so loudly rather than write a file that reads back wrong
ErrorLog( "Logic container: " + std::to_string( unknown ) + " events of a kind this build cannot cook" );
}
return writer.write( scene::logic::container_path( Sceneryfile ) );
}
bool
logic_stale( std::string const &Sceneryfile ) {
auto const path { scene::logic::container_path( Sceneryfile ) };
if( false == FileExists( path ) ) {
return true;
}
if( last_modified( scenery_file( Sceneryfile ) ) > last_modified( path ) ) {
WriteLog( "Logic container: \"" + path + "\" is older than the scenery, recooking" );
return true;
}
// the header says whether this build can read it at all, which covers a container
// cooked before a schema change
if( false == scene::logic::container_reader::readable( path ) ) {
WriteLog( "Logic container: \"" + path + "\" cannot be read by this build, recooking" );
return true;
}
return false;
}
bool
verify_logic( std::string const &Sceneryfile ) {
scene::logic::container_reader reader;
auto const path { scene::logic::container_path( Sceneryfile ) };
if( false == reader.open( path ) ) {
ErrorLog( "Logic container: cannot verify \"" + path + "\"" );
return false;
}
std::uint32_t mismatches { 0 };
auto const count { simulation::Events.event_count() };
if( reader.events() != count ) {
ErrorLog(
"Logic container: holds " + std::to_string( reader.events() )
+ " events, the scenery has " + std::to_string( count ) );
++mismatches;
}
auto const shared { std::min<std::size_t>( reader.events(), count ) };
for( std::size_t index = 0; index < shared; ++index ) {
auto const *loaded { simulation::Events.event_at( static_cast<std::uint32_t>( index ) ) };
if( loaded == nullptr ) { continue; }
auto const &record { reader.event( index ) };
auto const complain {
[ &mismatches, loaded ]( std::string const &What ) {
ErrorLog( "Logic container: event \"" + loaded->name() + "\" " + What );
++mismatches;
} };
if( reader.string( record.name ) != loaded->name() ) {
complain( "is called \"" + std::string( reader.string( record.name ) ) + "\" in the container" );
continue;
}
if( record.kind != static_cast<std::uint16_t>( scene::logic::kind_of( loaded->type_name() ) ) ) {
complain( "was cooked as a different kind" );
}
// delays are written as they are held, so they compare exactly. a difference here
// means the record lost precision, which is the whole thing worth catching
if( record.delay != loaded->m_delay ) {
complain( "has a different delay in the container" );
}
if( record.delayrandom != loaded->m_delayrandom ) {
complain( "has a different random delay in the container" );
}
auto const targets { loaded->target_names() };
if( record.targetcount != targets.size() ) {
complain(
"has " + std::to_string( record.targetcount ) + " targets in the container, "
+ std::to_string( targets.size() ) + " in the scenery" );
continue;
}
for( std::uint32_t target = 0; target < record.targetcount; ++target ) {
auto const &entry { reader.target( record.targetfirst + target ) };
if( reader.string( entry.name ) != targets[ target ] ) {
complain( "target " + std::to_string( target ) + " is \"" + std::string( reader.string( entry.name ) ) + "\", not \"" + targets[ target ] + "\"" );
continue;
}
// the index is only meaningful for the parse the container was cooked from, so
// the check that matters is whether it still lands on that same name
if( entry.index != scene::logic::no_index ) {
auto const table { static_cast<scene::logic::node_table>( entry.table ) };
if( index_in( table, targets[ target ] ) != entry.index ) {
complain( "target " + std::to_string( target ) + " \"" + targets[ target ] + "\" is at a different index now" );
}
}
}
}
if( mismatches == 0 ) {
WriteLog( "Logic container: \"" + path + "\" matches the loaded scenery, " + std::to_string( count ) + " events" );
return true;
}
ErrorLog( "Logic container: \"" + path + "\" disagrees with the loaded scenery in " + std::to_string( mismatches ) + " places" );
return false;
}
} // namespace simulation

42
simulation/logiccook.cppm Normal file
View File

@@ -0,0 +1,42 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <string>
export module eu07.simulation.logiccook;
export {
// Cooking the LOGIC container out of a scenery that has just been loaded.
//
// The plan called for a standalone tools/scn-cook reading .scn directly. It is cooked from
// inside the engine instead, and on purpose: an event's targets, siblings and indices are the
// product of the whole scenery deserializer, not of the parser, so a standalone cooker would
// have to grow a second copy of it. Cooking from the loaded state guarantees the thing that
// actually matters - that what is written is exactly what the text path produces - and the
// determinism the plan asks for comes from the writer, not from where it is called.
namespace simulation {
// writes the container next to the scenery. returns false and logs on failure
bool cook_logic( std::string const &Sceneryfile );
// reads a cooked container back and checks it against the events currently loaded. logs
// every disagreement and returns true only when there are none
bool verify_logic( std::string const &Sceneryfile );
// whether the container next to this scenery needs cooking: there is none, this build
// cannot read the one that is there, or the scenery has been edited since.
//
// Edits to an .inc the scenery pulls in do NOT make it stale - only the .scn's own
// timestamp is compared, because the list of files that went into a load is not kept. Cook
// by hand with -cooklogic after editing an include.
bool logic_stale( std::string const &Sceneryfile );
} // namespace simulation
}

View File

@@ -17,6 +17,7 @@ module;
#include <cstddef> #include <cstddef>
#include <cstdlib> #include <cstdlib>
#include <memory> #include <memory>
#include <optional>
#include <string> #include <string>
#include "global_include/interfaces/ITexture_macros.h" #include "global_include/interfaces/ITexture_macros.h"
#include "utilities/Globals_macros.h" #include "utilities/Globals_macros.h"
@@ -29,6 +30,7 @@ import eu07.utilities.classes;
import eu07.glm; import eu07.glm;
import eu07.utilities.globals; import eu07.utilities.globals;
import eu07.simulation.simulation; import eu07.simulation.simulation;
import eu07.simulation.loadprofile;
import eu07.simulation.simulationtime; import eu07.simulation.simulationtime;
import eu07.simulation.simulationsounds; import eu07.simulation.simulationsounds;
import eu07.simulation.simulationenvironment; import eu07.simulation.simulationenvironment;
@@ -80,6 +82,7 @@ state_serializer::deserialize_begin( std::string const &Scenariofile ) {
// TODO: check first for presence of serialized binary files // TODO: check first for presence of serialized binary files
// if this fails, fall back on the legacy text format // if this fails, fall back on the legacy text format
state->scratchpad.name = Scenariofile; state->scratchpad.name = Scenariofile;
loadprofile::reset();
if( true == Global.file_binary_terrain if( true == Global.file_binary_terrain
&& Scenariofile != "$.scn" ) { && Scenariofile != "$.scn" ) {
// compilation to binary file isn't supported for rainsted-created overrides // compilation to binary file isn't supported for rainsted-created overrides
@@ -99,6 +102,11 @@ state_serializer::deserialize_begin( std::string const &Scenariofile ) {
} }
scene::Groups.create(); scene::Groups.create();
// before a token is read: pick up a heightfield baked on an earlier run, or arm the
// bake when there is none. it has to happen here because the decision to skip terrain
// triangles is taken per node while parsing
terrainbake::begin( Scenariofile );
if( false == state->input.ok() ) { if( false == state->input.ok() ) {
if( false == Global.editor_startup ) { if( false == Global.editor_startup ) {
throw invalid_scenery_exception(); throw invalid_scenery_exception();
@@ -131,6 +139,7 @@ state_serializer::deserialize_begin( std::string const &Scenariofile ) {
{ "endgroup", &state_serializer::deserialize_endgroup }, { "endgroup", &state_serializer::deserialize_endgroup },
{ "light", &state_serializer::deserialize_light }, { "light", &state_serializer::deserialize_light },
{ "node", &state_serializer::deserialize_node }, { "node", &state_serializer::deserialize_node },
{ "prefab", &state_serializer::deserialize_prefab },
{ "origin", &state_serializer::deserialize_origin }, { "origin", &state_serializer::deserialize_origin },
{ "endorigin", &state_serializer::deserialize_endorigin }, { "endorigin", &state_serializer::deserialize_endorigin },
{ "scale", &state_serializer::deserialize_scale }, { "scale", &state_serializer::deserialize_scale },
@@ -177,11 +186,14 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
auto timenow = std::chrono::steady_clock::now(); auto timenow = std::chrono::steady_clock::now();
if( std::chrono::duration_cast<std::chrono::milliseconds>( timenow - timelast ).count() >= 200 ) { if( std::chrono::duration_cast<std::chrono::milliseconds>( timenow - timelast ).count() >= 200 ) {
Application.set_progress( Input.getProgress(), Input.getFullProgress() ); Application.set_progress( Input.getProgress(), Input.getFullProgress() );
loadprofile::add_total( std::chrono::duration<double>( timenow - timelast ).count() );
return true; return true;
} }
token = Input.getToken<std::string>(); token = Input.getToken<std::string>();
} }
loadprofile::add_total(
std::chrono::duration<double>( std::chrono::steady_clock::now() - timelast ).count() );
if( false == Scratchpad.initialized ) { if( false == Scratchpad.initialized ) {
// manually perform scenario initialization // manually perform scenario initialization
@@ -202,6 +214,22 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
Region->serialize( state->scenariofile ); Region->serialize( state->scenariofile );
} }
terrainbake::finish( state->scenariofile );
loadprofile::report( state->scenariofile );
if( true == Global.verify_logic ) {
verify_logic( state->scenariofile );
}
else if( ( true == Global.cook_logic )
|| ( true == logic_stale( state->scenariofile ) ) ) {
// cooked from the loaded state rather than from the text, and immediately read back,
// so a container that disagrees with what it was cooked from says so on the spot
if( true == cook_logic( state->scenariofile ) ) {
verify_logic( state->scenariofile );
}
}
return false; return false;
} }
@@ -457,6 +485,40 @@ state_serializer::deserialize_light( cParser &Input, scene::scratch_data &Scratc
skip_until( Input, "endlight" ); skip_until( Input, "endlight" );
} }
void
state_serializer::deserialize_prefab( cParser &Input, scene::scratch_data &Scratchpad ) {
// prefab <definition> <name> <x> <y> <z> <yaw> [<texture>] endprefab
scene::prefab_placement placement;
Input.getTokens( 1, false );
std::string definition; Input >> definition;
Input.getTokens( 1, false );
Input >> placement.name;
Input.getTokens( 4 );
Input
>> placement.x
>> placement.y
>> placement.z
>> placement.yaw;
auto token { Input.getToken<std::string>( false ) };
if( ( false == token.empty() ) && ( token != "endprefab" ) ) {
placement.texture = token;
token = Input.getToken<std::string>( false );
}
if( token != "endprefab" ) {
ErrorLog( "Bad scenario: prefab \"" + placement.name + "\" is missing its endprefab, found \"" + token + "\" instead" );
}
auto const scenery { Prefabs.expand( definition, placement ) };
if( true == scenery.empty() ) {
ErrorLog( "Bad scenario: prefab \"" + placement.name + "\" of unknown type \"" + definition + "\" was skipped" );
return;
}
// the expansion is ordinary scenery text, and goes in as an include would
Input.injectString( scenery );
}
void void
state_serializer::deserialize_node( cParser &Input, scene::scratch_data &Scratchpad ) { state_serializer::deserialize_node( cParser &Input, scene::scratch_data &Scratchpad ) {
@@ -471,6 +533,19 @@ state_serializer::deserialize_node( cParser &Input, scene::scratch_data &Scratch
>> nodedata.name >> nodedata.name
>> nodedata.type; >> nodedata.type;
if( nodedata.name == "none" ) { nodedata.name.clear(); } if( nodedata.name == "none" ) { nodedata.name.clear(); }
// the triangle family is timed in its own branch below; everything else is timed
// against the kind of node it is, since "not a triangle" covers models, tracks,
// traction and vehicles alike and they cost very different amounts
std::optional<loadprofile::scoped_timer<loadprofile::stage>> profiledstage;
std::optional<loadprofile::scoped_timer<loadprofile::kind>> profiledkindtime;
if( true == loadprofile::enabled() ) {
auto const profiledkind { loadprofile::kind_of( nodedata.type ) };
loadprofile::add_node( profiledkind );
if( profiledkind != loadprofile::kind::triangles ) {
profiledstage.emplace( loadprofile::stage::other_nodes );
profiledkindtime.emplace( profiledkind );
}
}
// type-based deserialization. not elegant but it'll do // type-based deserialization. not elegant but it'll do
if( nodedata.type == "dynamic" ) { if( nodedata.type == "dynamic" ) {
@@ -604,25 +679,71 @@ state_serializer::deserialize_node( cParser &Input, scene::scratch_data &Scratch
|| nodedata.type == "triangle_strip" || nodedata.type == "triangle_strip"
|| nodedata.type == "triangle_fan" ) { || nodedata.type == "triangle_fan" ) {
auto const file { Input.Name() };
auto const switchtrackbedfile {
// crude way to detect fixed switch trackbed geometry
true == Global.CreateSwitchTrackbeds
&& file.size() >= 15
&& file.starts_with( "scenery/zwr" )
&& file.ends_with( ".inc" ) };
auto const offset {
Scratchpad.location.offset.empty() ? glm::dvec3( 0.0 ) : Scratchpad.location.offset.top() };
auto const worldspace {
( offset == glm::dvec3( 0.0 ) ) && ( Scratchpad.location.rotation == glm::vec3( 0.0f ) ) };
// whether a heightfield stands in for this node, and whether a bake wants it
auto const decision {
( ( true == Scratchpad.binary.terrain ) || ( true == switchtrackbedfile ) )
? terrainbake::node_decision{}
: terrainbake::examine(
file, Input.Line(), Input.Parameters(), nodedata.type,
offset, Scratchpad.location.rotation, worldspace ) };
auto const skip { auto const skip {
// all shapes will be loaded from the binary version of the file // all shapes will be loaded from the binary version of the file
true == Scratchpad.binary.terrain ( true == Scratchpad.binary.terrain )
// crude way to detect fixed switch trackbed geometry // old trackbed geometry the engine replaces
|| ( true == Global.CreateSwitchTrackbeds || ( true == switchtrackbedfile )
&& Input.Name().size() >= 15 || ( true == decision.skip ) };
&& Input.Name().starts_with("scenery/zwr")
&& Input.Name().ends_with(".inc") ) };
if( false == skip ) { if( true == skip ) {
skip_until( Input, "endtri" );
simulation::Region->insert(
scene::shape_node().import(
Input, nodedata ),
Scratchpad,
true );
} }
else { else {
skip_until( Input, "endtri" ); auto shape { [&]() {
loadprofile::scoped_timer const importtime { loadprofile::stage::shape_import };
return scene::shape_node().import( Input, nodedata ); }() };
loadprofile::add_vertices( shape.data().vertices.size() );
if( true == decision.bake ) {
// the bake sees the node exactly where it is drawn. placing is done on a copy:
// the region applies the same placement to the shape itself when inserting it
auto const &data { shape.data() };
auto const material {
data.material != null_handle
? GfxRenderer->Material( data.material )->GetName()
: "none" };
// the old switch trackbeds are dropped from the scenery when generated ones
// replace them, so they are no part of the ground either
if( false == ( Global.CreateSwitchTrackbeds && scene::is_switch_trackbed( material ) ) ) {
auto placed { data.vertices };
scene::place_in_world( placed, Scratchpad );
terrainbake::add( decision.key, placed, material );
}
}
if( decision.plan.what == terrainbake::verdict::filter ) {
auto const &drawn { decision.plan.drawn };
shape.keep_triangles(
[ &drawn ]( std::size_t const Triangle ) {
return ( Triangle >= drawn.size() ) || drawn[ Triangle ]; } );
}
loadprofile::scoped_timer const inserttime { loadprofile::stage::shape_insert };
simulation::Region->insert(
std::move( shape ),
Scratchpad,
true );
} }
} }
else if( nodedata.type == "lines" else if( nodedata.type == "lines"
@@ -827,7 +948,14 @@ state_serializer::deserialize_terrain(cParser &Input, scene::scratch_data &Scrat
std::string line; std::string line;
Input.getTokens(1); Input.getTokens(1);
Input >> line; Input >> line;
if (Global.file_binary_terrain && line.ends_with(".sbt")) if (line.ends_with(".ehf"))
{
// a cooked heightfield. the renderer opens it and the scenery carries no terrain
// geometry at all, which is the point: the text it replaces is most of the file
Global.terrain_heightfields.push_back(line);
WriteLog("Terrain heightfield: " + line);
}
else if (Global.file_binary_terrain && line.ends_with(".sbt"))
{ {
Scratchpad.binary.terrain = Region->is_scene(line); Scratchpad.binary.terrain = Region->is_scene(line);
Global.file_binary_terrain_state = true; Global.file_binary_terrain_state = true;

View File

@@ -28,6 +28,9 @@ import eu07.glm;
import eu07.audio.sound; import eu07.audio.sound;
import eu07.utilities.parser; import eu07.utilities.parser;
import eu07.simcore; import eu07.simcore;
import eu07.scene.prefab;
import eu07.simulation.logiccook;
import eu07.simulation.terrainbake;
export { export {
@@ -86,6 +89,8 @@ private:
void deserialize_endgroup( cParser &Input, scene::scratch_data &Scratchpad ); void deserialize_endgroup( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_light( cParser &Input, scene::scratch_data &Scratchpad ); void deserialize_light( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_node( cParser &Input, scene::scratch_data &Scratchpad ); void deserialize_node( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_prefab( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_origin( cParser &Input, scene::scratch_data &Scratchpad ); void deserialize_origin( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_endorigin( cParser &Input, scene::scratch_data &Scratchpad ); void deserialize_endorigin( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_scale( cParser &Input, scene::scratch_data &Scratchpad ); void deserialize_scale( cParser &Input, scene::scratch_data &Scratchpad );

463
simulation/terrainbake.cpp Normal file
View File

@@ -0,0 +1,463 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <filesystem>
#include <fstream>
#include <string>
#include <string_view>
#include <system_error>
#include <unordered_map>
#include <vector>
#include "utilities/Globals_macros.h"
#include "scene/terraincooker.h"
#include "scene/heightfieldreader.h"
module eu07.simulation.terrainbake;
import eu07.utilities.logs;
import eu07.utilities.globals;
import eu07.utilities.utilities;
import eu07.simulation.loadprofile;
namespace simulation::terrainbake {
namespace {
// triangles are buffered until there are enough vertices to say what the grid step is, and
// only then rasterised. the step has to be known before the first sample is placed, and it
// is a property of the whole terrain rather than of the first triangle that turns up
constexpr std::size_t samplegoal { 200000 };
// the node table written beside a baked heightfield
constexpr char tablemagic[ 8 ] { 'E', 'U', '0', '7', 'H', 'F', 'N', 'D' };
// 2: nodes keyed by file, line, include parameters and placement rather than by their text
constexpr std::uint32_t tableversion { 2 };
constexpr std::uint32_t allshown { 0xffffffffu };
struct table_header {
char magic[ 8 ];
std::uint32_t version;
std::uint32_t selection;
std::uint64_t count;
std::uint64_t maskbytes;
};
struct table_record {
std::uint64_t key;
std::uint32_t triangles;
// offset into the mask blob, one bit per triangle set where it is still drawn; allshown
// when the heightfield shows every triangle and there is no mask
std::uint32_t mask;
};
struct file_stamp {
std::uint64_t size { 0 };
std::int64_t time { 0 };
};
struct buffered {
terrain::vertex v[ 3 ];
std::uint32_t node; // the node it came from, which holds its material
std::uint32_t id;
};
// one node as baked: where its triangles sit in the cooker's numbering
struct baked_node {
std::uint64_t key;
std::uint32_t first;
std::uint32_t count;
// kept once per node rather than once per buffered triangle
std::string material;
};
struct {
// baking
bool collecting { false };
bool configured { false };
terrain::cooker cooker;
std::vector<buffered> buffer;
std::vector<baked_node> nodes;
std::uint32_t triangles { 0 };
std::chrono::steady_clock::time_point started;
// planning
std::unordered_map<std::uint64_t, table_record> table;
std::vector<std::uint8_t> masks;
std::size_t skipped { 0 }, filtered { 0 }, drawn { 0 };
// size and modification time of every file a key was taken in, looked up once per load
std::unordered_map<std::string, file_stamp> files;
} state;
std::string
heightfield_path( std::string const &Sceneryfile ) {
return scenery_sidecar( Sceneryfile, ".ehf" );
}
std::string
table_path( std::string const &Heightfield ) {
return Heightfield + ".nodes";
}
bool
load_table( std::string const &Path ) {
std::ifstream input( Path, std::ios::binary );
if( false == input.good() ) { return false; }
table_header header {};
input.read( reinterpret_cast<char *>( &header ), sizeof( header ) );
if( ( false == static_cast<bool>( input ) )
|| ( 0 != std::memcmp( header.magic, tablemagic, sizeof( tablemagic ) ) )
|| ( header.version != tableversion )
|| ( header.selection != heightfield::selection_rules ) ) {
return false;
}
std::vector<table_record> records( header.count );
input.read( reinterpret_cast<char *>( records.data() ), static_cast<std::streamsize>( header.count * sizeof( table_record ) ) );
state.masks.resize( header.maskbytes );
input.read( reinterpret_cast<char *>( state.masks.data() ), static_cast<std::streamsize>( header.maskbytes ) );
if( false == static_cast<bool>( input ) ) { return false; }
state.table.reserve( records.size() );
for( auto const &record : records ) {
state.table.emplace( record.key, record );
}
return true;
}
void
write_table( std::string const &Path ) {
// a key met more than once is the same line of the same file, included with the same
// parameters at the same place - the same surface: a triangle counts as shown if any of
// the copies is
std::unordered_map<std::uint64_t, std::vector<bool>> shown;
for( auto const &node : state.nodes ) {
auto &entry { shown[ node.key ] };
entry.resize( node.count, false );
for( std::uint32_t index = 0; index < node.count; ++index ) {
if( true == state.cooker.represented( node.first + index ) ) { entry[ index ] = true; }
}
}
std::vector<table_record> records;
std::vector<std::uint8_t> masks;
for( auto const &node : state.nodes ) {
auto const lookup { shown.find( node.key ) };
if( lookup == shown.end() ) { continue; } // written already
auto const &entry { lookup->second };
std::uint32_t count { 0 };
for( auto const flag : entry ) { count += ( flag ? 1 : 0 ); }
if( count > 0 ) {
table_record record { node.key, node.count, allshown };
if( count < node.count ) {
record.mask = static_cast<std::uint32_t>( masks.size() );
auto const bytes { ( node.count + 7u ) / 8u };
masks.resize( masks.size() + bytes, 0u );
for( std::uint32_t index = 0; index < node.count; ++index ) {
if( false == entry[ index ] ) {
masks[ record.mask + index / 8u ] |= static_cast<std::uint8_t>( 1u << ( index % 8u ) );
}
}
}
records.emplace_back( record );
}
shown.erase( lookup );
}
table_header header {};
std::memcpy( header.magic, tablemagic, sizeof( tablemagic ) );
header.version = tableversion;
header.selection = heightfield::selection_rules;
header.count = records.size();
header.maskbytes = masks.size();
std::ofstream output( Path, std::ios::binary | std::ios::trunc );
output.write( reinterpret_cast<char const *>( &header ), sizeof( header ) );
output.write( reinterpret_cast<char const *>( records.data() ), static_cast<std::streamsize>( records.size() * sizeof( table_record ) ) );
output.write( reinterpret_cast<char const *>( masks.data() ), static_cast<std::streamsize>( masks.size() ) );
std::size_t full { 0 };
for( auto const &record : records ) { full += ( record.mask == allshown ? 1 : 0 ); }
WriteLog(
"Terrain bake: " + std::to_string( records.size() ) + " of " + std::to_string( state.nodes.size() )
+ " triangle nodes are shown by the heightfield, " + std::to_string( full ) + " of them entirely" );
}
void
rasterize( buffered const &Triangle ) {
state.cooker.rasterize(
Triangle.v[ 0 ], Triangle.v[ 1 ], Triangle.v[ 2 ], state.nodes[ Triangle.node ].material, Triangle.id );
}
// works the grid step out of what has been buffered, then puts the buffer through
void
configure_and_drain() {
std::vector<terrain::vertex> sample;
sample.reserve( state.buffer.size() * 3 );
for( auto const &triangle : state.buffer ) {
sample.emplace_back( triangle.v[ 0 ] );
sample.emplace_back( triangle.v[ 1 ] );
sample.emplace_back( triangle.v[ 2 ] );
}
auto const step { terrain::detect_step( sample ) };
WriteLog( "Terrain bake: grid step " + std::to_string( step ) + " m, from " + std::to_string( sample.size() ) + " sampled vertices" );
if( false == loadprofile::enabled() ) {
// the per-material survival table is what a bad bake is diagnosed from, so it is
// printed when the load is being measured anyway
state.cooker.quiet();
}
state.cooker.configure( step, true );
state.configured = true;
for( auto const &triangle : state.buffer ) {
rasterize( triangle );
}
state.buffer.clear();
state.buffer.shrink_to_fit();
}
} // anonymous namespace
namespace {
std::uint64_t
node_key(
std::string const &File, std::size_t const Line, std::vector<std::string> const &Parameters,
std::string_view const Type, glm::dvec3 const &Offset, glm::vec3 const &Rotation ) {
// fnv-1a. stable across runs and builds, which a std::hash is not required to be
std::uint64_t hash { 0xcbf29ce484222325ull };
auto const mix {
[ &hash ]( void const *Data, std::size_t const Size ) {
auto const *bytes { static_cast<unsigned char const *>( Data ) };
for( std::size_t index = 0; index < Size; ++index ) {
hash ^= bytes[ index ];
hash *= 0x100000001b3ull;
} } };
auto const text {
[ &mix ]( std::string_view const Text ) {
mix( Text.data(), Text.size() );
mix( "\n", 1 ); } };
// what the file is now: an edit anywhere in it changes every key it holds
auto lookup { state.files.find( File ) };
if( lookup == state.files.end() ) {
std::error_code error;
file_stamp stamp {};
auto const size { std::filesystem::file_size( File, error ) };
stamp.size = ( error ? 0 : static_cast<std::uint64_t>( size ) );
auto const time { std::filesystem::last_write_time( File, error ) };
stamp.time = ( error ? 0 : static_cast<std::int64_t>( time.time_since_epoch().count() ) );
lookup = state.files.emplace( File, stamp ).first;
}
text( File );
mix( &lookup->second.size, sizeof( lookup->second.size ) );
mix( &lookup->second.time, sizeof( lookup->second.time ) );
auto const line { static_cast<std::uint64_t>( Line ) };
mix( &line, sizeof( line ) );
for( auto const &parameter : Parameters ) { text( parameter ); }
text( Type );
mix( &Offset.x, sizeof( Offset.x ) ); mix( &Offset.y, sizeof( Offset.y ) ); mix( &Offset.z, sizeof( Offset.z ) );
mix( &Rotation.x, sizeof( Rotation.x ) ); mix( &Rotation.y, sizeof( Rotation.y ) ); mix( &Rotation.z, sizeof( Rotation.z ) );
return hash;
}
} // anonymous namespace
void
begin( std::string const &Sceneryfile ) {
state = {};
if( false == Global.terrain_heightfields.empty() ) {
// the scenery names its own, which always wins over anything baked beside it
return;
}
if( false == Global.bake_terrain ) {
// neither bake nor pick one up: the triangles as written
return;
}
auto const path { heightfield_path( Sceneryfile ) };
std::error_code error;
if( true == FileExists( path ) ) {
// a baked file is a cache: one this build cannot read, one cooked under older rules
// about what counts as terrain, or one whose node table is missing, is thrown away
// and baked again rather than used
heightfield::reader probe;
if( ( true == probe.open( path ) )
&& ( probe.selection() == heightfield::selection_rules )
&& ( true == load_table( table_path( path ) ) ) ) {
Global.terrain_heightfields.push_back( path );
WriteLog( "Terrain bake: using \"" + path + "\", baked earlier, " + std::to_string( state.table.size() ) + " triangle nodes replaced by it" );
return;
}
WriteLog( "Terrain bake: \"" + path + "\" was baked under different rules, baking it again" );
probe.close();
state.table.clear();
state.masks.clear();
std::filesystem::remove( path, error );
std::filesystem::remove( table_path( path ), error );
}
WriteLog( "Terrain bake: no heightfield for this scenery, baking one. This load will be slow; the next will not" );
state.collecting = true;
state.started = std::chrono::steady_clock::now();
}
namespace {
bool
planning() {
return false == state.table.empty();
}
node_plan
plan( std::uint64_t const Key ) {
node_plan result;
auto const lookup { state.table.find( Key ) };
if( lookup == state.table.end() ) {
++state.drawn;
return result;
}
auto const &record { lookup->second };
if( record.mask == allshown ) {
++state.skipped;
result.what = verdict::skip;
return result;
}
++state.filtered;
result.what = verdict::filter;
result.drawn.resize( record.triangles, false );
for( std::uint32_t index = 0; index < record.triangles; ++index ) {
auto const byte { record.mask + index / 8u };
result.drawn[ index ] = ( byte < state.masks.size() ) && ( ( state.masks[ byte ] >> ( index % 8u ) ) & 1u ) != 0;
}
return result;
}
// collecting for a bake, and not overruled by a heightfield the scenery named part way in
bool
baking() {
return state.collecting && Global.terrain_heightfields.empty();
}
} // anonymous namespace
node_decision
examine(
std::string const &File, std::size_t const Line, std::vector<std::string> const &Parameters,
std::string_view const Type, glm::dvec3 const &Offset, glm::vec3 const &Rotation, bool const Worldspace ) {
node_decision decision;
if( ( true == baking() ) || ( true == planning() ) ) {
decision.key = node_key( File, Line, Parameters, Type, Offset, Rotation );
decision.bake = baking();
if( false == decision.bake ) {
decision.plan = plan( decision.key );
decision.skip = ( decision.plan.what == verdict::skip );
}
return decision;
}
// a heightfield the scenery names itself, cooked by hand, replaces the world space
// triangles outright, as it always has
decision.skip = ( true == Worldspace ) && ( false == Global.terrain_heightfields.empty() );
return decision;
}
void
add( std::uint64_t const Key, std::vector<world_vertex> const &Vertices, std::string_view const Material ) {
if( false == state.collecting ) { return; }
auto const count { static_cast<std::uint32_t>( Vertices.size() / 3 ) };
auto const node { static_cast<std::uint32_t>( state.nodes.size() ) };
state.nodes.push_back( { Key, state.triangles, count, std::string { Material } } );
for( std::uint32_t triangle = 0; triangle < count; ++triangle ) {
auto const vertex = [ & ]( std::size_t const Index ) {
auto const &source { Vertices[ triangle * 3u + Index ] };
return terrain::vertex{ source.position.x, source.position.y, source.position.z, source.texture.x, source.texture.y }; };
buffered entry { { vertex( 0 ), vertex( 1 ), vertex( 2 ) }, node, state.triangles + triangle };
if( true == state.configured ) {
rasterize( entry );
continue;
}
state.buffer.emplace_back( entry );
}
state.triangles += count;
if( ( false == state.configured ) && ( state.buffer.size() * 3 >= samplegoal ) ) {
configure_and_drain();
}
}
void
finish( std::string const &Sceneryfile ) {
if( true == planning() ) {
WriteLog(
"Terrain bake: " + std::to_string( state.skipped ) + " triangle nodes left to the heightfield, "
+ std::to_string( state.filtered ) + " drawn in part, " + std::to_string( state.drawn ) + " drawn as written" );
}
if( false == state.collecting ) { return; }
if( false == Global.terrain_heightfields.empty() ) {
// the scenery named its own heightfield part way through, and that one wins
WriteLog( "Terrain bake: the scenery names its own heightfield, nothing baked" );
state = {};
return;
}
if( false == state.configured ) {
// a scenery with less terrain than the sample goal never triggered the drain
if( true == state.buffer.empty() ) {
WriteLog( "Terrain bake: the scenery holds no triangles, nothing baked" );
state.collecting = false;
return;
}
configure_and_drain();
}
auto const path { heightfield_path( Sceneryfile ) };
state.cooker.finish( path, {} );
write_table( table_path( path ) );
std::error_code error;
auto const bytes { std::filesystem::file_size( path, error ) };
auto const seconds {
std::chrono::duration<double>( std::chrono::steady_clock::now() - state.started ).count() };
WriteLog(
"Terrain bake: wrote \"" + path + "\" from " + std::to_string( state.triangles )
+ " triangles in " + std::to_string( seconds ) + " s"
+ ( error ? std::string() : ", " + std::to_string( bytes / 1048576 ) + " MB" ) );
// the triangles of this load were all drawn; the table applies from the next one
state.collecting = false;
state.buffer.clear();
state.nodes.clear();
}
} // namespace simulation::terrainbake

View File

@@ -0,0 +1,85 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <cstddef>
#include <cstdint>
#include <string>
#include <string_view>
#include <vector>
export module eu07.simulation.terrainbake;
import eu07.glm;
import eu07.model.vertex;
export {
// Baking a scenery's heightfield on its first run, and deciding on every later run which
// triangles it replaces.
//
// A cooked heightfield is what makes a large scenery affordable - l204 loads in 25 seconds
// against 219, in 1.9 GB against 9.1 - but until now it had to be produced by hand with the
// terraincook tool and named in the scenery text. This bakes it the first time the scenery is
// opened and picks it up by itself afterwards.
//
// Nothing here guesses what terrain is. Every triangle the scenery draws is laid into the
// heightfield where it is drawn, and afterwards each one is asked whether the heightfield still
// shows it. A tree, a wall or a gantry stands on its edge and never reaches a sample; a bridge
// deck loses its samples to the ground beneath; those are drawn as they always were. Only
// triangles the heightfield really shows stop being drawn on their own, so nothing can vanish
// from the scenery - the worst a bad bake can do is draw something twice.
//
// A triangle node is recognised by the file and line it is written at - with that file's size
// and modification time - the parameters its include was given, and the origin and rotation in
// force, not by its position in the load. An edited file changes the keys of everything in it,
// and those nodes are then simply drawn again instead of the wrong ones being dropped. Hashing
// the node's text instead would cost a second parse of every triangle node in the scenery.
//
// The rasterising is scene/terraincooker.h, the same code the tool uses.
namespace simulation::terrainbake {
// what to do with one triangle node
enum class verdict {
draw, // the heightfield does not show it: import and draw it as written
skip, // the heightfield shows all of it: do not even import it
filter // the heightfield shows some of it: import it and draw only the rest
};
struct node_plan {
verdict what { verdict::draw };
// for filter: one entry per triangle in import order, true where it is still drawn
std::vector<bool> drawn;
};
// called before the scenery is parsed. picks up a heightfield baked earlier together with its
// node table, or arms the bake when there is none
void begin( std::string const &Sceneryfile );
// what the load is to do with one triangle node, decided in one place: whether it is read at
// all, whether it goes into a bake, and which of its triangles are still drawn
struct node_decision {
bool skip { false }; // step over it without importing
bool bake { false }; // hand it to add() once imported
std::uint64_t key { 0 };
node_plan plan;
};
// Worldspace says whether the node stands in world coordinates, which is all that decides it
// when the scenery names a hand-cooked heightfield of its own
node_decision examine(
std::string const &File, std::size_t const Line, std::vector<std::string> const &Parameters,
std::string_view const Type, glm::dvec3 const &Offset, glm::vec3 const &Rotation, bool const Worldspace );
// one triangle node, in world space, as it is about to be drawn
void add( std::uint64_t const Key, std::vector<world_vertex> const &Vertices, std::string_view const Material );
// called once the scenery has been parsed. writes the heightfield and its node table
void finish( std::string const &Sceneryfile );
} // namespace simulation::terrainbake
}

View File

@@ -0,0 +1,24 @@
# Standalone analysis tool for phase 1 of the binary scene format plan. Deliberately
# free of engine dependencies so it configures and builds in seconds, on its own.
cmake_minimum_required(VERSION 3.20)
project(terraincook CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
find_package(zstd CONFIG QUIET)
if(NOT zstd_FOUND)
find_package(PkgConfig REQUIRED)
pkg_check_modules(ZSTD REQUIRED IMPORTED_TARGET libzstd)
add_library(zstd::libzstd ALIAS PkgConfig::ZSTD)
endif()
add_executable(terraincook main.cpp)
# the on-disk format lives with the engine, so the tool and the reader cannot drift apart
target_include_directories(terraincook PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/../..")
target_link_libraries(terraincook PRIVATE zstd::libzstd)
if(NOT MSVC)
target_compile_options(terraincook PRIVATE -O2 -Wall)
endif()

Binary file not shown.

879
tools/terraincook/main.cpp Normal file
View File

@@ -0,0 +1,879 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
// Terrain analysis and cooking outside the engine. Reads terrain .scm files, detects the base
// grid, reports how the triangle soup splits into a regular heightfield and an off-grid
// overlay - with greyscale previews, since whether the overlay is a narrow strip along the
// track has to be seen - and cooks, verifies or probes a tiled heightfield. The cooking
// itself is scene/terraincooker.h, shared with the engine's first-run bake.
#include <algorithm>
#include <array>
#include <charconv>
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <cstring>
#include <filesystem>
#include <fstream>
#include <iostream>
#include <limits>
#include <sstream>
#include <string>
#include <string_view>
#include <unordered_map>
#include <vector>
#include "scene/heightfieldformat.h"
#include "scene/heightfieldreader.h"
#include "scene/terraincooker.h"
namespace {
using terrain::conflictepsilon;
using terrain::on_grid;
using terrain::planar_length;
using terrain::vertex;
using terrain::writepreview;
struct triangle {
vertex v[ 3 ];
};
struct bounds {
double minx { 1e30 }, maxx { -1e30 };
double minz { 1e30 }, maxz { -1e30 };
void add( double const X, double const Z ) {
minx = std::min( minx, X ); maxx = std::max( maxx, X );
minz = std::min( minz, Z ); maxz = std::max( maxz, Z ); }
};
// cell sizes the covered area is measured at. vertices landing on a 1 m grid says
// nothing about how far apart neighbouring samples actually are, and the difference
// decides what a heightfield of this terrain would really cost
constexpr double coveragescales[] { 1.0, 2.0, 4.0, 8.0, 16.0, 32.0, 64.0, 128.0, 256.0, 512.0 };
constexpr std::size_t coveragescalecount { sizeof( coveragescales ) / sizeof( coveragescales[ 0 ] ) };
struct statistics {
std::size_t triangles { 0 };
std::size_t gridtriangles { 0 };
std::size_t vertices { 0 };
std::size_t gridvertices { 0 };
std::size_t conflicts { 0 };
double maxconflict { 0.0 };
bounds extent;
bounds overlayextent;
// triangle edge lengths, bucketed by powers of two: <1, <2, <4, <8, <16, <32, <64, rest
std::size_t edgebuckets[ 8 ] {};
double areasum { 0.0 };
};
using heightfield::tile_key;
bool
numeric( char const Character ) {
return ( Character >= '0' && Character <= '9' ) || Character == '-' || Character == '+' || Character == '.';
}
std::string
readfile( std::filesystem::path const &Path ) {
std::ifstream file( Path, std::ios::binary );
if( false == file.good() ) { return {}; }
file.seekg( 0, std::ios::end );
auto const size { static_cast<std::size_t>( file.tellg() ) };
file.seekg( 0, std::ios::beg );
std::string content;
content.resize( size );
file.read( content.data(), static_cast<std::streamsize>( size ) );
return content;
}
// walks the scenery text and hands every triangle of a triangle-family node to the sink.
// deliberately forgiving: anything that is not a node header or vertex data is skipped
template <typename Sink_>
void
parse( std::string const &Content, Sink_ &&Handler ) {
auto const *cursor { Content.data() };
auto const *end { cursor + Content.size() };
auto skipspace = [ & ]() {
while( cursor < end && ( static_cast<unsigned char>( *cursor ) <= ' ' ) ) { ++cursor; } };
auto token = [ & ]() -> std::string_view {
skipspace();
auto const *start { cursor };
while( cursor < end && ( static_cast<unsigned char>( *cursor ) > ' ' ) ) { ++cursor; }
return { start, static_cast<std::size_t>( cursor - start ) }; };
auto skipline = [ & ]() {
while( cursor < end && *cursor != '\n' ) { ++cursor; } };
std::vector<double> numbers;
numbers.reserve( 32 );
auto transformdepth { 0 };
auto rotated { false };
while( cursor < end ) {
auto word { token() };
if( word.empty() ) { break; }
if( word.starts_with( "//" ) ) { skipline(); continue; }
// the same selection the engine bakes with (heightfield::selection_rules): terrain
// is written in world space, and whatever sits under origin, rotate or scale is an
// object built from triangles rather than ground
if( word == "origin" ) { ++transformdepth; continue; }
if( word == "endorigin" ) { transformdepth = std::max( 0, transformdepth - 1 ); continue; }
if( word == "scale" ) { ++transformdepth; continue; }
if( word == "endscale" ) { transformdepth = std::max( 0, transformdepth - 1 ); continue; }
if( word == "rotate" ) {
auto const x { token() }, y { token() }, z { token() };
auto const zero = []( std::string_view const Value ) {
return std::strtod( std::string( Value ).c_str(), nullptr ) == 0.0; };
rotated = !( zero( x ) && zero( y ) && zero( z ) );
continue;
}
if( word != "node" ) { continue; }
if( ( transformdepth > 0 ) || ( true == rotated ) ) {
// an object, not terrain: step over its body without handing it on
while( cursor < end ) {
auto const body { token() };
if( body.empty() || ( body.starts_with( "end" ) && ( body != "end" ) ) ) { break; }
}
continue;
}
// node header: range_max range_min name type
token(); token(); token();
auto const type { token() };
if( ( type != "triangles" ) && ( type != "triangle_strip" ) && ( type != "triangle_fan" ) ) {
continue;
}
auto const material { token() };
numbers.clear();
while( cursor < end ) {
auto const value { token() };
if( value.empty() ) { break; }
if( value == "endtri" ) { break; }
if( false == numeric( value.front() ) ) { continue; } // "end" and friends
double parsed { 0.0 };
auto const result {
std::from_chars( value.data(), value.data() + value.size(), parsed ) };
if( result.ec == std::errc() ) { numbers.push_back( parsed ); }
}
// vertex is position, normal and texture coordinates; only the position matters here
auto const stride { 8u };
auto const vertexcount { numbers.size() / stride };
std::vector<vertex> vertices;
vertices.reserve( vertexcount );
for( auto index { 0u }; index < vertexcount; ++index ) {
vertices.push_back( {
numbers[ index * stride + 0 ],
numbers[ index * stride + 1 ],
numbers[ index * stride + 2 ],
numbers[ index * stride + 6 ],
numbers[ index * stride + 7 ] } );
}
if( type == "triangles" ) {
for( auto index { 0u }; index + 2 < vertices.size(); index += 3 ) {
Handler( vertices[ index ], vertices[ index + 1 ], vertices[ index + 2 ], material );
}
}
else if( type == "triangle_strip" ) {
for( auto index { 2u }; index < vertices.size(); ++index ) {
Handler( vertices[ index - 2 ], vertices[ index - 1 ], vertices[ index ], material );
}
}
else { // triangle_fan
for( auto index { 2u }; index < vertices.size(); ++index ) {
Handler( vertices[ 0 ], vertices[ index - 1 ], vertices[ index ], material );
}
}
}
}
// ---------------------------------------------------------------------------------
// track proximity. the question is whether the fine sampling of the terrain follows the
// railway or is spread evenly: that decides whether a coarse heightfield plus a corridor
// overlay is enough, or whether the base grid has to be fine everywhere
// distance field over a coarse grid, in metres, built from the sampled track centrelines
class distance_field {
public:
void
build( std::vector<vertex> const &Samples, double const Cell ) {
m_cell = Cell;
if( true == Samples.empty() ) { return; }
for( auto const &point : Samples ) { m_extent.add( point.x, point.z ); }
// a margin, so that terrain lying off to the side still gets a real distance
m_extent.minx -= margin; m_extent.maxx += margin;
m_extent.minz -= margin; m_extent.maxz += margin;
m_width = static_cast<std::size_t>( ( m_extent.maxx - m_extent.minx ) / m_cell ) + 1;
m_height = static_cast<std::size_t>( ( m_extent.maxz - m_extent.minz ) / m_cell ) + 1;
m_distance.assign( m_width * m_height, unreachable );
for( auto const &point : Samples ) {
auto const index { cell( point.x, point.z ) };
if( index != npos ) { m_distance[ index ] = 0.0f; }
}
// two-pass chamfer transform; weights 3/4 approximate euclidean within ~2%
constexpr float straight { 3.0f }, diagonal { 4.0f };
auto relax = [ & ]( std::size_t const Target, std::size_t const Source, float const Weight ) {
m_distance[ Target ] = std::min( m_distance[ Target ], m_distance[ Source ] + Weight ); };
for( std::size_t y = 0; y < m_height; ++y ) {
for( std::size_t x = 0; x < m_width; ++x ) {
auto const here { y * m_width + x };
if( y > 0 ) {
relax( here, here - m_width, straight );
if( x > 0 ) { relax( here, here - m_width - 1, diagonal ); }
if( x + 1 < m_width ) { relax( here, here - m_width + 1, diagonal ); }
}
if( x > 0 ) { relax( here, here - 1, straight ); }
}
}
for( std::size_t y = m_height; y-- > 0; ) {
for( std::size_t x = m_width; x-- > 0; ) {
auto const here { y * m_width + x };
if( y + 1 < m_height ) {
relax( here, here + m_width, straight );
if( x > 0 ) { relax( here, here + m_width - 1, diagonal ); }
if( x + 1 < m_width ) { relax( here, here + m_width + 1, diagonal ); }
}
if( x + 1 < m_width ) { relax( here, here + 1, straight ); }
}
}
}
// distance in metres, or a negative value when the point falls outside the field
double
at( double const X, double const Z ) const {
auto const index { cell( X, Z ) };
if( index == npos ) { return -1.0; }
auto const value { m_distance[ index ] };
if( value >= unreachable ) { return -1.0; }
return value * m_cell / 3.0; }
bool ready() const { return false == m_distance.empty(); }
private:
static constexpr std::size_t npos { static_cast<std::size_t>( -1 ) };
static constexpr float unreachable { 1e9f };
static constexpr double margin { 4000.0 };
std::size_t
cell( double const X, double const Z ) const {
if( ( X < m_extent.minx ) || ( X > m_extent.maxx )
|| ( Z < m_extent.minz ) || ( Z > m_extent.maxz ) ) { return npos; }
auto const x { static_cast<std::size_t>( ( X - m_extent.minx ) / m_cell ) };
auto const z { static_cast<std::size_t>( ( Z - m_extent.minz ) / m_cell ) };
if( ( x >= m_width ) || ( z >= m_height ) ) { return npos; }
return z * m_width + x; }
bounds m_extent;
double m_cell { 16.0 };
std::size_t m_width { 0 }, m_height { 0 };
std::vector<float> m_distance;
};
// reads track nodes and samples their centrelines. the scenery spells a track as a cubic
// bezier: two endpoints with roll, and two control vectors relative to them
std::vector<vertex>
sampletracks( std::vector<std::filesystem::path> const &Paths, double const Spacing ) {
std::vector<vertex> samples;
std::size_t nodes { 0 }, rejected { 0 };
double tracklength { 0.0 };
for( auto const &path : Paths ) {
auto const content { readfile( path ) };
auto const *cursor { content.data() };
auto const *end { cursor + content.size() };
auto token = [ & ]() -> std::string_view {
while( cursor < end && ( static_cast<unsigned char>( *cursor ) <= ' ' ) ) { ++cursor; }
auto const *start { cursor };
while( cursor < end && ( static_cast<unsigned char>( *cursor ) > ' ' ) ) { ++cursor; }
return { start, static_cast<std::size_t>( cursor - start ) }; };
auto number = [ & ]() -> double {
auto const word { token() };
if( ( true == word.empty() ) || ( false == numeric( word.front() ) ) ) { return 0.0; }
double parsed { 0.0 };
std::from_chars( word.data(), word.data() + word.size(), parsed );
return parsed; };
while( cursor < end ) {
auto const word { token() };
if( true == word.empty() ) { break; }
if( word != "track" ) { continue; }
// the header has to be walked token by token, in the order TTrack::Load reads
// it: skipping to "the first numbers after the keyword" lands in the middle of
// the parameter line and yields nonsense geometry
token(); // type
for( auto field { 0u }; field < 4u; ++field ) { number(); } // length, gauge, friction, sound distance
for( auto field { 0u }; field < 2u; ++field ) { number(); } // quality, damage
token(); // environment
auto const visibility { token() }; // vis / unvis
if( visibility == "vis" ) {
token(); // rail texture
number(); // texture tile length
token(); // sub or rail texture
for( auto field { 0u }; field < 3u; ++field ) { number(); } // tex height, width, slope
}
vertex p0 {}, c0 {}, c1 {}, p1 {};
p0.x = number(); p0.y = number(); p0.z = number(); number(); // roll
c0.x = number(); c0.y = number(); c0.z = number();
c1.x = number(); c1.y = number(); c1.z = number();
p1.x = number(); p1.y = number(); p1.z = number(); number(); // roll
vertex const control0 { p0.x + c0.x, p0.y + c0.y, p0.z + c0.z };
vertex const control1 { p1.x + c1.x, p1.y + c1.y, p1.z + c1.z };
auto const chord {
std::sqrt( ( p1.x - p0.x ) * ( p1.x - p0.x ) + ( p1.z - p0.z ) * ( p1.z - p0.z ) ) };
++nodes;
if( ( chord <= 0.0 ) || ( chord > 5000.0 ) ) { ++rejected; continue; }
tracklength += chord;
auto const steps { std::max<std::size_t>( 2u, static_cast<std::size_t>( chord / Spacing ) + 1 ) };
for( auto step { 0u }; step <= steps; ++step ) {
auto const t { static_cast<double>( step ) / static_cast<double>( steps ) };
auto const u { 1.0 - t };
auto const w0 { u * u * u }, w1 { 3.0 * u * u * t }, w2 { 3.0 * u * t * t }, w3 { t * t * t };
samples.push_back( {
w0 * p0.x + w1 * control0.x + w2 * control1.x + w3 * p1.x,
0.0,
w0 * p0.z + w1 * control0.z + w2 * control1.z + w3 * p1.z } );
}
}
}
std::printf( " track nodes %zu, rejected %zu, total chord length %.1f km\n",
nodes, rejected, tracklength / 1000.0 );
return samples;
}
// prints a cross section straight out of a cooked file. a screenshot can show that the
// ground has a cliff in it; only the samples say how tall it is and how many of them it
// takes, which is the difference between a slope that is too steep and one that is right
int
probe( std::filesystem::path const &Path, double const X, double const Z,
double const Dirx, double const Dirz, std::size_t const Count ) {
heightfield::reader reader;
if( false == reader.open( Path.string() ) ) {
std::printf( "probe: cannot open %s\n", Path.string().c_str() );
return 1;
}
auto const length { std::sqrt( Dirx * Dirx + Dirz * Dirz ) };
if( length < 1e-9 ) { std::printf( "probe: direction has no length\n" ); return 1; }
auto const stepx { Dirx / length * reader.gridstep() };
auto const stepz { Dirz / length * reader.gridstep() };
std::printf( "-- cross section from %.1f, %.1f along %.2f, %.2f, %zu samples of %.2f m\n",
X, Z, Dirx / length, Dirz / length, Count, reader.gridstep() );
std::vector<std::uint16_t> heights;
std::vector<std::uint8_t> materials;
auto previous { std::numeric_limits<double>::quiet_NaN() };
for( std::size_t index = 0; index < Count; ++index ) {
auto const x { X + stepx * static_cast<double>( index ) };
auto const z { Z + stepz * static_cast<double>( index ) };
auto const tilex { reader.tile_x( x ) };
auto const tilez { reader.tile_z( z ) };
if( false == reader.read_level( tilex, tilez, 0, heights, materials ) ) {
std::printf( " %8.1f %8.1f no tile\n", x, z );
continue;
}
auto const side { reader.samples_per_side( 0 ) };
auto const local {
static_cast<std::int64_t>( std::llround( x / reader.gridstep() ) )
- static_cast<std::int64_t>( tilex ) * reader.tilesamples() };
auto const localrow {
static_cast<std::int64_t>( std::llround( z / reader.gridstep() ) )
- static_cast<std::int64_t>( tilez ) * reader.tilesamples() };
if( ( local < 0 ) || ( localrow < 0 ) || ( local >= side ) || ( localrow >= side ) ) {
std::printf( " %8.1f %8.1f outside\n", x, z );
continue;
}
auto const sample { heights[ localrow * side + local ] };
if( sample == heightfield::nodata ) {
std::printf( " %8.1f %8.1f hole\n", x, z );
previous = std::numeric_limits<double>::quiet_NaN();
continue;
}
auto const height { reader.height( sample ) };
auto const material { materials[ localrow * side + local ] };
std::printf( " %8.1f %8.1f %8.3f m step %+7.3f %s\n",
x, z, height, std::isnan( previous ) ? 0.0 : height - previous,
material < reader.materials().size() ? reader.materials()[ material ].c_str() : "?" );
previous = height;
}
return 0;
}
// reads a cooked file back with the reader the engine uses, and checks the promises the
// format makes. the one that matters is the shared edge: neighbouring tiles must hold
// byte-identical samples along the boundary they share, at every mip level, or the
// terrain will show seams that no amount of skirting hides
int
verify( std::filesystem::path const &Path ) {
heightfield::reader reader;
if( false == reader.open( Path.string() ) ) {
std::printf( "verify: cannot open %s as a heightfield\n", Path.string().c_str() );
return 1;
}
std::printf( "-- verifying %s\n", Path.string().c_str() );
std::printf( " tiles %zu, levels %u, grid step %.2f m, tile side %.0f m, materials %zu\n",
reader.tiles().size(), reader.levels(), reader.gridstep(), reader.tilesize(), reader.materials().size() );
std::vector<std::uint16_t> heights, neighbourheights;
std::vector<std::uint8_t> materials, neighbourmaterials;
std::size_t unreadable { 0 }, seams { 0 }, checkededges { 0 }, holes { 0 }, samples { 0 };
for( auto const &tile : reader.tiles() ) {
for( std::uint32_t level = 0; level < reader.levels(); ++level ) {
if( false == reader.read_level( tile.x, tile.z, level, heights, materials ) ) {
++unreadable;
continue;
}
auto const side { reader.samples_per_side( level ) };
if( heights.size() != static_cast<std::size_t>( side ) * side ) {
++unreadable;
continue;
}
if( level == 0 ) {
samples += heights.size();
for( auto const value : heights ) {
if( value == heightfield::nodata ) { ++holes; }
}
}
// the tile to the east shares this tile's last column with its first
if( true == reader.contains( tile.x + 1, tile.z ) ) {
if( true == reader.read_level( tile.x + 1, tile.z, level, neighbourheights, neighbourmaterials ) ) {
++checkededges;
for( std::uint32_t row = 0; row < side; ++row ) {
if( heights[ row * side + ( side - 1 ) ] != neighbourheights[ row * side ] ) { ++seams; break; }
}
}
// re-reading this tile, since the reader keeps one decoded payload
reader.read_level( tile.x, tile.z, level, heights, materials );
}
// and the tile to the south shares the last row
if( true == reader.contains( tile.x, tile.z + 1 ) ) {
if( true == reader.read_level( tile.x, tile.z + 1, level, neighbourheights, neighbourmaterials ) ) {
++checkededges;
for( std::uint32_t column = 0; column < side; ++column ) {
if( heights[ ( side - 1 ) * side + column ] != neighbourheights[ column ] ) { ++seams; break; }
}
}
reader.read_level( tile.x, tile.z, level, heights, materials );
}
}
}
std::printf( " unreadable tile levels %zu\n", unreadable );
std::printf( " shared edges checked %zu\n", checkededges );
std::printf( " mismatched edges %zu\n", seams );
std::printf( " holes at finest level %zu of %zu samples (%.2f %%)\n",
holes, samples, samples == 0 ? 0.0 : 100.0 * static_cast<double>( holes ) / static_cast<double>( samples ) );
auto const failed { ( unreadable > 0 ) || ( seams > 0 ) };
std::printf( " %s\n", failed ? "FAILED" : "ok" );
return failed ? 1 : 0;
}
} // namespace
int
main( int argc, char *argv[] ) {
if( argc < 2 ) {
std::cout
<< "usage: terraincook <terrain .scm files or directory>\n"
<< " [-step <metres>] [-tracks <dir>] [-out <directory>]\n"
<< " [-cook] [-cookstep <metres>]\n"
<< " analyses the terrain: grid detection, heightfield/overlay split, sampling\n"
<< " density against distance from the nearest track, coverage by cell size\n"
<< " -cook additionally rasterises it into a tiled heightfield (terrain.ehf)\n"
<< " -raw leaves the cooked tiles uncompressed\n"
<< "usage: terraincook -verify <terrain.ehf>\n"
<< " reads a cooked file back and checks its tiles and shared edges\n"
<< "usage: terraincook -probe <terrain.ehf> <x,z,dirx,dirz,count>\n"
<< " prints cooked heights along a line, for looking at a cross section\n";
return 1;
}
for( auto index { 1 }; index < argc; ++index ) {
if( std::string { argv[ index ] } != "-probe" ) { continue; }
if( index + 2 >= argc ) {
std::cerr << "-probe needs <terrain.ehf> <x,z,dirx,dirz,count>\n";
return 1;
}
std::vector<double> fields;
std::istringstream stream { std::string { argv[ index + 2 ] } };
std::string field;
while( std::getline( stream, field, ',' ) ) { fields.push_back( std::strtod( field.c_str(), nullptr ) ); }
if( fields.size() < 5 ) {
std::cerr << "-probe wants x,z,dirx,dirz,count\n";
return 1;
}
return probe(
argv[ index + 1 ], fields[ 0 ], fields[ 1 ], fields[ 2 ], fields[ 3 ],
static_cast<std::size_t>( fields[ 4 ] ) );
}
// verification stands on its own: it reads a cooked file and needs no source data
for( auto index { 1 }; index < argc; ++index ) {
if( std::string { argv[ index ] } != "-verify" ) { continue; }
if( index + 1 >= argc ) {
std::cerr << "-verify needs a path to a cooked heightfield\n";
return 1;
}
return verify( argv[ index + 1 ] );
}
std::vector<std::filesystem::path> inputs;
std::vector<std::filesystem::path> trackinputs;
std::filesystem::path outputdirectory { "." };
double step { 0.0 };
double cookstep { 2.0 };
bool cooking { false };
bool compress { true };
for( auto index { 1 }; index < argc; ++index ) {
std::string const argument { argv[ index ] };
if( argument == "-step" && index + 1 < argc ) { step = std::strtod( argv[ ++index ], nullptr ); continue; }
if( argument == "-out" && index + 1 < argc ) { outputdirectory = argv[ ++index ]; continue; }
if( argument == "-cook" ) { cooking = true; continue; }
if( argument == "-raw" ) { compress = false; continue; }
if( argument == "-cookstep" && index + 1 < argc ) { cookstep = std::strtod( argv[ ++index ], nullptr ); cooking = true; continue; }
if( argument == "-tracks" && index + 1 < argc ) {
std::filesystem::path const trackpath { argv[ ++index ] };
if( std::filesystem::is_directory( trackpath ) ) {
for( auto const &entry : std::filesystem::directory_iterator( trackpath ) ) {
if( entry.path().extension() == ".scm" ) { trackinputs.push_back( entry.path() ); }
}
}
else { trackinputs.push_back( trackpath ); }
continue; }
std::filesystem::path const path { argument };
if( std::filesystem::is_directory( path ) ) {
for( auto const &entry : std::filesystem::directory_iterator( path ) ) {
if( entry.path().extension() == ".scm" ) { inputs.push_back( entry.path() ); }
}
}
else {
inputs.push_back( path );
}
}
std::sort( inputs.begin(), inputs.end() );
if( true == inputs.empty() ) {
std::cerr << "no input files\n";
return 1;
}
// first pass: sample vertices to pick the grid step, unless one was given
if( step <= 0.0 ) {
std::vector<vertex> sample;
auto const content { readfile( inputs.front() ) };
parse( content, [ & ]( vertex const &A, vertex const &B, vertex const &C, std::string_view ) {
if( sample.size() >= 300000 ) { return; }
sample.push_back( A ); sample.push_back( B ); sample.push_back( C ); } );
std::cout << "grid step detection, on " << sample.size() << " sampled vertices:\n";
step = terrain::detect_step(
sample,
[]( double const Candidate, double const Share ) {
std::printf( " %6.2f m %6.2f %%\n", Candidate, Share ); } );
std::cout << " chosen: " << step << " m\n\n";
}
// distance-to-track field, when track files were given
distance_field trackdistance;
if( false == trackinputs.empty() ) {
std::sort( trackinputs.begin(), trackinputs.end() );
auto const samples { sampletracks( trackinputs, 8.0 ) };
std::cout << "track centreline samples: " << samples.size() << "\n";
trackdistance.build( samples, 16.0 );
}
// triangles bucketed by distance from the nearest track, with their sampling scale
constexpr double distancebands[] { 25.0, 50.0, 100.0, 200.0, 400.0, 800.0, 1600.0 };
constexpr std::size_t bandcount { sizeof( distancebands ) / sizeof( distancebands[ 0 ] ) + 2 };
std::array<std::size_t, bandcount> bandtriangles {};
std::array<double, bandcount> bandedgesum {};
std::array<std::size_t, bandcount> bandfine {};
terrain::cooker cooker;
cooker.configure( cookstep, compress );
statistics stats;
std::vector<std::unordered_map<std::int64_t, char>> coverage( coveragescalecount );
std::unordered_map<std::int64_t, float> heights;
heights.reserve( 8u << 20 );
std::vector<triangle> overlay;
for( auto const &path : inputs ) {
auto const content { readfile( path ) };
if( true == content.empty() ) {
std::cerr << "cannot read " << path << "\n";
continue;
}
auto const before { stats.triangles };
parse( content, [ & ]( vertex const &A, vertex const &B, vertex const &C, std::string_view Material ) {
if( true == cooking ) { cooker.rasterize( A, B, C, Material ); }
++stats.triangles;
stats.vertices += 3;
vertex const points[ 3 ] { A, B, C };
auto gridpoints { 0 };
for( auto const &point : points ) {
stats.extent.add( point.x, point.z );
if( on_grid( point.x, step ) && on_grid( point.z, step ) ) { ++gridpoints; }
for( auto scaleindex { 0u }; scaleindex < coveragescalecount; ++scaleindex ) {
auto const cell { coveragescales[ scaleindex ] };
coverage[ scaleindex ].emplace(
tile_key(
static_cast<std::int32_t>( std::floor( point.x / cell ) ),
static_cast<std::int32_t>( std::floor( point.z / cell ) ) ),
char { 0 } );
}
}
// horizontal edge lengths and footprint area, to see the real sampling density
for( auto index { 0u }; index < 3u; ++index ) {
auto const length { planar_length( points[ index ], points[ ( index + 1 ) % 3 ] ) };
auto bucket { 0u };
while( ( bucket < 7u ) && ( length >= ( 1.0 * ( 1u << bucket ) ) ) ) { ++bucket; }
++stats.edgebuckets[ bucket ];
}
if( true == trackdistance.ready() ) {
auto const centroidx { ( points[ 0 ].x + points[ 1 ].x + points[ 2 ].x ) / 3.0 };
auto const centroidz { ( points[ 0 ].z + points[ 1 ].z + points[ 2 ].z ) / 3.0 };
auto edgesum { 0.0 };
for( auto index { 0u }; index < 3u; ++index ) {
edgesum += planar_length( points[ index ], points[ ( index + 1 ) % 3 ] );
}
auto const scale { edgesum / 3.0 };
auto const distance { trackdistance.at( centroidx, centroidz ) };
std::size_t band { bandcount - 1 }; // outside the field
if( distance >= 0.0 ) {
band = bandcount - 2; // beyond the last threshold
for( auto index { 0u }; index < bandcount - 2; ++index ) {
if( distance < distancebands[ index ] ) { band = index; break; }
}
}
++bandtriangles[ band ];
bandedgesum[ band ] += scale;
if( scale < 4.0 ) { ++bandfine[ band ]; }
}
stats.areasum += 0.5 * std::abs(
( points[ 1 ].x - points[ 0 ].x ) * ( points[ 2 ].z - points[ 0 ].z )
- ( points[ 2 ].x - points[ 0 ].x ) * ( points[ 1 ].z - points[ 0 ].z ) );
stats.gridvertices += static_cast<std::size_t>( gridpoints );
if( gridpoints == 3 ) {
++stats.gridtriangles;
for( auto const &point : points ) {
auto const key {
tile_key(
static_cast<std::int32_t>( std::llround( point.x / step ) ),
static_cast<std::int32_t>( std::llround( point.z / step ) ) ) };
auto const existing { heights.find( key ) };
if( existing == heights.end() ) {
heights.emplace( key, static_cast<float>( point.y ) );
}
else {
auto const difference { std::abs( static_cast<double>( existing->second ) - point.y ) };
if( difference > conflictepsilon ) {
++stats.conflicts;
stats.maxconflict = std::max( stats.maxconflict, difference );
}
}
}
}
else {
overlay.push_back( { { A, B, C } } );
for( auto const &point : points ) { stats.overlayextent.add( point.x, point.z ); }
}
} );
std::printf( "%-28s %10zu triangles\n",
path.filename().string().c_str(), stats.triangles - before );
}
auto const share = []( std::size_t const Part, std::size_t const Whole ) {
return Whole == 0 ? 0.0 : 100.0 * static_cast<double>( Part ) / static_cast<double>( Whole ); };
// the full bounding box is set by outliers, so derive the working extent from the
// occupied grid samples themselves, trimming the sparsest 0.1% at each end
std::vector<std::int32_t> columns, rows;
columns.reserve( heights.size() );
rows.reserve( heights.size() );
for( auto const & [ key, height ] : heights ) {
columns.push_back( static_cast<std::int32_t>( key >> 32 ) );
rows.push_back( static_cast<std::int32_t>( static_cast<std::uint32_t>( key & 0xffffffff ) ) );
}
auto trimmed = []( std::vector<std::int32_t> &Values ) -> std::pair<double, double> {
if( true == Values.empty() ) { return { 0.0, 0.0 }; }
auto const low { Values.size() / 1000 };
auto const high { Values.size() - 1 - low };
std::nth_element( Values.begin(), Values.begin() + static_cast<std::ptrdiff_t>( low ), Values.end() );
auto const minimum { Values[ low ] };
std::nth_element( Values.begin(), Values.begin() + static_cast<std::ptrdiff_t>( high ), Values.end() );
auto const maximum { Values[ high ] };
return { static_cast<double>( minimum ), static_cast<double>( maximum ) }; };
auto const [ columnmin, columnmax ] { trimmed( columns ) };
auto const [ rowmin, rowmax ] { trimmed( rows ) };
auto const gridwidth { static_cast<std::size_t>( columnmax - columnmin ) + 1 };
auto const gridheight { static_cast<std::size_t>( rowmax - rowmin ) + 1 };
auto const overlaytriangles { stats.triangles - stats.gridtriangles };
std::printf( "\n-- terrain decomposition, grid step %.2f m\n", step );
std::printf( " triangles total %12zu\n", stats.triangles );
std::printf( " on grid (heightfield)%12zu %6.2f %%\n", stats.gridtriangles, share( stats.gridtriangles, stats.triangles ) );
std::printf( " off grid (overlay) %12zu %6.2f %%\n", overlaytriangles, share( overlaytriangles, stats.triangles ) );
std::printf( " vertices on grid %12zu %6.2f %%\n", stats.gridvertices, share( stats.gridvertices, stats.vertices ) );
std::printf( " distinct grid samples%12zu\n", heights.size() );
std::printf( " stacked samples %12zu (max height difference %.2f m)\n", stats.conflicts, stats.maxconflict );
std::printf( " full bounding box %.0f .. %.0f x %.0f .. %.0f z\n",
stats.extent.minx, stats.extent.maxx, stats.extent.minz, stats.extent.maxz );
std::printf( " working extent %.0f .. %.0f x %.0f .. %.0f z (%zu x %zu grid)\n",
columnmin * step, columnmax * step, rowmin * step, rowmax * step, gridwidth, gridheight );
std::printf( " covered area %12.2f km2 (%.1f %% of the working extent)\n",
heights.size() * step * step / 1.0e6,
share( heights.size(), gridwidth * gridheight ) );
if( overlaytriangles > 0 ) {
std::printf( " overlay extent %.0f .. %.0f x %.0f .. %.0f z\n",
stats.overlayextent.minx, stats.overlayextent.maxx,
stats.overlayextent.minz, stats.overlayextent.maxz );
}
std::printf( "\n-- real sampling density\n" );
std::printf( " summed triangle footprint %8.2f km2\n", stats.areasum / 1.0e6 );
std::printf( " mean triangle footprint %8.2f m2\n",
stats.triangles == 0 ? 0.0 : stats.areasum / static_cast<double>( stats.triangles ) );
char const *edgelabels[ 8 ] { "<1 m", "<2 m", "<4 m", "<8 m", "<16 m", "<32 m", "<64 m", ">=64 m" };
std::size_t edgetotal { 0 };
for( auto const bucket : stats.edgebuckets ) { edgetotal += bucket; }
for( auto index { 0u }; index < 8u; ++index ) {
if( stats.edgebuckets[ index ] == 0 ) { continue; }
std::printf( " edges %-7s %12zu %6.2f %%\n",
edgelabels[ index ], stats.edgebuckets[ index ], share( stats.edgebuckets[ index ], edgetotal ) );
}
if( true == trackdistance.ready() ) {
std::printf( "\n-- sampling density against distance from the nearest track\n" );
std::printf( " %-14s %12s %12s %10s\n", "band", "triangles", "mean edge", "fine <4 m" );
for( auto index { 0u }; index < bandcount; ++index ) {
if( bandtriangles[ index ] == 0 ) { continue; }
char label[ 32 ];
if( index < bandcount - 2 ) {
std::snprintf( label, sizeof( label ), "< %.0f m", distancebands[ index ] );
}
else if( index == bandcount - 2 ) {
std::snprintf( label, sizeof( label ), ">= %.0f m", distancebands[ bandcount - 3 ] );
}
else {
std::snprintf( label, sizeof( label ), "off the field" );
}
std::printf( " %-14s %12zu %10.2f m %9.2f %%\n",
label, bandtriangles[ index ],
bandedgesum[ index ] / static_cast<double>( bandtriangles[ index ] ),
share( bandfine[ index ], bandtriangles[ index ] ) );
}
}
std::printf( "\n-- covered area by cell size, and what a heightfield at that resolution costs\n" );
for( auto scaleindex { 0u }; scaleindex < coveragescalecount; ++scaleindex ) {
auto const cell { coveragescales[ scaleindex ] };
auto const cells { coverage[ scaleindex ].size() };
std::printf( " %5.0f m %12zu cells %9.2f km2 %9.1f MB R16\n",
cell, cells, cells * cell * cell / 1.0e6, cells * 2u / 1048576.0 );
}
auto const heightfieldbytes { gridwidth * gridheight * 2u };
auto const overlaybytes { overlaytriangles * 3u * 20u };
std::printf( "\n-- storage estimate\n" );
std::printf( " dense heightfield R16%12.1f MB (whole bounding box)\n", heightfieldbytes / 1048576.0 );
std::printf( " occupied samples R16 %12.1f MB (sparse, what is actually covered)\n", heights.size() * 2u / 1048576.0 );
std::printf( " overlay at 20 B/vertex%11.1f MB\n", overlaybytes / 1048576.0 );
// previews, downsampled so that the wider side is at most 2048 pixels
std::filesystem::create_directories( outputdirectory );
auto const preview { std::max<std::size_t>( 1u, ( std::max( gridwidth, gridheight ) + 2047u ) / 2048u ) };
auto const previewwidth { gridwidth / preview + 1 };
auto const previewheight { gridheight / preview + 1 };
constexpr float sentinel { -1e30f };
std::vector<float> heightpreview( previewwidth * previewheight, sentinel );
for( auto const & [ key, height ] : heights ) {
auto const x { static_cast<std::int32_t>( key >> 32 ) };
auto const z { static_cast<std::int32_t>( static_cast<std::uint32_t>( key & 0xffffffff ) ) };
if( ( x < columnmin ) || ( x > columnmax ) || ( z < rowmin ) || ( z > rowmax ) ) { continue; }
auto const column { static_cast<std::size_t>( x - columnmin ) / preview };
auto const row { static_cast<std::size_t>( z - rowmin ) / preview };
if( column >= previewwidth || row >= previewheight ) { continue; }
auto &target { heightpreview[ row * previewwidth + column ] };
target = ( target == sentinel ? height : std::max( target, height ) );
}
writepreview( outputdirectory / "height.pgm", heightpreview, previewwidth, previewheight, sentinel );
std::vector<float> overlaypreview( previewwidth * previewheight, 0.0f );
for( auto const &piece : overlay ) {
for( auto const &point : piece.v ) {
auto const gridx { std::llround( point.x / step ) };
auto const gridz { std::llround( point.z / step ) };
if( ( gridx < columnmin ) || ( gridx > columnmax ) || ( gridz < rowmin ) || ( gridz > rowmax ) ) { continue; }
auto const column { static_cast<std::size_t>( gridx - columnmin ) / preview };
auto const row { static_cast<std::size_t>( gridz - rowmin ) / preview };
if( column >= previewwidth || row >= previewheight ) { continue; }
overlaypreview[ row * previewwidth + column ] += 1.0f;
}
}
writepreview( outputdirectory / "overlay.pgm", overlaypreview, previewwidth, previewheight, -1.0f );
if( true == cooking ) {
std::filesystem::create_directories( outputdirectory );
cooker.finish( outputdirectory / "terrain.ehf", outputdirectory / "cooked.pgm" );
}
std::printf( "\n previews written to %s (%zux%zu, 1 px = %.0f m)\n",
outputdirectory.string().c_str(), previewwidth, previewheight, step * preview );
return 0;
}

File diff suppressed because one or more lines are too long

View File

@@ -265,6 +265,12 @@ bool global_settings::ConfigParseGraphics(cParser& Parser, const std::string& to
return true; return true;
} }
if (token == "terrainrange")
{
ParseOne(Parser, TerrainRange, 1);
return true;
}
if (token == "vsync") if (token == "vsync")
{ {
ParseOne(Parser, VSync); ParseOne(Parser, VSync);
@@ -1533,6 +1539,7 @@ global_settings::export_as_text( std::ostream &Output ) const {
export_as_text( Output, "height", window_size.y ); export_as_text( Output, "height", window_size.y );
export_as_text( Output, "targetfps", targetfps ); export_as_text( Output, "targetfps", targetfps );
export_as_text( Output, "basedrawrange", BaseDrawRange ); export_as_text( Output, "basedrawrange", BaseDrawRange );
export_as_text( Output, "terrainrange", TerrainRange );
export_as_text( Output, "fullscreen", bFullScreen ); export_as_text( Output, "fullscreen", bFullScreen );
export_as_text( Output, "fullscreenmonitor", fullscreen_monitor ); export_as_text( Output, "fullscreenmonitor", fullscreen_monitor );
export_as_text( Output, "fullscreenwindowed", fullscreen_windowed ); export_as_text( Output, "fullscreenwindowed", fullscreen_windowed );

View File

@@ -122,6 +122,19 @@ struct global_settings {
int iConvertModels{ 0 }; // tworzenie plików binarnych int iConvertModels{ 0 }; // tworzenie plików binarnych
int iConvertIndexRange{ 1000 }; // range of duplicate vertex scan int iConvertIndexRange{ 1000 }; // range of duplicate vertex scan
bool file_binary_terrain{ true }; // enable binary terrain (de)serialization bool file_binary_terrain{ true }; // enable binary terrain (de)serialization
// -cooklogic: write the scenery's LOGIC container once it has loaded, then read it back
// and check it against what was loaded
bool cook_logic{ false };
// -verifylogic: read the container that is already there and check it against the
// scenery, without writing anything. this is how a stale container gets caught
bool verify_logic{ false };
// bake a heightfield for a scenery that has none, the first time it is opened, and use
// it from then on. -nobaketerrain turns it off
bool bake_terrain{ true };
// cooked heightfields drawn in place of the legacy terrain geometry, coarsest first
// once sorted. a scenery needs several: a regional elevation model sampled every
// hundred metres, and a corridor of its own for each line, sampled every two
std::vector<std::string> terrain_heightfields;
bool file_binary_terrain_state{true}; bool file_binary_terrain_state{true};
// logs // logs
bool priorityLoadText3D{false}; // ladowanie T3D priorytetowo bool priorityLoadText3D{false}; // ladowanie T3D priorytetowo
@@ -180,6 +193,9 @@ struct global_settings {
bool bWireFrame{ false }; bool bWireFrame{ false };
bool bAdjustScreenFreq{ true }; bool bAdjustScreenFreq{ true };
float BaseDrawRange{ 2500.f }; float BaseDrawRange{ 2500.f };
// how far terrain tiles are streamed and drawn, independent of BaseDrawRange: a baked
// heightfield is cheap far out, and seeing it to the horizon is the point of having it
float TerrainRange{ 20000.f };
int DynamicLightCount{ 7 }; int DynamicLightCount{ 7 };
bool ScaleSpecularValues{ true }; bool ScaleSpecularValues{ true };
std::string GfxRenderer{ "default" }; std::string GfxRenderer{ "default" };

View File

@@ -622,6 +622,11 @@ std::size_t cParser::Line() const
} }
} }
std::vector<std::string> const & cParser::Parameters() const
{
return mIncludeParser ? mIncludeParser->Parameters() : parameters;
}
int cParser::LineMain() const int cParser::LineMain() const
{ {
return mIncludeParser ? -1 : mLine; return mIncludeParser ? -1 : mLine;

View File

@@ -109,6 +109,8 @@ class cParser //: public std::stringstream
std::string Name() const; std::string Name() const;
// returns number of currently processed line // returns number of currently processed line
std::size_t Line() const; std::size_t Line() const;
// parameters of the innermost file being read, the (pN) values its include line gave
std::vector<std::string> const & Parameters() const;
// returns number of currently processed line in main file, -1 if inside include // returns number of currently processed line in main file, -1 if inside include
int LineMain() const; int LineMain() const;
bool expandIncludes = true; bool expandIncludes = true;

View File

@@ -415,6 +415,24 @@ std::time_t last_modified(std::string const &Filename)
} }
// potentially erases file extension from provided file name. returns: true if extension was removed, false otherwise // potentially erases file extension from provided file name. returns: true if extension was removed, false otherwise
std::string scenery_file(std::string const &Sceneryfile)
{
return Sceneryfile.find_first_of("/\\") == std::string::npos ? Global.asCurrentSceneryPath + Sceneryfile : Sceneryfile;
}
std::string scenery_sidecar(std::string const &Sceneryfile, std::string const &Extension)
{
auto path{scenery_file(Sceneryfile)};
auto const slash{path.find_last_of("/\\")};
auto const namestart{slash == std::string::npos ? 0 : slash + 1};
if ((namestart < path.size()) && (path[namestart] == '$'))
{
path.erase(namestart, 1);
}
erase_extension(path);
return path + Extension;
}
bool erase_extension(std::string &Filename) bool erase_extension(std::string &Filename)
{ {

View File

@@ -232,6 +232,14 @@ std::time_t last_modified(std::string const &Filename);
// potentially erases file extension from provided file name. returns: true if extension was removed, false otherwise // potentially erases file extension from provided file name. returns: true if extension was removed, false otherwise
bool erase_extension(std::string &Filename); bool erase_extension(std::string &Filename);
// where a scenery file lives: a bare name is looked up in the scenery directory
std::string scenery_file(std::string const &Sceneryfile);
// a file cooked for a scenery and kept beside it, with the given extension in place of the
// scenery's. named after the scenario rather than the $-prefixed copy of it the launcher
// writes on every start, so the launcher and a direct start share one
std::string scenery_sidecar(std::string const &Sceneryfile, std::string const &Extension);
// potentially erase leading slashes from provided file path // potentially erase leading slashes from provided file path
void erase_leading_slashes(std::string &Filename); void erase_leading_slashes(std::string &Filename);

View File

@@ -309,6 +309,17 @@ basic_event::~basic_event() {
m_sibling = nullptr; // nie usuwać podczepionych tutaj m_sibling = nullptr; // nie usuwać podczepionych tutaj
} }
std::vector<std::string>
basic_event::target_names() const {
std::vector<std::string> names;
names.reserve( m_targets.size() );
for( auto const &target : m_targets ) {
names.emplace_back( std::get<std::string>( target ) );
}
return names;
}
void void
basic_event::deserialize( cParser &Input, scene::scratch_data &Scratchpad ) { basic_event::deserialize( cParser &Input, scene::scratch_data &Scratchpad ) {
@@ -356,10 +367,10 @@ basic_event::deserialize_targets( std::string const &Input ) {
} }
void void
basic_event::run() { basic_event::run( TDynamicObject const *Activator ) {
WriteLog( "EVENT LAUNCHED" + ( m_activator ? " by " + m_activator->asName : "" ) + ": " + m_name ); WriteLog( "EVENT LAUNCHED" + ( Activator ? " by " + Activator->asName : "" ) + ": " + m_name );
run_(); run_( Activator );
} }
// sends basic content of the class in legacy (text) format to provided stream // sends basic content of the class in legacy (text) format to provided stream
@@ -537,7 +548,7 @@ updatevalues_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpa
// run() subclass details // run() subclass details
// TODO: update and copy values run_ methods are largely identical, refactor to a single helper // TODO: update and copy values run_ methods are largely identical, refactor to a single helper
void void
updatevalues_event::run_() { updatevalues_event::run_( TDynamicObject const *Activator ) {
if( false == m_conditions.test() ) { return; } if( false == m_conditions.test() ) { return; }
@@ -631,7 +642,7 @@ getvalues_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad )
// run() subclass details // run() subclass details
void void
getvalues_event::run_() { getvalues_event::run_( TDynamicObject const *Activator ) {
auto const *cell { m_input.data_cell() }; auto const *cell { m_input.data_cell() };
@@ -641,15 +652,15 @@ getvalues_event::run_() {
+ to_string( cell->Value1(), 2 ) + "] [" + to_string( cell->Value1(), 2 ) + "] ["
+ to_string( cell->Value2(), 2 ) + "]" ); + to_string( cell->Value2(), 2 ) + "]" );
if( m_activator == nullptr ) { return; } if( Activator == nullptr ) { return; }
cell->PutCommand( cell->PutCommand(
m_activator->Mechanik, Activator->Mechanik,
&cell->location() ); &cell->location() );
// potwierdzenie wykonania dla serwera (odczyt semafora już tak nie działa) // potwierdzenie wykonania dla serwera (odczyt semafora już tak nie działa)
if( Global.iMultiplayer ) { if( Global.iMultiplayer ) {
multiplayer::WyslijEvent( m_name, m_activator->name() ); multiplayer::WyslijEvent( m_name, Activator->name() );
} }
} }
@@ -787,7 +798,7 @@ putvalues_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad )
// run() subclass details // run() subclass details
void void
putvalues_event::run_() { putvalues_event::run_( TDynamicObject const *Activator ) {
WriteLog( WriteLog(
"Type: PutValues - [" "Type: PutValues - ["
@@ -795,7 +806,7 @@ putvalues_event::run_() {
+ to_string( m_input.data_value_1, 2 ) + "] [" + to_string( m_input.data_value_1, 2 ) + "] ["
+ to_string( m_input.data_value_2, 2 ) + "]" ); + to_string( m_input.data_value_2, 2 ) + "]" );
if( m_activator == nullptr ) { return; } if( Activator == nullptr ) { return; }
// zamiana, bo fizyka ma inaczej niż sceneria // zamiana, bo fizyka ma inaczej niż sceneria
// NOTE: y & z swap, negative x // NOTE: y & z swap, negative x
TLocation const loc { TLocation const loc {
@@ -803,22 +814,22 @@ putvalues_event::run_() {
m_input.location.z, m_input.location.z,
m_input.location.y }; m_input.location.y };
if( m_activator->Mechanik ) { if( Activator->Mechanik ) {
// przekazanie rozkazu do AI // przekazanie rozkazu do AI
WriteLog( " Vehicle: [" + m_activator->Mechanik->Vehicle()->name() + "]" ); WriteLog( " Vehicle: [" + Activator->Mechanik->Vehicle()->name() + "]" );
m_activator->Mechanik->PutCommand( Activator->Mechanik->PutCommand(
m_input.data_text, m_input.data_text,
m_input.data_value_1, m_input.data_value_1,
m_input.data_value_2, m_input.data_value_2,
loc ); loc );
} }
else if( m_activator->ctOwner else if( Activator->ctOwner
&& is_command_for_owner(m_input) ) { && is_command_for_owner(m_input) ) {
// send the command to consist owner, // send the command to consist owner,
// we're acting on presumption there's hardly ever need to issue command to unmanned vehicle // we're acting on presumption there's hardly ever need to issue command to unmanned vehicle
// and the intended recipient moved between vehicles after the event was queued // and the intended recipient moved between vehicles after the event was queued
WriteLog( " Vehicle: [" + m_activator->ctOwner->Vehicle()->name() + "]" ); WriteLog( " Vehicle: [" + Activator->ctOwner->Vehicle()->name() + "]" );
m_activator->ctOwner->PutCommand( Activator->ctOwner->PutCommand(
m_input.data_text, m_input.data_text,
m_input.data_value_1, m_input.data_value_1,
m_input.data_value_2, m_input.data_value_2,
@@ -826,8 +837,8 @@ putvalues_event::run_() {
} }
else { else {
// przekazanie do pojazdu // przekazanie do pojazdu
WriteLog( " Vehicle: [" + m_activator->name() +"]" ); WriteLog( " Vehicle: [" + Activator->name() +"]" );
m_activator->MoverParameters->PutCommand( Activator->MoverParameters->PutCommand(
m_input.data_text, m_input.data_text,
m_input.data_value_1, m_input.data_value_1,
m_input.data_value_2, m_input.data_value_2,
@@ -943,7 +954,7 @@ copyvalues_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad
// run() subclass details // run() subclass details
// TODO: update and copy values run_ methods are largely identical, refactor to a single helper // TODO: update and copy values run_ methods are largely identical, refactor to a single helper
void void
copyvalues_event::run_() { copyvalues_event::run_( TDynamicObject const *Activator ) {
// skopiowanie wartości z innej komórki // skopiowanie wartości z innej komórki
auto const *datasource { static_cast<TMemCell *>( std::get<scene::basic_node *>( m_input.data_source ) ) }; auto const *datasource { static_cast<TMemCell *>( std::get<scene::basic_node *>( m_input.data_source ) ) };
m_input.data_text = datasource->Text(); m_input.data_text = datasource->Text();
@@ -1035,9 +1046,9 @@ whois_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) {
// run() subclass details // run() subclass details
void void
whois_event::run_() { whois_event::run_( TDynamicObject const *Activator ) {
if( m_activator == nullptr ) { if( Activator == nullptr ) {
// no triggering vehicle to report on - most likely whois was launched from a // no triggering vehicle to report on - most likely whois was launched from a
// scenery event chain rather than a track/isolation crossing // scenery event chain rather than a track/isolation crossing
WriteLog( "Type: WhoIs (" + std::to_string( m_input.flags ) + ") - no activator, event ignored" ); WriteLog( "Type: WhoIs (" + std::to_string( m_input.flags ) + ") - no activator, event ignored" );
@@ -1059,9 +1070,9 @@ whois_event::run_() {
// next station name // next station name
if( m_input.flags & flags::mode_alt ) { if( m_input.flags & flags::mode_alt ) {
auto const *owner { ( auto const *owner { (
m_activator->Mechanik != nullptr && m_activator->Mechanik->primary() ? Activator->Mechanik != nullptr && Activator->Mechanik->primary() ?
m_activator->Mechanik : Activator->Mechanik :
m_activator->ctOwner ) }; Activator->ctOwner ) };
auto const nextstop { ( auto const nextstop { (
owner != nullptr ? owner != nullptr ?
owner->TrainTimetable().NextStop() : owner->TrainTimetable().NextStop() :
@@ -1086,14 +1097,14 @@ whois_event::run_() {
// vehicle name // vehicle name
else { else {
targetcell->UpdateValues( targetcell->UpdateValues(
m_activator->asName, // vehicle name Activator->asName, // vehicle name
0, // unused 0, // unused
0, // unused 0, // unused
m_input.flags & ( flags::text | flags::value1 | flags::value2 ) ); m_input.flags & ( flags::text | flags::value1 | flags::value2 ) );
WriteLog( WriteLog(
"Type: WhoIs (" + std::to_string( m_input.flags ) + ") - " "Type: WhoIs (" + std::to_string( m_input.flags ) + ") - "
+ "[name: " + m_activator->asName + "], " + "[name: " + Activator->asName + "], "
+ "[X], " + "[X], "
+ "[X]" ); + "[X]" );
} }
@@ -1105,9 +1116,9 @@ whois_event::run_() {
// jeśli typ pojazdu // jeśli typ pojazdu
// TODO: define and recognize individual request types // TODO: define and recognize individual request types
auto const owner { ( auto const owner { (
m_activator->Mechanik != nullptr && m_activator->Mechanik->primary() ? Activator->Mechanik != nullptr && Activator->Mechanik->primary() ?
m_activator->Mechanik : Activator->Mechanik :
m_activator->ctOwner ) }; Activator->ctOwner ) };
auto const consistbrakelevel { ( auto const consistbrakelevel { (
owner != nullptr ? owner != nullptr ?
owner->fReady : owner->fReady :
@@ -1118,45 +1129,45 @@ whois_event::run_() {
-1.0 ) }; -1.0 ) };
targetcell->UpdateValues( targetcell->UpdateValues(
m_activator->MoverParameters->TypeName, // typ pojazdu Activator->MoverParameters->TypeName, // typ pojazdu
consistbrakelevel, consistbrakelevel,
collisiondistance, collisiondistance,
m_input.flags & ( flags::text | flags::value1 | flags::value2 ) ); m_input.flags & ( flags::text | flags::value1 | flags::value2 ) );
WriteLog( WriteLog(
"Type: WhoIs (" + std::to_string( m_input.flags ) + ") - " "Type: WhoIs (" + std::to_string( m_input.flags ) + ") - "
+ "[type: " + m_activator->MoverParameters->TypeName + "], " + "[type: " + Activator->MoverParameters->TypeName + "], "
+ "[consist brake level: " + to_string( consistbrakelevel, 2 ) + "], " + "[consist brake level: " + to_string( consistbrakelevel, 2 ) + "], "
+ "[obstacle distance: " + to_string( collisiondistance, 2 ) + " m]" ); + "[obstacle distance: " + to_string( collisiondistance, 2 ) + " m]" );
} }
else { else {
// jeśli parametry ładunku // jeśli parametry ładunku
targetcell->UpdateValues( targetcell->UpdateValues(
m_activator->MoverParameters->LoadType.name, // nazwa ładunku Activator->MoverParameters->LoadType.name, // nazwa ładunku
m_activator->MoverParameters->LoadAmount, // aktualna ilość Activator->MoverParameters->LoadAmount, // aktualna ilość
m_activator->MoverParameters->MaxLoad, // maksymalna ilość Activator->MoverParameters->MaxLoad, // maksymalna ilość
m_input.flags & ( flags::text | flags::value1 | flags::value2 ) ); m_input.flags & ( flags::text | flags::value1 | flags::value2 ) );
WriteLog( WriteLog(
"Type: WhoIs (" + std::to_string( m_input.flags ) + ") - " "Type: WhoIs (" + std::to_string( m_input.flags ) + ") - "
+ "[load type: " + m_activator->MoverParameters->LoadType.name + "], " + "[load type: " + Activator->MoverParameters->LoadType.name + "], "
+ "[current load: " + to_string( m_activator->MoverParameters->LoadAmount, 2 ) + "], " + "[current load: " + to_string( Activator->MoverParameters->LoadAmount, 2 ) + "], "
+ "[max load: " + to_string( m_activator->MoverParameters->MaxLoad, 2 ) + "]" ); + "[max load: " + to_string( Activator->MoverParameters->MaxLoad, 2 ) + "]" );
} }
} }
// +8 // +8
else if( m_input.flags & flags::mode_alt ) { // jeśli miejsce docelowe pojazdu else if( m_input.flags & flags::mode_alt ) { // jeśli miejsce docelowe pojazdu
targetcell->UpdateValues( targetcell->UpdateValues(
m_activator->asDestination, // adres docelowy Activator->asDestination, // adres docelowy
m_activator->DirectionGet(), // kierunek pojazdu względem czoła składu (1=zgodny,-1=przeciwny) Activator->DirectionGet(), // kierunek pojazdu względem czoła składu (1=zgodny,-1=przeciwny)
m_activator->MoverParameters->Power, // moc pojazdu silnikowego: 0 dla wagonu Activator->MoverParameters->Power, // moc pojazdu silnikowego: 0 dla wagonu
m_input.flags & ( flags::text | flags::value1 | flags::value2 ) ); m_input.flags & ( flags::text | flags::value1 | flags::value2 ) );
WriteLog( WriteLog(
"Type: WhoIs (" + std::to_string( m_input.flags ) + ") - " "Type: WhoIs (" + std::to_string( m_input.flags ) + ") - "
+ "[destination: " + m_activator->asDestination + "], " + "[destination: " + Activator->asDestination + "], "
+ "[direction: " + std::to_string( m_activator->DirectionGet() ) + "], " + "[direction: " + std::to_string( Activator->DirectionGet() ) + "], "
+ "[engine power: " + to_string( m_activator->MoverParameters->Power, 2 ) + "]" ); + "[engine power: " + to_string( Activator->MoverParameters->Power, 2 ) + "]" );
} }
// +0 // +0
else { else {
@@ -1165,9 +1176,9 @@ whois_event::run_() {
// triggered the event ahead of its locomotive) - +0 used to require the activator's // triggered the event ahead of its locomotive) - +0 used to require the activator's
// own Mechanik and silently produce nothing for a wagon-triggered whois // own Mechanik and silently produce nothing for a wagon-triggered whois
auto const *owner { ( auto const *owner { (
m_activator->Mechanik != nullptr && m_activator->Mechanik->primary() ? Activator->Mechanik != nullptr && Activator->Mechanik->primary() ?
m_activator->Mechanik : Activator->Mechanik :
m_activator->ctOwner ) }; Activator->ctOwner ) };
if( owner != nullptr ) { if( owner != nullptr ) {
targetcell->UpdateValues( targetcell->UpdateValues(
owner->TrainName(), owner->TrainName(),
@@ -1218,7 +1229,7 @@ logvalues_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad )
// run() subclass details // run() subclass details
void void
logvalues_event::run_() { logvalues_event::run_( TDynamicObject const *Activator ) {
// zapisanie zawartości komórki pamięci do logu // zapisanie zawartości komórki pamięci do logu
if( m_targets.empty() ) { if( m_targets.empty() ) {
// lista wszystkich // lista wszystkich
@@ -1316,7 +1327,7 @@ multi_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) {
// run() subclass details // run() subclass details
void void
multi_event::run_() { multi_event::run_( TDynamicObject const *Activator ) {
auto const conditiontest { m_conditions.test() }; auto const conditiontest { m_conditions.test() };
if( conditiontest if( conditiontest
@@ -1330,12 +1341,12 @@ multi_event::run_() {
if( childevent != this ) { if( childevent != this ) {
// normalnie dodać // normalnie dodać
simulation::Events.AddToQuery( childevent, m_activator ); simulation::Events.AddToQuery( childevent, Activator );
} }
else { else {
// jeśli ma być rekurencja to musi mieć sensowny okres powtarzania // jeśli ma być rekurencja to musi mieć sensowny okres powtarzania
if( m_delay >= 5.0 ) { if( m_delay >= 5.0 ) {
simulation::Events.AddToQuery( this, m_activator ); simulation::Events.AddToQuery( this, Activator );
} }
} }
} }
@@ -1343,8 +1354,8 @@ multi_event::run_() {
// dajemy znać do serwera o wykonaniu // dajemy znać do serwera o wykonaniu
if( false == m_conditions.has_else ) { if( false == m_conditions.has_else ) {
// jednoznaczne tylko, gdy nie było else // jednoznaczne tylko, gdy nie było else
if( m_activator ) { if( Activator ) {
multiplayer::WyslijEvent( m_name, m_activator->name() ); multiplayer::WyslijEvent( m_name, Activator->name() );
} }
else { else {
multiplayer::WyslijEvent( m_name, "" ); multiplayer::WyslijEvent( m_name, "" );
@@ -1425,7 +1436,7 @@ sound_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) {
// run() subclass details // run() subclass details
void void
sound_event::run_() { sound_event::run_( TDynamicObject const *Activator ) {
WriteLog( WriteLog(
"Type: Sound - [" + std::string( m_soundmode == 1 ? "play" : m_soundmode == -1 ? "loop" : "stop" ) + "]" "Type: Sound - [" + std::string( m_soundmode == 1 ? "play" : m_soundmode == -1 ? "loop" : "stop" ) + "]"
@@ -1549,7 +1560,7 @@ texture_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) {
// run() subclass details // run() subclass details
void void
texture_event::run_() { texture_event::run_( TDynamicObject const *Activator ) {
material_handle material { null_handle }; material_handle material { null_handle };
@@ -1717,7 +1728,7 @@ animation_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad )
// run() subclass details // run() subclass details
void void
animation_event::run_() { animation_event::run_( TDynamicObject const *Activator ) {
WriteLog( "Type: Animation" ); WriteLog( "Type: Animation" );
// animation modes target specific submodels // animation modes target specific submodels
@@ -1816,7 +1827,7 @@ lights_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) {
// run() subclass details // run() subclass details
void void
lights_event::run_() { lights_event::run_( TDynamicObject const *Activator ) {
for( auto &target : m_targets ) { for( auto &target : m_targets ) {
auto *targetmodel = static_cast<TAnimModel *>( std::get<scene::basic_node *>( target ) ); auto *targetmodel = static_cast<TAnimModel *>( std::get<scene::basic_node *>( target ) );
@@ -1908,7 +1919,7 @@ switch_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) {
// run() subclass details // run() subclass details
void void
switch_event::run_() { switch_event::run_( TDynamicObject const *Activator ) {
for( auto &target : m_targets ) { for( auto &target : m_targets ) {
auto *targettrack { static_cast<TTrack *>( std::get<scene::basic_node *>( target ) ) }; auto *targettrack { static_cast<TTrack *>( std::get<scene::basic_node *>( target ) ) };
@@ -1975,7 +1986,7 @@ track_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) {
// run() subclass details // run() subclass details
void void
track_event::run_() { track_event::run_( TDynamicObject const *Activator ) {
WriteLog( "Type: TrackVel - [" + to_string( m_velocity, 2 ) + "]" ); WriteLog( "Type: TrackVel - [" + to_string( m_velocity, 2 ) + "]" );
for( auto &target : m_targets ) { for( auto &target : m_targets ) {
@@ -2024,7 +2035,7 @@ voltage_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) {
// run() subclass details // run() subclass details
void void
voltage_event::run_() { voltage_event::run_( TDynamicObject const *Activator ) {
// zmiana napięcia w zasilaczu (TractionPowerSource) // zmiana napięcia w zasilaczu (TractionPowerSource)
WriteLog( "Type: Voltage [" + to_string( m_voltage, 2 ) + "]" ); WriteLog( "Type: Voltage [" + to_string( m_voltage, 2 ) + "]" );
for( auto &target : m_targets ) { for( auto &target : m_targets ) {
@@ -2087,7 +2098,7 @@ visible_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) {
// run() subclass details // run() subclass details
void void
visible_event::run_() { visible_event::run_( TDynamicObject const *Activator ) {
for( auto &target : m_targets ) { for( auto &target : m_targets ) {
auto *targetnode = std::get<scene::basic_node *>( target ); auto *targetnode = std::get<scene::basic_node *>( target );
@@ -2131,7 +2142,7 @@ friction_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad )
// run() subclass details // run() subclass details
void void
friction_event::run_() { friction_event::run_( TDynamicObject const *Activator ) {
// zmiana tarcia na scenerii // zmiana tarcia na scenerii
WriteLog( "Type: Friction" ); WriteLog( "Type: Friction" );
Global.fFriction = m_friction; Global.fFriction = m_friction;
@@ -2171,10 +2182,10 @@ void lua_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad )
} }
// run() subclass details // run() subclass details
void lua_event::run_() { void lua_event::run_( TDynamicObject const *Activator ) {
try { try {
if (lua_func != LUA_NOREF) if (lua_func != LUA_NOREF)
lua::dispatch_event(lua_state, lua_func, this, m_activator); lua::dispatch_event(lua_state, lua_func, this, Activator);
} catch (...) { } catch (...) {
ErrorLog("lua: Runtime error: " + simulation::Lua.get_error()); ErrorLog("lua: Runtime error: " + simulation::Lua.get_error());
} }
@@ -2217,7 +2228,7 @@ message_event::deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) {
// run() subclass details // run() subclass details
void void
message_event::run_() { message_event::run_( TDynamicObject const *Activator ) {
// TODO: implement // TODO: implement
} }
@@ -2374,6 +2385,8 @@ event_manager::insert( basic_event *Event ) {
} }
} }
// the event's identity, fixed for its lifetime; the queue refers to it by this
Event->m_index = static_cast<std::uint32_t>( m_events.size() );
m_events.emplace_back( Event ); m_events.emplace_back( Event );
if( lookup == m_eventmap.end() ) { if( lookup == m_eventmap.end() ) {
// if it's first event with such name, it's potential candidate for the execution queue // if it's first event with such name, it's potential candidate for the execution queue
@@ -2420,118 +2433,131 @@ event_manager::FindEvent( std::string const &Name )
bool bool
event_manager::AddToQuery( basic_event *Event, TDynamicObject const *Owner, double delay ) { event_manager::AddToQuery( basic_event *Event, TDynamicObject const *Owner, double delay ) {
if( Event->m_passive ) { return false; } // jeśli może być dodany do kolejki (nie używany w skanowaniu) if( Event->m_passive ) { return false; } // not queued at all: scanning uses it in place
if( Event->m_inqueue > 0 ) { return false; } // jeśli nie dodany jeszcze do kolejki if( true == pending( Event ) ) { return false; } // already waiting its turn
// kolejka eventów jest posortowana względem (fStartTime)
Event->m_activator = Owner;
if( Event->is_instant() ) { if( Event->is_instant() ) {
// eventy AddValues trzeba wykonywać natychmiastowo, inaczej kolejka może zgubić jakieś dodawanie // AddValues and its kind have to run on the spot, or the queue loses additions
if( false == Event->m_ignored ) { if( false == Event->m_ignored ) {
Event->run(); Event->run( Owner );
} }
// jeśli jest kolejny o takiej samej nazwie, to idzie do kolejki (and if there's no joint event it'll be set to null and processing will end here) // an event sharing its name goes to the queue instead; skip the ones that cannot
do { do {
Event = Event->m_sibling; Event = Event->m_sibling;
// NOTE: we could've received a new event from joint event above, so we need to check conditions just in case and discard the bad events } while( ( Event != nullptr )
// TODO: refactor this arrangement, it's hardly optimal && ( Event->m_passive || pending( Event ) ) );
} while( Event != nullptr
&& ( Event->m_passive
|| Event->m_inqueue > 0 ) );
}
if( Event != nullptr
&& false == Event->m_ignored ) {
// standardowe dodanie do kolejki
++Event->m_inqueue; // zabezpieczenie przed podwójnym dodaniem do kolejki
WriteLog( "EVENT ADDED TO QUEUE" + ( Owner ? " by " + Owner->asName : "" ) + ": " + Event->m_name );
Event->m_launchtime = delay + std::abs( Event->m_delay ) + Timer::GetTime(); // czas od uruchomienia scenerii
if( Event->m_delayrandom > 0.0 ) {
// doliczenie losowego czasu opóźnienia
Event->m_launchtime += Event->m_delayrandom * Random();
}
if( Owner != nullptr
&& false == std::isnan(Event->m_delaydeparture) ) {
auto const *timetableowner { (
Owner->Mechanik != nullptr && Owner->Mechanik->primary() ?
Owner->Mechanik :
Owner->ctOwner ) };
if( timetableowner != nullptr ) {
auto const &timetable { timetableowner->TrainTimetable() };
auto const &time { simulation::Time.data() };
Event->m_launchtime +=
timetable.seconds_until_departure( time.wHour, time.wMinute + time.wSecond * 0.0167 )
+ Event->m_delaydeparture;
}
}
// NOTE: sanity check, as departure-based delay math can potentially produce negative overall delay
Event->m_launchtime = std::max( Event->m_launchtime, 0.0 );
if( QueryRootEvent != nullptr ) {
basic_event *target { QueryRootEvent };
basic_event *previous { nullptr };
while( Event->m_launchtime >= target->m_launchtime
&& target->m_next != nullptr ) {
previous = target;
target = target->m_next;
}
// the new event will be either before or after the one we located
if( Event->m_launchtime >= target->m_launchtime ) {
assert( target->m_next == nullptr );
target->m_next = Event;
// if we have resurrected event land at the end of list, the link from previous run could potentially "add" unwanted events to the queue
Event->m_next = nullptr;
}
else {
if( previous != nullptr ) {
previous->m_next = Event;
Event->m_next = target;
}
else {
// special case, we're inserting our event at the very start
Event->m_next = QueryRootEvent;
QueryRootEvent = Event;
}
}
}
else {
QueryRootEvent = Event;
QueryRootEvent->m_next = nullptr;
}
} }
if( ( Event == nullptr ) || ( true == Event->m_ignored ) ) { return true; }
WriteLog( "EVENT ADDED TO QUEUE" + ( Owner ? " by " + Owner->asName : "" ) + ": " + Event->m_name );
auto launchtime { delay + std::abs( Event->m_delay ) + Timer::GetTime() };
if( Event->m_delayrandom > 0.0 ) {
launchtime += Event->m_delayrandom * Random();
}
if( ( Owner != nullptr ) && ( false == std::isnan( Event->m_delaydeparture ) ) ) {
auto const *timetableowner { (
Owner->Mechanik != nullptr && Owner->Mechanik->primary() ?
Owner->Mechanik :
Owner->ctOwner ) };
if( timetableowner != nullptr ) {
auto const &timetable { timetableowner->TrainTimetable() };
auto const &time { simulation::Time.data() };
launchtime +=
timetable.seconds_until_departure( time.wHour, time.wMinute + time.wSecond * 0.0167 )
+ Event->m_delaydeparture;
}
}
// departure-based delays can work out negative overall
launchtime = std::max( launchtime, 0.0 );
push( { launchtime, Event->m_index, Owner } );
return true; return true;
} }
// legacy method, executes queued events // orders the queue: soonest first, and among events due at the same instant the one that
// appears first in the scenery. ties broken by index rather than by insertion order, so
// the same scenario runs the same way twice
bool
event_manager::later( queued_event const &Left, queued_event const &Right ) {
if( Left.launchtime != Right.launchtime ) { return Left.launchtime > Right.launchtime; }
return Left.index > Right.index;
}
bool
event_manager::pending( basic_event const *Event ) const {
return ( Event->m_index < m_queued.size() ) && ( m_queued[ Event->m_index ] != 0 );
}
void
event_manager::push( queued_event const &Entry ) {
if( Entry.index >= m_queued.size() ) { m_queued.resize( Entry.index + 1, 0 ); }
m_queued[ Entry.index ] = 1;
m_queue.emplace_back( Entry );
std::push_heap( std::begin( m_queue ), std::end( m_queue ), later );
}
std::uint32_t
event_manager::sibling_of( std::uint32_t const Index ) const {
// every event knows its own position, so following the sibling chain costs a pointer
// hop rather than a search
auto const *event { event_at( Index ) };
if( ( event == nullptr ) || ( event->m_sibling == nullptr ) ) {
return static_cast<std::uint32_t>( m_events.size() );
}
return event->m_sibling->m_index;
}
std::vector<event_manager::queued_event>
event_manager::queue_snapshot() const {
auto ordered { m_queue };
std::sort( std::begin( ordered ), std::end( ordered ),
[]( queued_event const &Left, queued_event const &Right ) { return later( Right, Left ); } );
return ordered;
}
bool bool
event_manager::CheckQuery() { event_manager::CheckQuery() {
while( QueryRootEvent != nullptr while( ( false == m_queue.empty() )
&& QueryRootEvent->m_launchtime < Timer::GetTime() ) && ( m_queue.front().launchtime < Timer::GetTime() ) ) {
{ // eventy są posortowana wg czasu wykonania
m_workevent = QueryRootEvent; // wyjęcie eventu z kolejki std::pop_heap( std::begin( m_queue ), std::end( m_queue ), later );
if (QueryRootEvent->m_sibling) // jeśli jest kolejny o takiej samej nazwie auto const entry { m_queue.back() };
{ // to teraz on będzie następny do wykonania m_queue.pop_back();
QueryRootEvent = QueryRootEvent->m_sibling; // następny będzie ten doczepiony m_queued[ entry.index ] = 0;
QueryRootEvent->m_next = m_workevent->m_next; // pamiętając o następnym z kolejki
QueryRootEvent->m_launchtime = m_workevent->m_launchtime; // czas musi być ten sam, bo nie jest aktualizowany m_workevent = event_at( entry.index );
QueryRootEvent->m_activator = m_workevent->m_activator; // pojazd aktywujący if( m_workevent == nullptr ) { continue; }
QueryRootEvent->m_inqueue = 1;
// w sumie można by go dodać normalnie do kolejki, ale trzeba te połączone posortować wg czasu wykonania // an event sharing this one's name follows it at the same instant. it goes
// through the queue like any other, rather than being spliced into the list head
auto const *sibling { m_workevent->m_sibling };
if( ( sibling != nullptr ) && ( false == sibling->m_passive ) && ( false == pending( sibling ) ) ) {
push( { entry.launchtime, sibling->m_index, entry.activator } );
} }
else // a jak nazwa jest unikalna, to kolejka idzie dalej
QueryRootEvent = QueryRootEvent->m_next; // NULL w skrajnym przypadku if( ( false == m_workevent->m_ignored ) && ( false == m_workevent->m_passive ) ) {
if( false == m_workevent->m_ignored && false == m_workevent->m_passive ) { m_workevent->run( entry.activator );
// w zasadzie te wyłączone są skanowane i nie powinny się nigdy w kolejce znaleźć }
--m_workevent->m_inqueue; // teraz moze być ponownie dodany do kolejki }
m_workevent->run();
} // if (tmpEvent->bEnabled)
} // while
return true; return true;
} }
// legacy method, initializes events after deserialization from scenario file // legacy method, initializes events after deserialization from scenario file
void void
event_manager::InitEvents() { event_manager::InitEvents() {
// how many events a scenery carries, which is worth knowing: they are not written by
// hand but expanded from parameterised includes, a dozen or more per signal
WriteLog( "Events: " + std::to_string( m_events.size() ) + " defined" );
//łączenie eventów z pozostałymi obiektami //łączenie eventów z pozostałymi obiektami
for( auto *event : m_events ) { for( auto *event : m_events ) {
event->init(); event->init();

View File

@@ -79,10 +79,10 @@ public:
virtual virtual
void void
init() = 0; init() = 0;
// executes event // executes the event on behalf of the vehicle that triggered it, if any
virtual virtual
void void
run(); run( TDynamicObject const *Activator );
// sends basic content of the class in legacy (text) format to provided stream // sends basic content of the class in legacy (text) format to provided stream
virtual virtual
void void
@@ -110,15 +110,20 @@ public:
void group( scene::group_handle Group ); void group( scene::group_handle Group );
scene::group_handle group() const; scene::group_handle group() const;
std::string const &name() const { return m_name; } std::string const &name() const { return m_name; }
// what this event calls itself in a scenery, for anything that has to write it back out
std::string type_name() const { return type(); }
// the names this event was written to act on, in the order it was written with them
std::vector<std::string> target_names() const;
// members // members
basic_event *m_next { nullptr }; // następny w kolejce // TODO: replace with event list in the manager
basic_event *m_sibling { nullptr }; // kolejny event z tą samą nazwą - od wersji 378 basic_event *m_sibling { nullptr }; // kolejny event z tą samą nazwą - od wersji 378
std::string m_name; std::string m_name;
bool m_ignored { false }; // replacement for tp_ignored bool m_ignored { false }; // replacement for tp_ignored
bool m_passive { false }; // false gdy ma nie być dodawany do kolejki (skanowanie sygnałów) bool m_passive { false }; // false gdy ma nie być dodawany do kolejki (skanowanie sygnałów)
int m_inqueue { 0 }; // ile razy dodany do kolejki // position in the manager's collection, assigned once when the event is inserted.
TDynamicObject const *m_activator { nullptr }; // it is the event's identity, not its state: the queue keys on it, so that two
double m_launchtime { 0.0 }; // events due at the same instant run in an order that does not depend on the order
// they happened to be queued in
std::uint32_t m_index { 0 };
double m_delay { 0.0 }; double m_delay { 0.0 };
double m_delayrandom { 0.0 }; // zakres dodatkowego opóźnienia // standardowo nie będzie dodatkowego losowego opóźnienia double m_delayrandom { 0.0 }; // zakres dodatkowego opóźnienia // standardowo nie będzie dodatkowego losowego opóźnienia
double m_delaydeparture { std::numeric_limits<double>::quiet_NaN() }; // departure-based event delay double m_delaydeparture { std::numeric_limits<double>::quiet_NaN() }; // departure-based event delay
@@ -168,7 +173,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
virtual void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) = 0; virtual void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) = 0;
// run() subclass details // run() subclass details
virtual void run_() = 0; virtual void run_( TDynamicObject const *Activator ) = 0;
// export_as_text() subclass details // export_as_text() subclass details
virtual void export_as_text_( std::ostream &Output ) const = 0; virtual void export_as_text_( std::ostream &Output ) const = 0;
// members // members
@@ -220,7 +225,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -243,7 +248,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
}; };
@@ -273,7 +278,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
}; };
@@ -299,7 +304,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
//determines whether provided input should be passed to consist owner //determines whether provided input should be passed to consist owner
@@ -322,7 +327,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
}; };
@@ -343,7 +348,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
}; };
@@ -369,7 +374,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -398,7 +403,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -432,7 +437,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -459,7 +464,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -488,7 +493,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -511,7 +516,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -536,7 +541,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -559,7 +564,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -582,7 +587,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -605,7 +610,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -622,7 +627,7 @@ public:
private: private:
std::string type() const override; std::string type() const override;
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
void run_() override; void run_( TDynamicObject const *Activator ) override;
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
bool is_instant() const override; bool is_instant() const override;
@@ -645,7 +650,7 @@ private:
// deserialize() subclass details // deserialize() subclass details
void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override; void deserialize_( cParser &Input, scene::scratch_data &Scratchpad ) override;
// run() subclass details // run() subclass details
void run_() override; void run_( TDynamicObject const *Activator ) override;
// export_as_text() subclass details // export_as_text() subclass details
void export_as_text_( std::ostream &Output ) const override; void export_as_text_( std::ostream &Output ) const override;
// members // members
@@ -687,11 +692,39 @@ public:
inline void purge (TEventLauncher *Launcher) { inline void purge (TEventLauncher *Launcher) {
m_radiodrivenlaunchers.purge(Launcher); m_radiodrivenlaunchers.purge(Launcher);
m_inputdrivenlaunchers.purge(Launcher); } m_inputdrivenlaunchers.purge(Launcher); }
// returns first event in the queue // one entry of the pending queue. the event itself holds none of this: when it runs
inline // and what asked for it are properties of this request, not of the event
struct queued_event {
double launchtime { 0.0 };
std::uint32_t index { 0 };
TDynamicObject const *activator { nullptr };
};
// the queue in the order it will run, for anything that wants to look at it.
// returned by value: this is for display and debugging, not for the hot path
std::vector<queued_event>
queue_snapshot() const;
// the event an entry refers to, or null if the index is stale
basic_event * basic_event *
begin() { event_at( std::uint32_t const Index ) const {
return QueryRootEvent; } return Index < m_events.size() ? m_events[ Index ] : nullptr; }
bool
queue_empty() const { return m_queue.empty(); }
// how many events the manager holds. with event_at() this is the whole read-only view
// of the collection, which is all a cooker needs
std::uint32_t
event_count() const { return static_cast<std::uint32_t>( m_events.size() ); }
// the index of an event's next sibling - the next event registered under the same name
// - or an out of range value when it has none
std::uint32_t
sibling_of( std::uint32_t const Index ) const;
private:
static bool later( queued_event const &Left, queued_event const &Right );
bool pending( basic_event const *Event ) const;
void push( queued_event const &Entry );
public:
basic_event* basic_event*
FindEventById(uint32_t id); FindEventById(uint32_t id);
@@ -735,8 +768,14 @@ private:
// NOTE: disabled until event class refactoring // NOTE: disabled until event class refactoring
event_sequence m_eventqueue; event_sequence m_eventqueue;
*/ */
// legacy version of the above // pending events, kept as a binary heap ordered by launch time and then by event
basic_event *QueryRootEvent { nullptr }; // index. a sorted linked list threaded through the events themselves cost a linear
// walk on every insertion, put queue state inside the event definition, and left the
// order of events due at the same instant depending on insertion history
std::vector<queued_event> m_queue;
// whether an event is already pending, indexed by event index. guards against an
// event being queued twice over
std::vector<std::uint8_t> m_queued;
basic_event *m_workevent { nullptr }; basic_event *m_workevent { nullptr };
event_map m_eventmap; event_map m_eventmap;
basic_table<TEventLauncher> m_inputdrivenlaunchers; basic_table<TEventLauncher> m_inputdrivenlaunchers;