eu07/maszyna
17
0
Fork 0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-11-03 07:20:22 +01:00
Code Issues 157 Wiki Activity

Stream the scenery: a baked terrain heightfield, a logic container, and prefabs

Browse Source 
Today's work on the binary scene format plan (binary-scene-format-plan.md), from the
first measurement to a terrain that bakes itself on first run and streams from a
thread. The step-by-step history is kept on backup/scene-streaming-format-presquash-0916.

Measuring first. -loadprofile times a scenery load by stage and by node kind and reports
node counts and resident memory. On l204 terrain is 95% of the bytes and 99.8% of the
nodes; on quarkmce2007 vehicles, not models, dominate what is left. The terrain turned
out to be a heightfield in all but name, a ribbon covering 12% of its bounding box.

The heightfield. scene/heightfieldformat.h is the .ehf layout: Morton-ordered tiles
of 128 samples with one shared edge sample, a point-sampled mip chain, zstd per tile,
a nodata sentinel, a material palette with a measured tiling per material, and a stamp
of the rules that decided what counted as terrain. scene/heightfieldreader.h reads it
for the tool, the renderer and the loader alike, and scene/terraincooker.h writes it,
shared by the terraincook tool and the engine so a first-run bake is the file the tool
would have made. The cooker rasterises every triangle where it is drawn (through
scene::place_in_world, the placement the region applies too), keeps an upper surface
laid within half a metre over another and the lower of two further apart, and then
asks each triangle whether the heightfield shows it: laid, reaching a sample, keeping
nearly all it claimed, and with the ground around it drawn at every level out to 2 km.

Baking on first run. simulation/terrainbake bakes <scenario>.ehf the first time a
scenery is opened, choosing the grid step from the first 200,000 vertices, and on later
loads leaves out the triangles its node table says the heightfield shows. Everything
else - trees, grass tufts, gantries, bridge decks - is drawn as written, so a bad bake
can at worst draw something twice, and the heightfield sits 2 cm under the triangles so
the original wins there. Nodes are keyed by file, line, include parameters and
placement; an edited file changes its keys and those nodes are drawn again. A bake under
older rules, or without its table, is thrown away and redone. A launcher copy named
$scenario shares the scenario's files. -nobaketerrain shows the triangles as written.
A scenery can still name hand-cooked fields, several at different steps.

Drawing and streaming. rendering/terrainclipmap draws resident tiles at the mip level
their distance earns, deriving the grid from gl_VertexID and texturing in world space
with the scenery's own materials. rendering/terraintileloader reads and decodes tiles on
a thread of its own; the renderer rescans what is in range only after moving a quarter
of a tile, asks for what is missing nearest first, uploads at most 32 tiles a frame,
drops arrivals the camera has moved past, and keeps drawing a tile at the level it has
until the right one arrives. The terrain has its own range, terrainrange in the ini and
a slider in the debug panel, 20 km by default and independent of basedrawrange; the far
plane of an ordinary view is pushed out to it. The debug panel shows the range, tiles in
view, on the gpu, wanted and queued, and how many sit at each level.

Events. The event queue moved out of the events into a binary heap with a deterministic
tie-break, and an event is handed its activator instead of storing it, so basic_event
is a definition and nothing else.

The logic container. scene/logicformat.h and scene/logiccontainer write a scenery's
events, their targets and sibling chains as flat, deterministic arrays with a schema
hash that refuses a mismatched build. simulation/logiccook cooks it from the loaded
state whenever it is missing, stale or unreadable, reads it straight back to check it,
and resolves each target to a position in the registry its event kind searches.
-cooklogic forces a cook, -verifylogic only checks. Nothing reads it at load yet.

Prefabs. scene/prefab describes a signal by the aspects it can show - lights, speeds,
animation channels, a sound - instead of pasting an .inc per variant, and expands an
instance to ordinary scenery text. Tried on a light signal and a shape signal.

Screenshots without a person present: -shot x,y,z[,yaw[,pitch]], -shotsettle, -shotstay.
This commit is contained in:
maj00r
2026-09-16 23:18:09 +02:00
parent 366645ef59
commit 2660d14565
47 changed files with 7117 additions and 240 deletions
Download Patch File Download Diff File Expand all files Collapse all files

231
simulation/loadprofile.cpp Normal file
View File

@@ -0,0 +1,231 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <array>
#include <atomic>
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <fstream>
#include <iomanip>
#include <sstream>
#include <string>
#include <string_view>
#include <unordered_map>
#ifdef __linux__
#include <unistd.h>
#endif
module eu07.simulation.loadprofile;
import eu07.utilities.logs;
namespace loadprofile {
namespace {
bool profiling { false };
std::array<double, static_cast<std::size_t>( stage::count )> stagetime {};
std::array<std::size_t, static_cast<std::size_t>( kind::count )> nodecount {};
std::array<double, static_cast<std::size_t>( kind::count )> kindtime {};
std::size_t vertexcount { 0 };
std::size_t splitcount { 0 }; // shapes the trimmer had to divide
std::size_t splitpieces { 0 }; // extra pieces the trimmer produced
std::size_t memoryatstart { 0 };
double totaltime { 0.0 };
// resident set size in kilobytes, 0 when it cannot be read. linux only; on other
// platforms the memory lines of the report simply stay at zero
std::size_t
resident_memory() {
#ifdef __linux__
std::ifstream statm( "/proc/self/statm" );
if( false == statm.good() ) { return 0; }
std::size_t total { 0 }, resident { 0 };
statm >> total >> resident;
return resident * ( static_cast<std::size_t>( ::sysconf( _SC_PAGESIZE ) ) / 1024 );
#else
return 0;
#endif
}
std::string const stagenames[] {
"token loop (rest)",
"triangle import",
"triangle insert",
" of which divider",
"other node types" };
std::string const kindnames[] {
"triangles", "lines", "track", "traction", "model",
"dynamic", "memcell", "eventlauncher", "sound", "other" };
std::string
megabytes( std::size_t const Kilobytes ) {
std::ostringstream converted;
converted << std::fixed << std::setprecision( 1 ) << ( Kilobytes / 1024.0 ) << " MB";
return converted.str();
}
std::string
seconds( double const Seconds ) {
std::ostringstream converted;
converted << std::fixed << std::setprecision( 2 ) << Seconds << " s";
return converted.str();
}
} // namespace
bool
enabled() {
return profiling;
}
void
enable( bool const Enable ) {
profiling = Enable;
}
void
reset() {
stagetime.fill( 0.0 );
nodecount.fill( 0 );
kindtime.fill( 0.0 );
totaltime = 0.0;
vertexcount = 0;
splitcount = 0;
splitpieces = 0;
memoryatstart = resident_memory();
}
void
add_time( stage const Stage, double const Seconds ) {
if( false == profiling ) { return; }
stagetime[ static_cast<std::size_t>( Stage ) ] += Seconds;
}
void
add_total( double const Seconds ) {
if( false == profiling ) { return; }
totaltime += Seconds;
}
void
add_time( kind const Kind, double const Seconds ) {
if( false == profiling ) { return; }
kindtime[ static_cast<std::size_t>( Kind ) ] += Seconds;
}
void
add_node( kind const Kind ) {
if( false == profiling ) { return; }
++nodecount[ static_cast<std::size_t>( Kind ) ];
}
void
add_vertices( std::size_t const Vertices ) {
if( false == profiling ) { return; }
vertexcount += Vertices;
}
void
add_split( std::size_t const Pieces ) {
if( false == profiling ) { return; }
++splitcount;
splitpieces += Pieces;
}
kind
kind_of( std::string_view const Type ) {
if( ( Type == "triangles" ) || ( Type == "triangle_strip" ) || ( Type == "triangle_fan" ) ) { return kind::triangles; }
if( ( Type == "lines" ) || ( Type == "line_strip" ) || ( Type == "line_loop" ) ) { return kind::lines; }
if( Type == "track" ) { return kind::track; }
if( Type == "traction" ) { return kind::traction; }
if( Type == "model" ) { return kind::model; }
if( Type == "dynamic" ) { return kind::dynamic; }
if( Type == "memcell" ) { return kind::memcell; }
if( Type == "eventlauncher" ) { return kind::eventlauncher; }
if( Type == "sound" ) { return kind::sound; }
return kind::other;
}
void
report( std::string const &Scenariofile ) {
if( false == profiling ) { return; }
auto const memoryatend { resident_memory() };
std::ostringstream output;
output << "-- scenery load profile: " << Scenariofile << "\n";
// the node stages are measured inside the token loop, so what is left of the loop
// time once they are taken out is the parsing itself. the divider is a subset of
// the insert stage and stays out of both sums
auto accounted { 0.0 };
for( auto stageindex { 0u }; stageindex < static_cast<std::size_t>( stage::count ); ++stageindex ) {
if( ( stageindex != static_cast<std::size_t>( stage::shape_divide ) )
&& ( stageindex != static_cast<std::size_t>( stage::parse ) ) ) {
accounted += stagetime[ stageindex ];
}
}
stagetime[ static_cast<std::size_t>( stage::parse ) ] = ( totaltime > accounted ? totaltime - accounted : 0.0 );
auto const total { totaltime > accounted ? totaltime : accounted };
for( auto stageindex { 0u }; stageindex < static_cast<std::size_t>( stage::count ); ++stageindex ) {
auto const share {
total > 0.0 ?
100.0 * stagetime[ stageindex ] / total :
0.0 };
output
<< " " << std::left << std::setw( 20 ) << stagenames[ stageindex ]
<< std::right << std::setw( 10 ) << seconds( stagetime[ stageindex ] )
<< std::setw( 8 ) << std::fixed << std::setprecision( 1 ) << share << " %\n";
}
output << " " << std::left << std::setw( 20 ) << "total" << std::right << std::setw( 10 ) << seconds( total ) << "\n";
output << "-- nodes\n";
for( auto kindindex { 0u }; kindindex < static_cast<std::size_t>( kind::count ); ++kindindex ) {
if( nodecount[ kindindex ] == 0 ) { continue; }
output
<< " " << std::left << std::setw( 20 ) << kindnames[ kindindex ]
<< std::right << std::setw( 12 ) << nodecount[ kindindex ] << "\n";
}
output << "-- time by node kind\n";
for( auto kindindex { 0u }; kindindex < static_cast<std::size_t>( kind::count ); ++kindindex ) {
if( kindtime[ kindindex ] < 0.005 ) { continue; }
output
<< " " << std::left << std::setw( 20 ) << kindnames[ kindindex ]
<< std::right << std::setw( 10 ) << seconds( kindtime[ kindindex ] )
<< std::setw( 12 ) << nodecount[ kindindex ] << " nodes\n";
}
output << "-- nodes\n";
output << " " << std::left << std::setw( 20 ) << "shape vertices" << std::right << std::setw( 12 ) << vertexcount << "\n";
output << " " << std::left << std::setw( 20 ) << "shapes divided" << std::right << std::setw( 12 ) << splitcount
<< " (extra pieces: " << splitpieces << ")\n";
output << "-- memory (resident)\n";
output << " " << std::left << std::setw( 20 ) << "before load" << std::right << std::setw( 12 ) << megabytes( memoryatstart ) << "\n";
output << " " << std::left << std::setw( 20 ) << "after load" << std::right << std::setw( 12 ) << megabytes( memoryatend ) << "\n";
output << " " << std::left << std::setw( 20 ) << "delta" << std::right << std::setw( 12 )
<< megabytes( memoryatend > memoryatstart ? memoryatend - memoryatstart : 0 ) << "\n";
WriteLog( output.str() );
std::ofstream file( "loadprofile.txt", std::ios::app );
if( true == file.good() ) {
file << output.str() << std::endl;
}
}
} // loadprofile

108
simulation/loadprofile.cppm Normal file
View File

@@ -0,0 +1,108 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <array>
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <string>
#include <string_view>
export module eu07.simulation.loadprofile;
export {
// Scenery load profiler. Answers the questions phase 0 of the binary format plan asks:
// where the load time goes, how many nodes of each kind the scenery holds, and how much
// memory the result occupies. Measurement only - nothing here changes what gets loaded.
namespace loadprofile {
// stages the load time is split into. shape_import covers parsing the vertex data of a
// triangle node, shape_insert the region side of it: trimming to cell boundaries and
// handing the geometry to its bank
enum class stage : std::size_t {
parse, // token loop remainder: total time less every stage below
shape_import,
shape_insert,
shape_divide, // RaTriangleDivider, the subset of shape_insert spent splitting
other_nodes, // every node type other than the triangle family
count
};
// node kinds counted separately; anything else lands in 'other'
enum class kind : std::size_t {
triangles,
lines,
track,
traction,
model,
dynamic,
memcell,
eventlauncher,
sound,
other,
count
};
// discards everything gathered so far and takes the starting memory reading
void reset();
// accumulates elapsed time in the given stage
void add_time( stage Stage, double const Seconds );
// accumulates elapsed time against a node kind, so that "everything else" can be told
// apart into what it actually is
void add_time( kind const Kind, double const Seconds );
// accumulates elapsed time of the whole token loop. the parse stage is reported as
// what is left of it once the node stages are taken out
void add_total( double const Seconds );
// registers a node of the given kind
void add_node( kind const Kind );
// adds to the tally of vertices held by shape nodes
void add_vertices( std::size_t const Vertices );
// registers a shape split into extra pieces by the cell-boundary trimmer
void add_split( std::size_t const Pieces );
// true when profiling is switched on; the instrumentation is skipped otherwise
bool enabled();
// switches profiling on or off. off by default, turned on by the -loadprofile switch
void enable( bool const Enable );
// resolves the node type name used by the scenario format to a counter
kind kind_of( std::string_view const Type );
// writes the gathered numbers to the log and to loadprofile.txt in the working directory
void report( std::string const &Scenariofile );
// RAII helper accumulating the lifetime of the scope against a stage or a node kind. the
// clock is not read at all while profiling is off, since this sits on per-node paths
template <typename Counter_>
class scoped_timer {
public:
explicit scoped_timer( Counter_ const Counter ) :
m_counter( Counter ) {
if( true == enabled() ) {
m_start = std::chrono::steady_clock::now();
m_running = true;
} }
~scoped_timer() {
if( false == m_running ) { return; }
add_time(
m_counter,
std::chrono::duration<double>( std::chrono::steady_clock::now() - m_start ).count() ); }
scoped_timer( scoped_timer const & ) = delete;
scoped_timer &operator=( scoped_timer const & ) = delete;
private:
Counter_ m_counter;
std::chrono::steady_clock::time_point m_start;
bool m_running { false };
};
} // loadprofile
} // export

264
simulation/logiccook.cpp Normal file
View File

@@ -0,0 +1,264 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <cstdint>
#include <string>
#include <algorithm>
#include <set>
#include <utility>
#include <vector>
#include "utilities/Globals_macros.h"
#include "global_include/interfaces/ITexture_macros.h"
#include "scene/logicformat.h"
module eu07.simulation.logiccook;
import eu07.scene.logiccontainer;
import eu07.simcore;
import eu07.simulation.simulation;
import eu07.simulation.simulationsounds;
import eu07.utilities.logs;
import eu07.utilities.globals;
import eu07.utilities.utilities;
namespace simulation {
namespace {
// the flat record an event becomes. everything here is read off the event itself, so the
// container cannot drift from what the text path built
scene::logic::event_record
record_of( basic_event const &Event, scene::logic::event_kind const Kind, std::uint32_t const Sibling, std::uint32_t const Name ) {
scene::logic::event_record record {};
record.kind = static_cast<std::uint16_t>( Kind );
record.name = Name;
record.sibling = ( Sibling < simulation::Events.event_count() ? Sibling : scene::logic::no_index );
record.group = ( Event.group() == null_handle ? scene::logic::no_index : static_cast<std::uint32_t>( Event.group() ) );
record.delay = Event.m_delay;
record.delayrandom = Event.m_delayrandom;
record.delaydeparture = Event.m_delaydeparture;
record.ignored = ( Event.m_ignored ? 1 : 0 );
record.passive = ( Event.m_passive ? 1 : 0 );
return record;
}
// where an event of this kind looks for its targets, and where it looks next. this mirrors
// each subclass's init() rather than guessing: an event that resolves against the memory
// cells will never find its target among the tracks, and saying otherwise here would bind
// it to whatever happened to share the name
std::pair<scene::logic::node_table, scene::logic::node_table>
tables_for( scene::logic::event_kind const Kind ) {
using kind = scene::logic::event_kind;
using table = scene::logic::node_table;
switch( Kind ) {
case kind::updatevalues:
case kind::addvalues:
case kind::copyvalues:
case kind::getvalues:
case kind::putvalues:
case kind::whois:
case kind::logvalues:
case kind::multiple: return { table::memory, table::none };
case kind::texture:
case kind::animation:
case kind::lights: return { table::instance, table::none };
case kind::switchevent:
case kind::trackvel: return { table::path, table::none };
case kind::voltage: return { table::powergrid, table::none };
case kind::sound: return { table::sound, table::none };
// a visible event takes model instances and tracks alike, and looks in that order
case kind::visible: return { table::instance, table::path };
default: return { table::none, table::none };
}
}
std::uint32_t
index_in( scene::logic::node_table const Table, std::string const &Name ) {
using table = scene::logic::node_table;
switch( Table ) {
case table::memory: return simulation::Memory.find_id( Name );
case table::instance: return simulation::Instances.find_id( Name );
case table::path: return simulation::Paths.find_id( Name );
case table::sound: return simulation::Sounds.find_id( Name );
case table::powergrid: return simulation::Powergrid.find_id( Name );
default: return static_cast<std::uint32_t>( -1 );
}
}
// the entries an event's targets become, each one carrying the registry it was found in
std::vector<scene::logic::target_entry>
targets_of( basic_event const &Event, scene::logic::event_kind const Kind, scene::logic::container_writer &Writer ) {
auto const kind { Kind };
auto const [ first, second ] { tables_for( kind ) };
std::vector<scene::logic::target_entry> entries;
for( auto const &name : Event.target_names() ) {
scene::logic::target_entry entry {};
entry.name = Writer.intern( name );
entry.table = static_cast<std::uint16_t>( scene::logic::node_table::none );
entry.index = scene::logic::no_index;
for( auto const table : { first, second } ) {
if( table == scene::logic::node_table::none ) { continue; }
auto const index { index_in( table, name ) };
if( index != static_cast<std::uint32_t>( -1 ) ) {
entry.table = static_cast<std::uint16_t>( table );
entry.index = index;
break;
}
}
entries.emplace_back( entry );
}
return entries;
}
} // anonymous namespace
bool
cook_logic( std::string const &Sceneryfile ) {
scene::logic::container_writer writer;
auto const count { simulation::Events.event_count() };
std::uint32_t unknown { 0 };
std::set<std::string> unknownkinds;
for( std::uint32_t index = 0; index < count; ++index ) {
auto const *cooked { simulation::Events.event_at( index ) };
if( cooked == nullptr ) { continue; }
auto const name { writer.intern( cooked->name() ) };
auto const kind { scene::logic::kind_of( cooked->type_name() ) };
auto const record { record_of( *cooked, kind, simulation::Events.sibling_of( index ), name ) };
if( record.kind == static_cast<std::uint16_t>( scene::logic::event_kind::unknown ) ) {
++unknown;
if( unknownkinds.insert( cooked->type_name() ).second ) {
ErrorLog( "Logic container: no cooked kind for event type \"" + cooked->type_name() + "\" (first seen on \"" + cooked->name() + "\")" );
}
}
writer.add_event( record, targets_of( *cooked, kind, writer ) );
}
if( unknown > 0 ) {
// a kind the container has no value for would come back as something else entirely,
// so say so loudly rather than write a file that reads back wrong
ErrorLog( "Logic container: " + std::to_string( unknown ) + " events of a kind this build cannot cook" );
}
return writer.write( scene::logic::container_path( Sceneryfile ) );
}
bool
logic_stale( std::string const &Sceneryfile ) {
auto const path { scene::logic::container_path( Sceneryfile ) };
if( false == FileExists( path ) ) {
return true;
}
if( last_modified( scenery_file( Sceneryfile ) ) > last_modified( path ) ) {
WriteLog( "Logic container: \"" + path + "\" is older than the scenery, recooking" );
return true;
}
// the header says whether this build can read it at all, which covers a container
// cooked before a schema change
if( false == scene::logic::container_reader::readable( path ) ) {
WriteLog( "Logic container: \"" + path + "\" cannot be read by this build, recooking" );
return true;
}
return false;
}
bool
verify_logic( std::string const &Sceneryfile ) {
scene::logic::container_reader reader;
auto const path { scene::logic::container_path( Sceneryfile ) };
if( false == reader.open( path ) ) {
ErrorLog( "Logic container: cannot verify \"" + path + "\"" );
return false;
}
std::uint32_t mismatches { 0 };
auto const count { simulation::Events.event_count() };
if( reader.events() != count ) {
ErrorLog(
"Logic container: holds " + std::to_string( reader.events() )
+ " events, the scenery has " + std::to_string( count ) );
++mismatches;
}
auto const shared { std::min<std::size_t>( reader.events(), count ) };
for( std::size_t index = 0; index < shared; ++index ) {
auto const *loaded { simulation::Events.event_at( static_cast<std::uint32_t>( index ) ) };
if( loaded == nullptr ) { continue; }
auto const &record { reader.event( index ) };
auto const complain {
[ &mismatches, loaded ]( std::string const &What ) {
ErrorLog( "Logic container: event \"" + loaded->name() + "\" " + What );
++mismatches;
} };
if( reader.string( record.name ) != loaded->name() ) {
complain( "is called \"" + std::string( reader.string( record.name ) ) + "\" in the container" );
continue;
}
if( record.kind != static_cast<std::uint16_t>( scene::logic::kind_of( loaded->type_name() ) ) ) {
complain( "was cooked as a different kind" );
}
// delays are written as they are held, so they compare exactly. a difference here
// means the record lost precision, which is the whole thing worth catching
if( record.delay != loaded->m_delay ) {
complain( "has a different delay in the container" );
}
if( record.delayrandom != loaded->m_delayrandom ) {
complain( "has a different random delay in the container" );
}
auto const targets { loaded->target_names() };
if( record.targetcount != targets.size() ) {
complain(
"has " + std::to_string( record.targetcount ) + " targets in the container, "
+ std::to_string( targets.size() ) + " in the scenery" );
continue;
}
for( std::uint32_t target = 0; target < record.targetcount; ++target ) {
auto const &entry { reader.target( record.targetfirst + target ) };
if( reader.string( entry.name ) != targets[ target ] ) {
complain( "target " + std::to_string( target ) + " is \"" + std::string( reader.string( entry.name ) ) + "\", not \"" + targets[ target ] + "\"" );
continue;
}
// the index is only meaningful for the parse the container was cooked from, so
// the check that matters is whether it still lands on that same name
if( entry.index != scene::logic::no_index ) {
auto const table { static_cast<scene::logic::node_table>( entry.table ) };
if( index_in( table, targets[ target ] ) != entry.index ) {
complain( "target " + std::to_string( target ) + " \"" + targets[ target ] + "\" is at a different index now" );
}
}
}
}
if( mismatches == 0 ) {
WriteLog( "Logic container: \"" + path + "\" matches the loaded scenery, " + std::to_string( count ) + " events" );
return true;
}
ErrorLog( "Logic container: \"" + path + "\" disagrees with the loaded scenery in " + std::to_string( mismatches ) + " places" );
return false;
}
} // namespace simulation

42
simulation/logiccook.cppm Normal file
View File

@@ -0,0 +1,42 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <string>
export module eu07.simulation.logiccook;
export {
// Cooking the LOGIC container out of a scenery that has just been loaded.
//
// The plan called for a standalone tools/scn-cook reading .scn directly. It is cooked from
// inside the engine instead, and on purpose: an event's targets, siblings and indices are the
// product of the whole scenery deserializer, not of the parser, so a standalone cooker would
// have to grow a second copy of it. Cooking from the loaded state guarantees the thing that
// actually matters - that what is written is exactly what the text path produces - and the
// determinism the plan asks for comes from the writer, not from where it is called.
namespace simulation {
// writes the container next to the scenery. returns false and logs on failure
bool cook_logic( std::string const &Sceneryfile );
// reads a cooked container back and checks it against the events currently loaded. logs
// every disagreement and returns true only when there are none
bool verify_logic( std::string const &Sceneryfile );
// whether the container next to this scenery needs cooking: there is none, this build
// cannot read the one that is there, or the scenery has been edited since.
//
// Edits to an .inc the scenery pulls in do NOT make it stale - only the .scn's own
// timestamp is compared, because the list of files that went into a load is not kept. Cook
// by hand with -cooklogic after editing an include.
bool logic_stale( std::string const &Sceneryfile );
} // namespace simulation
}

158
simulation/simulationstateserializer.cpp
View File

@@ -17,6 +17,7 @@ module;
#include <cstddef>
#include <cstdlib>
#include <memory>
#include <optional>
#include <string>
#include "global_include/interfaces/ITexture_macros.h"
#include "utilities/Globals_macros.h"
@@ -29,6 +30,7 @@ import eu07.utilities.classes;
import eu07.glm;
import eu07.utilities.globals;
import eu07.simulation.simulation;
import eu07.simulation.loadprofile;
import eu07.simulation.simulationtime;
import eu07.simulation.simulationsounds;
import eu07.simulation.simulationenvironment;
@@ -80,6 +82,7 @@ state_serializer::deserialize_begin( std::string const &Scenariofile ) {
// TODO: check first for presence of serialized binary files
// if this fails, fall back on the legacy text format
state->scratchpad.name = Scenariofile;
loadprofile::reset();
if( true == Global.file_binary_terrain
&& Scenariofile != "$.scn" ) {
// compilation to binary file isn't supported for rainsted-created overrides
@@ -99,6 +102,11 @@ state_serializer::deserialize_begin( std::string const &Scenariofile ) {
}
scene::Groups.create();
// before a token is read: pick up a heightfield baked on an earlier run, or arm the
// bake when there is none. it has to happen here because the decision to skip terrain
// triangles is taken per node while parsing
terrainbake::begin( Scenariofile );
if( false == state->input.ok() ) {
if( false == Global.editor_startup ) {
throw invalid_scenery_exception();
@@ -131,6 +139,7 @@ state_serializer::deserialize_begin( std::string const &Scenariofile ) {
{ "endgroup", &state_serializer::deserialize_endgroup },
{ "light", &state_serializer::deserialize_light },
{ "node", &state_serializer::deserialize_node },
{ "prefab", &state_serializer::deserialize_prefab },
{ "origin", &state_serializer::deserialize_origin },
{ "endorigin", &state_serializer::deserialize_endorigin },
{ "scale", &state_serializer::deserialize_scale },
@@ -177,11 +186,14 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
auto timenow = std::chrono::steady_clock::now();
if( std::chrono::duration_cast<std::chrono::milliseconds>( timenow - timelast ).count() >= 200 ) {
Application.set_progress( Input.getProgress(), Input.getFullProgress() );
loadprofile::add_total( std::chrono::duration<double>( timenow - timelast ).count() );
return true;
}
token = Input.getToken<std::string>();
}
loadprofile::add_total(
std::chrono::duration<double>( std::chrono::steady_clock::now() - timelast ).count() );
if( false == Scratchpad.initialized ) {
// manually perform scenario initialization
@@ -202,6 +214,22 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
Region->serialize( state->scenariofile );
}
terrainbake::finish( state->scenariofile );
loadprofile::report( state->scenariofile );
if( true == Global.verify_logic ) {
verify_logic( state->scenariofile );
}
else if( ( true == Global.cook_logic )
|| ( true == logic_stale( state->scenariofile ) ) ) {
// cooked from the loaded state rather than from the text, and immediately read back,
// so a container that disagrees with what it was cooked from says so on the spot
if( true == cook_logic( state->scenariofile ) ) {
verify_logic( state->scenariofile );
}
}
return false;
}
@@ -457,6 +485,40 @@ state_serializer::deserialize_light( cParser &Input, scene::scratch_data &Scratc
skip_until( Input, "endlight" );
}
void
state_serializer::deserialize_prefab( cParser &Input, scene::scratch_data &Scratchpad ) {
// prefab <definition> <name> <x> <y> <z> <yaw> [<texture>] endprefab
scene::prefab_placement placement;
Input.getTokens( 1, false );
std::string definition; Input >> definition;
Input.getTokens( 1, false );
Input >> placement.name;
Input.getTokens( 4 );
Input
>> placement.x
>> placement.y
>> placement.z
>> placement.yaw;
auto token { Input.getToken<std::string>( false ) };
if( ( false == token.empty() ) && ( token != "endprefab" ) ) {
placement.texture = token;
token = Input.getToken<std::string>( false );
}
if( token != "endprefab" ) {
ErrorLog( "Bad scenario: prefab \"" + placement.name + "\" is missing its endprefab, found \"" + token + "\" instead" );
}
auto const scenery { Prefabs.expand( definition, placement ) };
if( true == scenery.empty() ) {
ErrorLog( "Bad scenario: prefab \"" + placement.name + "\" of unknown type \"" + definition + "\" was skipped" );
return;
}
// the expansion is ordinary scenery text, and goes in as an include would
Input.injectString( scenery );
}
void
state_serializer::deserialize_node( cParser &Input, scene::scratch_data &Scratchpad ) {
@@ -471,6 +533,19 @@ state_serializer::deserialize_node( cParser &Input, scene::scratch_data &Scratch
>> nodedata.name
>> nodedata.type;
if( nodedata.name == "none" ) { nodedata.name.clear(); }
// the triangle family is timed in its own branch below; everything else is timed
// against the kind of node it is, since "not a triangle" covers models, tracks,
// traction and vehicles alike and they cost very different amounts
std::optional<loadprofile::scoped_timer<loadprofile::stage>> profiledstage;
std::optional<loadprofile::scoped_timer<loadprofile::kind>> profiledkindtime;
if( true == loadprofile::enabled() ) {
auto const profiledkind { loadprofile::kind_of( nodedata.type ) };
loadprofile::add_node( profiledkind );
if( profiledkind != loadprofile::kind::triangles ) {
profiledstage.emplace( loadprofile::stage::other_nodes );
profiledkindtime.emplace( profiledkind );
}
}
// type-based deserialization. not elegant but it'll do
if( nodedata.type == "dynamic" ) {
@@ -604,25 +679,71 @@ state_serializer::deserialize_node( cParser &Input, scene::scratch_data &Scratch
|| nodedata.type == "triangle_strip"
|| nodedata.type == "triangle_fan" ) {
auto const file { Input.Name() };
auto const switchtrackbedfile {
// crude way to detect fixed switch trackbed geometry
true == Global.CreateSwitchTrackbeds
&& file.size() >= 15
&& file.starts_with( "scenery/zwr" )
&& file.ends_with( ".inc" ) };
auto const offset {
Scratchpad.location.offset.empty() ? glm::dvec3( 0.0 ) : Scratchpad.location.offset.top() };
auto const worldspace {
( offset == glm::dvec3( 0.0 ) ) && ( Scratchpad.location.rotation == glm::vec3( 0.0f ) ) };
// whether a heightfield stands in for this node, and whether a bake wants it
auto const decision {
( ( true == Scratchpad.binary.terrain ) || ( true == switchtrackbedfile ) )
? terrainbake::node_decision{}
: terrainbake::examine(
file, Input.Line(), Input.Parameters(), nodedata.type,
offset, Scratchpad.location.rotation, worldspace ) };
auto const skip {
// all shapes will be loaded from the binary version of the file
true == Scratchpad.binary.terrain
// crude way to detect fixed switch trackbed geometry
|| ( true == Global.CreateSwitchTrackbeds
&& Input.Name().size() >= 15
&& Input.Name().starts_with("scenery/zwr")
&& Input.Name().ends_with(".inc") ) };
( true == Scratchpad.binary.terrain )
// old trackbed geometry the engine replaces
|| ( true == switchtrackbedfile )
|| ( true == decision.skip ) };
if( false == skip ) {
simulation::Region->insert(
scene::shape_node().import(
Input, nodedata ),
Scratchpad,
true );
if( true == skip ) {
skip_until( Input, "endtri" );
}
else {
skip_until( Input, "endtri" );
auto shape { [&]() {
loadprofile::scoped_timer const importtime { loadprofile::stage::shape_import };
return scene::shape_node().import( Input, nodedata ); }() };
loadprofile::add_vertices( shape.data().vertices.size() );
if( true == decision.bake ) {
// the bake sees the node exactly where it is drawn. placing is done on a copy:
// the region applies the same placement to the shape itself when inserting it
auto const &data { shape.data() };
auto const material {
data.material != null_handle
? GfxRenderer->Material( data.material )->GetName()
: "none" };
// the old switch trackbeds are dropped from the scenery when generated ones
// replace them, so they are no part of the ground either
if( false == ( Global.CreateSwitchTrackbeds && scene::is_switch_trackbed( material ) ) ) {
auto placed { data.vertices };
scene::place_in_world( placed, Scratchpad );
terrainbake::add( decision.key, placed, material );
}
}
if( decision.plan.what == terrainbake::verdict::filter ) {
auto const &drawn { decision.plan.drawn };
shape.keep_triangles(
[ &drawn ]( std::size_t const Triangle ) {
return ( Triangle >= drawn.size() ) || drawn[ Triangle ]; } );
}
loadprofile::scoped_timer const inserttime { loadprofile::stage::shape_insert };
simulation::Region->insert(
std::move( shape ),
Scratchpad,
true );
}
}
else if( nodedata.type == "lines"
@@ -827,7 +948,14 @@ state_serializer::deserialize_terrain(cParser &Input, scene::scratch_data &Scrat
std::string line;
Input.getTokens(1);
Input >> line;
if (Global.file_binary_terrain && line.ends_with(".sbt"))
if (line.ends_with(".ehf"))
{
// a cooked heightfield. the renderer opens it and the scenery carries no terrain
// geometry at all, which is the point: the text it replaces is most of the file
Global.terrain_heightfields.push_back(line);
WriteLog("Terrain heightfield: " + line);
}
else if (Global.file_binary_terrain && line.ends_with(".sbt"))
{
Scratchpad.binary.terrain = Region->is_scene(line);
Global.file_binary_terrain_state = true;

5
simulation/simulationstateserializer.cppm
View File

@@ -28,6 +28,9 @@ import eu07.glm;
import eu07.audio.sound;
import eu07.utilities.parser;
import eu07.simcore;
import eu07.scene.prefab;
import eu07.simulation.logiccook;
import eu07.simulation.terrainbake;
export {
@@ -86,6 +89,8 @@ private:
void deserialize_endgroup( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_light( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_node( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_prefab( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_origin( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_endorigin( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_scale( cParser &Input, scene::scratch_data &Scratchpad );

463
simulation/terrainbake.cpp Normal file
View File

@@ -0,0 +1,463 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <filesystem>
#include <fstream>
#include <string>
#include <string_view>
#include <system_error>
#include <unordered_map>
#include <vector>
#include "utilities/Globals_macros.h"
#include "scene/terraincooker.h"
#include "scene/heightfieldreader.h"
module eu07.simulation.terrainbake;
import eu07.utilities.logs;
import eu07.utilities.globals;
import eu07.utilities.utilities;
import eu07.simulation.loadprofile;
namespace simulation::terrainbake {
namespace {
// triangles are buffered until there are enough vertices to say what the grid step is, and
// only then rasterised. the step has to be known before the first sample is placed, and it
// is a property of the whole terrain rather than of the first triangle that turns up
constexpr std::size_t samplegoal { 200000 };
// the node table written beside a baked heightfield
constexpr char tablemagic[ 8 ] { 'E', 'U', '0', '7', 'H', 'F', 'N', 'D' };
// 2: nodes keyed by file, line, include parameters and placement rather than by their text
constexpr std::uint32_t tableversion { 2 };
constexpr std::uint32_t allshown { 0xffffffffu };
struct table_header {
char magic[ 8 ];
std::uint32_t version;
std::uint32_t selection;
std::uint64_t count;
std::uint64_t maskbytes;
};
struct table_record {
std::uint64_t key;
std::uint32_t triangles;
// offset into the mask blob, one bit per triangle set where it is still drawn; allshown
// when the heightfield shows every triangle and there is no mask
std::uint32_t mask;
};
struct file_stamp {
std::uint64_t size { 0 };
std::int64_t time { 0 };
};
struct buffered {
terrain::vertex v[ 3 ];
std::uint32_t node; // the node it came from, which holds its material
std::uint32_t id;
};
// one node as baked: where its triangles sit in the cooker's numbering
struct baked_node {
std::uint64_t key;
std::uint32_t first;
std::uint32_t count;
// kept once per node rather than once per buffered triangle
std::string material;
};
struct {
// baking
bool collecting { false };
bool configured { false };
terrain::cooker cooker;
std::vector<buffered> buffer;
std::vector<baked_node> nodes;
std::uint32_t triangles { 0 };
std::chrono::steady_clock::time_point started;
// planning
std::unordered_map<std::uint64_t, table_record> table;
std::vector<std::uint8_t> masks;
std::size_t skipped { 0 }, filtered { 0 }, drawn { 0 };
// size and modification time of every file a key was taken in, looked up once per load
std::unordered_map<std::string, file_stamp> files;
} state;
std::string
heightfield_path( std::string const &Sceneryfile ) {
return scenery_sidecar( Sceneryfile, ".ehf" );
}
std::string
table_path( std::string const &Heightfield ) {
return Heightfield + ".nodes";
}
bool
load_table( std::string const &Path ) {
std::ifstream input( Path, std::ios::binary );
if( false == input.good() ) { return false; }
table_header header {};
input.read( reinterpret_cast<char *>( &header ), sizeof( header ) );
if( ( false == static_cast<bool>( input ) )
|| ( 0 != std::memcmp( header.magic, tablemagic, sizeof( tablemagic ) ) )
|| ( header.version != tableversion )
|| ( header.selection != heightfield::selection_rules ) ) {
return false;
}
std::vector<table_record> records( header.count );
input.read( reinterpret_cast<char *>( records.data() ), static_cast<std::streamsize>( header.count * sizeof( table_record ) ) );
state.masks.resize( header.maskbytes );
input.read( reinterpret_cast<char *>( state.masks.data() ), static_cast<std::streamsize>( header.maskbytes ) );
if( false == static_cast<bool>( input ) ) { return false; }
state.table.reserve( records.size() );
for( auto const &record : records ) {
state.table.emplace( record.key, record );
}
return true;
}
void
write_table( std::string const &Path ) {
// a key met more than once is the same line of the same file, included with the same
// parameters at the same place - the same surface: a triangle counts as shown if any of
// the copies is
std::unordered_map<std::uint64_t, std::vector<bool>> shown;
for( auto const &node : state.nodes ) {
auto &entry { shown[ node.key ] };
entry.resize( node.count, false );
for( std::uint32_t index = 0; index < node.count; ++index ) {
if( true == state.cooker.represented( node.first + index ) ) { entry[ index ] = true; }
}
}
std::vector<table_record> records;
std::vector<std::uint8_t> masks;
for( auto const &node : state.nodes ) {
auto const lookup { shown.find( node.key ) };
if( lookup == shown.end() ) { continue; } // written already
auto const &entry { lookup->second };
std::uint32_t count { 0 };
for( auto const flag : entry ) { count += ( flag ? 1 : 0 ); }
if( count > 0 ) {
table_record record { node.key, node.count, allshown };
if( count < node.count ) {
record.mask = static_cast<std::uint32_t>( masks.size() );
auto const bytes { ( node.count + 7u ) / 8u };
masks.resize( masks.size() + bytes, 0u );
for( std::uint32_t index = 0; index < node.count; ++index ) {
if( false == entry[ index ] ) {
masks[ record.mask + index / 8u ] |= static_cast<std::uint8_t>( 1u << ( index % 8u ) );
}
}
}
records.emplace_back( record );
}
shown.erase( lookup );
}
table_header header {};
std::memcpy( header.magic, tablemagic, sizeof( tablemagic ) );
header.version = tableversion;
header.selection = heightfield::selection_rules;
header.count = records.size();
header.maskbytes = masks.size();
std::ofstream output( Path, std::ios::binary | std::ios::trunc );
output.write( reinterpret_cast<char const *>( &header ), sizeof( header ) );
output.write( reinterpret_cast<char const *>( records.data() ), static_cast<std::streamsize>( records.size() * sizeof( table_record ) ) );
output.write( reinterpret_cast<char const *>( masks.data() ), static_cast<std::streamsize>( masks.size() ) );
std::size_t full { 0 };
for( auto const &record : records ) { full += ( record.mask == allshown ? 1 : 0 ); }
WriteLog(
"Terrain bake: " + std::to_string( records.size() ) + " of " + std::to_string( state.nodes.size() )
+ " triangle nodes are shown by the heightfield, " + std::to_string( full ) + " of them entirely" );
}
void
rasterize( buffered const &Triangle ) {
state.cooker.rasterize(
Triangle.v[ 0 ], Triangle.v[ 1 ], Triangle.v[ 2 ], state.nodes[ Triangle.node ].material, Triangle.id );
}
// works the grid step out of what has been buffered, then puts the buffer through
void
configure_and_drain() {
std::vector<terrain::vertex> sample;
sample.reserve( state.buffer.size() * 3 );
for( auto const &triangle : state.buffer ) {
sample.emplace_back( triangle.v[ 0 ] );
sample.emplace_back( triangle.v[ 1 ] );
sample.emplace_back( triangle.v[ 2 ] );
}
auto const step { terrain::detect_step( sample ) };
WriteLog( "Terrain bake: grid step " + std::to_string( step ) + " m, from " + std::to_string( sample.size() ) + " sampled vertices" );
if( false == loadprofile::enabled() ) {
// the per-material survival table is what a bad bake is diagnosed from, so it is
// printed when the load is being measured anyway
state.cooker.quiet();
}
state.cooker.configure( step, true );
state.configured = true;
for( auto const &triangle : state.buffer ) {
rasterize( triangle );
}
state.buffer.clear();
state.buffer.shrink_to_fit();
}
} // anonymous namespace
namespace {
std::uint64_t
node_key(
std::string const &File, std::size_t const Line, std::vector<std::string> const &Parameters,
std::string_view const Type, glm::dvec3 const &Offset, glm::vec3 const &Rotation ) {
// fnv-1a. stable across runs and builds, which a std::hash is not required to be
std::uint64_t hash { 0xcbf29ce484222325ull };
auto const mix {
[ &hash ]( void const *Data, std::size_t const Size ) {
auto const *bytes { static_cast<unsigned char const *>( Data ) };
for( std::size_t index = 0; index < Size; ++index ) {
hash ^= bytes[ index ];
hash *= 0x100000001b3ull;
} } };
auto const text {
[ &mix ]( std::string_view const Text ) {
mix( Text.data(), Text.size() );
mix( "\n", 1 ); } };
// what the file is now: an edit anywhere in it changes every key it holds
auto lookup { state.files.find( File ) };
if( lookup == state.files.end() ) {
std::error_code error;
file_stamp stamp {};
auto const size { std::filesystem::file_size( File, error ) };
stamp.size = ( error ? 0 : static_cast<std::uint64_t>( size ) );
auto const time { std::filesystem::last_write_time( File, error ) };
stamp.time = ( error ? 0 : static_cast<std::int64_t>( time.time_since_epoch().count() ) );
lookup = state.files.emplace( File, stamp ).first;
}
text( File );
mix( &lookup->second.size, sizeof( lookup->second.size ) );
mix( &lookup->second.time, sizeof( lookup->second.time ) );
auto const line { static_cast<std::uint64_t>( Line ) };
mix( &line, sizeof( line ) );
for( auto const &parameter : Parameters ) { text( parameter ); }
text( Type );
mix( &Offset.x, sizeof( Offset.x ) ); mix( &Offset.y, sizeof( Offset.y ) ); mix( &Offset.z, sizeof( Offset.z ) );
mix( &Rotation.x, sizeof( Rotation.x ) ); mix( &Rotation.y, sizeof( Rotation.y ) ); mix( &Rotation.z, sizeof( Rotation.z ) );
return hash;
}
} // anonymous namespace
void
begin( std::string const &Sceneryfile ) {
state = {};
if( false == Global.terrain_heightfields.empty() ) {
// the scenery names its own, which always wins over anything baked beside it
return;
}
if( false == Global.bake_terrain ) {
// neither bake nor pick one up: the triangles as written
return;
}
auto const path { heightfield_path( Sceneryfile ) };
std::error_code error;
if( true == FileExists( path ) ) {
// a baked file is a cache: one this build cannot read, one cooked under older rules
// about what counts as terrain, or one whose node table is missing, is thrown away
// and baked again rather than used
heightfield::reader probe;
if( ( true == probe.open( path ) )
&& ( probe.selection() == heightfield::selection_rules )
&& ( true == load_table( table_path( path ) ) ) ) {
Global.terrain_heightfields.push_back( path );
WriteLog( "Terrain bake: using \"" + path + "\", baked earlier, " + std::to_string( state.table.size() ) + " triangle nodes replaced by it" );
return;
}
WriteLog( "Terrain bake: \"" + path + "\" was baked under different rules, baking it again" );
probe.close();
state.table.clear();
state.masks.clear();
std::filesystem::remove( path, error );
std::filesystem::remove( table_path( path ), error );
}
WriteLog( "Terrain bake: no heightfield for this scenery, baking one. This load will be slow; the next will not" );
state.collecting = true;
state.started = std::chrono::steady_clock::now();
}
namespace {
bool
planning() {
return false == state.table.empty();
}
node_plan
plan( std::uint64_t const Key ) {
node_plan result;
auto const lookup { state.table.find( Key ) };
if( lookup == state.table.end() ) {
++state.drawn;
return result;
}
auto const &record { lookup->second };
if( record.mask == allshown ) {
++state.skipped;
result.what = verdict::skip;
return result;
}
++state.filtered;
result.what = verdict::filter;
result.drawn.resize( record.triangles, false );
for( std::uint32_t index = 0; index < record.triangles; ++index ) {
auto const byte { record.mask + index / 8u };
result.drawn[ index ] = ( byte < state.masks.size() ) && ( ( state.masks[ byte ] >> ( index % 8u ) ) & 1u ) != 0;
}
return result;
}
// collecting for a bake, and not overruled by a heightfield the scenery named part way in
bool
baking() {
return state.collecting && Global.terrain_heightfields.empty();
}
} // anonymous namespace
node_decision
examine(
std::string const &File, std::size_t const Line, std::vector<std::string> const &Parameters,
std::string_view const Type, glm::dvec3 const &Offset, glm::vec3 const &Rotation, bool const Worldspace ) {
node_decision decision;
if( ( true == baking() ) || ( true == planning() ) ) {
decision.key = node_key( File, Line, Parameters, Type, Offset, Rotation );
decision.bake = baking();
if( false == decision.bake ) {
decision.plan = plan( decision.key );
decision.skip = ( decision.plan.what == verdict::skip );
}
return decision;
}
// a heightfield the scenery names itself, cooked by hand, replaces the world space
// triangles outright, as it always has
decision.skip = ( true == Worldspace ) && ( false == Global.terrain_heightfields.empty() );
return decision;
}
void
add( std::uint64_t const Key, std::vector<world_vertex> const &Vertices, std::string_view const Material ) {
if( false == state.collecting ) { return; }
auto const count { static_cast<std::uint32_t>( Vertices.size() / 3 ) };
auto const node { static_cast<std::uint32_t>( state.nodes.size() ) };
state.nodes.push_back( { Key, state.triangles, count, std::string { Material } } );
for( std::uint32_t triangle = 0; triangle < count; ++triangle ) {
auto const vertex = [ & ]( std::size_t const Index ) {
auto const &source { Vertices[ triangle * 3u + Index ] };
return terrain::vertex{ source.position.x, source.position.y, source.position.z, source.texture.x, source.texture.y }; };
buffered entry { { vertex( 0 ), vertex( 1 ), vertex( 2 ) }, node, state.triangles + triangle };
if( true == state.configured ) {
rasterize( entry );
continue;
}
state.buffer.emplace_back( entry );
}
state.triangles += count;
if( ( false == state.configured ) && ( state.buffer.size() * 3 >= samplegoal ) ) {
configure_and_drain();
}
}
void
finish( std::string const &Sceneryfile ) {
if( true == planning() ) {
WriteLog(
"Terrain bake: " + std::to_string( state.skipped ) + " triangle nodes left to the heightfield, "
+ std::to_string( state.filtered ) + " drawn in part, " + std::to_string( state.drawn ) + " drawn as written" );
}
if( false == state.collecting ) { return; }
if( false == Global.terrain_heightfields.empty() ) {
// the scenery named its own heightfield part way through, and that one wins
WriteLog( "Terrain bake: the scenery names its own heightfield, nothing baked" );
state = {};
return;
}
if( false == state.configured ) {
// a scenery with less terrain than the sample goal never triggered the drain
if( true == state.buffer.empty() ) {
WriteLog( "Terrain bake: the scenery holds no triangles, nothing baked" );
state.collecting = false;
return;
}
configure_and_drain();
}
auto const path { heightfield_path( Sceneryfile ) };
state.cooker.finish( path, {} );
write_table( table_path( path ) );
std::error_code error;
auto const bytes { std::filesystem::file_size( path, error ) };
auto const seconds {
std::chrono::duration<double>( std::chrono::steady_clock::now() - state.started ).count() };
WriteLog(
"Terrain bake: wrote \"" + path + "\" from " + std::to_string( state.triangles )
+ " triangles in " + std::to_string( seconds ) + " s"
+ ( error ? std::string() : ", " + std::to_string( bytes / 1048576 ) + " MB" ) );
// the triangles of this load were all drawn; the table applies from the next one
state.collecting = false;
state.buffer.clear();
state.nodes.clear();
}
} // namespace simulation::terrainbake

85
simulation/terrainbake.cppm Normal file
View File

@@ -0,0 +1,85 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <cstddef>
#include <cstdint>
#include <string>
#include <string_view>
#include <vector>
export module eu07.simulation.terrainbake;
import eu07.glm;
import eu07.model.vertex;
export {
// Baking a scenery's heightfield on its first run, and deciding on every later run which
// triangles it replaces.
//
// A cooked heightfield is what makes a large scenery affordable - l204 loads in 25 seconds
// against 219, in 1.9 GB against 9.1 - but until now it had to be produced by hand with the
// terraincook tool and named in the scenery text. This bakes it the first time the scenery is
// opened and picks it up by itself afterwards.
//
// Nothing here guesses what terrain is. Every triangle the scenery draws is laid into the
// heightfield where it is drawn, and afterwards each one is asked whether the heightfield still
// shows it. A tree, a wall or a gantry stands on its edge and never reaches a sample; a bridge
// deck loses its samples to the ground beneath; those are drawn as they always were. Only
// triangles the heightfield really shows stop being drawn on their own, so nothing can vanish
// from the scenery - the worst a bad bake can do is draw something twice.
//
// A triangle node is recognised by the file and line it is written at - with that file's size
// and modification time - the parameters its include was given, and the origin and rotation in
// force, not by its position in the load. An edited file changes the keys of everything in it,
// and those nodes are then simply drawn again instead of the wrong ones being dropped. Hashing
// the node's text instead would cost a second parse of every triangle node in the scenery.
//
// The rasterising is scene/terraincooker.h, the same code the tool uses.
namespace simulation::terrainbake {
// what to do with one triangle node
enum class verdict {
draw, // the heightfield does not show it: import and draw it as written
skip, // the heightfield shows all of it: do not even import it
filter // the heightfield shows some of it: import it and draw only the rest
};
struct node_plan {
verdict what { verdict::draw };
// for filter: one entry per triangle in import order, true where it is still drawn
std::vector<bool> drawn;
};
// called before the scenery is parsed. picks up a heightfield baked earlier together with its
// node table, or arms the bake when there is none
void begin( std::string const &Sceneryfile );
// what the load is to do with one triangle node, decided in one place: whether it is read at
// all, whether it goes into a bake, and which of its triangles are still drawn
struct node_decision {
bool skip { false }; // step over it without importing
bool bake { false }; // hand it to add() once imported
std::uint64_t key { 0 };
node_plan plan;
};
// Worldspace says whether the node stands in world coordinates, which is all that decides it
// when the scenery names a hand-cooked heightfield of its own
node_decision examine(
std::string const &File, std::size_t const Line, std::vector<std::string> const &Parameters,
std::string_view const Type, glm::dvec3 const &Offset, glm::vec3 const &Rotation, bool const Worldspace );
// one triangle node, in world space, as it is about to be drawn
void add( std::uint64_t const Key, std::vector<world_vertex> const &Vertices, std::string_view const Material );
// called once the scenery has been parsed. writes the heightfield and its node table
void finish( std::string const &Sceneryfile );
} // namespace simulation::terrainbake
}