eu07/maszyna
16
0
Fork 0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-07-18 01:59:19 +02:00
Code Issues 121 Wiki Activity

Stream deferred visual nodes in camera-distance order (nearest first)

Browse Source 
The progressive load previously streamed the deferred visual nodes (3d model
instances + terrain shapes/lines) in file order, so distant scenery could load
before the player's surroundings. This builds them nearest-camera first instead.

The visual pass now runs in two steps. Enumeration replays the twin and captures
each visual node verbatim (its resolved tokens as text -- numbers round-trip
losslessly through cParser) together with the transform/group context it was read
under and, for models, its transformed world position. Once the replay is
exhausted the records are sorted by squared distance to the camera (terrain shapes
first so the ground appears before the props on it), then built a budgeted slice
per frame through the normal node path with the captured transform and group
restored -- so placement, grouping and the per-cell instance buckets come out
identical to an in-order load.

Two supporting fixes make out-of-order/late insertion correct:
- a cell/section whose geometry was already baked (the renderer finalised it
  before a deferred node arrived) now appends the new shape/lines straight into
  its live geometry bank instead of merging into vertex-freed geometry, which
  would silently drop it; create_geometry() remembers the bank for every cell.
- events that bind to visual model instances (lights/animation/texture/visible)
  are deferred from InitEvents() to a new InitInstanceEvents() run after the
  visual nodes are built, so their target models exist when they initialise.

Verified on td.scn: playable ~2s, 540 deferred nodes enumerated and built
nearest-first ~0.5s later, no duplicate instances.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
maj00r
2026-06-23 17:11:39 +02:00
parent c10fadbfc9
commit 1160bfecac
8 changed files with 326 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

222
simulation/simulationstateserializer.cpp
View File

@@ -29,6 +29,10 @@ http://mozilla.org/MPL/2.0/.
#include "rendering/renderer.h"
#include "utilities/Logs.h"
#include <algorithm>
#include <cctype>
#include <cstdlib>
namespace simulation {
std::shared_ptr<deserializer_state>
@@ -50,9 +54,15 @@ state_serializer::deserialize_begin( std::string const &Scenariofile ) {
state->scenariofile = Scenariofile;
state->scratchpad.name = Scenariofile;
// first pass loads infrastructure (tracks/traction/events/memcells/sounds + directives);
// visual nodes are skipped by the reader and loaded in a second pass. for a text/compile
// load (no twin) this is a no-op and everything loads in a single pass.
state->input.setReplayPass( scene::scenery_load_pass::infrastructure );
// visual nodes are skipped by the reader and loaded in a second pass. this two-pass split
// is only valid when the top-level file is itself a replayable twin, because the visual
// pass is started via restartReplay() which needs a top-level reader. for a text/compile
// load (no top twin) we MUST stay in a single 'all' pass and load everything at once;
// otherwise visual nodes served by included twins (.incb) would be skipped in the infra
// pass and never rebuilt (restartReplay returns false), and all those models go missing.
if( true == state->input.isReplaying() ) {
state->input.setReplayPass( scene::scenery_load_pass::infrastructure );
}
scene::Groups.create();
if( false == state->input.ok() )
@@ -108,6 +118,12 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
cParser &Input = state->input;
scene::scratch_data &Scratchpad = state->scratchpad;
// camera-ordered build phase: the visual replay has been enumerated into records and
// sorted; build them nearest-camera first, a budgeted slice per call.
if( state->enumdone ) {
return build_visual_records( state );
}
// stateful directives that build objects/lists; on the visual (second) pass they are
// skipped wholesale so their side effects (trainsets, events, cameras, ...) don't
// duplicate. transform/group directives (origin/rotate/scale/group) and idempotent
@@ -137,6 +153,18 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
token = Input.getToken<std::string>();
continue;
}
if( ( true == state->enumerate ) && ( token == "node" ) ) {
// capture the visual node (text + transform/group snapshot) for later,
// camera-ordered building, instead of building it in file order now
enumerate_visual_node( *state );
auto timenow = std::chrono::steady_clock::now();
if( std::chrono::duration_cast<std::chrono::milliseconds>( timenow - timelast ).count() >= 8 ) {
Application.set_progress( Input.getProgress(), Input.getFullProgress() );
return true;
}
token = Input.getToken<std::string>();
continue;
}
}
auto lookup = state->functionmap.find( token );
@@ -159,6 +187,28 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
token = Input.getToken<std::string>();
}
if( ( true == state->visualphase ) && ( true == state->enumerate ) ) {
// visual replay exhausted: order the captured nodes by distance to the camera so
// the player's surroundings stream in first (terrain shapes ahead of models, so the
// ground appears before the props on it), then switch to the build phase.
auto const eye { Global.pCamera.Pos };
for( auto &rec : state->records ) {
auto const d { rec.worldpos - eye };
rec.sortkey =
( rec.isshape ?
-1.0 : // shapes (terrain) first, regardless of distance
( d.x * d.x + d.y * d.y + d.z * d.z ) );
}
std::stable_sort(
std::begin( state->records ), std::end( state->records ),
[]( visual_record const &L, visual_record const &R ) {
return L.sortkey < R.sortkey; } );
WriteLog( "Progressive visual load: " + std::to_string( state->records.size() ) + " deferred nodes enumerated, building nearest-camera first" );
state->enumerate = false;
state->enumdone = true;
return true; // proceed to the build phase
}
if( false == Scratchpad.initialized ) {
// manually perform scenario initialization
deserialize_firstinit( Input, Scratchpad );
@@ -172,6 +222,7 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
if( ( false == state->visualphase )
&& ( true == Input.restartReplay( scene::scenery_load_pass::visual ) ) ) {
state->visualphase = true;
state->enumerate = true; // capture deferred visual nodes, build them camera-ordered
// rebuild the transform state from scratch for the visual pass: the directives
// (origin/rotate/scale) are replayed in order, so resetting here reproduces the
// single-pass placement exactly. without this, an unbalanced origin left on the
@@ -188,6 +239,11 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
scene::Groups.update_map();
Region->create_map_geometry();
// all nodes (including visual model instances) are now loaded, so initialise the
// events that bind to model instances; in a single-pass (text/compile) load nothing
// was deferred, but InitEvents() still skipped them, so do it here too
simulation::Events.InitInstanceEvents();
// loading finished: flush the top-level scenario's binary twin now rather than
// waiting for the parser to be destroyed (the loader keeps the state around)
Input.flushBinaryTwin();
@@ -199,6 +255,166 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
return false;
}
// captures one visual node (the "node" token already consumed) into a record for later,
// camera-ordered building: stores the node's verbatim tokens plus the transform/group
// context it was read under. numbers come through cParser losslessly, so the rebuild is
// exact. only model/triangles/lines reach the visual pass.
void
state_serializer::enumerate_visual_node( deserializer_state &State ) {
cParser &Input = State.input;
scene::scratch_data &Scratchpad = State.scratchpad;
// node header, original case preserved (model file paths are case-sensitive at replay)
auto const smax { Input.getToken<std::string>( false ) };
auto const smin { Input.getToken<std::string>( false ) };
auto const sname { Input.getToken<std::string>( false ) };
auto const stype { Input.getToken<std::string>( false ) };
auto lower = []( std::string s ) {
for( auto &c : s ) { c = static_cast<char>( std::tolower( static_cast<unsigned char>( c ) ) ); }
return s; };
auto const typelc { lower( stype ) };
std::string endtok;
bool ismodel { false };
bool istriangles { false };
if( typelc == "model" ) {
endtok = "endmodel"; ismodel = true;
}
else if( ( typelc == "triangles" ) || ( typelc == "triangle_strip" ) || ( typelc == "triangle_fan" ) ) {
endtok = "endtri"; istriangles = true;
}
else if( ( typelc == "lines" ) || ( typelc == "line_strip" ) || ( typelc == "line_loop" ) ) {
endtok = "endline";
}
// mirror deserialize_node's switch-trackbed skip here, while the source file name is
// still known (Input.Name() is the live include during replay; the per-record rebuild
// parser has no name). without this, fixed trackbed geometry from scenery/zwr*.inc that
// the in-order load drops would be rebuilt and duplicate the procedural trackbeds.
bool skipnode { false };
if( ( true == istriangles ) && ( true == Global.CreateSwitchTrackbeds ) ) {
auto const name { Input.Name() };
skipnode =
( name.size() >= 15 )
&& name.starts_with( "scenery/zwr" )
&& name.ends_with( ".inc" );
}
// appends one token to the rebuilt node text, re-quoting it if it carries a token break
// (a quoted source token such as a name/path with spaces arrives here unquoted; without
// re-quoting it the rebuild parser would split it into several tokens)
auto appendtok = []( std::string &Text, std::string const &Token ) {
bool const needsquote {
Token.empty()
|| ( Token.find_first_of( "\n\r\t ;" ) != std::string::npos ) };
Text.push_back( ' ' );
if( true == needsquote ) { Text.push_back( '\"' ); }
Text.append( Token );
if( true == needsquote ) { Text.push_back( '\"' ); } };
std::string text;
text.reserve( 96 );
text.append( "node" );
appendtok( text, smax );
appendtok( text, smin );
appendtok( text, sname );
appendtok( text, stype );
glm::dvec3 localpos { 0.0 };
int posread { 0 };
// read the rest of the node verbatim until its terminator (or the first end* token if the
// type is unrecognised), capturing a model's local position (the first 3 numbers)
for( ;; ) {
auto const tok { Input.getToken<std::string>( false ) };
if( tok.empty() ) { break; } // safety: input ran out
appendtok( text, tok );
if( ( true == ismodel ) && ( posread < 3 ) ) {
localpos[ posread ] = std::atof( tok.c_str() );
++posread;
}
auto const toklc { lower( tok ) };
if( false == endtok.empty() ) {
if( toklc == endtok ) { break; }
}
else if( ( toklc.size() >= 3 ) && ( 0 == toklc.compare( 0, 3, "end" ) ) ) {
break;
}
}
if( true == skipnode ) {
// node consumed from the stream above; intentionally not recorded (matches the
// in-order load, which skips this geometry in favour of procedural trackbeds)
return;
}
visual_record rec;
rec.text = std::move( text );
rec.group = scene::Groups.handle();
rec.has_offset = ( false == Scratchpad.location.offset.empty() );
if( true == rec.has_offset ) { rec.offset = Scratchpad.location.offset.top(); }
rec.has_scale = ( false == Scratchpad.location.scale.empty() );
if( true == rec.has_scale ) { rec.scale = Scratchpad.location.scale.top(); }
rec.rotation = Scratchpad.location.rotation;
rec.isshape = ( false == ismodel );
if( true == ismodel ) {
rec.worldpos = transform( localpos, Scratchpad );
}
State.records.emplace_back( std::move( rec ) );
}
// builds the enumerated visual records in (already-sorted) nearest-camera-first order, a
// budgeted slice per call. each node is rebuilt through the normal node path with its
// captured transform/group restored, so placement, grouping and the per-cell instance
// buckets come out identical to an in-order load.
bool
state_serializer::build_visual_records( std::shared_ptr<deserializer_state> state ) {
scene::scratch_data &Scratchpad = state->scratchpad;
auto timelast { std::chrono::steady_clock::now() };
while( state->buildcursor < state->records.size() ) {
auto &rec = state->records[ state->buildcursor ];
// restore the transform context captured for this node
Scratchpad.location.offset = {};
if( true == rec.has_offset ) { Scratchpad.location.offset.push( rec.offset ); }
Scratchpad.location.scale = {};
if( true == rec.has_scale ) { Scratchpad.location.scale.push( rec.scale ); }
Scratchpad.location.rotation = rec.rotation;
// re-attach to the node's original group and build via the normal node path (this
// routes instanceable models into their per-cell instance bucket automatically)
scene::Groups.push_active( rec.group );
{
cParser nodeparser( rec.text, cParser::buffer_TEXT );
nodeparser.getToken<std::string>(); // consume "node"
deserialize_node( nodeparser, Scratchpad );
}
scene::Groups.pop_active();
std::string().swap( rec.text ); // release the captured text as we go
++state->buildcursor;
auto timenow = std::chrono::steady_clock::now();
if( std::chrono::duration_cast<std::chrono::milliseconds>( timenow - timelast ).count() >= 8 ) {
return true; // yield; resume next frame
}
}
// every deferred visual node is now built: finalise the scenario
scene::Groups.close();
scene::Groups.update_map();
Region->create_map_geometry();
// the visual model instances now exist, so initialise the events that bind to them
// (lights/animation/texture/visible) -- these were deferred during the infrastructure
// pass when their target models hadn't been streamed in yet
simulation::Events.InitInstanceEvents();
state->input.flushBinaryTwin();
scene::scenerybinary_wait_all();
std::vector<visual_record>().swap( state->records );
state->done = true;
return false;
}
void
state_serializer::deserialize_isolated( cParser &Input, scene::scratch_data &Scratchpad ) {
// first parameter specifies name of parent piece...