eu07/maszyna
16
0
Fork 0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-07-18 00:49:19 +02:00
Code Issues 121 Wiki Activity

Stream deferred visual nodes in camera-distance order (nearest first)

Browse Source 
The progressive load previously streamed the deferred visual nodes (3d model
instances + terrain shapes/lines) in file order, so distant scenery could load
before the player's surroundings. This builds them nearest-camera first instead.

The visual pass now runs in two steps. Enumeration replays the twin and captures
each visual node verbatim (its resolved tokens as text -- numbers round-trip
losslessly through cParser) together with the transform/group context it was read
under and, for models, its transformed world position. Once the replay is
exhausted the records are sorted by squared distance to the camera (terrain shapes
first so the ground appears before the props on it), then built a budgeted slice
per frame through the normal node path with the captured transform and group
restored -- so placement, grouping and the per-cell instance buckets come out
identical to an in-order load.

Two supporting fixes make out-of-order/late insertion correct:
- a cell/section whose geometry was already baked (the renderer finalised it
  before a deferred node arrived) now appends the new shape/lines straight into
  its live geometry bank instead of merging into vertex-freed geometry, which
  would silently drop it; create_geometry() remembers the bank for every cell.
- events that bind to visual model instances (lights/animation/texture/visible)
  are deferred from InitEvents() to a new InitInstanceEvents() run after the
  visual nodes are built, so their target models exist when they initialise.

Verified on td.scn: playable ~2s, 540 deferred nodes enumerated and built
nearest-first ~0.5s later, no duplicate instances.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
maj00r
2026-06-23 17:11:39 +02:00
parent c10fadbfc9
commit 1160bfecac
8 changed files with 326 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
scene/scene.cpp
View File

@@ -275,6 +275,16 @@ basic_cell::insert( shape_node Shape ) {
shapedata.translucent ?
m_shapestranslucent :
m_shapesopaque );
// if this cell's geometry was already baked (deferred visual streaming inserting into a
// section the renderer already finalised), don't try to merge into an existing shape
// whose CPU-side vertices were freed at bake time -- add the shape standalone and upload
// it straight into the live bank, otherwise it would never become visible.
if( m_geometrybank != null_handle ) {
Shape.origin( m_area.center );
shapes.emplace_back( Shape );
shapes.back().create_geometry( m_geometrybank );
return;
}
for( auto &targetshape : shapes ) {
// try to merge shapes with matching view ranges...
auto const &targetshapedata { targetshape.data() };
@@ -302,6 +312,14 @@ basic_cell::insert( lines_node Lines ) {
m_active = true;
auto const &linesdata { Lines.data() };
// see the matching note in insert( shape_node ): once the cell is baked, append the new
// lines straight into the live bank rather than merging into vertex-freed geometry.
if( m_geometrybank != null_handle ) {
Lines.origin( m_area.center );
m_lines.emplace_back( Lines );
m_lines.back().create_geometry( m_geometrybank );
return;
}
for( auto &targetlines : m_lines ) {
// try to merge shapes with matching view ranges...
auto const &targetlinesdata { targetlines.data() };
@@ -632,6 +650,10 @@ basic_cell::center( glm::dvec3 Center ) {
void
basic_cell::create_geometry( gfx::geometrybank_handle const &Bank ) {
// remember the bank for *all* cells (even ones empty at bake time): a deferred visual
// node may activate this cell later, and insert() needs the live bank to upload into.
m_geometrybank = Bank;
if( false == m_active ) { return; } // nothing to do here
for( auto &shape : m_shapesopaque ) { shape.create_geometry( Bank ); }
@@ -850,6 +872,14 @@ basic_section::insert( shape_node Shape ) {
}
else {
// large, opaque shapes are placed on section level
// if the section was already baked (deferred visual streaming), append straight into
// the live bank instead of merging into vertex-freed geometry -- see basic_cell::insert.
if( true == m_geometrycreated ) {
Shape.origin( m_area.center );
m_shapes.emplace_back( Shape );
m_shapes.back().create_geometry( m_geometrybank );
return;
}
for( auto &shape : m_shapes ) {
// check first if the shape can't be merged with one of the shapes already present in the section
if( true == shape.merge( Shape ) ) {

5
scene/scene.h
View File

@@ -234,6 +234,11 @@ public:
} m_directories;
// animation of owned items (legacy code, clean up along with track refactoring)
bool m_geometrycreated { false };
// bank this cell's geometry was baked into; remembered so shapes/lines inserted
// *after* the section was first rendered (deferred visual streaming) can be appended
// straight into the live bank instead of being silently dropped. null until the owning
// section bakes its geometry, which doubles as the "already baked" signal.
gfx::geometrybank_handle m_geometrybank {};
unsigned int m_framestamp { 0 }; // id of last rendered gfx frame
TTrack *tTrackAnim = nullptr; // obiekty do przeliczenia animacji
command_relay m_relay;

7
scene/scenenodegroups.h
View File

@@ -40,6 +40,13 @@ public:
// returns current active group, or null_handle if group stack is empty
group_handle
handle() const;
// makes Group the active group for subsequent inserts, without creating a new group.
// used by deferred (camera-ordered) visual loading to re-attach a node to the group it
// was originally read under during the enumeration pass. must be paired with pop_active().
void
push_active( scene::group_handle const Group ) { m_activegroup.push( Group ); }
void
pop_active() { if( false == m_activegroup.empty() ) { m_activegroup.pop(); } }
// places provided node in specified group
void
insert( scene::group_handle const Group, scene::basic_node *Node );

222
simulation/simulationstateserializer.cpp
View File

@@ -29,6 +29,10 @@ http://mozilla.org/MPL/2.0/.
#include "rendering/renderer.h"
#include "utilities/Logs.h"
#include <algorithm>
#include <cctype>
#include <cstdlib>
namespace simulation {
std::shared_ptr<deserializer_state>
@@ -50,9 +54,15 @@ state_serializer::deserialize_begin( std::string const &Scenariofile ) {
state->scenariofile = Scenariofile;
state->scratchpad.name = Scenariofile;
// first pass loads infrastructure (tracks/traction/events/memcells/sounds + directives);
// visual nodes are skipped by the reader and loaded in a second pass. for a text/compile
// load (no twin) this is a no-op and everything loads in a single pass.
state->input.setReplayPass( scene::scenery_load_pass::infrastructure );
// visual nodes are skipped by the reader and loaded in a second pass. this two-pass split
// is only valid when the top-level file is itself a replayable twin, because the visual
// pass is started via restartReplay() which needs a top-level reader. for a text/compile
// load (no top twin) we MUST stay in a single 'all' pass and load everything at once;
// otherwise visual nodes served by included twins (.incb) would be skipped in the infra
// pass and never rebuilt (restartReplay returns false), and all those models go missing.
if( true == state->input.isReplaying() ) {
state->input.setReplayPass( scene::scenery_load_pass::infrastructure );
}
scene::Groups.create();
if( false == state->input.ok() )
@@ -108,6 +118,12 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
cParser &Input = state->input;
scene::scratch_data &Scratchpad = state->scratchpad;
// camera-ordered build phase: the visual replay has been enumerated into records and
// sorted; build them nearest-camera first, a budgeted slice per call.
if( state->enumdone ) {
return build_visual_records( state );
}
// stateful directives that build objects/lists; on the visual (second) pass they are
// skipped wholesale so their side effects (trainsets, events, cameras, ...) don't
// duplicate. transform/group directives (origin/rotate/scale/group) and idempotent
@@ -137,6 +153,18 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
token = Input.getToken<std::string>();
continue;
}
if( ( true == state->enumerate ) && ( token == "node" ) ) {
// capture the visual node (text + transform/group snapshot) for later,
// camera-ordered building, instead of building it in file order now
enumerate_visual_node( *state );
auto timenow = std::chrono::steady_clock::now();
if( std::chrono::duration_cast<std::chrono::milliseconds>( timenow - timelast ).count() >= 8 ) {
Application.set_progress( Input.getProgress(), Input.getFullProgress() );
return true;
}
token = Input.getToken<std::string>();
continue;
}
}
auto lookup = state->functionmap.find( token );
@@ -159,6 +187,28 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
token = Input.getToken<std::string>();
}
if( ( true == state->visualphase ) && ( true == state->enumerate ) ) {
// visual replay exhausted: order the captured nodes by distance to the camera so
// the player's surroundings stream in first (terrain shapes ahead of models, so the
// ground appears before the props on it), then switch to the build phase.
auto const eye { Global.pCamera.Pos };
for( auto &rec : state->records ) {
auto const d { rec.worldpos - eye };
rec.sortkey =
( rec.isshape ?
-1.0 : // shapes (terrain) first, regardless of distance
( d.x * d.x + d.y * d.y + d.z * d.z ) );
}
std::stable_sort(
std::begin( state->records ), std::end( state->records ),
[]( visual_record const &L, visual_record const &R ) {
return L.sortkey < R.sortkey; } );
WriteLog( "Progressive visual load: " + std::to_string( state->records.size() ) + " deferred nodes enumerated, building nearest-camera first" );
state->enumerate = false;
state->enumdone = true;
return true; // proceed to the build phase
}
if( false == Scratchpad.initialized ) {
// manually perform scenario initialization
deserialize_firstinit( Input, Scratchpad );
@@ -172,6 +222,7 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
if( ( false == state->visualphase )
&& ( true == Input.restartReplay( scene::scenery_load_pass::visual ) ) ) {
state->visualphase = true;
state->enumerate = true; // capture deferred visual nodes, build them camera-ordered
// rebuild the transform state from scratch for the visual pass: the directives
// (origin/rotate/scale) are replayed in order, so resetting here reproduces the
// single-pass placement exactly. without this, an unbalanced origin left on the
@@ -188,6 +239,11 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
scene::Groups.update_map();
Region->create_map_geometry();
// all nodes (including visual model instances) are now loaded, so initialise the
// events that bind to model instances; in a single-pass (text/compile) load nothing
// was deferred, but InitEvents() still skipped them, so do it here too
simulation::Events.InitInstanceEvents();
// loading finished: flush the top-level scenario's binary twin now rather than
// waiting for the parser to be destroyed (the loader keeps the state around)
Input.flushBinaryTwin();
@@ -199,6 +255,166 @@ state_serializer::deserialize_continue(std::shared_ptr<deserializer_state> state
return false;
}
// captures one visual node (the "node" token already consumed) into a record for later,
// camera-ordered building: stores the node's verbatim tokens plus the transform/group
// context it was read under. numbers come through cParser losslessly, so the rebuild is
// exact. only model/triangles/lines reach the visual pass.
void
state_serializer::enumerate_visual_node( deserializer_state &State ) {
cParser &Input = State.input;
scene::scratch_data &Scratchpad = State.scratchpad;
// node header, original case preserved (model file paths are case-sensitive at replay)
auto const smax { Input.getToken<std::string>( false ) };
auto const smin { Input.getToken<std::string>( false ) };
auto const sname { Input.getToken<std::string>( false ) };
auto const stype { Input.getToken<std::string>( false ) };
auto lower = []( std::string s ) {
for( auto &c : s ) { c = static_cast<char>( std::tolower( static_cast<unsigned char>( c ) ) ); }
return s; };
auto const typelc { lower( stype ) };
std::string endtok;
bool ismodel { false };
bool istriangles { false };
if( typelc == "model" ) {
endtok = "endmodel"; ismodel = true;
}
else if( ( typelc == "triangles" ) || ( typelc == "triangle_strip" ) || ( typelc == "triangle_fan" ) ) {
endtok = "endtri"; istriangles = true;
}
else if( ( typelc == "lines" ) || ( typelc == "line_strip" ) || ( typelc == "line_loop" ) ) {
endtok = "endline";
}
// mirror deserialize_node's switch-trackbed skip here, while the source file name is
// still known (Input.Name() is the live include during replay; the per-record rebuild
// parser has no name). without this, fixed trackbed geometry from scenery/zwr*.inc that
// the in-order load drops would be rebuilt and duplicate the procedural trackbeds.
bool skipnode { false };
if( ( true == istriangles ) && ( true == Global.CreateSwitchTrackbeds ) ) {
auto const name { Input.Name() };
skipnode =
( name.size() >= 15 )
&& name.starts_with( "scenery/zwr" )
&& name.ends_with( ".inc" );
}
// appends one token to the rebuilt node text, re-quoting it if it carries a token break
// (a quoted source token such as a name/path with spaces arrives here unquoted; without
// re-quoting it the rebuild parser would split it into several tokens)
auto appendtok = []( std::string &Text, std::string const &Token ) {
bool const needsquote {
Token.empty()
|| ( Token.find_first_of( "\n\r\t ;" ) != std::string::npos ) };
Text.push_back( ' ' );
if( true == needsquote ) { Text.push_back( '\"' ); }
Text.append( Token );
if( true == needsquote ) { Text.push_back( '\"' ); } };
std::string text;
text.reserve( 96 );
text.append( "node" );
appendtok( text, smax );
appendtok( text, smin );
appendtok( text, sname );
appendtok( text, stype );
glm::dvec3 localpos { 0.0 };
int posread { 0 };
// read the rest of the node verbatim until its terminator (or the first end* token if the
// type is unrecognised), capturing a model's local position (the first 3 numbers)
for( ;; ) {
auto const tok { Input.getToken<std::string>( false ) };
if( tok.empty() ) { break; } // safety: input ran out
appendtok( text, tok );
if( ( true == ismodel ) && ( posread < 3 ) ) {
localpos[ posread ] = std::atof( tok.c_str() );
++posread;
}
auto const toklc { lower( tok ) };
if( false == endtok.empty() ) {
if( toklc == endtok ) { break; }
}
else if( ( toklc.size() >= 3 ) && ( 0 == toklc.compare( 0, 3, "end" ) ) ) {
break;
}
}
if( true == skipnode ) {
// node consumed from the stream above; intentionally not recorded (matches the
// in-order load, which skips this geometry in favour of procedural trackbeds)
return;
}
visual_record rec;
rec.text = std::move( text );
rec.group = scene::Groups.handle();
rec.has_offset = ( false == Scratchpad.location.offset.empty() );
if( true == rec.has_offset ) { rec.offset = Scratchpad.location.offset.top(); }
rec.has_scale = ( false == Scratchpad.location.scale.empty() );
if( true == rec.has_scale ) { rec.scale = Scratchpad.location.scale.top(); }
rec.rotation = Scratchpad.location.rotation;
rec.isshape = ( false == ismodel );
if( true == ismodel ) {
rec.worldpos = transform( localpos, Scratchpad );
}
State.records.emplace_back( std::move( rec ) );
}
// builds the enumerated visual records in (already-sorted) nearest-camera-first order, a
// budgeted slice per call. each node is rebuilt through the normal node path with its
// captured transform/group restored, so placement, grouping and the per-cell instance
// buckets come out identical to an in-order load.
bool
state_serializer::build_visual_records( std::shared_ptr<deserializer_state> state ) {
scene::scratch_data &Scratchpad = state->scratchpad;
auto timelast { std::chrono::steady_clock::now() };
while( state->buildcursor < state->records.size() ) {
auto &rec = state->records[ state->buildcursor ];
// restore the transform context captured for this node
Scratchpad.location.offset = {};
if( true == rec.has_offset ) { Scratchpad.location.offset.push( rec.offset ); }
Scratchpad.location.scale = {};
if( true == rec.has_scale ) { Scratchpad.location.scale.push( rec.scale ); }
Scratchpad.location.rotation = rec.rotation;
// re-attach to the node's original group and build via the normal node path (this
// routes instanceable models into their per-cell instance bucket automatically)
scene::Groups.push_active( rec.group );
{
cParser nodeparser( rec.text, cParser::buffer_TEXT );
nodeparser.getToken<std::string>(); // consume "node"
deserialize_node( nodeparser, Scratchpad );
}
scene::Groups.pop_active();
std::string().swap( rec.text ); // release the captured text as we go
++state->buildcursor;
auto timenow = std::chrono::steady_clock::now();
if( std::chrono::duration_cast<std::chrono::milliseconds>( timenow - timelast ).count() >= 8 ) {
return true; // yield; resume next frame
}
}
// every deferred visual node is now built: finalise the scenario
scene::Groups.close();
scene::Groups.update_map();
Region->create_map_geometry();
// the visual model instances now exist, so initialise the events that bind to them
// (lights/animation/texture/visible) -- these were deferred during the infrastructure
// pass when their target models hadn't been streamed in yet
simulation::Events.InitInstanceEvents();
state->input.flushBinaryTwin();
scene::scenerybinary_wait_all();
std::vector<visual_record>().swap( state->records );
state->done = true;
return false;
}
void
state_serializer::deserialize_isolated( cParser &Input, scene::scratch_data &Scratchpad ) {
// first parameter specifies name of parent piece...

29
simulation/simulationstateserializer.h
View File

@@ -14,6 +14,24 @@ http://mozilla.org/MPL/2.0/.
namespace simulation {
// a deferred visual node captured during the visual enumeration pass, to be (re)built
// later in camera-distance order. holds the node's fully resolved tokens as text -- numbers
// come through cParser losslessly (std::to_chars shortest round-trip), so rebuilding from
// this text reproduces the exact same node -- plus a snapshot of the transform/group context
// it was read under, so the out-of-order rebuild places and groups it identically.
struct visual_record {
std::string text; // "node <range> <range> <name> <type> ... end<type>"
glm::dvec3 offset { 0.0 }; // top of the origin stack at capture (identity if none)
glm::vec3 scale { 1.f }; // top of the scale stack at capture (identity if none)
glm::vec3 rotation { 0.f }; // active rotation at capture
bool has_offset { false };
bool has_scale { false };
scene::group_handle group {}; // group the node was read under
glm::dvec3 worldpos { 0.0 }; // transformed position (for models; sort key source)
bool isshape { false }; // terrain shape/lines -> load before models (ground first)
double sortkey { 0.0 }; // squared distance to camera; lower = built sooner
};
struct deserializer_state {
std::string scenariofile;
cParser input;
@@ -27,6 +45,13 @@ struct deserializer_state {
bool visualphase { false };
// set once the whole load (both passes / single text pass) has fully finished
bool done { false };
// visual phase sub-state: while enumerate is true the visual pass captures deferred
// nodes into records instead of building them; once the replay is exhausted they are
// sorted by camera distance and built in that order (enumdone).
bool enumerate { false };
bool enumdone { false };
std::vector<visual_record> records;
std::size_t buildcursor { 0 };
deserializer_state(std::string const &File, cParser::buffertype const Type, const std::string &Path, bool const Loadtraction)
: scenariofile(File), input(File, Type, Path, Loadtraction) { }
@@ -68,6 +93,10 @@ private:
void deserialize_endgroup( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_light( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_node( cParser &Input, scene::scratch_data &Scratchpad );
// visual streaming (camera-ordered deferred load): capture one visual node (already
// past its "node" token) into a record; build budgeted records in sorted order.
void enumerate_visual_node( deserializer_state &State );
bool build_visual_records( std::shared_ptr<deserializer_state> State );
void deserialize_origin( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_endorigin( cParser &Input, scene::scratch_data &Scratchpad );
void deserialize_scale( cParser &Input, scene::scratch_data &Scratchpad );

3
utilities/parser.h
View File

@@ -54,6 +54,9 @@ class cParser //: public std::stringstream
// open include child. used to run a second pass (visual) over an already-loaded twin.
// returns false if this parser isn't replaying a twin (no second pass possible).
bool restartReplay( scene::scenery_load_pass Pass );
// true when this (top-level) file is served from a binary twin, i.e. a second
// (visual) pass via restartReplay() is possible. false for a text/compile load.
bool isReplaying() const { return m_replay; }
// methods:
template <typename Type_>
cParser &

15
world/Event.cpp
View File

@@ -2493,6 +2493,21 @@ void
event_manager::InitEvents() {
//łączenie eventów z pozostałymi obiektami
for( auto *event : m_events ) {
// events binding to model instances are deferred: with progressive loading their
// target models stream in after this (infrastructure) pass, so binding them now
// would fail. InitInstanceEvents() initialises them once the visuals are in place.
if( true == event->binds_to_instances() ) { continue; }
event->init();
if( event->m_delay < 0 ) { AddToQuery( event, nullptr ); }
}
}
// initializes the events deferred by InitEvents() (those binding to model instances),
// after the (progressively loaded) visual nodes have been built so the models exist.
void
event_manager::InitInstanceEvents() {
for( auto *event : m_events ) {
if( false == event->binds_to_instances() ) { continue; }
event->init();
if( event->m_delay < 0 ) { AddToQuery( event, nullptr ); }
}

19
world/Event.h
View File

@@ -55,6 +55,13 @@ public:
virtual
void
init() = 0;
// true for events that bind to visual model instances (lights/animation/texture/
// visible). With progressive loading these models are streamed in after the
// infrastructure pass, so such events must be initialised only once the deferred
// visual nodes exist -- not during the early InitEvents() of the infrastructure pass.
virtual
bool
binds_to_instances() const { return false; }
// executes event
virtual
void
@@ -393,6 +400,7 @@ public:
// methods
// prepares event for use
void init() override;
bool binds_to_instances() const override { return true; }
private:
// types
@@ -427,6 +435,7 @@ public:
// methods
// prepares event for use
void init() override;
bool binds_to_instances() const override { return true; }
private:
// methods
@@ -456,6 +465,7 @@ public:
// methods
// prepares event for use
void init() override;
bool binds_to_instances() const override { return true; }
private:
// methods
@@ -550,6 +560,7 @@ public:
// methods
// prepares event for use
void init() override;
bool binds_to_instances() const override { return true; }
private:
// methods
@@ -690,9 +701,15 @@ public:
// legacy method, executes queued events
bool
CheckQuery();
// legacy method, initializes events after deserialization from scenario file
// legacy method, initializes events after deserialization from scenario file.
// events that bind to visual model instances are skipped here and initialised later
// by InitInstanceEvents(), once the (progressively loaded) visual nodes are in place.
void
InitEvents();
// initializes the events skipped by InitEvents() (those binding to model instances).
// called once the deferred visual nodes have been built, so their target models exist.
void
InitInstanceEvents();
// legacy method, initializes event launchers after deserialization from scenario file
void
InitLaunchers();