The progressive load previously streamed the deferred visual nodes (3d model
instances + terrain shapes/lines) in file order, so distant scenery could load
before the player's surroundings. This builds them nearest-camera first instead.
The visual pass now runs in two steps. Enumeration replays the twin and captures
each visual node verbatim (its resolved tokens as text -- numbers round-trip
losslessly through cParser) together with the transform/group context it was read
under and, for models, its transformed world position. Once the replay is
exhausted the records are sorted by squared distance to the camera (terrain shapes
first so the ground appears before the props on it), then built a budgeted slice
per frame through the normal node path with the captured transform and group
restored -- so placement, grouping and the per-cell instance buckets come out
identical to an in-order load.
Two supporting fixes make out-of-order/late insertion correct:
- a cell/section whose geometry was already baked (the renderer finalised it
before a deferred node arrived) now appends the new shape/lines straight into
its live geometry bank instead of merging into vertex-freed geometry, which
would silently drop it; create_geometry() remembers the bank for every cell.
- events that bind to visual model instances (lights/animation/texture/visible)
are deferred from InitEvents() to a new InitInstanceEvents() run after the
visual nodes are built, so their target models exist when they initialise.
Verified on td.scn: playable ~2s, 540 deferred nodes enumerated and built
nearest-first ~0.5s later, no duplicate instances.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When replaying a binary twin, load in two passes over the same data: the first
pass loads infrastructure (tracks/traction/events/memcells/sounds + directives),
the second pass loads the visual nodes (3d models, terrain shapes/lines) that the
reader skipped via the v6 node-class markers.
- cParser: setReplayPass() selects the served node class (propagated to include
children); restartReplay() rewinds the twin for the second pass.
- state_serializer: first pass uses the infrastructure pass; on completion it
restarts the twin for the visual pass. Stateful directives (trainset, event,
camera, light, sky, time, ...) are skipped on the visual pass so their side
effects do not duplicate; transform/group directives re-run so deferred visual
nodes get correct placement. A text/compile load (no twin) stays single-pass.
Verified: td.scn replays through both passes with no duplicate vehicles/events
and reaches the normal load endpoint. This is the sequential foundation; moving
the visual pass into the driver (so play starts after the infrastructure pass) is
the next step.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each top-level node is wrapped in a marker carrying its class (infrastructure vs
visual) and byte span, so the reader can serve or skip a whole node per load pass
without knowing its terminator token. The bake recognizes nodes by the "node"
keyword + type token (map type->terminator confirmed against the deserializers)
and classifies them; the writer buffers a node's entries and emits the marker.
The reader gains a pass selector (all / infrastructure / visual) and skips nodes
not in the pass by advancing past their byte span.
This is the foundation only: the default pass is "all", so loading is identical
to v5 (markers are transparent) -- verified by loading td.scn to the same point.
The eager/visual split is wired up by the upcoming loader/driver integration.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The buffer-based tokenizer reads the source into memory once at construction, so
the underlying stream fail bit never flips at end-of-input. cParser::ok() was
not stream.fail(), so while( parser.ok() ) loops (TMoverParameters::LoadFIZ and
similar) never terminated -- they spun on empty tokens past EOF, hanging the
vehicle/.fiz load. Redefine ok() as "input still remains" (buffer not exhausted,
or for replay not exhausted), which terminates those loops while staying true
right after opening a non-empty file (the open checks if(!ok())).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Reader is buffer-based and streaming: the twin is read once and parsed by
pointer, entries decoded on demand (no per-byte stream, no vector-of-entry
materialisation), string tokens served as views into the buffer.
- Format v5 is markedly smaller without compression: string interning with
varint indices, entry type packed into the head varint, integers as zig-zag
varint and fractional numbers as f32 (f64 only when needed). Tokens are stored
in original case with a quoted flag and lower-cased per consumer at replay, so
capture is grammar-independent.
- Writer encodes entries incrementally into a compact buffer instead of holding
a struct per token, keeping memory bounded when baking huge files in parallel.
- New headless bake mode (-bake) precompiles a scenario and all its includes into
twins on a thread pool, with no window/renderer/scene; each file is tokenised in
isolation and baked exactly once.
- Fix stack overflow on files with long runs of consecutive includes by handling
include directives iteratively instead of recursively (also hardens the normal
text load).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Text scenery component files (.scn/.inc/.scm/.ctr) are compiled into
per-file binary twins (.scnb/.incb/.scmb/.ctrb), handled transparently
at the cParser layer: a fresh twin (mtime-checked, version-matched) is
replayed instead of re-tokenizing text; otherwise the text is parsed and
a twin is compiled alongside it for next time.
Format details:
- per-file string interning: keywords/paths stored once, referenced by
varint index (so node/endmodel/... are not repeated as text)
- numeric tokens stored as 8-byte IEEE doubles, not ASCII
- includes kept as references with parameters; random sets stored verbatim
and re-evaluated on every load (choice not frozen at compile time)
- twin writing offloaded to a bounded thread pool so baking overlaps
scene construction instead of blocking the load
The legacy terrain-only .sbt path is removed; terrain now loads as
ordinary scenery content.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
