17
0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-10-17 21:29:19 +02:00
Files
maszyna/scene/scene.cppm
maj00r 2660d14565 Stream the scenery: a baked terrain heightfield, a logic container, and prefabs
Today's work on the binary scene format plan (binary-scene-format-plan.md), from the
first measurement to a terrain that bakes itself on first run and streams from a
thread. The step-by-step history is kept on backup/scene-streaming-format-presquash-0916.

Measuring first. -loadprofile times a scenery load by stage and by node kind and reports
node counts and resident memory. On l204 terrain is 95% of the bytes and 99.8% of the
nodes; on quarkmce2007 vehicles, not models, dominate what is left. The terrain turned
out to be a heightfield in all but name, a ribbon covering 12% of its bounding box.

The heightfield. scene/heightfieldformat.h is the .ehf layout: Morton-ordered tiles
of 128 samples with one shared edge sample, a point-sampled mip chain, zstd per tile,
a nodata sentinel, a material palette with a measured tiling per material, and a stamp
of the rules that decided what counted as terrain. scene/heightfieldreader.h reads it
for the tool, the renderer and the loader alike, and scene/terraincooker.h writes it,
shared by the terraincook tool and the engine so a first-run bake is the file the tool
would have made. The cooker rasterises every triangle where it is drawn (through
scene::place_in_world, the placement the region applies too), keeps an upper surface
laid within half a metre over another and the lower of two further apart, and then
asks each triangle whether the heightfield shows it: laid, reaching a sample, keeping
nearly all it claimed, and with the ground around it drawn at every level out to 2 km.

Baking on first run. simulation/terrainbake bakes <scenario>.ehf the first time a
scenery is opened, choosing the grid step from the first 200,000 vertices, and on later
loads leaves out the triangles its node table says the heightfield shows. Everything
else - trees, grass tufts, gantries, bridge decks - is drawn as written, so a bad bake
can at worst draw something twice, and the heightfield sits 2 cm under the triangles so
the original wins there. Nodes are keyed by file, line, include parameters and
placement; an edited file changes its keys and those nodes are drawn again. A bake under
older rules, or without its table, is thrown away and redone. A launcher copy named
$scenario shares the scenario's files. -nobaketerrain shows the triangles as written.
A scenery can still name hand-cooked fields, several at different steps.

Drawing and streaming. rendering/terrainclipmap draws resident tiles at the mip level
their distance earns, deriving the grid from gl_VertexID and texturing in world space
with the scenery's own materials. rendering/terraintileloader reads and decodes tiles on
a thread of its own; the renderer rescans what is in range only after moving a quarter
of a tile, asks for what is missing nearest first, uploads at most 32 tiles a frame,
drops arrivals the camera has moved past, and keeps drawing a tile at the level it has
until the right one arrives. The terrain has its own range, terrainrange in the ini and
a slider in the debug panel, 20 km by default and independent of basedrawrange; the far
plane of an ordinary view is pushed out to it. The debug panel shows the range, tiles in
view, on the gpu, wanted and queued, and how many sit at each level.

Events. The event queue moved out of the events into a binary heap with a deterministic
tie-break, and an event is handed its activator instead of storing it, so basic_event
is a definition and nothing else.

The logic container. scene/logicformat.h and scene/logiccontainer write a scenery's
events, their targets and sibling chains as flat, deterministic arrays with a schema
hash that refuses a mismatched build. simulation/logiccook cooks it from the loaded
state whenever it is missing, stale or unreadable, reads it straight back to check it,
and resolves each target to a position in the registry its event kind searches.
-cooklogic forces a cook, -verifylogic only checks. Nothing reads it at load yet.

Prefabs. scene/prefab describes a signal by the aspects it can show - lights, speeds,
animation channels, a sound - instead of pasting an .inc per variant, and expands an
instance to ordinary scenery text. Tried on a light signal and a shape signal.

Screenshots without a person present: -shot x,y,z[,yaw[,pitch]], -shotsettle, -shotstay.
2026-09-16 23:18:09 +02:00

514 lines
21 KiB
C++
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
module;
#include <utility>
#include <chrono>
#include <limits>
#include <mutex>
#include <sstream>
#include <type_traits>
#include <cstddef>
#include <functional>
#include <istream>
#include <ostream>
#include <string>
#include <tuple>
#include <iosfwd>
#include <cmath>
#include <vector>
#include <deque>
#include <array>
#include <stack>
#include <unordered_set>
#include <unordered_map>
#include <map>
#include "utilities/utilities_macros.h"
export module eu07.simcore:scene;
import :fwd;
import eu07.input.command;
import eu07.utilities.parser;
import :scenenode;
import :track;
import :traction;
import eu07.utilities.utilities;
export {
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
namespace scene {
int constexpr EU07_CELLSIZE = 250;
int constexpr EU07_SECTIONSIZE = 1000;
int constexpr EU07_REGIONSIDESECTIONCOUNT = 500; // number of sections along a side of square region
struct scratch_data {
struct binary_data {
bool terrain{ false };
bool terrain_included{false};
} binary;
struct location_data {
std::stack<glm::dvec3> offset;
// per-axis scale stack — mirrors `offset` for the `scale`/`endscale`
// scenario directives. Effective scale at any nesting depth is the
// component-wise product of all stack entries (outer `scale 2 2 2` ×
// inner `scale 1.5 1.5 1.5` yields (3,3,3)). Empty stack means (1,1,1).
std::stack<glm::vec3> scale;
glm::vec3 rotation;
} location;
struct trainset_data {
std::string name;
std::string track;
float offset { 0.f };
float velocity { 0.f };
std::vector<TDynamicObject *> vehicles;
std::vector<int> couplings;
TDynamicObject * driver { nullptr };
bool is_open { false };
std::unordered_map<std::string, std::string> assignment;
} trainset;
std::string name;
std::string terrain_name;
bool initialized { false };
bool time_initialized { false };
};
// basic element of rudimentary partitioning scheme for the section. fixed size, no further subdivision
// TBD, TODO: replace with quadtree scheme?
class basic_cell {
public:
// constructors
basic_cell() = default;
// methods
// potentially activates event handler with the same name as provided node, and within handler activation range
void
on_click( TAnimModel const *Instance );
// legacy method, finds and assigns traction piece to specified pantograph of provided vehicle
void
update_traction( TDynamicObject *Vehicle, int const Pantographindex );
// legacy method, polls event launchers within radius around specified point
void
update_events();
// legacy method, updates sounds within radius around specified point
void
update_sounds();
// legacy method, triggers radio-stop procedure for all vehicles located on paths in the cell
void
radio_stop();
// legacy method, adds specified path to the list of pieces undergoing state change
bool
RaTrackAnimAdd( TTrack *Track );
// legacy method, updates geometry for pieces in the animation list
void
RaAnimate( unsigned int const Framestamp );
// sends content of the class to provided stream
void
serialize( std::ostream &Output ) const;
// restores content of the class from provided stream
void
deserialize( std::istream &Input );
// sends content of the class in legacy (text) format to provided stream
void
export_as_text( std::ostream &Output ) const;
// adds provided shape to the cell
void
insert( shape_node Shape );
// adds provided lines to the cell
void
insert( lines_node Lines );
// adds provided path to the cell
void
insert( TTrack *Path );
// adds provided path to the cell
void
insert( TTraction *Traction );
// adds provided model instance to the cell
void
insert( TAnimModel *Instance );
// adds provided sound instance to the cell
void
insert( sound_source *Sound );
// adds provided event launcher to the cell
void
insert( TEventLauncher *Launcher );
// adds provided memory cell to the cell
void
insert( TMemCell *Memorycell );
// registers provided path in the lookup directory of the cell
void
register_end( TTrack *Path );
// registers provided traction piece in the lookup directory of the cell
void
register_end( TTraction *Traction );
// removes provided model instance from the cell
void
erase( TAnimModel *Instance );
// removes provided memory cell from the cell
void
erase( TMemCell *Memorycell );
// find a vehicle located nearest to specified point, within specified radius. reurns: located vehicle and distance
std::tuple<TDynamicObject *, float>
find( glm::dvec3 const &Point, float const Radius, bool const Onlycontrolled, bool const Findbycoupler ) const;
// finds a path with one of its ends located in specified point. returns: located path and id of the matching endpoint
std::tuple<TTrack *, int>
find( glm::dvec3 const &Point, TTrack const *Exclude ) const;
// finds a traction piece with one of its ends located in specified point. returns: located traction piece and id of the matching endpoint
std::tuple<TTraction *, int>
find( glm::dvec3 const &Point, TTraction const *Exclude ) const;
// finds a traction piece located nearest to specified point, sharing section with specified other piece and powered in specified direction. returns: located traction piece
std::tuple<TTraction *, int, float>
find( glm::dvec3 const &Point, TTraction const *Other, int const Currentdirection ) const;
// sets center point of the cell
void
center( glm::dvec3 Center );
// generates renderable version of held non-instanced geometry in specified geometry bank
void
create_geometry( gfx::geometrybank_handle const &Bank );
void
create_map_geometry(std::vector<gfx::basic_vertex> &Bank, const gfx::geometrybank_handle Extra);
void
get_map_active_paths(map_colored_paths &handles);
glm::vec3 find_nearest_track_point(const glm::dvec3 &pos);
// provides access to bounding area data
bounding_area const &
area() const {
return m_area; }
//private:
// types
using path_sequence = std::vector<TTrack *>;
using shapenode_sequence = std::vector<shape_node>;
using linesnode_sequence = std::vector<lines_node>;
using traction_sequence = std::vector<TTraction *>;
using instance_sequence = std::vector<TAnimModel *>;
// Composite key: instances of the same TModel3d sharing the same replacable
// skin set can be GPU-batched together, but instances with different skins
// must go to different buckets so each batched draw call uses one consistent
// material binding. Sub-bucketing here keeps Render_Instanced correct without
// forcing per-instance material switching inside a single batch.
struct instance_bucket_key {
TModel3d *pModel { nullptr };
std::array<material_handle, 5> skins {};
bool operator==( instance_bucket_key const &other ) const {
return pModel == other.pModel && skins == other.skins;
}
};
struct instance_bucket_key_hash {
std::size_t operator()( instance_bucket_key const &k ) const {
std::size_t h = std::hash<TModel3d *>()( k.pModel );
for( auto s : k.skins ) {
// boost-style hash combine
h ^= std::hash<int>()( s ) + 0x9e3779b9 + ( h << 6 ) + ( h >> 2 );
}
return h;
}
};
using instance_bucket_map = std::unordered_map< instance_bucket_key, std::vector<TAnimModel *>, instance_bucket_key_hash >;
using sound_sequence = std::vector<sound_source *>;
using eventlauncher_sequence = std::vector<TEventLauncher *>;
using memorycell_sequence = std::vector<TMemCell *>;
// methods
void
launch_event(TEventLauncher *Launcher, bool local_only);
void
enclose_area( scene::basic_node *Node );
// members
scene::bounding_area m_area { glm::dvec3(), static_cast<float>( 0.5 * M_SQRT2 * EU07_CELLSIZE ) };
bool m_active { false }; // whether the cell holds any actual data content
shapenode_sequence m_shapesopaque; // opaque pieces of geometry
shapenode_sequence m_shapestranslucent; // translucent pieces of geometry
linesnode_sequence m_lines;
path_sequence m_paths; // path pieces
instance_sequence m_instancesopaque;
instance_sequence m_instancetranslucent;
// batched instance buckets keyed by shared TModel3d*; populated alongside
// m_instancesopaque for nodes whose TAnimModel::m_instanceable == true.
// The renderer uses these to amortise per-model state setup across many instances.
instance_bucket_map m_instancebuckets_opaque;
traction_sequence m_traction;
sound_sequence m_sounds;
eventlauncher_sequence m_eventlaunchers;
memorycell_sequence m_memorycells;
// search helpers
struct lookup_data {
path_sequence paths;
traction_sequence traction;
} m_directories;
// animation of owned items (legacy code, clean up along with track refactoring)
bool m_geometrycreated { false };
unsigned int m_framestamp { 0 }; // id of last rendered gfx frame
TTrack *tTrackAnim = nullptr; // obiekty do przeliczenia animacji
command_relay m_relay;
};
// basic scene partitioning structure, holds terrain geometry and collection of cells
class basic_section {
public:
// constructors
basic_section() = default;
// methods
// potentially activates event handler with the same name as provided node, and within handler activation range
void
on_click( TAnimModel const *Instance );
// legacy method, finds and assigns traction piece to specified pantograph of provided vehicle
void
update_traction( TDynamicObject *Vehicle, int const Pantographindex );
// legacy method, updates sounds and polls event launchers within radius around specified point
void
update_events( glm::dvec3 const &Location, float const Radius );
// legacy method, updates sounds and polls event launchers within radius around specified point
void
update_sounds( glm::dvec3 const &Location, float const Radius );
// legacy method, triggers radio-stop procedure for all vehicles in 2km radius around specified location
void
radio_stop( glm::dvec3 const &Location, float const Radius );
// sends content of the class to provided stream
void
serialize( std::ostream &Output ) const;
// restores content of the class from provided stream
void
deserialize( std::istream &Input );
// sends content of the class in legacy (text) format to provided stream
void
export_as_text( std::ostream &Output ) const;
// adds provided shape to the section
void
insert( shape_node Shape );
// adds provided lines to the section
void
insert( lines_node Lines );
// adds provided node to the section
template <class Type_>
void
insert( Type_ *Node ) {
auto &targetcell { cell( Node->location() ) };
targetcell.insert( Node );
// some node types can extend bounding area of the target cell
m_area.radius = std::max(
m_area.radius,
static_cast<float>( glm::length( m_area.center - targetcell.area().center ) + targetcell.area().radius ) ); }
// erases provided node from the section
template <class Type_>
void
erase( Type_ *Node ) {
auto &targetcell { cell( Node->location() ) };
// TODO: re-calculate bounding area after removal
targetcell.erase( Node ); }
// registers provided node in the lookup directory of the section enclosing specified point
template <class Type_>
void
register_node( Type_ *Node, glm::dvec3 const &Point ) {
cell( Point ).register_end( Node ); }
// find a vehicle located nearest to specified point, within specified radius. reurns: located vehicle and distance
std::tuple<TDynamicObject *, float>
find( glm::dvec3 const &Point, float const Radius, bool const Onlycontrolled, bool const Findbycoupler );
// finds a path with one of its ends located in specified point. returns: located path and id of the matching endpoint
std::tuple<TTrack *, int>
find( glm::dvec3 const &Point, TTrack const *Exclude );
// finds a traction piece with one of its ends located in specified point. returns: located traction piece and id of the matching endpoint
std::tuple<TTraction *, int>
find( glm::dvec3 const &Point, TTraction const *Exclude );
// finds a traction piece located nearest to specified point, sharing section with specified other piece and powered in specified direction. returns: located traction piece
std::tuple<TTraction *, int, float>
find( glm::dvec3 const &Point, TTraction const *Other, int const Currentdirection );
// sets center point of the section
void
center( glm::dvec3 Center );
// generates renderable version of held non-instanced geometry
void
create_geometry();
void
create_map_geometry(const gfx::geometrybank_handle handle);
void
get_map_active_paths(map_colored_paths &handles);
// provides access to bounding area data
bounding_area const &
area() const {
return m_area; }
const gfx::geometrybank_handle get_map_geometry()
{ return m_map_geometryhandle;}
glm::vec3 find_nearest_track_point(const glm::dvec3 &point);
//private:
// types
using cell_array = std::array<basic_cell, EU07_SECTIONSIZE / EU07_CELLSIZE * (EU07_SECTIONSIZE / EU07_CELLSIZE)>;
using shapenode_sequence = std::vector<shape_node>;
// methods
// provides access to section enclosing specified point
basic_cell &
cell(glm::dvec3 const &Location, const glm::ivec2 &offset = glm::ivec2(0));
// members
// placement and visibility
scene::bounding_area m_area { glm::dvec3(), static_cast<float>( 0.5 * M_SQRT2 * EU07_SECTIONSIZE ) };
// content
cell_array m_cells; // partitioning scheme
shapenode_sequence m_shapes; // large pieces of opaque geometry and (legacy) terrain
// TODO: implement dedicated, higher fidelity, fixed resolution terrain mesh item
// gfx renderer data
gfx::geometrybank_handle m_geometrybank;
bool m_geometrycreated { false };
gfx::geometrybank_handle m_map_geometryhandle;
};
// top-level of scene spatial structure, holds collection of sections
// whether a material belongs to the old hand-made switch trackbeds, which the engine
// replaces with generated ones when CreateSwitchTrackbeds is on
bool is_switch_trackbed( std::string const &Materialname );
// moves shape vertices from the coordinates they were written in to world space, applying the
// rotate and origin in force where the node stood. the one place this happens, so that anything
// else looking at a shape - the terrain bake - sees it where it is drawn
void place_in_world( std::vector<world_vertex> &Vertices, scratch_data const &Scratchpad );
class basic_region {
public:
// constructors
basic_region();
// destructor
~basic_region();
// methods
// potentially activates event handler with the same name as provided node, and within handler activation range
void
on_click( TAnimModel const *Instance );
// legacy method, finds and assigns traction piece to specified pantograph of provided vehicle
void
update_traction( TDynamicObject *Vehicle, int const Pantographindex );
// legacy method, polls event launchers around camera
void
update_events();
// legacy method, updates sounds around camera
void
update_sounds();
// checks whether specified file is a valid region data file
bool
is_scene( std::string const &Scenariofile ) const;
// stores content of the class in file with specified name
void
serialize( std::string const &Scenariofile ) const;
// restores content of the class from file with specified name. returns: true on success, false otherwise
bool
deserialize( std::string const &Scenariofile );
// sends content of the class in legacy (text) format to provided stream
void
export_as_text( std::ostream &Output ) const;
// legacy method, links specified path piece with potential neighbours
void
TrackJoin( TTrack *Track );
// legacy method, triggers radio-stop procedure for all vehicles in 2km radius around specified location
void
RadioStop( glm::dvec3 const &Location );
// inserts provided shape in the region
void
insert( shape_node Shape, scratch_data &Scratchpad, bool const Transform );
// inserts provided lines in the region
void
insert( lines_node Lines, scratch_data &Scratchpad );
// inserts provided node in the region
template <class Type_>
void
insert( Type_ *Node ) {
auto const location { Node->location() };
if( false == point_inside( location ) ) {
// NOTE: nodes placed outside of region boundaries are discarded
// TBD, TODO: clamp coordinates to region boundaries?
return; }
section( location ).insert( Node ); }
// inserts provided node in the region and registers its ends in lookup directory
template <class Type_>
void
insert_and_register( Type_ *Node ) {
insert( Node );
for( auto const &point : Node->endpoints() ) {
if( point_inside( point ) ) {
section( point ).register_node( Node, point ); } } }
// removes specified node from the region
template <class Type_>
void
erase( Type_ *Node ) {
auto const location{ Node->location() };
if( point_inside( location ) ) {
section( location ).erase( Node ); } }
// find a vehicle located nearest to specified point, within specified radius. reurns: located vehicle and distance
std::tuple<TDynamicObject *, float>
find_vehicle( glm::dvec3 const &Point, float const Radius, bool const Onlycontrolled, bool const Findbycoupler );
// finds a path with one of its ends located in specified point. returns: located path and id of the matching endpoint
std::tuple<TTrack *, int>
find_path( glm::dvec3 const &Point, TTrack const *Exclude );
// finds a traction piece with one of its ends located in specified point. returns: located traction piece and id of the matching endpoint
std::tuple<TTraction *, int>
find_traction( glm::dvec3 const &Point, TTraction const *Exclude );
// finds a traction piece located nearest to specified point, sharing section with specified other piece and powered in specified direction. returns: located traction piece
std::tuple<TTraction *, int>
find_traction( glm::dvec3 const &Point, TTraction const *Other, int const Currentdirection );
// finds sections inside specified sphere. returns: list of sections
std::vector<basic_section *> const &
sections( glm::dvec3 const &Point, float const Radius );
void
create_map_geometry();
void
update_poi_geometry();
basic_section* get_section(size_t section)
{ return m_sections[section]; }
gfx::geometrybank_handle
get_map_poi_geometry() { return m_map_poipoints; }
glm::vec3 find_nearest_track_point(const glm::dvec3 &pos)
{ return section(pos).find_nearest_track_point(pos); }
//private:
// types
using section_array = std::array<basic_section *, sq(EU07_REGIONSIDESECTIONCOUNT)>;
struct region_scratchpad {
std::vector<basic_section *> sections;
};
gfx::geometrybank_handle m_map_geometrybank;
gfx::geometrybank_handle m_map_poipoints;
// methods
// checks whether specified point is within boundaries of the region
bool
point_inside( glm::dvec3 const &Location );
// legacy method, trims provided shape to fit into a section. adds trimmed part at the end of provided list, returns true if changes were made
static
bool
RaTriangleDivider( shape_node &Shape, std::deque<shape_node> &Shapes );
// provides access to section enclosing specified point
basic_section &
section( glm::dvec3 const &Location );
// members
section_array m_sections;
region_scratchpad m_scratchpad;
};
// global hierarchy map for scene nodes
extern std::map<std::string, basic_node *> Hierarchy;
} // scene
//---------------------------------------------------------------------------
} // export