eu07/maszyna
17
0
Fork 0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-11-04 10:09:31 +01:00
Code Issues 156 Wiki Activity

Stream the scenery: a baked terrain heightfield, a logic container, and prefabs

Browse Source 
Today's work on the binary scene format plan (binary-scene-format-plan.md), from the
first measurement to a terrain that bakes itself on first run and streams from a
thread. The step-by-step history is kept on backup/scene-streaming-format-presquash-0916.

Measuring first. -loadprofile times a scenery load by stage and by node kind and reports
node counts and resident memory. On l204 terrain is 95% of the bytes and 99.8% of the
nodes; on quarkmce2007 vehicles, not models, dominate what is left. The terrain turned
out to be a heightfield in all but name, a ribbon covering 12% of its bounding box.

The heightfield. scene/heightfieldformat.h is the .ehf layout: Morton-ordered tiles
of 128 samples with one shared edge sample, a point-sampled mip chain, zstd per tile,
a nodata sentinel, a material palette with a measured tiling per material, and a stamp
of the rules that decided what counted as terrain. scene/heightfieldreader.h reads it
for the tool, the renderer and the loader alike, and scene/terraincooker.h writes it,
shared by the terraincook tool and the engine so a first-run bake is the file the tool
would have made. The cooker rasterises every triangle where it is drawn (through
scene::place_in_world, the placement the region applies too), keeps an upper surface
laid within half a metre over another and the lower of two further apart, and then
asks each triangle whether the heightfield shows it: laid, reaching a sample, keeping
nearly all it claimed, and with the ground around it drawn at every level out to 2 km.

Baking on first run. simulation/terrainbake bakes <scenario>.ehf the first time a
scenery is opened, choosing the grid step from the first 200,000 vertices, and on later
loads leaves out the triangles its node table says the heightfield shows. Everything
else - trees, grass tufts, gantries, bridge decks - is drawn as written, so a bad bake
can at worst draw something twice, and the heightfield sits 2 cm under the triangles so
the original wins there. Nodes are keyed by file, line, include parameters and
placement; an edited file changes its keys and those nodes are drawn again. A bake under
older rules, or without its table, is thrown away and redone. A launcher copy named
$scenario shares the scenario's files. -nobaketerrain shows the triangles as written.
A scenery can still name hand-cooked fields, several at different steps.

Drawing and streaming. rendering/terrainclipmap draws resident tiles at the mip level
their distance earns, deriving the grid from gl_VertexID and texturing in world space
with the scenery's own materials. rendering/terraintileloader reads and decodes tiles on
a thread of its own; the renderer rescans what is in range only after moving a quarter
of a tile, asks for what is missing nearest first, uploads at most 32 tiles a frame,
drops arrivals the camera has moved past, and keeps drawing a tile at the level it has
until the right one arrives. The terrain has its own range, terrainrange in the ini and
a slider in the debug panel, 20 km by default and independent of basedrawrange; the far
plane of an ordinary view is pushed out to it. The debug panel shows the range, tiles in
view, on the gpu, wanted and queued, and how many sit at each level.

Events. The event queue moved out of the events into a binary heap with a deterministic
tie-break, and an event is handed its activator instead of storing it, so basic_event
is a definition and nothing else.

The logic container. scene/logicformat.h and scene/logiccontainer write a scenery's
events, their targets and sibling chains as flat, deterministic arrays with a schema
hash that refuses a mismatched build. simulation/logiccook cooks it from the loaded
state whenever it is missing, stale or unreadable, reads it straight back to check it,
and resolves each target to a position in the registry its event kind searches.
-cooklogic forces a cook, -verifylogic only checks. Nothing reads it at load yet.

Prefabs. scene/prefab describes a signal by the aspects it can show - lights, speeds,
animation channels, a sound - instead of pasting an .inc per variant, and expands an
instance to ordinary scenery text. Tried on a light signal and a shape signal.

Screenshots without a person present: -shot x,y,z[,yaw[,pitch]], -shotsettle, -shotstay.
This commit is contained in:
maj00r
2026-09-16 23:18:09 +02:00
parent 366645ef59
commit 2660d14565
47 changed files with 7117 additions and 240 deletions
Download Patch File Download Diff File Expand all files Collapse all files

80
application/application.cpp
View File

@@ -66,6 +66,8 @@ import eu07.simcore;
import eu07.utilities.globals;
import eu07.utilities.utilities;
import eu07.simulation.simulation;
import eu07.simulation.loadprofile;
import eu07.application.shotrunner;
import eu07.simulation.simulationsounds;
import eu07.vehicle.train;
import eu07.utilities.dictionary;
@@ -1171,6 +1173,9 @@ int eu07_application::init_settings(int Argc, char *Argv[])
// process command line arguments
auto scenerywasasked = false;
std::vector<shot_runner::shot> shots;
auto shotsettle = 3.0;
auto shotquit = true;
for (int i = 1; i < Argc; ++i)
{
@@ -1202,16 +1207,91 @@ int eu07_application::init_settings(int Argc, char *Argv[])
{
Global.editor_startup = true;
}
else if (token == "-terrain")
{
if (i + 1 < Argc)
{
// cooked heightfield produced by tools/terraincook; may be repeated
Global.terrain_heightfields.push_back(Argv[++i]);
}
}
else if (token == "-shot")
{
// unattended screenshot: "x,y,z[,yaw[,pitch]]", repeatable for a sequence
if (i + 1 < Argc)
{
shot_runner::shot shot;
if (shot_runner::parse(Argv[++i], shot))
{
shots.push_back(shot);
}
else
{
std::cout << "bad -shot viewpoint: " << Argv[i] << std::endl;
return -1;
}
}
}
else if (token == "-shotsettle")
{
// seconds to wait at each viewpoint before saving, so terrain tiles and
// models have time to stream in
if (i + 1 < Argc)
{
shotsettle = std::atof(Argv[++i]);
}
}
else if (token == "-shotstay")
{
// leave the simulation running once the sequence is done
shotquit = false;
}
else if (token == "-loadprofile")
{
// scenery load profiling: where the load time goes, what the scenery holds,
// what it costs in memory. writes to the log and to loadprofile.txt
loadprofile::enable(true);
}
else if (token == "-cooklogic")
{
// cook the scenery's LOGIC container once it has loaded, then read it back and
// check it against the events in memory
Global.cook_logic = true;
}
else if (token == "-nobaketerrain")
{
// load the scenery's terrain triangles as written, without baking or using a
// heightfield baked beside it. the way to see the original for comparison
Global.bake_terrain = false;
}
else if (token == "-verifylogic")
{
// read the LOGIC container that is already on disk and check it against the
// scenery, writing nothing
Global.verify_logic = true;
}
else
{
std::cout << "usage: " << std::string(Argv[0]) << " [-s sceneryfilepath]"
<< " [-v vehiclename]"
<< " [-editor]"
<< " [-terrain terrain.ehf]"
<< " [-shot x,y,z[,yaw[,pitch]]]"
<< " [-shotsettle seconds] [-shotstay]"
<< " [-loadprofile]"
<< " [-cooklogic]"
<< " [-verifylogic]"
<< " [-nobaketerrain]"
<< " (every switch also takes a double dash: --editor)" << std::endl;
return -1;
}
}
if (false == shots.empty())
{
Shots.configure(shots, shotsettle, shotquit);
}
if (Global.editor_startup && false == scenerywasasked)
{
// the editor opens on empty ground unless a scenery was actually asked for. whatever the ini

35
application/drivermode.cpp
View File

@@ -22,6 +22,7 @@ module;
#include "vehicle/DynObj_macros.h"
module eu07.application.drivermode;
import eu07.application.shotrunner;
import eu07.console; // Console:: is used under _WIN32
import eu07.application.uitranscripts;
import eu07.application.uilayer;
@@ -194,6 +195,40 @@ bool driver_mode::update()
simulation::State.update_scripting_interface();
simulation::Environment.update();
// unattended screenshot sequence, when one was asked for on the command line
if (true == Shots.active())
{
switch (Shots.update(Timer::GetDeltaTime()))
{
case shot_runner::action::place:
{
Camera.Init(Shots.current().position, Shots.current().angle, nullptr);
Global.pCamera = Camera;
FreeFlyModeFlag = true;
break;
}
case shot_runner::action::capture:
{
// where the lens actually is, which need not be where it was put: anything
// that moves the camera between placing and capturing would otherwise make
// the sequence quietly unrepeatable
WriteLog(
"Shot: camera at " + to_string(Camera.Pos)
+ " looking " + to_string(glm::dvec3{Camera.Angle} * 180.0 / M_PI)
+ (FreeFlyModeFlag ? " (free)" : " (attached)"));
Application.queue_screenshot();
break;
}
case shot_runner::action::finish:
{
Application.queue_quit(false);
break;
}
case shot_runner::action::none:
break;
}
}
if (deltatime != 0.0 || false == simulation::is_ready)
{
// jak pauza, to nie ma po co tego przeliczać

61
application/driveruipanels.cpp
View File

@@ -47,6 +47,7 @@ import eu07.utilities.logs;
import eu07.widgets.vehicleparams;
import eu07.utilities.u8;
import eu07.application.driverhints;
import eu07.rendering.terrainstatus;
#ifdef WITH_UART
import eu07.utilities.uart;
using namespace Mtable;
@@ -765,6 +766,14 @@ debug_panel::render_section_scenario() {
Global.BaseDrawRange = std::clamp(std::exp(drawrange), 100.0f, 50000.0f);
}
}
// terrain streaming range slider, separate from the draw range
{
auto terrainrange = std::log(Global.TerrainRange);
if (ImGui::SliderFloat(
(to_string(std::exp(terrainrange), 0, 5) + " m###terrainrange").c_str(), &terrainrange, std::log(500.0f), std::log(100000.0f), "Terrain range")) {
Global.TerrainRange = std::clamp(std::exp(terrainrange), 500.0f, 100000.0f);
}
}
}
return true;
@@ -1368,28 +1377,33 @@ debug_panel::update_section_eventqueue( std::vector<text_line> &Output ) {
// current event queue
auto const time { Timer::GetTime() };
auto const *event { simulation::Events.begin() };
auto const searchfilter { std::string( m_eventsearch.data() ) };
// a copy in running order; when and by whom an event was queued belongs to the
// queue entry now, not to the event
auto const queue { simulation::Events.queue_snapshot() };
Output.emplace_back( "Delay: Event:", Global.UITextColor );
while( event != nullptr
&& Output.size() < 30 ) {
for( auto const &entry : queue ) {
if( Output.size() >= 30 ) { break; }
auto const *event { simulation::Events.event_at( entry.index ) };
if( event == nullptr ) { continue; }
if( false == event->m_ignored
&& false == event->m_passive
&& ( false == m_eventqueueactivevehicleonly
|| event->m_activator == m_input.vehicle ) ) {
|| entry.activator == m_input.vehicle ) ) {
auto const label { event->m_name + ( event->m_activator ? " (by: " + event->m_activator->asName + ")" : "" ) };
auto const label { event->m_name + ( entry.activator ? " (by: " + entry.activator->asName + ")" : "" ) };
if( false == searchfilter.empty()
&& false == contains(label, searchfilter) ) {
event = event->m_next;
continue;
}
auto const delay { " " + to_string( std::max( 0.0, event->m_launchtime - time ), 1 ) };
auto const delay { " " + to_string( std::max( 0.0, entry.launchtime - time ), 1 ) };
textline =
delay.substr( delay.length() - 6 )
+ " "
@@ -1397,11 +1411,10 @@ debug_panel::update_section_eventqueue( std::vector<text_line> &Output ) {
Output.emplace_back( textline, Global.UITextColor );
}
event = event->m_next;
}
if( Output.size() == 1 ) {
// event queue can be empty either because no event got through active filters, or because it is genuinely empty
Output.front().data = simulation::Events.begin() == nullptr ? "(no queued events)" : "(no matching events)";
Output.front().data = simulation::Events.queue_empty() ? "(no queued events)" : "(no matching events)";
}
}
@@ -1490,6 +1503,36 @@ debug_panel::update_section_renderer( std::vector<text_line> &Output ) {
Output.emplace_back( textline, Global.UITextColor );
// terrain streaming: how far tiles are loaded, how many are held, and how far
// behind the camera the loader is
for( auto const &field : TerrainStatus ) {
auto const &stats { field.stats };
auto const slash { field.path.find_last_of( "/\\" ) };
Output.emplace_back(
"Terrain: " + ( slash == std::string::npos ? field.path : field.path.substr( slash + 1 ) )
+ ", grid " + to_string( field.gridstep, 1 ) + " m, tile " + to_string( field.tilesize, 0 ) + " m"
+ ", loaded out to " + to_string( field.range, 0 ) + " m",
Global.UITextColor );
Output.emplace_back(
" tiles: " + std::to_string( stats.inview ) + " in view, "
+ std::to_string( stats.resident ) + " on gpu (" + std::to_string( stats.texturebytes / 1048576 ) + " MB), "
+ std::to_string( stats.wanted ) + " wanted, " + std::to_string( field.backlog ) + " in loader, "
+ std::to_string( stats.uploaded ) + " uploaded, " + std::to_string( stats.dropped ) + " dropped stale",
Global.UITextColor );
std::string levels { " levels:" };
for( std::uint32_t level = 0; ( level < field.levels ) && ( level < stats.perlevel.size() ); ++level ) {
auto const last { level + 1 == field.levels };
levels +=
" L" + std::to_string( level )
+ ( last ? " beyond " + to_string( terrain_level_distance( level > 0 ? level - 1 : 0 ), 0 )
: " <" + to_string( terrain_level_distance( level ), 0 ) )
+ " m (" + to_string( field.gridstep * static_cast<float>( 1u << level ), 0 ) + " m grid): "
+ std::to_string( stats.perlevel[ level ] );
if( false == last ) { levels += " |"; }
}
Output.emplace_back( levels, Global.UITextColor );
}
textline += "\nRendering mode: ";
if (Global.GfxRenderer == "default")

105
application/shotrunner.cpp Normal file
View File

@@ -0,0 +1,105 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <cstddef>
#include <string>
#include <vector>
module eu07.application.shotrunner;
import eu07.glm;
import eu07.utilities.logs;
import eu07.utilities.utilities;
shot_runner Shots;
bool
shot_runner::parse( std::string const &Text, shot &Result ) {
std::vector<double> fields;
for( auto const &field : Split( Text, ',' ) ) {
try {
fields.push_back( std::stod( field ) );
}
catch( ... ) {
return false;
}
}
if( fields.size() < 3 ) { return false; }
Result.position = { fields[ 0 ], fields[ 1 ], fields[ 2 ] };
// the free camera takes its angles as pitch, yaw, roll in radians; a viewpoint is
// written in degrees, because nobody wants to type radians on a command line
Result.angle = {
static_cast<float>( glm::radians( fields.size() > 4 ? fields[ 4 ] : 0.0 ) ),
static_cast<float>( glm::radians( fields.size() > 3 ? fields[ 3 ] : 0.0 ) ),
0.f };
Result.label = Text;
return true;
}
void
shot_runner::configure( std::vector<shot> Shots, double const Settle, bool const Quit ) {
m_shots = std::move( Shots );
m_settle = Settle;
m_quit = Quit;
m_index = 0;
m_stage = stage::placing;
m_elapsed = 0.0;
m_done = m_shots.empty();
}
shot_runner::action
shot_runner::update( double const Deltatime ) {
if( false == active() ) { return action::none; }
switch( m_stage ) {
case stage::placing: {
m_stage = stage::settling;
m_elapsed = 0.0;
WriteLog( "Shot " + std::to_string( m_index + 1 ) + "/" + std::to_string( m_shots.size() )
+ ": moving to " + current().label );
return action::place;
}
case stage::settling: {
m_elapsed += Deltatime;
if( m_elapsed < m_settle ) { return action::none; }
m_stage = stage::capturing;
return action::none;
}
case stage::capturing: {
m_stage = stage::advancing;
WriteLog( "Shot " + std::to_string( m_index + 1 ) + ": capturing" );
return action::capture;
}
case stage::advancing: {
// a frame's grace after the capture, since the image is read back and written
// on another thread and quitting immediately would cut it short
m_elapsed += Deltatime;
if( m_elapsed < m_settle + 1.0 ) { return action::none; }
++m_index;
if( m_index < m_shots.size() ) {
m_stage = stage::placing;
return action::none;
}
m_done = true;
WriteLog( "Shot sequence finished" );
return m_quit ? action::finish : action::none;
}
}
return action::none;
}

76
application/shotrunner.cppm Normal file
View File

@@ -0,0 +1,76 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
module;
#include <cstddef>
#include <string>
#include <vector>
export module eu07.application.shotrunner;
import eu07.glm;
export {
// Drives an unattended sequence of screenshots: place the camera, let the world settle,
// save a frame, move on, quit. Exists so that a visual change can be checked without a
// person sitting at the machine pressing keys, and checked the same way twice.
//
// The runner owns the sequence and its timing and nothing else. It does not know what a
// camera is or how a frame is saved: it reports what should happen next and the caller,
// which does know, carries it out.
class shot_runner {
public:
struct shot {
glm::dvec3 position { 0.0, 0.0, 0.0 };
glm::vec3 angle { 0.f, 0.f, 0.f }; // pitch, yaw, roll in degrees, as the free camera takes them
std::string label; // for the log, so a saved frame can be tied to a viewpoint
};
enum class action {
none, // nothing to do this frame
place, // move the camera to current()
capture, // save a frame
finish // the sequence is done
};
// parses a "x,y,z[,yaw[,pitch]]" viewpoint. returns false on anything malformed,
// leaving Result untouched
static bool parse( std::string const &Text, shot &Result );
void configure( std::vector<shot> Shots, double const Settle, bool const Quit );
bool active() const { return ( false == m_shots.empty() ) && ( false == m_done ); }
shot const &current() const { return m_shots[ m_index ]; }
// advances the sequence by one frame. Deltatime is wall time, so that the settling
// period covers tiles streaming in rather than a fixed number of frames
action update( double const Deltatime );
private:
enum class stage {
placing,
settling,
capturing,
advancing
};
std::vector<shot> m_shots;
std::size_t m_index { 0 };
stage m_stage { stage::placing };
double m_settle { 3.0 };
double m_elapsed { 0.0 };
bool m_quit { true };
bool m_done { false };
};
// the session's sequence, configured from the command line and driven by the driver mode.
// idle and harmless when no viewpoints were given
extern shot_runner Shots;
} // export