mirror of
https://github.com/MaSzyna-EU07/maszyna.git
synced 2026-11-07 18:04:47 +01:00
Stream the scenery: a baked terrain heightfield, a logic container, and prefabs
Today's work on the binary scene format plan (binary-scene-format-plan.md), from the first measurement to a terrain that bakes itself on first run and streams from a thread. The step-by-step history is kept on backup/scene-streaming-format-presquash-0916. Measuring first. -loadprofile times a scenery load by stage and by node kind and reports node counts and resident memory. On l204 terrain is 95% of the bytes and 99.8% of the nodes; on quarkmce2007 vehicles, not models, dominate what is left. The terrain turned out to be a heightfield in all but name, a ribbon covering 12% of its bounding box. The heightfield. scene/heightfieldformat.h is the .ehf layout: Morton-ordered tiles of 128 samples with one shared edge sample, a point-sampled mip chain, zstd per tile, a nodata sentinel, a material palette with a measured tiling per material, and a stamp of the rules that decided what counted as terrain. scene/heightfieldreader.h reads it for the tool, the renderer and the loader alike, and scene/terraincooker.h writes it, shared by the terraincook tool and the engine so a first-run bake is the file the tool would have made. The cooker rasterises every triangle where it is drawn (through scene::place_in_world, the placement the region applies too), keeps an upper surface laid within half a metre over another and the lower of two further apart, and then asks each triangle whether the heightfield shows it: laid, reaching a sample, keeping nearly all it claimed, and with the ground around it drawn at every level out to 2 km. Baking on first run. simulation/terrainbake bakes <scenario>.ehf the first time a scenery is opened, choosing the grid step from the first 200,000 vertices, and on later loads leaves out the triangles its node table says the heightfield shows. Everything else - trees, grass tufts, gantries, bridge decks - is drawn as written, so a bad bake can at worst draw something twice, and the heightfield sits 2 cm under the triangles so the original wins there. Nodes are keyed by file, line, include parameters and placement; an edited file changes its keys and those nodes are drawn again. A bake under older rules, or without its table, is thrown away and redone. A launcher copy named $scenario shares the scenario's files. -nobaketerrain shows the triangles as written. A scenery can still name hand-cooked fields, several at different steps. Drawing and streaming. rendering/terrainclipmap draws resident tiles at the mip level their distance earns, deriving the grid from gl_VertexID and texturing in world space with the scenery's own materials. rendering/terraintileloader reads and decodes tiles on a thread of its own; the renderer rescans what is in range only after moving a quarter of a tile, asks for what is missing nearest first, uploads at most 32 tiles a frame, drops arrivals the camera has moved past, and keeps drawing a tile at the level it has until the right one arrives. The terrain has its own range, terrainrange in the ini and a slider in the debug panel, 20 km by default and independent of basedrawrange; the far plane of an ordinary view is pushed out to it. The debug panel shows the range, tiles in view, on the gpu, wanted and queued, and how many sit at each level. Events. The event queue moved out of the events into a binary heap with a deterministic tie-break, and an event is handed its activator instead of storing it, so basic_event is a definition and nothing else. The logic container. scene/logicformat.h and scene/logiccontainer write a scenery's events, their targets and sibling chains as flat, deterministic arrays with a schema hash that refuses a mismatched build. simulation/logiccook cooks it from the loaded state whenever it is missing, stale or unreadable, reads it straight back to check it, and resolves each target to a position in the registry its event kind searches. -cooklogic forces a cook, -verifylogic only checks. Nothing reads it at load yet. Prefabs. scene/prefab describes a signal by the aspects it can show - lights, speeds, animation channels, a sound - instead of pasting an .inc per variant, and expands an instance to ordinary scenery text. Tried on a light signal and a shape signal. Screenshots without a person present: -shot x,y,z[,yaw[,pitch]], -shotsettle, -shotstay.
This commit is contained in:
maj00r
167
scene/heightfieldformat.h
Normal file
167
scene/heightfieldformat.h
Normal file
@@ -0,0 +1,167 @@
|
||||
/*
|
||||
This Source Code Form is subject to the
|
||||
terms of the Mozilla Public License, v.
|
||||
2.0. If a copy of the MPL was not
|
||||
distributed with this file, You can
|
||||
obtain one at
|
||||
http://mozilla.org/MPL/2.0/.
|
||||
*/
|
||||
|
||||
#pragma once
|
||||
|
||||
// On-disk layout of the cooked terrain heightfield. Single source of truth: the cooker
|
||||
// (tools/terraincook) writes it, the engine reads it, and neither owns a private copy.
|
||||
// Plain structs and free functions only - no engine types, no allocation, no i/o - so
|
||||
// that the tool keeps building without the engine and the format stays readable on its own.
|
||||
//
|
||||
// File layout:
|
||||
// header
|
||||
// tile directory, one entry per tile, ordered by morton code
|
||||
// material table, each entry a uint16 length, that many name bytes, and a float
|
||||
// giving how many metres of ground one repeat of the texture covers
|
||||
// tile payloads, at the offsets the directory gives
|
||||
//
|
||||
// A tile payload holds a mip chain, finest level first. Level l has
|
||||
// samples_at_level( tilesamples, l ) values along a side, laid out as that many rows of
|
||||
// uint16 heights, followed by the same grid of uint8 material indices.
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
namespace heightfield {
|
||||
|
||||
// "EU07HFLD", spelled out so the header needs no string functions
|
||||
inline constexpr char magic[ 8 ] { 'E', 'U', '0', '7', 'H', 'F', 'L', 'D' };
|
||||
inline constexpr std::uint32_t version { 5 };
|
||||
// terrain selection rules, stamped into every cooked file.
|
||||
// 0 - every triangle node the scenery holds
|
||||
// 1 - world space triangle nodes only: anything under origin, rotate or scale is an object
|
||||
// 2 - every triangle as drawn; a triangle stays drawn unless the heightfield shows it, and a
|
||||
// triangle bordering a hole always stays drawn
|
||||
// 3 - as 2, with a hole checked for diagonally and at the coarser levels too
|
||||
inline constexpr std::uint32_t selection_rules { 3 };
|
||||
|
||||
// height value reserved to mean "this sample carries no terrain"; a cooked map is a
|
||||
// ribbon inside a much larger rectangle, so most edge tiles are partly empty
|
||||
inline constexpr std::uint16_t nodata { 0xffff };
|
||||
// the range the remaining values span, shared by every tile through the header's bias
|
||||
// and scale. over l204's 107 m of relief this puts a sample within 2 mm of the source
|
||||
inline constexpr std::uint32_t heightrange { 0xfffe };
|
||||
|
||||
enum class codec : std::uint16_t {
|
||||
raw = 0,
|
||||
zstd = 1
|
||||
};
|
||||
|
||||
#pragma pack( push, 1 )
|
||||
|
||||
struct file_header {
|
||||
char magic[ 8 ];
|
||||
std::uint32_t version;
|
||||
// samples along a tile side, not counting the shared edge. a tile therefore stores
|
||||
// one more sample in each direction than it owns
|
||||
std::uint32_t tilesamples;
|
||||
float gridstep; // metres between samples at the finest level
|
||||
float heightbias; // world height of stored value 0
|
||||
float heightscale; // metres per stored unit
|
||||
std::int32_t minx, minz, maxx, maxz; // tile coordinates present in the file, inclusive
|
||||
std::uint32_t tilecount;
|
||||
std::uint32_t materialcount;
|
||||
std::uint32_t miplevels; // levels stored per tile, level 0 being the finest
|
||||
// which rules decided what counted as terrain when this was cooked. the layout does
|
||||
// not depend on it, so a file with an older value still reads; a heightfield the
|
||||
// engine baked by itself is rebaked when it disagrees, one named by the scenery is not
|
||||
std::uint32_t selection;
|
||||
std::uint32_t reserved;
|
||||
};
|
||||
|
||||
struct tile_entry {
|
||||
std::int32_t x, z;
|
||||
std::uint64_t offset; // payload position in the file
|
||||
std::uint32_t storedsize; // bytes occupied in the file
|
||||
std::uint32_t plainsize; // bytes once decoded; equals storedsize for codec::raw
|
||||
float minheight, maxheight;
|
||||
std::uint32_t coverage; // samples carrying terrain at the finest level
|
||||
std::uint16_t tilecodec; // codec enumerator
|
||||
std::uint16_t reserved;
|
||||
};
|
||||
|
||||
#pragma pack( pop )
|
||||
|
||||
static_assert( sizeof( file_header ) == 64 );
|
||||
static_assert( sizeof( tile_entry ) == 40 );
|
||||
|
||||
// identity of a tile in any map keyed by tile coordinates. the cooker, the reader and the
|
||||
// renderer all key tiles this way, so it is spelled out once
|
||||
constexpr std::int64_t
|
||||
tile_key( std::int32_t const X, std::int32_t const Z ) {
|
||||
return ( static_cast<std::int64_t>( X ) << 32 ) ^ static_cast<std::uint32_t>( Z );
|
||||
}
|
||||
|
||||
// integer division rounding towards negative infinity. tile and sample coordinates go
|
||||
// negative, and every place that maps one onto the other has to round the same way for
|
||||
// neighbouring tiles to share an edge exactly
|
||||
constexpr std::int64_t
|
||||
floor_div( std::int64_t const Value, std::int64_t const Divisor ) {
|
||||
auto const quotient { Value / Divisor };
|
||||
return ( ( Value % Divisor != 0 ) && ( ( Value < 0 ) != ( Divisor < 0 ) ) ) ? quotient - 1 : quotient;
|
||||
}
|
||||
|
||||
// samples along a tile side at the given mip level, shared edge included
|
||||
constexpr std::uint32_t
|
||||
samples_at_level( std::uint32_t const Tilesamples, std::uint32_t const Level ) {
|
||||
return ( Tilesamples >> Level ) + 1;
|
||||
}
|
||||
|
||||
// bytes one mip level occupies: a grid of heights followed by a grid of material indices
|
||||
constexpr std::uint64_t
|
||||
level_bytes( std::uint32_t const Tilesamples, std::uint32_t const Level ) {
|
||||
std::uint64_t const side { samples_at_level( Tilesamples, Level ) };
|
||||
return side * side * ( sizeof( std::uint16_t ) + sizeof( std::uint8_t ) );
|
||||
}
|
||||
|
||||
// bytes a decoded tile payload occupies, mip chain included
|
||||
constexpr std::uint64_t
|
||||
tile_bytes( std::uint32_t const Tilesamples, std::uint32_t const Miplevels ) {
|
||||
std::uint64_t total { 0 };
|
||||
for( std::uint32_t level = 0; level < Miplevels; ++level ) {
|
||||
total += level_bytes( Tilesamples, level );
|
||||
}
|
||||
return total;
|
||||
}
|
||||
|
||||
// byte offset of a mip level inside a decoded tile payload
|
||||
constexpr std::uint64_t
|
||||
level_offset( std::uint32_t const Tilesamples, std::uint32_t const Level ) {
|
||||
return tile_bytes( Tilesamples, Level );
|
||||
}
|
||||
|
||||
// deepest level that still has at least Minimumside samples along a side
|
||||
constexpr std::uint32_t
|
||||
level_count( std::uint32_t const Tilesamples, std::uint32_t const Minimumside ) {
|
||||
std::uint32_t levels { 1 };
|
||||
while( ( ( Tilesamples >> levels ) >= 1 )
|
||||
&& ( samples_at_level( Tilesamples, levels ) >= Minimumside ) ) {
|
||||
++levels;
|
||||
}
|
||||
return levels;
|
||||
}
|
||||
|
||||
// interleaves tile coordinates so that tiles sorted by this value are read in an order
|
||||
// that keeps a moving camera reading the file forwards rather than seeking
|
||||
constexpr std::uint64_t
|
||||
morton( std::int32_t const X, std::int32_t const Z ) {
|
||||
auto const spread = []( std::uint32_t Value ) constexpr -> std::uint64_t {
|
||||
std::uint64_t result { Value };
|
||||
result = ( result | ( result << 16 ) ) & 0x0000ffff0000ffffull;
|
||||
result = ( result | ( result << 8 ) ) & 0x00ff00ff00ff00ffull;
|
||||
result = ( result | ( result << 4 ) ) & 0x0f0f0f0f0f0f0f0full;
|
||||
result = ( result | ( result << 2 ) ) & 0x3333333333333333ull;
|
||||
result = ( result | ( result << 1 ) ) & 0x5555555555555555ull;
|
||||
return result; };
|
||||
// biased into unsigned so that negative tile coordinates keep their relative order
|
||||
std::uint32_t const x { static_cast<std::uint32_t>( X + ( 1 << 20 ) ) };
|
||||
std::uint32_t const z { static_cast<std::uint32_t>( Z + ( 1 << 20 ) ) };
|
||||
return spread( x ) | ( spread( z ) << 1 );
|
||||
}
|
||||
|
||||
} // namespace heightfield
|
||||
237
scene/heightfieldreader.h
Normal file
237
scene/heightfieldreader.h
Normal file
@@ -0,0 +1,237 @@
|
||||
/*
|
||||
This Source Code Form is subject to the
|
||||
terms of the Mozilla Public License, v.
|
||||
2.0. If a copy of the MPL was not
|
||||
distributed with this file, You can
|
||||
obtain one at
|
||||
http://mozilla.org/MPL/2.0/.
|
||||
*/
|
||||
|
||||
#pragma once
|
||||
|
||||
// Reader for the cooked terrain heightfield. Header-only and free of engine types, so
|
||||
// that the cooker can verify its own output with the same code the engine runs - one
|
||||
// reader, not two that drift.
|
||||
//
|
||||
// The class owns the file handle and the directory; sample data it decodes into buffers
|
||||
// the caller provides and recycles. Nothing about the on-disk layout escapes past
|
||||
// read_level(): callers see tile coordinates, mip levels and samples.
|
||||
|
||||
#include <cmath>
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
#include <fstream>
|
||||
#include <string>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
|
||||
#include <zstd.h>
|
||||
|
||||
#include "scene/heightfieldformat.h"
|
||||
|
||||
namespace heightfield {
|
||||
|
||||
class reader {
|
||||
|
||||
public:
|
||||
struct tile_description {
|
||||
std::int32_t x { 0 }, z { 0 };
|
||||
float minheight { 0.f }, maxheight { 0.f };
|
||||
std::uint32_t coverage { 0 }; // samples carrying terrain at the finest level
|
||||
};
|
||||
|
||||
reader() = default;
|
||||
~reader() { ZSTD_freeDCtx( m_context ); }
|
||||
reader( reader const & ) = delete;
|
||||
reader &operator=( reader const & ) = delete;
|
||||
|
||||
// returns false and leaves the reader closed when the file is missing, truncated,
|
||||
// or written by a version this build does not know
|
||||
bool
|
||||
open( std::string const &Path ) {
|
||||
|
||||
close();
|
||||
m_file.open( Path, std::ios::binary );
|
||||
if( false == m_file.good() ) { return false; }
|
||||
|
||||
if( false == read_at( 0, &m_header, sizeof( m_header ) ) ) { return close_failed(); }
|
||||
if( 0 != std::memcmp( m_header.magic, magic, sizeof( magic ) ) ) { return close_failed(); }
|
||||
if( m_header.version != version ) { return close_failed(); }
|
||||
if( ( m_header.tilesamples == 0 ) || ( m_header.miplevels == 0 ) ) { return close_failed(); }
|
||||
|
||||
std::vector<tile_entry> directory( m_header.tilecount );
|
||||
if( false == read_at(
|
||||
sizeof( file_header ), directory.data(),
|
||||
directory.size() * sizeof( tile_entry ) ) ) { return close_failed(); }
|
||||
|
||||
m_entries.reserve( directory.size() );
|
||||
m_index.reserve( directory.size() );
|
||||
m_descriptions.reserve( directory.size() );
|
||||
for( auto const &entry : directory ) {
|
||||
m_index.emplace( tile_key( entry.x, entry.z ), m_entries.size() );
|
||||
m_entries.push_back( entry );
|
||||
m_descriptions.push_back( { entry.x, entry.z, entry.minheight, entry.maxheight, entry.coverage } );
|
||||
}
|
||||
|
||||
auto offset { sizeof( file_header ) + directory.size() * sizeof( tile_entry ) };
|
||||
m_materials.reserve( m_header.materialcount );
|
||||
for( std::uint32_t index = 0; index < m_header.materialcount; ++index ) {
|
||||
std::uint16_t length { 0 };
|
||||
if( false == read_at( offset, &length, sizeof( length ) ) ) { return close_failed(); }
|
||||
offset += sizeof( length );
|
||||
std::string name( length, '\0' );
|
||||
if( ( length > 0 ) && ( false == read_at( offset, name.data(), length ) ) ) { return close_failed(); }
|
||||
offset += length;
|
||||
float scale { 4.f };
|
||||
if( false == read_at( offset, &scale, sizeof( scale ) ) ) { return close_failed(); }
|
||||
offset += sizeof( scale );
|
||||
m_materials.push_back( std::move( name ) );
|
||||
m_materialscales.push_back( scale );
|
||||
}
|
||||
|
||||
m_open = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
void
|
||||
close() {
|
||||
m_file.close();
|
||||
m_file.clear();
|
||||
m_entries.clear();
|
||||
m_descriptions.clear();
|
||||
m_index.clear();
|
||||
m_materials.clear();
|
||||
m_materialscales.clear();
|
||||
m_header = {};
|
||||
m_open = false; }
|
||||
|
||||
bool ready() const { return m_open; }
|
||||
|
||||
// metres between samples at the finest level
|
||||
float gridstep() const { return m_header.gridstep; }
|
||||
// samples a tile owns along a side, shared edge excluded
|
||||
std::uint32_t tilesamples() const { return m_header.tilesamples; }
|
||||
// side of a tile in metres
|
||||
float tilesize() const { return m_header.gridstep * static_cast<float>( m_header.tilesamples ); }
|
||||
std::uint32_t levels() const { return m_header.miplevels; }
|
||||
std::uint32_t samples_per_side( std::uint32_t const Level ) const {
|
||||
return samples_at_level( m_header.tilesamples, Level ); }
|
||||
|
||||
std::vector<tile_description> const &tiles() const { return m_descriptions; }
|
||||
std::vector<std::string> const &materials() const { return m_materials; }
|
||||
// metres of ground covered by one repeat of a material's texture, measured by the
|
||||
// cooker from the texture coordinates the source geometry carried
|
||||
std::vector<float> const &material_scales() const { return m_materialscales; }
|
||||
|
||||
// tile coordinates covering a world position
|
||||
std::int32_t tile_x( double const X ) const { return tile_of( X ); }
|
||||
std::int32_t tile_z( double const Z ) const { return tile_of( Z ); }
|
||||
|
||||
// the terrain selection rules the file was cooked under
|
||||
std::uint32_t
|
||||
selection() const { return m_header.selection; }
|
||||
|
||||
bool
|
||||
contains( std::int32_t const X, std::int32_t const Z ) const {
|
||||
return m_index.find( tile_key( X, Z ) ) != m_index.end(); }
|
||||
|
||||
// description of one tile, or nullptr when the file does not hold it
|
||||
tile_description const *
|
||||
find( std::int32_t const X, std::int32_t const Z ) const {
|
||||
auto const lookup { m_index.find( tile_key( X, Z ) ) };
|
||||
return lookup == m_index.end() ? nullptr : &m_descriptions[ lookup->second ]; }
|
||||
|
||||
// world height of a stored sample; Value must not be nodata
|
||||
float
|
||||
height( std::uint16_t const Value ) const {
|
||||
return m_header.heightbias + static_cast<float>( Value ) * m_header.heightscale; }
|
||||
|
||||
// decodes one mip level of one tile into the caller's buffers, which are resized to
|
||||
// the level's sample count. returns false when the tile is absent or the payload is
|
||||
// damaged; the buffers are then left untouched
|
||||
bool
|
||||
read_level(
|
||||
std::int32_t const X, std::int32_t const Z, std::uint32_t const Level,
|
||||
std::vector<std::uint16_t> &Heights, std::vector<std::uint8_t> &Materials ) {
|
||||
|
||||
if( false == m_open ) { return false; }
|
||||
if( Level >= m_header.miplevels ) { return false; }
|
||||
auto const lookup { m_index.find( tile_key( X, Z ) ) };
|
||||
if( lookup == m_index.end() ) { return false; }
|
||||
auto const &entry { m_entries[ lookup->second ] };
|
||||
|
||||
if( false == decode( entry ) ) { return false; }
|
||||
|
||||
auto const side { samples_at_level( m_header.tilesamples, Level ) };
|
||||
auto const samples { static_cast<std::size_t>( side ) * side };
|
||||
auto const offset { static_cast<std::size_t>( level_offset( m_header.tilesamples, Level ) ) };
|
||||
if( offset + samples * 3u > m_plain.size() ) { return false; }
|
||||
|
||||
Heights.resize( samples );
|
||||
Materials.resize( samples );
|
||||
std::memcpy( Heights.data(), m_plain.data() + offset, samples * sizeof( std::uint16_t ) );
|
||||
std::memcpy( Materials.data(), m_plain.data() + offset + samples * sizeof( std::uint16_t ), samples );
|
||||
return true;
|
||||
}
|
||||
|
||||
private:
|
||||
std::int32_t
|
||||
tile_of( double const World ) const {
|
||||
return static_cast<std::int32_t>( std::floor( World / static_cast<double>( tilesize() ) ) ); }
|
||||
|
||||
bool
|
||||
read_at( std::uint64_t const Offset, void *Target, std::size_t const Bytes ) {
|
||||
if( Bytes == 0 ) { return true; }
|
||||
m_file.seekg( static_cast<std::streamoff>( Offset ), std::ios::beg );
|
||||
m_file.read( static_cast<char *>( Target ), static_cast<std::streamsize>( Bytes ) );
|
||||
return m_file.good(); }
|
||||
|
||||
bool
|
||||
close_failed() {
|
||||
close();
|
||||
return false; }
|
||||
|
||||
// keeps the most recently decoded payload, because a caller walking the mip chain of
|
||||
// one tile would otherwise decompress it once per level
|
||||
bool
|
||||
decode( tile_entry const &Entry ) {
|
||||
|
||||
if( ( true == m_decoded ) && ( m_decodedoffset == Entry.offset ) ) { return true; }
|
||||
m_decoded = false;
|
||||
|
||||
m_stored.resize( Entry.storedsize );
|
||||
if( false == read_at( Entry.offset, m_stored.data(), Entry.storedsize ) ) { return false; }
|
||||
|
||||
if( static_cast<codec>( Entry.tilecodec ) == codec::raw ) {
|
||||
m_plain.swap( m_stored );
|
||||
}
|
||||
else {
|
||||
m_plain.resize( Entry.plainsize );
|
||||
// one context for the life of the reader: creating one per tile costs more than
|
||||
// decoding a small tile does
|
||||
if( m_context == nullptr ) { m_context = ZSTD_createDCtx(); }
|
||||
auto const written {
|
||||
ZSTD_decompressDCtx( m_context, m_plain.data(), m_plain.size(), m_stored.data(), m_stored.size() ) };
|
||||
if( ( 0 != ZSTD_isError( written ) ) || ( written != Entry.plainsize ) ) { return false; }
|
||||
}
|
||||
|
||||
m_decoded = true;
|
||||
m_decodedoffset = Entry.offset;
|
||||
return true;
|
||||
}
|
||||
|
||||
std::ifstream m_file;
|
||||
ZSTD_DCtx *m_context { nullptr };
|
||||
file_header m_header {};
|
||||
std::vector<tile_entry> m_entries;
|
||||
std::vector<tile_description> m_descriptions;
|
||||
std::unordered_map<std::int64_t, std::size_t> m_index;
|
||||
std::vector<std::string> m_materials;
|
||||
std::vector<float> m_materialscales;
|
||||
std::vector<std::uint8_t> m_stored, m_plain;
|
||||
std::uint64_t m_decodedoffset { 0 };
|
||||
bool m_decoded { false };
|
||||
bool m_open { false };
|
||||
};
|
||||
|
||||
} // namespace heightfield
|
||||
283
scene/logiccontainer.cpp
Normal file
283
scene/logiccontainer.cpp
Normal file
@@ -0,0 +1,283 @@
|
||||
/*
|
||||
This Source Code Form is subject to the
|
||||
terms of the Mozilla Public License, v.
|
||||
2.0. If a copy of the MPL was not
|
||||
distributed with this file, You can
|
||||
obtain one at
|
||||
http://mozilla.org/MPL/2.0/.
|
||||
*/
|
||||
|
||||
module;
|
||||
#include <array>
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
#include <fstream>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
#include <vector>
|
||||
#include "scene/logicformat.h"
|
||||
|
||||
module eu07.scene.logiccontainer;
|
||||
import eu07.utilities.logs;
|
||||
import eu07.utilities.utilities;
|
||||
|
||||
namespace scene::logic {
|
||||
|
||||
namespace {
|
||||
|
||||
// pads the stream out to the next section boundary with zeroes, so a cooked file has no
|
||||
// uninitialised bytes in it and two cooks of the same scenery compare equal
|
||||
void
|
||||
pad( std::ofstream &Output ) {
|
||||
|
||||
std::array<char, alignment> zeroes {};
|
||||
auto const position { static_cast<std::size_t>( Output.tellp() ) };
|
||||
auto const padding { aligned( position ) - position };
|
||||
if( padding > 0 ) {
|
||||
Output.write( zeroes.data(), static_cast<std::streamsize>( padding ) );
|
||||
}
|
||||
}
|
||||
|
||||
template <typename Type_>
|
||||
void
|
||||
write_array( std::ofstream &Output, std::vector<Type_> const &Data ) {
|
||||
|
||||
if( false == Data.empty() ) {
|
||||
Output.write( reinterpret_cast<char const *>( Data.data() ), static_cast<std::streamsize>( Data.size() * sizeof( Type_ ) ) );
|
||||
}
|
||||
}
|
||||
|
||||
template <typename Type_>
|
||||
bool
|
||||
read_array( std::ifstream &Input, section_entry const &Section, std::vector<Type_> &Data ) {
|
||||
|
||||
if( Section.elementsize != sizeof( Type_ ) ) { return false; }
|
||||
if( Section.bytes != Section.count * sizeof( Type_ ) ) { return false; }
|
||||
Data.resize( Section.count );
|
||||
Input.seekg( static_cast<std::streamoff>( Section.offset ) );
|
||||
if( Section.count > 0 ) {
|
||||
Input.read( reinterpret_cast<char *>( Data.data() ), static_cast<std::streamsize>( Section.bytes ) );
|
||||
}
|
||||
return static_cast<bool>( Input );
|
||||
}
|
||||
|
||||
} // anonymous namespace
|
||||
|
||||
std::uint32_t
|
||||
container_writer::intern( std::string const &Text ) {
|
||||
|
||||
auto const lookup { m_interned.find( Text ) };
|
||||
if( lookup != m_interned.end() ) {
|
||||
return lookup->second;
|
||||
}
|
||||
auto const index { static_cast<std::uint32_t>( m_strings.size() ) };
|
||||
m_strings.emplace_back( Text );
|
||||
m_interned.emplace( Text, index );
|
||||
return index;
|
||||
}
|
||||
|
||||
void
|
||||
container_writer::add_event( event_record Record, std::vector<target_entry> const &Targets ) {
|
||||
|
||||
Record.targetfirst = static_cast<std::uint32_t>( m_targets.size() );
|
||||
Record.targetcount = static_cast<std::uint32_t>( Targets.size() );
|
||||
m_targets.insert( m_targets.end(), Targets.begin(), Targets.end() );
|
||||
m_events.emplace_back( Record );
|
||||
}
|
||||
|
||||
bool
|
||||
container_writer::write( std::string const &Path ) const {
|
||||
|
||||
std::ofstream output( Path, std::ios::binary | std::ios::trunc );
|
||||
if( false == output.good() ) {
|
||||
ErrorLog( "Logic container: cannot write \"" + Path + "\"" );
|
||||
return false;
|
||||
}
|
||||
|
||||
// the strings blob and its table are built first, since the header has to know how big
|
||||
// every section is before any of them is written
|
||||
std::vector<char> blob;
|
||||
std::vector<string_entry> table;
|
||||
table.reserve( m_strings.size() );
|
||||
for( auto const &text : m_strings ) {
|
||||
table.emplace_back( string_entry{ static_cast<std::uint32_t>( blob.size() ), static_cast<std::uint32_t>( text.size() ) } );
|
||||
blob.insert( blob.end(), text.begin(), text.end() );
|
||||
}
|
||||
|
||||
std::array<section_entry, static_cast<std::size_t>( section::count )> sections {};
|
||||
auto offset { aligned( sizeof( file_header ) + sizeof( sections ) ) };
|
||||
|
||||
auto const place {
|
||||
[ &offset, §ions ]( section const Kind, std::size_t const Elementsize, std::size_t const Count ) {
|
||||
auto &entry { sections[ static_cast<std::size_t>( Kind ) ] };
|
||||
entry.kind = static_cast<std::uint32_t>( Kind );
|
||||
entry.elementsize = static_cast<std::uint32_t>( Elementsize );
|
||||
entry.count = Count;
|
||||
entry.bytes = Elementsize * Count;
|
||||
entry.offset = offset;
|
||||
offset = aligned( offset + entry.bytes );
|
||||
} };
|
||||
|
||||
place( section::strings, 1, blob.size() );
|
||||
place( section::stringtable, sizeof( string_entry ), table.size() );
|
||||
place( section::events, sizeof( event_record ), m_events.size() );
|
||||
place( section::targets, sizeof( target_entry ), m_targets.size() );
|
||||
|
||||
file_header header {};
|
||||
header.magic = magic;
|
||||
header.version = version;
|
||||
header.schema_hash = schema_hash;
|
||||
header.endian = endian_marker;
|
||||
header.sectioncount = static_cast<std::uint32_t>( section::count );
|
||||
header.filesize = offset;
|
||||
|
||||
output.write( reinterpret_cast<char const *>( &header ), sizeof( header ) );
|
||||
output.write( reinterpret_cast<char const *>( sections.data() ), sizeof( sections ) );
|
||||
pad( output );
|
||||
write_array( output, blob );
|
||||
pad( output );
|
||||
write_array( output, table );
|
||||
pad( output );
|
||||
write_array( output, m_events );
|
||||
pad( output );
|
||||
write_array( output, m_targets );
|
||||
pad( output );
|
||||
|
||||
if( false == output.good() ) {
|
||||
ErrorLog( "Logic container: failed while writing \"" + Path + "\"" );
|
||||
return false;
|
||||
}
|
||||
WriteLog(
|
||||
"Logic container: wrote \"" + Path + "\", "
|
||||
+ std::to_string( m_events.size() ) + " events, "
|
||||
+ std::to_string( m_targets.size() ) + " targets, "
|
||||
+ std::to_string( m_strings.size() ) + " strings, "
|
||||
+ std::to_string( offset ) + " bytes" );
|
||||
return true;
|
||||
}
|
||||
|
||||
// reads the header and checks it, and the file's size, against what this build writes. a
|
||||
// cooked container is a cache, so anything unexpected means fall back to the text, not guess
|
||||
// at what an older writer meant
|
||||
static bool
|
||||
accepted( std::ifstream &Input, std::string const &Path ) {
|
||||
|
||||
if( false == Input.good() ) { return false; }
|
||||
|
||||
file_header header {};
|
||||
Input.read( reinterpret_cast<char *>( &header ), sizeof( header ) );
|
||||
if( false == static_cast<bool>( Input ) ) { return false; }
|
||||
|
||||
if( header.magic != magic ) {
|
||||
ErrorLog( "Logic container \"" + Path + "\" is not one" );
|
||||
return false;
|
||||
}
|
||||
if( ( header.version != version ) || ( header.schema_hash != schema_hash ) ) {
|
||||
WriteLog( "Logic container \"" + Path + "\" was cooked by a different build, ignoring it" );
|
||||
return false;
|
||||
}
|
||||
if( header.endian != endian_marker ) {
|
||||
ErrorLog( "Logic container \"" + Path + "\" was cooked on the other endianness" );
|
||||
return false;
|
||||
}
|
||||
if( header.sectioncount != static_cast<std::uint32_t>( section::count ) ) {
|
||||
WriteLog( "Logic container \"" + Path + "\" holds a different set of sections, ignoring it" );
|
||||
return false;
|
||||
}
|
||||
Input.seekg( 0, std::ios::end );
|
||||
if( static_cast<std::uint64_t>( Input.tellg() ) != header.filesize ) {
|
||||
ErrorLog( "Logic container \"" + Path + "\" is truncated" );
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool
|
||||
container_reader::readable( std::string const &Path ) {
|
||||
|
||||
std::ifstream input( Path, std::ios::binary );
|
||||
return accepted( input, Path );
|
||||
}
|
||||
|
||||
bool
|
||||
container_reader::open( std::string const &Path ) {
|
||||
|
||||
close();
|
||||
|
||||
std::ifstream input( Path, std::ios::binary );
|
||||
if( false == accepted( input, Path ) ) { return false; }
|
||||
input.seekg( sizeof( file_header ) );
|
||||
|
||||
std::array<section_entry, static_cast<std::size_t>( section::count )> sections {};
|
||||
input.read( reinterpret_cast<char *>( sections.data() ), sizeof( sections ) );
|
||||
if( false == static_cast<bool>( input ) ) { return false; }
|
||||
|
||||
auto const &strings { sections[ static_cast<std::size_t>( section::strings ) ] };
|
||||
m_strings.resize( strings.count );
|
||||
input.seekg( static_cast<std::streamoff>( strings.offset ) );
|
||||
if( strings.count > 0 ) {
|
||||
input.read( m_strings.data(), static_cast<std::streamsize>( strings.bytes ) );
|
||||
}
|
||||
|
||||
if( ( false == read_array( input, sections[ static_cast<std::size_t>( section::stringtable ) ], m_stringtable ) )
|
||||
|| ( false == read_array( input, sections[ static_cast<std::size_t>( section::events ) ], m_events ) )
|
||||
|| ( false == read_array( input, sections[ static_cast<std::size_t>( section::targets ) ], m_targets ) ) ) {
|
||||
ErrorLog( "Logic container \"" + Path + "\" has a section this build cannot read" );
|
||||
close();
|
||||
return false;
|
||||
}
|
||||
|
||||
m_ready = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
void
|
||||
container_reader::close() {
|
||||
|
||||
m_ready = false;
|
||||
m_strings.clear();
|
||||
m_stringtable.clear();
|
||||
m_events.clear();
|
||||
m_targets.clear();
|
||||
}
|
||||
|
||||
std::string_view
|
||||
container_reader::string( std::uint32_t const Index ) const {
|
||||
|
||||
if( Index >= m_stringtable.size() ) { return {}; }
|
||||
auto const &entry { m_stringtable[ Index ] };
|
||||
return { m_strings.data() + entry.offset, entry.length };
|
||||
}
|
||||
|
||||
std::string
|
||||
container_path( std::string const &Sceneryfile ) {
|
||||
|
||||
return scenery_sidecar( Sceneryfile, ".logic" );
|
||||
}
|
||||
|
||||
event_kind
|
||||
kind_of( std::string const &Type ) {
|
||||
|
||||
if( Type == "updatevalues" ) { return event_kind::updatevalues; }
|
||||
else if( Type == "addvalues" ) { return event_kind::addvalues; }
|
||||
else if( Type == "copyvalues" ) { return event_kind::copyvalues; }
|
||||
else if( Type == "getvalues" ) { return event_kind::getvalues; }
|
||||
else if( Type == "putvalues" ) { return event_kind::putvalues; }
|
||||
else if( Type == "whois" ) { return event_kind::whois; }
|
||||
else if( Type == "logvalues" ) { return event_kind::logvalues; }
|
||||
else if( Type == "multiple" ) { return event_kind::multiple; }
|
||||
else if( Type == "switch" ) { return event_kind::switchevent; }
|
||||
else if( Type == "trackvel" ) { return event_kind::trackvel; }
|
||||
else if( Type == "sound" ) { return event_kind::sound; }
|
||||
else if( Type == "texture" ) { return event_kind::texture; }
|
||||
else if( Type == "animation" ) { return event_kind::animation; }
|
||||
else if( Type == "lights" ) { return event_kind::lights; }
|
||||
else if( Type == "voltage" ) { return event_kind::voltage; }
|
||||
else if( Type == "visible" ) { return event_kind::visible; }
|
||||
else if( Type == "friction" ) { return event_kind::friction; }
|
||||
else if( Type == "message" ) { return event_kind::message; }
|
||||
else if( Type == "lua" ) { return event_kind::lua; }
|
||||
return event_kind::unknown;
|
||||
}
|
||||
|
||||
} // namespace scene::logic
|
||||
84
scene/logiccontainer.cppm
Normal file
84
scene/logiccontainer.cppm
Normal file
@@ -0,0 +1,84 @@
|
||||
/*
|
||||
This Source Code Form is subject to the
|
||||
terms of the Mozilla Public License, v.
|
||||
2.0. If a copy of the MPL was not
|
||||
distributed with this file, You can
|
||||
obtain one at
|
||||
http://mozilla.org/MPL/2.0/.
|
||||
*/
|
||||
|
||||
module;
|
||||
#include <cstdint>
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
#include <vector>
|
||||
#include "scene/logicformat.h"
|
||||
|
||||
export module eu07.scene.logiccontainer;
|
||||
|
||||
export {
|
||||
|
||||
// Writing and reading the LOGIC container. The layout it produces is described entirely by
|
||||
// logicformat.h; this is the part that knows how to get bytes in and out of it.
|
||||
//
|
||||
// The writer is deterministic: the same scenery cooked twice produces the same file, byte for
|
||||
// byte. Strings are interned in first-seen order rather than by a hash, records are written in
|
||||
// the order the manager holds them, and the padding between sections is zeroed. That is worth
|
||||
// the small cost, because it is what makes a cooked file comparable - a diff that is empty
|
||||
// proves a refactor changed nothing.
|
||||
namespace scene::logic {
|
||||
|
||||
class container_writer {
|
||||
|
||||
public:
|
||||
// interns a string, returning its index. the same text always gets the same index
|
||||
std::uint32_t intern( std::string const &Text );
|
||||
// adds one event and its targets, already resolved to the registry that holds them
|
||||
void add_event( event_record Record, std::vector<target_entry> const &Targets );
|
||||
// writes the container. returns false and logs on any failure
|
||||
bool write( std::string const &Path ) const;
|
||||
|
||||
std::size_t events() const { return m_events.size(); }
|
||||
|
||||
private:
|
||||
std::vector<std::string> m_strings;
|
||||
std::map<std::string, std::uint32_t> m_interned;
|
||||
std::vector<event_record> m_events;
|
||||
std::vector<target_entry> m_targets;
|
||||
};
|
||||
|
||||
class container_reader {
|
||||
|
||||
public:
|
||||
// reads the whole container into memory. refuses anything whose magic, version, schema
|
||||
// hash, endianness or size disagrees with what this build expects
|
||||
bool open( std::string const &Path );
|
||||
// whether this build can read the container, judged from its header and size alone
|
||||
static bool readable( std::string const &Path );
|
||||
void close();
|
||||
bool ready() const { return m_ready; }
|
||||
|
||||
std::size_t events() const { return m_events.size(); }
|
||||
event_record const & event( std::size_t const Index ) const { return m_events[ Index ]; }
|
||||
std::size_t targets() const { return m_targets.size(); }
|
||||
target_entry const & target( std::size_t const Index ) const { return m_targets[ Index ]; }
|
||||
// a view into the string blob, valid while the reader is open
|
||||
std::string_view string( std::uint32_t const Index ) const;
|
||||
|
||||
private:
|
||||
bool m_ready { false };
|
||||
std::vector<char> m_strings;
|
||||
std::vector<string_entry> m_stringtable;
|
||||
std::vector<event_record> m_events;
|
||||
std::vector<target_entry> m_targets;
|
||||
};
|
||||
|
||||
// the container's name for a scenery file: alongside it, with the extension replaced
|
||||
std::string container_path( std::string const &Sceneryfile );
|
||||
// maps the word an event writes as its type onto the cooked kind
|
||||
event_kind kind_of( std::string const &Type );
|
||||
|
||||
} // namespace scene::logic
|
||||
|
||||
}
|
||||
172
scene/logicformat.h
Normal file
172
scene/logicformat.h
Normal file
@@ -0,0 +1,172 @@
|
||||
/*
|
||||
This Source Code Form is subject to the
|
||||
terms of the Mozilla Public License, v.
|
||||
2.0. If a copy of the MPL was not
|
||||
distributed with this file, You can
|
||||
obtain one at
|
||||
http://mozilla.org/MPL/2.0/.
|
||||
*/
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
|
||||
// On-disk layout of the scenery's LOGIC container. Single source of truth for both the
|
||||
// cooker and the engine, so it deliberately knows nothing about either: no engine types, no
|
||||
// includes beyond the fixed-width integers.
|
||||
//
|
||||
// The container holds what the simulation needs before it can run - events, and the tables
|
||||
// that bind them to each other - as flat arrays indexed by position. Nothing here is a
|
||||
// pointer and nothing is looked up by name at load: an event's targets are a range in a
|
||||
// shared array, and a sibling is an index. That is the whole point of the section. The text
|
||||
// path resolves those names in init_targets() every time a scenery loads, and it is the part
|
||||
// of the load that cannot be made cheaper while the names are still there.
|
||||
//
|
||||
// Refusing to load is the only migration story. A container whose magic, version or schema
|
||||
// hash disagrees with this header is rejected and the scenery falls back to the text path,
|
||||
// because a cooked file is a cache and can always be rebuilt.
|
||||
namespace scene::logic {
|
||||
|
||||
// "EU07LOGC", little-endian, so a file opened in a hex editor names itself
|
||||
inline constexpr std::uint64_t magic { 0x43474f4c37305545ull };
|
||||
inline constexpr std::uint32_t version { 1 };
|
||||
// stamped into the header and compared on load. bump it whenever a record below changes
|
||||
// shape, so a stale container is refused rather than misread
|
||||
inline constexpr std::uint32_t schema_hash { 0x2f74be09u };
|
||||
// written as 1; a reader seeing 0x01000000 knows it is looking at the wrong endianness
|
||||
inline constexpr std::uint32_t endian_marker { 1 };
|
||||
// every section starts on this boundary, so records can be read in place
|
||||
inline constexpr std::size_t alignment { 16 };
|
||||
// index meaning "no such element", used where a record refers to another and need not
|
||||
inline constexpr std::uint32_t no_index { 0xffffffffu };
|
||||
|
||||
enum class section : std::uint32_t {
|
||||
strings = 0, // the string blob every name indexes into
|
||||
stringtable = 1, // one string_entry per interned string
|
||||
events = 2, // one event_record per event, in the order the manager holds them
|
||||
targets = 3, // target_entry array; an event owns a contiguous run of it
|
||||
count
|
||||
};
|
||||
|
||||
struct file_header {
|
||||
std::uint64_t magic;
|
||||
std::uint32_t version;
|
||||
std::uint32_t schema_hash;
|
||||
std::uint32_t endian;
|
||||
std::uint32_t sectioncount;
|
||||
// total size of the file as written, so a truncated container is caught before it is read
|
||||
std::uint64_t filesize;
|
||||
std::uint64_t reserved[ 3 ];
|
||||
};
|
||||
|
||||
struct section_entry {
|
||||
std::uint32_t kind;
|
||||
std::uint32_t elementsize;
|
||||
std::uint64_t offset;
|
||||
std::uint64_t bytes;
|
||||
std::uint64_t count;
|
||||
};
|
||||
|
||||
// one interned string: where it starts in the strings blob and how long it is. strings are
|
||||
// not terminated, so a reader hands out views straight into the mapped buffer
|
||||
struct string_entry {
|
||||
std::uint32_t offset;
|
||||
std::uint32_t length;
|
||||
};
|
||||
|
||||
// what an event does. the values are written to disk, so they are assigned explicitly and
|
||||
// never reordered - a new kind goes on the end
|
||||
enum class event_kind : std::uint16_t {
|
||||
unknown = 0,
|
||||
updatevalues = 1,
|
||||
copyvalues = 2,
|
||||
getvalues = 3,
|
||||
putvalues = 4,
|
||||
whois = 5,
|
||||
logvalues = 6,
|
||||
multiple = 7,
|
||||
switchevent = 8, // written "switch" in the scenery, which is a keyword here
|
||||
trackvel = 9,
|
||||
sound = 10,
|
||||
texture = 11,
|
||||
animation = 12,
|
||||
lights = 13,
|
||||
voltage = 14,
|
||||
visible = 15,
|
||||
friction = 16,
|
||||
message = 17,
|
||||
lua = 18,
|
||||
// the same class as updatevalues, told apart by a flag the base does not hold. it gets
|
||||
// its own value rather than being cooked as updatevalues, because until the flag word is
|
||||
// cooked too that is the only thing keeping the two apart
|
||||
addvalues = 19,
|
||||
};
|
||||
|
||||
// the part of an event that every kind has. what the kind itself needs beyond this lives in
|
||||
// the payload range, which the cooker writes and the matching subclass reads back
|
||||
struct event_record {
|
||||
std::uint16_t kind;
|
||||
// the per-kind flag word lives in the subclasses, not in the base, so it is cooked with
|
||||
// the rest of the payload once the sections for it exist. zero until then
|
||||
std::uint16_t reserved0;
|
||||
std::uint32_t name; // index into the string table
|
||||
std::uint32_t targetfirst; // first entry in the targets section
|
||||
std::uint32_t targetcount;
|
||||
std::uint32_t sibling; // index of the next event sharing this name, or no_index
|
||||
std::uint32_t group; // scene group handle as cooked, or no_index
|
||||
double delay;
|
||||
double delayrandom;
|
||||
// NaN when the event is not tied to a departure. written as the exact bit pattern the
|
||||
// text path produces, so a round trip compares equal bit for bit
|
||||
double delaydeparture;
|
||||
std::uint8_t ignored;
|
||||
std::uint8_t passive;
|
||||
std::uint8_t padding[ 6 ];
|
||||
};
|
||||
|
||||
// one target of one event: the name it was written with, and the index of whatever it
|
||||
// resolved to. the name is kept because the editor exports text again, and because an
|
||||
// unresolved target has to be reported by the name the author wrote
|
||||
// which of the engine's node registries a target was found in. an event resolves its
|
||||
// targets against one or two of them, decided by what kind of event it is - and for a
|
||||
// visible event, decided per target, since it looks in the instances first and the tracks
|
||||
// after. the values are written to disk, so they are assigned explicitly
|
||||
enum class node_table : std::uint16_t {
|
||||
none = 0,
|
||||
memory = 1,
|
||||
instance = 2,
|
||||
path = 3,
|
||||
sound = 4,
|
||||
powergrid = 5,
|
||||
};
|
||||
|
||||
// one target of one event: which registry held it and where in that registry, plus the
|
||||
// name it was written with.
|
||||
//
|
||||
// The index is the target's position in that registry, which is the order the scenery
|
||||
// built it in - so it is only meaningful for the parse this container was cooked from.
|
||||
// The name is kept as the check on that: a loader compares it against the node the index
|
||||
// lands on and falls back to a lookup when they disagree, which is what stops a container
|
||||
// that went stale without being noticed from binding events to the wrong nodes.
|
||||
struct target_entry {
|
||||
std::uint32_t name;
|
||||
std::uint16_t table;
|
||||
std::uint16_t reserved;
|
||||
std::uint32_t index;
|
||||
};
|
||||
static_assert( sizeof( file_header ) == 56, "logic file_header changed shape" );
|
||||
static_assert( sizeof( section_entry ) == 32, "logic section_entry changed shape" );
|
||||
static_assert( sizeof( string_entry ) == 8, "logic string_entry changed shape" );
|
||||
static_assert( sizeof( event_record ) == 56, "logic event_record changed shape" );
|
||||
static_assert( sizeof( target_entry ) == 12, "logic target_entry changed shape" );
|
||||
static_assert( alignof( event_record ) <= alignment, "logic records must fit the section alignment" );
|
||||
|
||||
// where a section's data starts, given what has been written so far
|
||||
inline constexpr std::size_t
|
||||
aligned( std::size_t const Offset ) {
|
||||
|
||||
return ( Offset + alignment - 1 ) & ~( alignment - 1 );
|
||||
}
|
||||
|
||||
} // namespace scene::logic
|
||||
278
scene/prefab.cpp
Normal file
278
scene/prefab.cpp
Normal file
@@ -0,0 +1,278 @@
|
||||
/*
|
||||
This Source Code Form is subject to the
|
||||
terms of the Mozilla Public License, v.
|
||||
2.0. If a copy of the MPL was not
|
||||
distributed with this file, You can
|
||||
obtain one at
|
||||
http://mozilla.org/MPL/2.0/.
|
||||
*/
|
||||
|
||||
module;
|
||||
#include <sstream>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
#include "utilities/Globals_macros.h"
|
||||
|
||||
module eu07.scene.prefab;
|
||||
import eu07.utilities.parser;
|
||||
import eu07.utilities.logs;
|
||||
import eu07.utilities.globals;
|
||||
|
||||
namespace scene {
|
||||
|
||||
namespace {
|
||||
|
||||
// a light pattern as a lights event writes it: one value per channel, each preceded by a
|
||||
// space, channels the given values do not reach set to Fill
|
||||
std::string
|
||||
pattern( std::vector<int> const &Lights, std::size_t const Chambers, int const Fill = 0 ) {
|
||||
|
||||
std::string result;
|
||||
for( std::size_t chamber = 0; chamber < Chambers; ++chamber ) {
|
||||
result += ' ' + std::to_string( chamber < Lights.size() ? Lights[ chamber ] : Fill );
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
void
|
||||
parse_aspect( cParser &Input, prefab_definition const &Definition, prefab_aspect &Aspect ) {
|
||||
|
||||
auto token { Input.getToken<std::string>() };
|
||||
while( ( false == token.empty() ) && ( token != "endaspect" ) ) {
|
||||
|
||||
if( token == "lights" ) {
|
||||
for( std::size_t chamber = 0; chamber < Definition.chambers; ++chamber ) {
|
||||
Aspect.lights.emplace_back( Input.getToken<int>() );
|
||||
}
|
||||
}
|
||||
else if( token == "lightdelay" ) {
|
||||
Aspect.lightdelay = Input.getToken<std::string>();
|
||||
}
|
||||
else if( token == "velocity" ) {
|
||||
Aspect.velocity = Input.getToken<std::string>();
|
||||
Aspect.velocitynext = Input.getToken<std::string>();
|
||||
}
|
||||
else if( token == "animate" ) {
|
||||
prefab_animation animation;
|
||||
animation.kind = Input.getToken<std::string>();
|
||||
animation.submodel = Input.getToken<std::string>( false );
|
||||
animation.x = Input.getToken<std::string>();
|
||||
animation.y = Input.getToken<std::string>();
|
||||
animation.z = Input.getToken<std::string>();
|
||||
animation.speed = Input.getToken<std::string>();
|
||||
Aspect.animations.emplace_back( animation );
|
||||
}
|
||||
else {
|
||||
ErrorLog( "Prefab \"" + Definition.name + "\": unexpected \"" + token + "\" in aspect \"" + Aspect.name + "\"" );
|
||||
}
|
||||
token = Input.getToken<std::string>();
|
||||
}
|
||||
}
|
||||
|
||||
} // anonymous namespace
|
||||
|
||||
bool
|
||||
prefab_registry::load( std::string const &Name ) {
|
||||
|
||||
auto const path { Global.asCurrentSceneryPath + Name + ".pfb" };
|
||||
cParser input( path, cParser::buffer_FILE );
|
||||
if( false == input.ok() ) {
|
||||
ErrorLog( "Prefab \"" + Name + "\" not found (" + path + ")" );
|
||||
return false;
|
||||
}
|
||||
|
||||
prefab_definition definition;
|
||||
definition.name = Name;
|
||||
|
||||
input.expectToken( "prefab" );
|
||||
auto token { input.getToken<std::string>() };
|
||||
while( ( false == token.empty() ) && ( token != "endprefab" ) ) {
|
||||
|
||||
if( token == "model" ) {
|
||||
definition.model = input.getToken<std::string>( false );
|
||||
}
|
||||
else if( token == "modelangle" ) {
|
||||
definition.modelangle = input.getToken<std::string>();
|
||||
}
|
||||
else if( token == "attach" ) {
|
||||
prefab_attachment attachment;
|
||||
attachment.rangemax = input.getToken<std::string>();
|
||||
attachment.rangemin = input.getToken<std::string>();
|
||||
attachment.model = input.getToken<std::string>( false );
|
||||
attachment.texture = input.getToken<std::string>( false );
|
||||
definition.attachments.emplace_back( attachment );
|
||||
}
|
||||
else if( token == "chambers" ) {
|
||||
definition.chambers = input.getToken<int>();
|
||||
}
|
||||
else if( token == "initial" ) {
|
||||
definition.initial = input.getToken<std::string>();
|
||||
}
|
||||
else if( token == "initiallights" ) {
|
||||
for( std::size_t chamber = 0; chamber < definition.chambers; ++chamber ) {
|
||||
definition.initiallights.emplace_back( input.getToken<int>() );
|
||||
}
|
||||
}
|
||||
else if( token == "sound" ) {
|
||||
definition.sound = input.getToken<std::string>( false );
|
||||
}
|
||||
else if( token == "lightmodes" ) {
|
||||
definition.lightmodes = true;
|
||||
}
|
||||
else if( token == "aspect" ) {
|
||||
prefab_aspect aspect;
|
||||
aspect.name = input.getToken<std::string>();
|
||||
parse_aspect( input, definition, aspect );
|
||||
definition.aspects.emplace_back( aspect );
|
||||
}
|
||||
else {
|
||||
ErrorLog( "Prefab \"" + Name + "\": unexpected \"" + token + "\"" );
|
||||
}
|
||||
token = input.getToken<std::string>();
|
||||
}
|
||||
|
||||
if( definition.chambers == 0 ) {
|
||||
// the chamber count has to come before the aspects, since it says how many light
|
||||
// values each of them reads. an aspect list with no lights in it means it did not
|
||||
ErrorLog( "Prefab \"" + Name + "\" declares no chambers" );
|
||||
return false;
|
||||
}
|
||||
|
||||
m_definitions.emplace( Name, definition );
|
||||
WriteLog( "Prefab \"" + Name + "\": " + std::to_string( definition.aspects.size() ) + " aspects, " + std::to_string( definition.chambers ) + " chambers" );
|
||||
return true;
|
||||
}
|
||||
|
||||
prefab_definition const *
|
||||
prefab_registry::definition( std::string const &Name ) {
|
||||
|
||||
auto const lookup { m_definitions.find( Name ) };
|
||||
if( lookup != m_definitions.end() ) {
|
||||
return &( lookup->second );
|
||||
}
|
||||
if( true == m_missing[ Name ] ) {
|
||||
return nullptr;
|
||||
}
|
||||
if( false == load( Name ) ) {
|
||||
m_missing[ Name ] = true;
|
||||
return nullptr;
|
||||
}
|
||||
return &( m_definitions.find( Name )->second );
|
||||
}
|
||||
|
||||
std::string
|
||||
prefab_registry::expand( std::string const &Name, prefab_placement const &Placement ) {
|
||||
|
||||
auto const *definition { this->definition( Name ) };
|
||||
if( definition == nullptr ) {
|
||||
return {};
|
||||
}
|
||||
|
||||
auto const &instance { Placement.name };
|
||||
auto const position {
|
||||
std::to_string( Placement.x ) + " "
|
||||
+ std::to_string( Placement.y ) + " "
|
||||
+ std::to_string( Placement.z ) };
|
||||
auto const texture { Placement.texture.empty() ? std::string( "none" ) : Placement.texture };
|
||||
|
||||
// what the signal shows until something sets it: the node's own pattern if the
|
||||
// definition gives one, otherwise the initial aspect's, otherwise dark
|
||||
auto const *initiallights { &definition->initiallights };
|
||||
if( true == initiallights->empty() ) {
|
||||
for( auto const &aspect : definition->aspects ) {
|
||||
if( aspect.name == definition->initial ) {
|
||||
initiallights = &aspect.lights;
|
||||
}
|
||||
}
|
||||
}
|
||||
auto const initial { pattern( *initiallights, definition->chambers ) };
|
||||
|
||||
std::ostringstream scenery;
|
||||
scenery
|
||||
<< "origin " << position << "\n"
|
||||
<< "rotate 0 " << Placement.yaw << " 0\n"
|
||||
<< "node 1000 0 " << instance << " model 0 0 0 " << definition->modelangle << " "
|
||||
<< definition->model << " " << texture << " lights" << initial << " endmodel\n";
|
||||
|
||||
for( auto const &attachment : definition->attachments ) {
|
||||
scenery
|
||||
<< "node " << attachment.rangemax << " " << attachment.rangemin
|
||||
<< " none model 0 0 0 " << definition->modelangle << " " << attachment.model << " "
|
||||
<< ( attachment.texture == "instance" ? texture : attachment.texture ) << " endmodel\n";
|
||||
}
|
||||
|
||||
scenery
|
||||
<< "rotate 0 0 0\n"
|
||||
<< "endorigin\n"
|
||||
// the speed the driver reads off the signal. the aspect events write it, the track
|
||||
// event beside the signal reads it
|
||||
<< "node -1 0 " << instance << "_sem_mem memcell " << position << " SetVelocity 0.0 0.0 none endmemcell\n"
|
||||
<< "event " << instance << "_sem_info getvalues 0 " << instance << "_sem_mem endevent\n";
|
||||
|
||||
if( false == definition->sound.empty() ) {
|
||||
scenery
|
||||
<< "node 50 0 " << instance << "_dzwiek sound " << position << " " << definition->sound << " endsound\n"
|
||||
<< "event " << instance << "_dzwiek sound 0.0 " << instance << "_dzwiek 1 endevent\n";
|
||||
}
|
||||
|
||||
// one state event per aspect, and that is the whole public surface of the instance:
|
||||
// whatever sets the signal launches <name>_<aspect>, and the lights, speeds, animation
|
||||
// and sound behind it belong to the prefab
|
||||
for( auto const &aspect : definition->aspects ) {
|
||||
scenery
|
||||
<< "event " << instance << "_" << aspect.name << " multiple 0 none "
|
||||
<< instance << "_sem_ligh_" << aspect.name << " "
|
||||
<< instance << "_sem_vel_" << aspect.name;
|
||||
if( false == aspect.animations.empty() ) {
|
||||
scenery << " " << instance << "_sem_anim_" << aspect.name;
|
||||
}
|
||||
if( false == definition->sound.empty() ) {
|
||||
scenery << " " << instance << "_dzwiek";
|
||||
}
|
||||
scenery
|
||||
<< " endevent\n"
|
||||
<< "event " << instance << "_sem_ligh_" << aspect.name << " lights " << aspect.lightdelay << " "
|
||||
<< instance << pattern( aspect.lights, definition->chambers ) << " endevent\n"
|
||||
<< "event " << instance << "_sem_vel_" << aspect.name << " updatevalues 0 "
|
||||
<< instance << "_sem_mem SetVelocity " << aspect.velocity << " " << aspect.velocitynext << " endevent\n";
|
||||
|
||||
if( false == aspect.animations.empty() ) {
|
||||
// the templates split this into groups of eight, which a multi_event has not
|
||||
// needed for a long time - it holds a vector. one list, one event
|
||||
scenery << "event " << instance << "_sem_anim_" << aspect.name << " multiple 0 none";
|
||||
for( std::size_t channel = 0; channel < aspect.animations.size(); ++channel ) {
|
||||
scenery << " " << instance << "_sem_anim_" << aspect.name << "_" << channel;
|
||||
}
|
||||
scenery << " endevent\n";
|
||||
std::size_t channel { 0 };
|
||||
for( auto const &animation : aspect.animations ) {
|
||||
scenery
|
||||
<< "event " << instance << "_sem_anim_" << aspect.name << "_" << channel
|
||||
<< " animation 0 " << instance << " " << animation.kind << " " << animation.submodel << " "
|
||||
<< animation.x << " " << animation.y << " " << animation.z << " " << animation.speed << " endevent\n";
|
||||
++channel;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// a failed signal is dark and stops what comes at it
|
||||
scenery
|
||||
<< "event " << instance << "_uszk multiple 0 none "
|
||||
<< instance << "_sem_ligh0 " << instance << "_sem_vel_stop endevent\n"
|
||||
<< "event " << instance << "_sem_ligh0 lights 0.0 " << instance << pattern( {}, definition->chambers ) << " endevent\n"
|
||||
<< "event " << instance << "_sem_vel_stop updatevalues 0 "
|
||||
<< instance << "_sem_mem SetVelocity 0.0 0.0 endevent\n";
|
||||
|
||||
if( true == definition->lightmodes ) {
|
||||
// channel zero picks day, night or automatic lighting; the rest is left alone
|
||||
auto const untouched { pattern( {}, definition->chambers > 0 ? definition->chambers - 1 : 0, -1 ) };
|
||||
scenery
|
||||
<< "event " << instance << "_dzienny lights 0.0 " << instance << " 0" << untouched << " endevent\n"
|
||||
<< "event " << instance << "_nocny lights 0.0 " << instance << " 1" << untouched << " endevent\n"
|
||||
<< "event " << instance << "_auto lights 0.0 " << instance << " 3" << untouched << " endevent\n";
|
||||
}
|
||||
|
||||
return scenery.str();
|
||||
}
|
||||
|
||||
} // namespace scene
|
||||
118
scene/prefab.cppm
Normal file
118
scene/prefab.cppm
Normal file
@@ -0,0 +1,118 @@
|
||||
/*
|
||||
This Source Code Form is subject to the
|
||||
terms of the Mozilla Public License, v.
|
||||
2.0. If a copy of the MPL was not
|
||||
distributed with this file, You can
|
||||
obtain one at
|
||||
http://mozilla.org/MPL/2.0/.
|
||||
*/
|
||||
|
||||
module;
|
||||
#include <string>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
|
||||
export module eu07.scene.prefab;
|
||||
|
||||
export {
|
||||
|
||||
// Shared archetypes. A prefab is a model plus the events, memcell, animation channels and
|
||||
// sounds that give it behaviour, described once and instanced many times. It exists because
|
||||
// the .inc templates it replaces are not templates at all: cParser pastes text and
|
||||
// substitutes (pN), so a signal that differs only by which aspects it can display needs a
|
||||
// whole new file. The scenery directory holds over twelve hundred of them.
|
||||
//
|
||||
// What makes one prefab enough for the whole signal family is that its parameter is a list,
|
||||
// not a number. Every signal template is the same shape - a state event, a light pattern and
|
||||
// a speed the driver reads, for each aspect it can display, wrapped in a fixed core of
|
||||
// _sem_info, _uszk and the dark state. Give the prefab the aspects and the rest follows. A
|
||||
// shape signal adds animation channels to each aspect and a sound to the transition, which
|
||||
// is more per aspect but not a different shape.
|
||||
//
|
||||
// Instances expand to scenery text and go through the ordinary node and event parsing. The
|
||||
// binary format will compile them directly; until then this keeps one description of the
|
||||
// signal family instead of one per variant.
|
||||
namespace scene {
|
||||
|
||||
// one submodel driven by an aspect. kind is the word the animation event takes, translate or
|
||||
// rotate; target is where the submodel goes, in metres or degrees; speed is how fast
|
||||
struct prefab_animation {
|
||||
std::string kind;
|
||||
std::string submodel;
|
||||
std::string x{ "0" }, y{ "0" }, z{ "0" };
|
||||
std::string speed{ "0" };
|
||||
};
|
||||
|
||||
// one thing the signal can show. lights is the pattern the model's light channels take, one
|
||||
// entry per channel, -1 leaving a channel alone; velocity is what the driver reads from the
|
||||
// memcell, -1 meaning no limit
|
||||
struct prefab_aspect {
|
||||
std::string name;
|
||||
std::vector<int> lights;
|
||||
std::string lightdelay{ "0.0" };
|
||||
std::string velocity{ "-1" };
|
||||
std::string velocitynext{ "-1" };
|
||||
std::vector<prefab_animation> animations;
|
||||
};
|
||||
|
||||
// a second model hung on the instance: the lattice mast a shape signal stands on, its
|
||||
// distant lod, the plate carrying the signal's symbol
|
||||
struct prefab_attachment {
|
||||
std::string rangemax{ "50" };
|
||||
std::string rangemin{ "0" };
|
||||
std::string model;
|
||||
// the instance's own texture when "instance", otherwise a literal, "none" included
|
||||
std::string texture{ "none" };
|
||||
};
|
||||
|
||||
struct prefab_definition {
|
||||
std::string name;
|
||||
std::string model;
|
||||
std::string modelangle{ "0" };
|
||||
std::vector<prefab_attachment> attachments;
|
||||
// how many light channels the model has, which is how many values an aspect's pattern holds
|
||||
std::size_t chambers{ 0 };
|
||||
std::vector<prefab_aspect> aspects;
|
||||
// aspect the signal shows before anything sets it, by name; empty lights every channel off
|
||||
std::string initial;
|
||||
// the node's own starting pattern, when it is not simply the initial aspect: a shape
|
||||
// signal starts with channel zero on automatic, which is not a thing any aspect says
|
||||
std::vector<int> initiallights;
|
||||
// played when the signal changes aspect. shape signals have one, light signals do not
|
||||
std::string sound;
|
||||
// emit the _dzienny / _nocny / _auto events which switch channel zero between day,
|
||||
// night and automatic lighting. only models with that channel want them
|
||||
bool lightmodes{ false };
|
||||
};
|
||||
|
||||
// arguments one instance supplies
|
||||
struct prefab_placement {
|
||||
std::string name;
|
||||
double x{ 0.0 }, y{ 0.0 }, z{ 0.0 };
|
||||
double yaw{ 0.0 };
|
||||
std::string texture;
|
||||
};
|
||||
|
||||
class prefab_registry {
|
||||
|
||||
public:
|
||||
// looks the definition up, loading <Name>.pfb from the scenery directory on first use.
|
||||
// returns nullptr and logs if there is no such prefab
|
||||
prefab_definition const * definition( std::string const &Name );
|
||||
// scenery text realising the instance: the model node, its attachments, memcell, sound
|
||||
// and events. empty if the definition is missing
|
||||
std::string expand( std::string const &Name, prefab_placement const &Placement );
|
||||
|
||||
private:
|
||||
bool load( std::string const &Name );
|
||||
|
||||
std::unordered_map<std::string, prefab_definition> m_definitions;
|
||||
// names which failed to load, so a scenery full of them complains once each
|
||||
std::unordered_map<std::string, bool> m_missing;
|
||||
};
|
||||
|
||||
} // namespace scene
|
||||
|
||||
inline scene::prefab_registry Prefabs;
|
||||
|
||||
}
|
||||
@@ -34,6 +34,7 @@ module;
|
||||
|
||||
module eu07.simcore;
|
||||
import eu07.simulation.simulation;
|
||||
import eu07.simulation.loadprofile;
|
||||
import eu07.rendering.renderer;
|
||||
import :mover;
|
||||
import eu07.utilities.globals;
|
||||
@@ -1306,17 +1307,50 @@ std::vector<std::string> switchtrackbedtextures {
|
||||
"zwrotl65r1200pods-new",
|
||||
"zwrotp65r1200pods-new" };
|
||||
|
||||
bool
|
||||
is_switch_trackbed( std::string const &Materialname ) {
|
||||
|
||||
for( auto const &switchtrackbedtexture : switchtrackbedtextures ) {
|
||||
if( contains( Materialname, switchtrackbedtexture ) ) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
void
|
||||
place_in_world( std::vector<world_vertex> &Vertices, scratch_data const &Scratchpad ) {
|
||||
|
||||
if( Scratchpad.location.rotation != glm::vec3( 0, 0, 0 ) ) {
|
||||
// rotate...
|
||||
auto const rotation = glm::radians( Scratchpad.location.rotation );
|
||||
for( auto &vertex : Vertices ) {
|
||||
vertex.position = glm::rotateZ<double>( vertex.position, rotation.z );
|
||||
vertex.position = glm::rotateX<double>( vertex.position, rotation.x );
|
||||
vertex.position = glm::rotateY<double>( vertex.position, rotation.y );
|
||||
vertex.normal = glm::rotateZ( vertex.normal, rotation.z );
|
||||
vertex.normal = glm::rotateX( vertex.normal, rotation.x );
|
||||
vertex.normal = glm::rotateY( vertex.normal, rotation.y );
|
||||
}
|
||||
}
|
||||
if( false == Scratchpad.location.offset.empty()
|
||||
&& Scratchpad.location.offset.top() != glm::dvec3( 0, 0, 0 ) ) {
|
||||
// ...and move
|
||||
auto const &offset = Scratchpad.location.offset.top();
|
||||
for( auto &vertex : Vertices ) {
|
||||
vertex.position += offset;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void
|
||||
basic_region::insert( shape_node Shape, scratch_data &Scratchpad, bool const Transform ) {
|
||||
|
||||
if( Global.CreateSwitchTrackbeds ) {
|
||||
|
||||
auto const materialname{GfxRenderer->Material(Shape.data().material)->GetName()};
|
||||
for( auto const &switchtrackbedtexture : switchtrackbedtextures ) {
|
||||
if( contains( materialname, switchtrackbedtexture ) ) {
|
||||
// geometry with blacklisted texture, part of old switch trackbed; ignore it
|
||||
return;
|
||||
}
|
||||
if( true == is_switch_trackbed( GfxRenderer->Material( Shape.data().material )->GetName() ) ) {
|
||||
// geometry with blacklisted texture, part of old switch trackbed; ignore it
|
||||
return;
|
||||
}
|
||||
}
|
||||
// shape might need to be split into smaller pieces, so we create list of nodes instead of just single one
|
||||
@@ -1328,37 +1362,24 @@ basic_region::insert( shape_node Shape, scratch_data &Scratchpad, bool const Tra
|
||||
// adjust input if necessary:
|
||||
if( true == Transform ) {
|
||||
// shapes generated from legacy terrain come with world space coordinates and don't need processing
|
||||
if( Scratchpad.location.rotation != glm::vec3( 0, 0, 0 ) ) {
|
||||
// rotate...
|
||||
auto const rotation = glm::radians( Scratchpad.location.rotation );
|
||||
for( auto &vertex : shape.m_data.vertices ) {
|
||||
vertex.position = glm::rotateZ<double>( vertex.position, rotation.z );
|
||||
vertex.position = glm::rotateX<double>( vertex.position, rotation.x );
|
||||
vertex.position = glm::rotateY<double>( vertex.position, rotation.y );
|
||||
vertex.normal = glm::rotateZ( vertex.normal, rotation.z );
|
||||
vertex.normal = glm::rotateX( vertex.normal, rotation.x );
|
||||
vertex.normal = glm::rotateY( vertex.normal, rotation.y );
|
||||
}
|
||||
}
|
||||
if( false == Scratchpad.location.offset.empty()
|
||||
&& Scratchpad.location.offset.top() != glm::dvec3(0, 0, 0) ) {
|
||||
// ...and move
|
||||
auto const& offset = Scratchpad.location.offset.top();
|
||||
for( auto &vertex : shape.m_data.vertices ) {
|
||||
vertex.position += offset;
|
||||
}
|
||||
}
|
||||
place_in_world( shape.m_data.vertices, Scratchpad );
|
||||
// calculate bounding area
|
||||
for( auto const &vertex : shape.m_data.vertices ) {
|
||||
shape.m_data.area.center += vertex.position;
|
||||
}
|
||||
shape.m_data.area.center /= shape.m_data.vertices.size();
|
||||
// trim the shape if needed. trimmed parts will be added to list as separate nodes
|
||||
for( std::size_t index = 0; index < shapes.size(); ++index ) {
|
||||
while( true == RaTriangleDivider( shapes[ index ], shapes ) ) {
|
||||
; // all work is done during expression check
|
||||
{
|
||||
loadprofile::scoped_timer const dividetime { loadprofile::stage::shape_divide };
|
||||
for( std::size_t index = 0; index < shapes.size(); ++index ) {
|
||||
while( true == RaTriangleDivider( shapes[ index ], shapes ) ) {
|
||||
; // all work is done during expression check
|
||||
}
|
||||
}
|
||||
}
|
||||
if( shapes.size() > 1 ) {
|
||||
loadprofile::add_split( shapes.size() - 1 );
|
||||
}
|
||||
}
|
||||
// move the data into appropriate section(s)
|
||||
for( auto &shape : shapes ) {
|
||||
|
||||
@@ -371,6 +371,15 @@ public:
|
||||
};
|
||||
|
||||
// top-level of scene spatial structure, holds collection of sections
|
||||
// whether a material belongs to the old hand-made switch trackbeds, which the engine
|
||||
// replaces with generated ones when CreateSwitchTrackbeds is on
|
||||
bool is_switch_trackbed( std::string const &Materialname );
|
||||
|
||||
// moves shape vertices from the coordinates they were written in to world space, applying the
|
||||
// rotate and origin in force where the node stood. the one place this happens, so that anything
|
||||
// else looking at a shape - the terrain bake - sees it where it is drawn
|
||||
void place_in_world( std::vector<world_vertex> &Vertices, scratch_data const &Scratchpad );
|
||||
|
||||
class basic_region {
|
||||
public:
|
||||
// constructors
|
||||
|
||||
@@ -167,6 +167,19 @@ public:
|
||||
// data access
|
||||
shapenode_data const &
|
||||
data() const;
|
||||
// keeps only the triangles a predicate accepts, by their position in the vertex list.
|
||||
// used to leave out what a baked heightfield already shows
|
||||
template <typename Predicate_>
|
||||
void
|
||||
keep_triangles( Predicate_ &&Keep ) {
|
||||
std::vector<world_vertex> kept;
|
||||
kept.reserve( m_data.vertices.size() );
|
||||
for( std::size_t index = 0; index + 2 < m_data.vertices.size(); index += 3 ) {
|
||||
if( true == Keep( index / 3 ) ) {
|
||||
kept.insert( kept.end(), m_data.vertices.begin() + index, m_data.vertices.begin() + index + 3 );
|
||||
}
|
||||
}
|
||||
m_data.vertices = std::move( kept ); }
|
||||
// get bounding radius
|
||||
// NOTE: use this method instead of direct access to the data member, due to lazy radius evaluation
|
||||
float radius();
|
||||
|
||||
789
scene/terraincooker.h
Normal file
789
scene/terraincooker.h
Normal file
@@ -0,0 +1,789 @@
|
||||
/*
|
||||
This Source Code Form is subject to the
|
||||
terms of the Mozilla Public License, v.
|
||||
2.0. If a copy of the MPL was not
|
||||
distributed with this file, You can
|
||||
obtain one at
|
||||
http://mozilla.org/MPL/2.0/.
|
||||
*/
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
#include <cstring>
|
||||
#include <climits>
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
#include <functional>
|
||||
#include <limits>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
#include <zstd.h>
|
||||
|
||||
#include "scene/heightfieldformat.h"
|
||||
|
||||
// Rasterising a scenery's terrain triangles into a tiled heightfield.
|
||||
//
|
||||
// Both the standalone terraincook tool and the engine cook with this, which is the point of
|
||||
// it being here: a heightfield baked on first run has to come out identical to one baked by
|
||||
// the tool, and the only way to be sure of that is for there to be one implementation.
|
||||
//
|
||||
// The cooker knows nothing about where triangles come from. Feed it rasterize() per triangle
|
||||
// and call finish(); the tool reads them out of scenery text, the engine hands them over as
|
||||
// it parses.
|
||||
namespace terrain {
|
||||
|
||||
|
||||
// candidate grid steps tried when the step is not given; the coarsest one that still holds
|
||||
// for practically every vertex names the grid (see detect_step).
|
||||
// the range has to reach far past a corridor's metre grid: a regional elevation model
|
||||
// is sampled every hundred metres, and a candidate list that stops at ten would report
|
||||
// it as a ten metre grid, because every point of a 100 m grid sits on a 10 m one too
|
||||
constexpr double gridsteps[] { 0.5, 1.0, 2.0, 2.5, 4.0, 5.0, 10.0, 20.0, 25.0, 50.0, 100.0, 200.0 };
|
||||
// how far off a grid line a coordinate may sit and still count as on it
|
||||
constexpr double gridepsilon { 0.01 };
|
||||
// height difference at one grid point above which the samples are treated as a genuine
|
||||
// conflict - two surfaces stacked over each other, which a heightfield cannot express
|
||||
constexpr double conflictepsilon { 0.25 };
|
||||
// share of sampled vertices a candidate step has to cover to count as the base grid
|
||||
constexpr double gridthreshold { 99.0 };
|
||||
|
||||
struct vertex {
|
||||
double x, y, z;
|
||||
double u { 0.0 }, v { 0.0 }; // texture coordinates, kept to measure material scale
|
||||
};
|
||||
|
||||
// length of the segment between two vertices as seen from above
|
||||
inline double
|
||||
planar_length( vertex const &From, vertex const &To ) {
|
||||
return std::sqrt( ( To.x - From.x ) * ( To.x - From.x ) + ( To.z - From.z ) * ( To.z - From.z ) );
|
||||
}
|
||||
|
||||
// whether a coordinate lies on a grid of the given step
|
||||
inline bool
|
||||
on_grid( double const Value, double const Step ) {
|
||||
return std::fabs( Value / Step - std::round( Value / Step ) ) * Step <= gridepsilon;
|
||||
}
|
||||
|
||||
// writes a greyscale preview. Values holds Width * Height samples, absent ones marked
|
||||
// by the sentinel; those come out black
|
||||
inline void
|
||||
writepreview(
|
||||
std::filesystem::path const &Path,
|
||||
std::vector<float> const &Values,
|
||||
std::size_t const Width, std::size_t const Height,
|
||||
float const Sentinel ) {
|
||||
|
||||
auto minimum { 1e30f }, maximum { -1e30f };
|
||||
for( auto const value : Values ) {
|
||||
if( value == Sentinel ) { continue; }
|
||||
minimum = std::min( minimum, value );
|
||||
maximum = std::max( maximum, value );
|
||||
}
|
||||
if( minimum > maximum ) { return; }
|
||||
auto const range { std::max( 1e-6f, maximum - minimum ) };
|
||||
|
||||
std::ofstream file( Path, std::ios::binary );
|
||||
file << "P5\n" << Width << " " << Height << "\n255\n";
|
||||
std::vector<unsigned char> row( Width );
|
||||
for( auto y { 0u }; y < Height; ++y ) {
|
||||
for( auto x { 0u }; x < Width; ++x ) {
|
||||
auto const value { Values[ y * Width + x ] };
|
||||
row[ x ] = (
|
||||
value == Sentinel ?
|
||||
0u :
|
||||
static_cast<unsigned char>( 1.0f + 254.0f * ( value - minimum ) / range ) );
|
||||
}
|
||||
file.write( reinterpret_cast<char const *>( row.data() ), static_cast<std::streamsize>( Width ) );
|
||||
}
|
||||
}
|
||||
|
||||
class cooker {
|
||||
|
||||
public:
|
||||
// samples along a tile side, shared edge excluded; with a 2 m grid this is a 256 m tile
|
||||
static constexpr std::uint32_t tilesamples { 128 };
|
||||
static constexpr std::uint32_t tilestride { heightfield::samples_at_level( tilesamples, 0 ) };
|
||||
// mip chain down to a 5 sample side, which is where a tile stops being worth its entry
|
||||
static constexpr std::uint32_t miplevels { heightfield::level_count( tilesamples, 5 ) };
|
||||
// triangles larger than this are not terrain: the water plane spans the whole map
|
||||
static constexpr double maxedge { 200.0 };
|
||||
// two surfaces closer than this are the same ground, one laid over the other, rather
|
||||
// than a deck carried above it. a kerb or a slab stands centimetres proud; a viaduct
|
||||
// stands metres
|
||||
static constexpr double surfaceepsilon { 0.5 };
|
||||
|
||||
// stop printing the cooking report. the tool wants the tables; the engine, baking on
|
||||
// a first run, wants one line in its own log instead
|
||||
void
|
||||
quiet() { m_verbose = false; }
|
||||
|
||||
void
|
||||
configure( double const Step, bool const Compress ) {
|
||||
m_step = Step;
|
||||
m_compress = Compress; }
|
||||
|
||||
// triangle identity for representation tracking. callers that do not track it pass
|
||||
// nothing and get notriangle, which is never reported as represented
|
||||
static constexpr std::uint32_t notriangle { 0xffffffffu };
|
||||
|
||||
void
|
||||
rasterize( vertex const &A, vertex const &B, vertex const &C, std::string_view const Material,
|
||||
std::uint32_t const Triangle = notriangle ) {
|
||||
|
||||
m_current = Triangle;
|
||||
if( Triangle != notriangle ) {
|
||||
if( Triangle >= m_triangleclaimed.size() ) {
|
||||
auto const size { static_cast<std::size_t>( Triangle ) + 1u };
|
||||
m_triangleclaimed.resize( size, 0u );
|
||||
m_trianglekept.resize( size, 0u );
|
||||
m_trianglerejected.resize( size, 0u );
|
||||
}
|
||||
}
|
||||
|
||||
vertex const points[ 3 ] { A, B, C };
|
||||
|
||||
for( auto index { 0u }; index < 3u; ++index ) {
|
||||
if( planar_length( points[ index ], points[ ( index + 1 ) % 3 ] ) > maxedge ) {
|
||||
++m_oversized;
|
||||
reject();
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// barycentric setup in the horizontal plane; a triangle standing on its edge
|
||||
// covers no samples and would divide by zero
|
||||
auto const area { ( B.x - A.x ) * ( C.z - A.z ) - ( C.x - A.x ) * ( B.z - A.z ) };
|
||||
if( std::abs( area ) < 1e-9 ) { ++m_degenerate; reject(); return; }
|
||||
auto const inversearea { 1.0 / area };
|
||||
// a palette slot is taken only by a material that reaches this far, so triangles
|
||||
// standing on their edge do not fill the palette
|
||||
auto const material { material_index( Material ) };
|
||||
measure_scale( material, points );
|
||||
|
||||
auto const firstcolumn { static_cast<std::int64_t>( std::ceil( std::min( { A.x, B.x, C.x } ) / m_step ) ) };
|
||||
auto const lastcolumn { static_cast<std::int64_t>( std::floor( std::max( { A.x, B.x, C.x } ) / m_step ) ) };
|
||||
auto const firstrow { static_cast<std::int64_t>( std::ceil( std::min( { A.z, B.z, C.z } ) / m_step ) ) };
|
||||
auto const lastrow { static_cast<std::int64_t>( std::floor( std::max( { A.z, B.z, C.z } ) / m_step ) ) };
|
||||
|
||||
std::size_t claimed { 0 };
|
||||
for( auto row { firstrow }; row <= lastrow; ++row ) {
|
||||
for( auto column { firstcolumn }; column <= lastcolumn; ++column ) {
|
||||
|
||||
auto const x { column * m_step };
|
||||
auto const z { row * m_step };
|
||||
auto const weightb { ( ( x - A.x ) * ( C.z - A.z ) - ( C.x - A.x ) * ( z - A.z ) ) * inversearea };
|
||||
auto const weightc { ( ( B.x - A.x ) * ( z - A.z ) - ( x - A.x ) * ( B.z - A.z ) ) * inversearea };
|
||||
auto const weighta { 1.0 - weightb - weightc };
|
||||
constexpr double slack { -1e-9 };
|
||||
if( ( weighta < slack ) || ( weightb < slack ) || ( weightc < slack ) ) { continue; }
|
||||
|
||||
++claimed;
|
||||
store( column, row, weighta * A.y + weightb * B.y + weightc * C.y, material );
|
||||
}
|
||||
}
|
||||
|
||||
// every sample this triangle wanted, whether or not it ended up keeping it.
|
||||
// comparing this with what the finished grid holds says how much of a
|
||||
// material was lost to triangles stacked over it
|
||||
if( m_current != notriangle ) { m_triangleclaimed[ m_current ] += static_cast<std::uint32_t>( claimed ); }
|
||||
tally( m_materialclaimed, material, claimed );
|
||||
if( material >= m_materialwhere.size() ) { m_materialwhere.resize( material + 1u, { 0.0, 0.0, false } ); }
|
||||
if( false == m_materialwhere[ material ].seen ) {
|
||||
// one place the material was laid, so a report line can be looked at
|
||||
m_materialwhere[ material ] = { ( A.x + B.x + C.x ) / 3.0, ( A.z + B.z + C.z ) / 3.0, true };
|
||||
}
|
||||
if( claimed == 0 ) {
|
||||
// smaller than the gap between samples: it left no trace at all
|
||||
tally( m_materialmissed, material, 1 );
|
||||
++m_toosmall;
|
||||
}
|
||||
}
|
||||
|
||||
// whether the heightfield shows this triangle well enough for it not to be drawn on its
|
||||
// own: it was laid at all, it reached at least one sample, and it still holds nearly
|
||||
// every sample it wanted once everything else has been laid. valid after finish()
|
||||
bool
|
||||
represented( std::uint32_t const Triangle ) const {
|
||||
if( Triangle >= m_triangleclaimed.size() ) { return false; }
|
||||
if( m_trianglerejected[ Triangle ] != 0 ) { return false; }
|
||||
auto const claimed { m_triangleclaimed[ Triangle ] };
|
||||
if( claimed == 0 ) { return false; }
|
||||
return static_cast<double>( m_trianglekept[ Triangle ] ) >= representedshare * static_cast<double>( claimed ); }
|
||||
|
||||
void
|
||||
finish( std::filesystem::path const &Path, std::filesystem::path const &Previewpath ) {
|
||||
|
||||
if( true == m_tiles.empty() ) {
|
||||
report( " nothing to cook\n" );
|
||||
return; }
|
||||
|
||||
// every sample counted once, in the tile that owns it rather than in the
|
||||
// neighbours carrying a copy of the shared edge
|
||||
for( auto const & [ key, entry ] : m_tiles ) {
|
||||
for( std::uint32_t row = 0; row < tilesamples; ++row ) {
|
||||
for( std::uint32_t column = 0; column < tilesamples; ++column ) {
|
||||
auto const owner { entry.owner[ row * tilestride + column ] };
|
||||
if( ( owner == notriangle ) || ( owner >= m_trianglekept.size() ) ) { continue; }
|
||||
++m_trianglekept[ owner ];
|
||||
// the heightfield is drawn only between samples that all carry ground,
|
||||
// so a triangle reaching past the last full cell leaves a strip nothing
|
||||
// covers. one holding a sample next to a hole keeps being drawn itself
|
||||
auto const globalcolumn { static_cast<std::int64_t>( entry.x ) * tilesamples + column };
|
||||
auto const globalrow { static_cast<std::int64_t>( entry.z ) * tilesamples + row };
|
||||
if( m_trianglerejected[ owner ] != 0 ) { continue; }
|
||||
if( false == drawn_around( globalcolumn, globalrow ) ) {
|
||||
m_trianglerejected[ owner ] = 1u;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
auto const scale { std::max( 1.0, m_maxheight - m_minheight ) / heightfield::heightrange };
|
||||
|
||||
std::vector<tile const *> ordered;
|
||||
ordered.reserve( m_tiles.size() );
|
||||
for( auto const & [ key, entry ] : m_tiles ) { ordered.push_back( &entry ); }
|
||||
std::sort( ordered.begin(), ordered.end(),
|
||||
[]( tile const *Left, tile const *Right ) {
|
||||
return heightfield::morton( Left->x, Left->z ) < heightfield::morton( Right->x, Right->z ); } );
|
||||
|
||||
std::ofstream file( Path, std::ios::binary );
|
||||
if( false == file.good() ) {
|
||||
report( " cannot write %s\n", Path.string().c_str() );
|
||||
return; }
|
||||
|
||||
heightfield::file_header header {};
|
||||
std::memcpy( header.magic, heightfield::magic, sizeof( header.magic ) );
|
||||
header.version = heightfield::version;
|
||||
header.tilesamples = tilesamples;
|
||||
header.gridstep = static_cast<float>( m_step );
|
||||
header.heightbias = static_cast<float>( m_minheight );
|
||||
header.heightscale = static_cast<float>( scale );
|
||||
header.tilecount = static_cast<std::uint32_t>( ordered.size() );
|
||||
header.materialcount = static_cast<std::uint32_t>( m_materials.size() );
|
||||
header.miplevels = miplevels;
|
||||
header.selection = heightfield::selection_rules;
|
||||
header.minx = header.minz = INT32_MAX;
|
||||
header.maxx = header.maxz = INT32_MIN;
|
||||
for( auto const *entry : ordered ) {
|
||||
header.minx = std::min( header.minx, entry->x ); header.maxx = std::max( header.maxx, entry->x );
|
||||
header.minz = std::min( header.minz, entry->z ); header.maxz = std::max( header.maxz, entry->z );
|
||||
}
|
||||
|
||||
std::size_t materialbytes { 0 };
|
||||
for( auto const &name : m_materials ) {
|
||||
materialbytes += sizeof( std::uint16_t ) + name.size() + sizeof( float );
|
||||
}
|
||||
auto const directorybytes { ordered.size() * sizeof( heightfield::tile_entry ) };
|
||||
auto const payloadstart { sizeof( heightfield::file_header ) + directorybytes + materialbytes };
|
||||
|
||||
// the payloads are encoded up front: the directory records where each one
|
||||
// lands and how large it ended up, so it cannot be written before they exist
|
||||
std::vector<heightfield::tile_entry> directory;
|
||||
std::vector<std::vector<std::uint8_t>> payloads;
|
||||
directory.reserve( ordered.size() );
|
||||
payloads.reserve( ordered.size() );
|
||||
|
||||
auto offset { payloadstart };
|
||||
auto const plainsize { heightfield::tile_bytes( tilesamples, miplevels ) };
|
||||
std::vector<std::uint8_t> plain( plainsize );
|
||||
|
||||
for( auto const *entry : ordered ) {
|
||||
|
||||
encode( *entry, scale, plain );
|
||||
auto payload { m_compress ? compress( plain ) : plain };
|
||||
m_storedbytes += payload.size();
|
||||
m_plainbytes += plain.size();
|
||||
|
||||
heightfield::tile_entry record {};
|
||||
record.x = entry->x;
|
||||
record.z = entry->z;
|
||||
record.offset = offset;
|
||||
record.storedsize = static_cast<std::uint32_t>( payload.size() );
|
||||
record.plainsize = static_cast<std::uint32_t>( plain.size() );
|
||||
record.minheight = entry->minheight;
|
||||
record.maxheight = entry->maxheight;
|
||||
record.coverage = static_cast<std::uint32_t>( entry->coverage );
|
||||
record.tilecodec = static_cast<std::uint16_t>(
|
||||
m_compress ? heightfield::codec::zstd : heightfield::codec::raw );
|
||||
offset += payload.size();
|
||||
|
||||
directory.push_back( record );
|
||||
payloads.push_back( std::move( payload ) );
|
||||
}
|
||||
|
||||
file.write( reinterpret_cast<char const *>( &header ), sizeof( header ) );
|
||||
file.write( reinterpret_cast<char const *>( directory.data() ), static_cast<std::streamsize>( directorybytes ) );
|
||||
for( std::size_t index = 0; index < m_materials.size(); ++index ) {
|
||||
auto const &name { m_materials[ index ] };
|
||||
std::uint16_t const length { static_cast<std::uint16_t>( name.size() ) };
|
||||
file.write( reinterpret_cast<char const *>( &length ), sizeof( length ) );
|
||||
file.write( name.data(), static_cast<std::streamsize>( name.size() ) );
|
||||
auto const scale { static_cast<float>( material_scale( index ) ) };
|
||||
file.write( reinterpret_cast<char const *>( &scale ), sizeof( scale ) );
|
||||
}
|
||||
for( auto const &payload : payloads ) {
|
||||
file.write( reinterpret_cast<char const *>( payload.data() ), static_cast<std::streamsize>( payload.size() ) );
|
||||
}
|
||||
|
||||
file.close();
|
||||
|
||||
report( Path, header, offset );
|
||||
if( false == Previewpath.empty() ) {
|
||||
writecookedpreview( Previewpath, header );
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
struct tile {
|
||||
std::int32_t x { 0 }, z { 0 };
|
||||
std::vector<float> height;
|
||||
std::vector<std::uint8_t> material;
|
||||
std::vector<std::uint8_t> covered;
|
||||
// which triangle the sample came from, so that once everything is laid each
|
||||
// triangle can be asked whether the heightfield still shows it
|
||||
std::vector<std::uint32_t> owner;
|
||||
float minheight { 1e30f }, maxheight { -1e30f };
|
||||
std::size_t coverage { 0 };
|
||||
};
|
||||
|
||||
// the palette slot of a material, allocated on first use. looked up by view, so a
|
||||
// material already known costs no allocation
|
||||
std::uint8_t
|
||||
material_index( std::string_view const Name ) {
|
||||
auto const lookup { m_materialindex.find( Name ) };
|
||||
if( lookup != m_materialindex.end() ) { return lookup->second; }
|
||||
if( m_materials.size() >= 255 ) { ++m_materialoverflow; return 255; }
|
||||
auto const index { static_cast<std::uint8_t>( m_materials.size() ) };
|
||||
m_materials.emplace_back( Name );
|
||||
m_materialindex.emplace( std::string { Name }, index );
|
||||
return index; }
|
||||
|
||||
// the source carries texture coordinates, so the tiling of every material can be
|
||||
// measured instead of guessed: the ratio of world distance to texture distance along
|
||||
// a triangle edge is how many metres one repeat covers. edges too short or with no
|
||||
// texture movement say nothing and are left out
|
||||
void
|
||||
measure_scale( std::uint8_t const Material, vertex const ( &Points )[ 3 ] ) {
|
||||
|
||||
if( Material >= m_scalesums.size() ) {
|
||||
m_scalesums.resize( Material + 1, 0.0 );
|
||||
m_scalesquares.resize( Material + 1, 0.0 );
|
||||
m_scalecounts.resize( Material + 1, 0 );
|
||||
}
|
||||
for( auto index { 0u }; index < 3u; ++index ) {
|
||||
auto const &from { Points[ index ] };
|
||||
auto const &to { Points[ ( index + 1 ) % 3 ] };
|
||||
auto const world { planar_length( from, to ) };
|
||||
auto const texture {
|
||||
std::sqrt( ( to.u - from.u ) * ( to.u - from.u ) + ( to.v - from.v ) * ( to.v - from.v ) ) };
|
||||
if( ( world < 0.5 ) || ( texture < 1e-4 ) ) { continue; }
|
||||
auto const ratio { world / texture };
|
||||
m_scalesums[ Material ] += ratio;
|
||||
m_scalesquares[ Material ] += ratio * ratio;
|
||||
++m_scalecounts[ Material ];
|
||||
}
|
||||
}
|
||||
|
||||
void
|
||||
store(
|
||||
std::int64_t const Column, std::int64_t const Row, double const Height,
|
||||
std::uint8_t const Material ) {
|
||||
|
||||
// a sample on a tile boundary belongs to both neighbours. writing it to each
|
||||
// of them is what makes the shared edge identical rather than merely close
|
||||
auto const basex { static_cast<std::int32_t>( heightfield::floor_div( Column, tilesamples ) ) };
|
||||
auto const basez { static_cast<std::int32_t>( heightfield::floor_div( Row, tilesamples ) ) };
|
||||
auto const onxedge { Column == static_cast<std::int64_t>( basex ) * tilesamples };
|
||||
auto const onzedge { Row == static_cast<std::int64_t>( basez ) * tilesamples };
|
||||
|
||||
for( auto duplicate { 0u }; duplicate < 4u; ++duplicate ) {
|
||||
|
||||
if( ( ( duplicate & 1u ) != 0 ) && ( false == onxedge ) ) { continue; }
|
||||
if( ( ( duplicate & 2u ) != 0 ) && ( false == onzedge ) ) { continue; }
|
||||
|
||||
auto const tilex { basex - ( ( duplicate & 1u ) != 0 ? 1 : 0 ) };
|
||||
auto const tilez { basez - ( ( duplicate & 2u ) != 0 ? 1 : 0 ) };
|
||||
auto const column { static_cast<std::size_t>( Column - static_cast<std::int64_t>( tilex ) * tilesamples ) };
|
||||
auto const row { static_cast<std::size_t>( Row - static_cast<std::int64_t>( tilez ) * tilesamples ) };
|
||||
if( ( column >= tilestride ) || ( row >= tilestride ) ) { continue; }
|
||||
|
||||
auto &target { tileat( tilex, tilez ) };
|
||||
auto const index { row * tilestride + column };
|
||||
if( target.covered[ index ] == 0 ) {
|
||||
target.covered[ index ] = 1;
|
||||
target.height[ index ] = static_cast<float>( Height );
|
||||
target.material[ index ] = Material;
|
||||
target.owner[ index ] = m_current;
|
||||
++target.coverage;
|
||||
++m_covered;
|
||||
}
|
||||
else {
|
||||
auto const difference { Height - static_cast<double>( target.height[ index ] ) };
|
||||
if( std::abs( difference ) > conflictepsilon ) { ++m_stacked; }
|
||||
|
||||
if( std::abs( difference ) <= surfaceepsilon ) {
|
||||
// near enough to be the same ground: a slab, a path, a patch of
|
||||
// gravel laid over it. what is laid on top is what one sees, so
|
||||
// the upper surface takes the sample. comparing heights rather
|
||||
// than arrival order keeps the result independent of file order
|
||||
if( difference > 0.0 ) {
|
||||
target.height[ index ] = static_cast<float>( Height );
|
||||
target.material[ index ] = Material;
|
||||
target.owner[ index ] = m_current;
|
||||
}
|
||||
else if( ( std::abs( difference ) <= coincident )
|
||||
&& ( duplicate == 0u )
|
||||
&& ( m_current != notriangle ) ) {
|
||||
// two triangles meeting along an edge both claim the samples on
|
||||
// it, at the same height. the one that arrived second shows there
|
||||
// just as much as the first, so it is credited with the sample
|
||||
++m_trianglekept[ m_current ];
|
||||
}
|
||||
}
|
||||
else if( difference < 0.0 ) {
|
||||
// far apart: one of them is carried above the ground on a
|
||||
// structure, and the ground is the lower of the two
|
||||
target.height[ index ] = static_cast<float>( Height );
|
||||
target.material[ index ] = Material;
|
||||
target.owner[ index ] = m_current;
|
||||
}
|
||||
}
|
||||
target.minheight = std::min( target.minheight, static_cast<float>( Height ) );
|
||||
target.maxheight = std::max( target.maxheight, static_cast<float>( Height ) );
|
||||
}
|
||||
m_minheight = std::min( m_minheight, Height );
|
||||
m_maxheight = std::max( m_maxheight, Height );
|
||||
}
|
||||
|
||||
// the tile at the given coordinates, created on first use. the last one returned is
|
||||
// remembered: consecutive samples nearly always fall in the same tile, and a map's
|
||||
// elements stay where they are when it grows
|
||||
tile &
|
||||
tileat( std::int32_t const X, std::int32_t const Z ) {
|
||||
auto const key { heightfield::tile_key( X, Z ) };
|
||||
if( ( m_lasttile != nullptr ) && ( m_lasttilekey == key ) ) { return *m_lasttile; }
|
||||
m_lasttilekey = key;
|
||||
auto const lookup { m_tiles.find( key ) };
|
||||
if( lookup != m_tiles.end() ) { return *( m_lasttile = &lookup->second ); }
|
||||
tile fresh;
|
||||
fresh.x = X;
|
||||
fresh.z = Z;
|
||||
fresh.height.assign( tilestride * tilestride, 0.0f );
|
||||
fresh.material.assign( tilestride * tilestride, 0u );
|
||||
fresh.covered.assign( tilestride * tilestride, 0u );
|
||||
fresh.owner.assign( tilestride * tilestride, notriangle );
|
||||
return *( m_lasttile = &m_tiles.emplace( key, std::move( fresh ) ).first->second ); }
|
||||
|
||||
// lays the mip chain of one tile into the buffer, finest level first.
|
||||
// coarse levels are point sampled rather than averaged, deliberately: every level
|
||||
// keeps the samples that sit on the tile boundary exactly as its neighbour has them,
|
||||
// so the seam between two tiles stays identical at every level instead of needing
|
||||
// skirts to hide a mismatch. the cost is aliasing on distant terrain
|
||||
void
|
||||
encode( tile const &Tile, double const Scale, std::vector<std::uint8_t> &Buffer ) const {
|
||||
|
||||
auto *cursor { Buffer.data() };
|
||||
for( auto level { 0u }; level < miplevels; ++level ) {
|
||||
|
||||
auto const side { heightfield::samples_at_level( tilesamples, level ) };
|
||||
auto const stepping { std::size_t { 1 } << level };
|
||||
auto *heights { reinterpret_cast<std::uint16_t *>( cursor ) };
|
||||
auto *materials { cursor + static_cast<std::size_t>( side ) * side * sizeof( std::uint16_t ) };
|
||||
|
||||
for( auto row { 0u }; row < side; ++row ) {
|
||||
for( auto column { 0u }; column < side; ++column ) {
|
||||
auto const source { ( row * stepping ) * tilestride + ( column * stepping ) };
|
||||
auto const target { static_cast<std::size_t>( row ) * side + column };
|
||||
heights[ target ] = (
|
||||
Tile.covered[ source ] == 0 ?
|
||||
heightfield::nodata :
|
||||
static_cast<std::uint16_t>( ( Tile.height[ source ] - m_minheight ) / Scale ) );
|
||||
materials[ target ] = Tile.material[ source ];
|
||||
}
|
||||
}
|
||||
cursor += heightfield::level_bytes( tilesamples, level );
|
||||
}
|
||||
}
|
||||
|
||||
std::vector<std::uint8_t>
|
||||
compress( std::vector<std::uint8_t> const &Plain ) {
|
||||
|
||||
// one context for the whole cook: creating one per tile costs more than a tile
|
||||
if( m_context == nullptr ) { m_context.reset( ZSTD_createCCtx() ); }
|
||||
std::vector<std::uint8_t> encoded( ZSTD_compressBound( Plain.size() ) );
|
||||
auto const written {
|
||||
ZSTD_compressCCtx( m_context.get(), encoded.data(), encoded.size(), Plain.data(), Plain.size(), compressionlevel ) };
|
||||
if( 0 != ZSTD_isError( written ) ) {
|
||||
// a tile that will not compress is still a valid tile; it just stays raw
|
||||
return Plain;
|
||||
}
|
||||
encoded.resize( written );
|
||||
return encoded;
|
||||
}
|
||||
|
||||
void
|
||||
report(
|
||||
std::filesystem::path const &Path,
|
||||
heightfield::file_header const &Header,
|
||||
std::size_t const Bytes ) const {
|
||||
|
||||
auto const tilearea {
|
||||
( static_cast<std::int64_t>( Header.maxx ) - Header.minx + 1 )
|
||||
* ( static_cast<std::int64_t>( Header.maxz ) - Header.minz + 1 ) };
|
||||
auto const storedsamples {
|
||||
static_cast<std::size_t>( Header.tilecount ) * tilestride * tilestride };
|
||||
|
||||
report( "\n-- cooked heightfield\n" );
|
||||
report( " grid step %12.2f m\n", m_step );
|
||||
report( " tiles %12u (%u x %u samples, %.0f m each)\n",
|
||||
Header.tilecount, tilestride, tilestride, tilesamples * m_step );
|
||||
report( " mip levels %12u (finest %u, coarsest %u samples)\n",
|
||||
miplevels, tilestride, heightfield::samples_at_level( tilesamples, miplevels - 1 ) );
|
||||
report( " tile span %12d .. %d x %d .. %d z\n",
|
||||
Header.minx, Header.maxx, Header.minz, Header.maxz );
|
||||
report( " occupancy %12.1f %% of the tile bounding box\n",
|
||||
100.0 * static_cast<double>( Header.tilecount ) / static_cast<double>( tilearea ) );
|
||||
report( " samples with data %12zu %6.2f %% of the finest level\n",
|
||||
m_covered, 100.0 * static_cast<double>( m_covered ) / static_cast<double>( storedsamples ) );
|
||||
report( " height range %12.2f .. %.2f m (%.4f m per unit)\n",
|
||||
m_minheight, m_maxheight, Header.heightscale );
|
||||
report( " materials %12zu%s\n", m_materials.size(),
|
||||
m_materialoverflow > 0 ? " (palette full, some triangles share the last slot)" : "" );
|
||||
auto const coverage { material_coverage() };
|
||||
report( "\n-- what survived, per material\n" );
|
||||
report( " %-28s %10s %10s %8s %8s %s\n", "material", "claimed", "kept", "kept %", "vanished", "seen at x,z" );
|
||||
for( std::size_t index = 0; index < m_materials.size(); ++index ) {
|
||||
auto const claimed { index < m_materialclaimed.size() ? m_materialclaimed[ index ] : 0 };
|
||||
auto const kept { index < coverage.size() ? coverage[ index ] : 0 };
|
||||
auto const missed { index < m_materialmissed.size() ? m_materialmissed[ index ] : 0 };
|
||||
auto const where { index < m_materialwhere.size() ? m_materialwhere[ index ] : place{} };
|
||||
report( " %-28s %10zu %10zu %7.1f %% %8zu %.0f,%.0f\n",
|
||||
m_materials[ index ].c_str(), claimed, kept,
|
||||
claimed == 0 ? 0.0 : 100.0 * static_cast<double>( kept ) / static_cast<double>( claimed ),
|
||||
missed, where.x, where.z );
|
||||
}
|
||||
report( " triangles too small to reach a sample: %zu\n", m_toosmall );
|
||||
|
||||
report( "\n-- material tiling\n" );
|
||||
for( std::size_t index = 0; index < m_materials.size(); ++index ) {
|
||||
report( " %-28s %6.2f m per repeat, spread %5.1f %%%s\n",
|
||||
m_materials[ index ].c_str(), material_scale( index ), 100.0 * scale_spread( index ),
|
||||
( index < m_scalecounts.size() ) && ( m_scalecounts[ index ] > 0 ) ? "" : " (assumed)" );
|
||||
}
|
||||
report( " oversized skipped %12zu (edges over %.0f m, the water plane)\n", m_oversized, maxedge );
|
||||
report( " degenerate skipped %12zu\n", m_degenerate );
|
||||
report( " stacked samples %12zu (surfaces more than %.2f m apart)\n", m_stacked, conflictepsilon );
|
||||
if( true == m_compress ) {
|
||||
report( " payload %12.1f MB from %.1f MB (%.2fx, zstd level %d)\n",
|
||||
m_storedbytes / 1048576.0, m_plainbytes / 1048576.0,
|
||||
m_plainbytes > 0 ? static_cast<double>( m_plainbytes ) / static_cast<double>( m_storedbytes ) : 0.0,
|
||||
compressionlevel );
|
||||
}
|
||||
report( " file size %12.1f MB -> %s\n", Bytes / 1048576.0, Path.string().c_str() );
|
||||
}
|
||||
|
||||
void
|
||||
writecookedpreview( std::filesystem::path const &Path, heightfield::file_header const &Header ) const {
|
||||
|
||||
auto const width { static_cast<std::size_t>( Header.maxx - Header.minx + 1 ) };
|
||||
auto const height { static_cast<std::size_t>( Header.maxz - Header.minz + 1 ) };
|
||||
// one preview pixel per coarsest mip sample, capped so the image stays viewable
|
||||
auto pixelspertile { heightfield::samples_at_level( tilesamples, miplevels - 1 ) - 1 };
|
||||
while( ( pixelspertile > 1 ) && ( std::max( width, height ) * pixelspertile > 2048 ) ) { pixelspertile /= 2; }
|
||||
auto const previewwidth { width * pixelspertile };
|
||||
auto const previewheight { height * pixelspertile };
|
||||
auto const samplesper { tilesamples / pixelspertile };
|
||||
constexpr float sentinel { -1e30f };
|
||||
std::vector<float> preview( previewwidth * previewheight, sentinel );
|
||||
|
||||
for( auto const & [ key, tile ] : m_tiles ) {
|
||||
for( auto y { 0u }; y < pixelspertile; ++y ) {
|
||||
for( auto x { 0u }; x < pixelspertile; ++x ) {
|
||||
auto sum { 0.0 };
|
||||
std::size_t count { 0 };
|
||||
for( auto sy { 0u }; sy < samplesper; ++sy ) {
|
||||
for( auto sx { 0u }; sx < samplesper; ++sx ) {
|
||||
auto const index { ( y * samplesper + sy ) * tilestride + ( x * samplesper + sx ) };
|
||||
if( tile.covered[ index ] == 0 ) { continue; }
|
||||
sum += tile.height[ index ];
|
||||
++count;
|
||||
}
|
||||
}
|
||||
if( count == 0 ) { continue; }
|
||||
auto const column { static_cast<std::size_t>( tile.x - Header.minx ) * pixelspertile + x };
|
||||
auto const row { static_cast<std::size_t>( tile.z - Header.minz ) * pixelspertile + y };
|
||||
if( ( column >= previewwidth ) || ( row >= previewheight ) ) { continue; }
|
||||
preview[ row * previewwidth + column ] = static_cast<float>( sum / static_cast<double>( count ) );
|
||||
}
|
||||
}
|
||||
}
|
||||
writepreview( Path, preview, previewwidth, previewheight, sentinel );
|
||||
}
|
||||
|
||||
// metres of ground per texture repeat; materials the measurement never caught fall
|
||||
// back to something plausible rather than to zero, which would divide in the shader
|
||||
double
|
||||
material_scale( std::size_t const Index ) const {
|
||||
if( ( Index >= m_scalecounts.size() ) || ( m_scalecounts[ Index ] == 0 ) ) { return 4.0; }
|
||||
return m_scalesums[ Index ] / static_cast<double>( m_scalecounts[ Index ] ); }
|
||||
|
||||
// how much the measured tiling varies within one material, relative to its mean.
|
||||
// a single global scale per texture is only faithful where this is small; a material
|
||||
// the source scaled differently from place to place cannot be served by one number
|
||||
double
|
||||
scale_spread( std::size_t const Index ) const {
|
||||
if( ( Index >= m_scalecounts.size() ) || ( m_scalecounts[ Index ] < 2 ) ) { return 0.0; }
|
||||
auto const count { static_cast<double>( m_scalecounts[ Index ] ) };
|
||||
auto const mean { m_scalesums[ Index ] / count };
|
||||
auto const variance { std::max( 0.0, m_scalesquares[ Index ] / count - mean * mean ) };
|
||||
return mean > 1e-6 ? std::sqrt( variance ) / mean : 0.0; }
|
||||
|
||||
static void
|
||||
tally( std::vector<std::size_t> &Counters, std::uint8_t const Material, std::size_t const Amount ) {
|
||||
if( Material >= Counters.size() ) { Counters.resize( Material + 1, 0 ); }
|
||||
Counters[ Material ] += Amount; }
|
||||
|
||||
// what the finished grid actually holds, counted once the last triangle is in
|
||||
std::vector<std::size_t>
|
||||
material_coverage() const {
|
||||
std::vector<std::size_t> counters( m_materials.size(), 0 );
|
||||
for( auto const & [ key, tile ] : m_tiles ) {
|
||||
// only the samples a tile owns, so the shared edge is not counted twice
|
||||
for( std::uint32_t row = 0; row < tilesamples; ++row ) {
|
||||
for( std::uint32_t column = 0; column < tilesamples; ++column ) {
|
||||
auto const index { row * tilestride + column };
|
||||
if( tile.covered[ index ] == 0 ) { continue; }
|
||||
auto const material { tile.material[ index ] };
|
||||
if( material < counters.size() ) { ++counters[ material ]; }
|
||||
}
|
||||
}
|
||||
}
|
||||
return counters; }
|
||||
|
||||
template <typename... Arguments_>
|
||||
void
|
||||
report( char const *Format, Arguments_... Values ) const {
|
||||
if( true == m_verbose ) { std::printf( Format, Values... ); } }
|
||||
|
||||
static constexpr int compressionlevel { 9 };
|
||||
bool m_verbose { true };
|
||||
|
||||
double m_step { 2.0 };
|
||||
bool m_compress { true };
|
||||
std::unordered_map<std::int64_t, tile> m_tiles;
|
||||
std::vector<std::string> m_materials;
|
||||
// looked up by string_view, so finding a material does not build a string
|
||||
struct name_hash {
|
||||
using is_transparent = void;
|
||||
std::size_t operator()( std::string_view const Name ) const { return std::hash<std::string_view>{}( Name ); } };
|
||||
std::unordered_map<std::string, std::uint8_t, name_hash, std::equal_to<>> m_materialindex;
|
||||
double m_minheight { 1e30 }, m_maxheight { -1e30 };
|
||||
std::size_t m_covered { 0 }, m_stacked { 0 }, m_oversized { 0 }, m_degenerate { 0 };
|
||||
std::size_t m_materialoverflow { 0 };
|
||||
|
||||
// whether the sample at a grid position carries ground, wherever it is stored. the last
|
||||
// tile looked in is remembered, since neighbouring samples nearly always share one
|
||||
bool
|
||||
covered_at( std::int64_t const Column, std::int64_t const Row ) const {
|
||||
auto const tilex { static_cast<std::int32_t>( heightfield::floor_div( Column, tilesamples ) ) };
|
||||
auto const tilez { static_cast<std::int32_t>( heightfield::floor_div( Row, tilesamples ) ) };
|
||||
auto const tilekey { heightfield::tile_key( tilex, tilez ) };
|
||||
if( ( m_lastcovered == nullptr ) || ( m_lastcoveredkey != tilekey ) ) {
|
||||
auto const lookup { m_tiles.find( tilekey ) };
|
||||
if( lookup == m_tiles.end() ) { return false; }
|
||||
m_lastcovered = &lookup->second;
|
||||
m_lastcoveredkey = tilekey;
|
||||
}
|
||||
auto const column { static_cast<std::size_t>( Column - static_cast<std::int64_t>( tilex ) * tilesamples ) };
|
||||
auto const row { static_cast<std::size_t>( Row - static_cast<std::int64_t>( tilez ) * tilesamples ) };
|
||||
return m_lastcovered->covered[ row * tilestride + column ] != 0; }
|
||||
|
||||
// whether the ground around a sample is drawn at every level a triangle node is likely to
|
||||
// be seen at. the heightfield draws a cell only when all four of its corners carry ground,
|
||||
// and a coarser level has larger cells, so a sample can be drawn up close and fall into a
|
||||
// hole further out. corners are checked on every cell touching the sample, diagonals too
|
||||
bool
|
||||
drawn_around( std::int64_t const Column, std::int64_t const Row ) const {
|
||||
for( std::uint32_t level = 0; level < checkedlevels; ++level ) {
|
||||
auto const stride { static_cast<std::int64_t>( 1 ) << level };
|
||||
auto const basecolumn { heightfield::floor_div( Column, stride ) * stride };
|
||||
auto const baserow { heightfield::floor_div( Row, stride ) * stride };
|
||||
// a sample on a cell line touches the cells either side of it
|
||||
auto const firstcolumn { basecolumn == Column ? basecolumn - stride : basecolumn };
|
||||
auto const firstrow { baserow == Row ? baserow - stride : baserow };
|
||||
for( auto row { firstrow }; row <= baserow + stride; row += stride ) {
|
||||
for( auto column { firstcolumn }; column <= basecolumn + stride; column += stride ) {
|
||||
if( false == covered_at( column, row ) ) { return false; }
|
||||
}
|
||||
}
|
||||
}
|
||||
return true; }
|
||||
|
||||
void
|
||||
reject() {
|
||||
if( m_current != notriangle ) { m_trianglerejected[ m_current ] = 1u; } }
|
||||
|
||||
// levels up to 2048 m, the farthest a triangle node of the terrain is normally drawn
|
||||
static constexpr std::uint32_t checkedlevels { 3 };
|
||||
mutable tile const *m_lastcovered { nullptr };
|
||||
mutable std::int64_t m_lastcoveredkey { 0 };
|
||||
|
||||
// heights this close are one surface rather than two
|
||||
static constexpr double coincident { 0.05 };
|
||||
// share of its samples a triangle has to keep to count as shown by the heightfield
|
||||
static constexpr double representedshare { 0.9 };
|
||||
|
||||
std::uint32_t m_current { notriangle };
|
||||
std::vector<std::uint32_t> m_triangleclaimed, m_trianglekept;
|
||||
std::vector<std::uint8_t> m_trianglerejected;
|
||||
std::vector<std::size_t> m_materialclaimed, m_materialmissed;
|
||||
struct place { double x { 0.0 }, z { 0.0 }; bool seen { false }; };
|
||||
std::vector<place> m_materialwhere;
|
||||
std::size_t m_toosmall { 0 };
|
||||
std::vector<double> m_scalesums, m_scalesquares;
|
||||
std::vector<std::size_t> m_scalecounts;
|
||||
std::size_t m_storedbytes { 0 }, m_plainbytes { 0 };
|
||||
tile *m_lasttile { nullptr };
|
||||
std::int64_t m_lasttilekey { 0 };
|
||||
struct context_release {
|
||||
void operator()( ZSTD_CCtx *Context ) const { ZSTD_freeCCtx( Context ); } };
|
||||
std::unique_ptr<ZSTD_CCtx, context_release> m_context;
|
||||
};
|
||||
|
||||
// the coarsest candidate step that still holds for practically every sampled vertex.
|
||||
// every vertex sitting on a 2 m grid also sits on a 1 m and a 0.5 m one, so the finest
|
||||
// candidate always scores best and says nothing - it is the coarsest that names the grid
|
||||
// Report is called with each candidate and the share of vertices on it, for a caller that
|
||||
// wants to show the working
|
||||
template <typename Sample_, typename Report_>
|
||||
double
|
||||
detect_step( Sample_ const &Sample, Report_ &&Report ) {
|
||||
|
||||
auto best { 0.0 };
|
||||
for( auto const candidate : gridsteps ) {
|
||||
std::size_t hits { 0 };
|
||||
for( auto const &point : Sample ) {
|
||||
if( on_grid( point.x, candidate ) && on_grid( point.z, candidate ) ) { ++hits; }
|
||||
}
|
||||
auto const share { std::empty( Sample ) ? 0.0 : 100.0 * static_cast<double>( hits ) / static_cast<double>( std::size( Sample ) ) };
|
||||
Report( candidate, share );
|
||||
if( share >= gridthreshold ) { best = candidate; }
|
||||
}
|
||||
return ( best > 0.0 ? best : 1.0 );
|
||||
}
|
||||
|
||||
template <typename Sample_>
|
||||
double
|
||||
detect_step( Sample_ const &Sample ) {
|
||||
return detect_step( Sample, []( double, double ) {} );
|
||||
}
|
||||
|
||||
} // namespace terrain
|
||||
Reference in New Issue
Block a user