eu07/maszyna
17
0
Fork 0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-10-28 18:07:29 +01:00
Code Issues 154 Wiki Activity

Bake only the ground a heightfield replaces, at a step suited to each triangle

Browse Source 
Three changes to what goes into a baked heightfield and how it meets the geometry drawn
beside it.

The heightfield held every triangle laid in it, including those it does not replace and
that are therefore drawn as geometry too - so they were drawn twice, and a roof or a
bridge deck left in it bent the ground. Now the first pass lays everything and decides
what is replaced; the second lays the replaced triangles together with the neighbours
that support them - the owners of the corner samples of the cells around them, without
which those cells are not drawn - and only that is written. A first attempt that
repeated passes, dropping whatever bordered a hole, eroded the whole terrain away one
ring per pass: the edge of the ground is always a hole. The replaced triangles stay
replaced in the second pass, since their supports are all there and they have fewer
rivals. The triangles bake no longer keep in memory; they wait in a spill file beside
the heightfield while the scenery parses.

Triangles were refused for an edge longer than 200 m, meant for a water plane under the
whole map. On baltyk and tomaszewo that refused thousands of ordinary fields of a
hand-modelled terrain, and nothing there comes near a whole-map plane. Only triangles
standing on their edge - walls, trees, fences - are refused now. The bake log says why
the triangles not replaced are not: standing on edge, too small, under other surfaces,
next to a hole.

Hand-modelled terrain has triangles hundreds of metres long, elevation-model terrain
small ones along the line, and one grid step either loses the detail or samples a field
every metre. A bake now writes three heightfields a factor of four apart -
<scenario>.ehf, <scenario>.1.ehf, <scenario>.2.ehf - and each triangle goes to the
coarsest one that still puts sixteen samples along its longest edge, and to every
coarser one too, so the coarsest holds all the ground and a triangle at the edge of a
finer field can be replaced by a coarser field beneath it. All of them are picked up
together, and a missing or stale one sends the whole set to be baked again.

Where fields overlap each other or the geometry, the depth test decides: the terrain is
drawn with a polygon offset, negative for the reversed depth buffer and larger for each
coarser field, instead of being lowered by 2 cm - so geometry lying on the ground wins
at any distance and on any slope, and a finer field wins over a coarser one.

On tomaszewo the heightfields went from 176 MB to 10 MB, the load that bakes them from
5:16 to 3:11 and its peak memory from 15.7 GB to 8.5 GB - the scenery alone takes 8.0 -
while the triangles replaced went from 37,130 to 40,402. On baltyk from 19 MB to 2 MB,
with 6,670 triangles replaced against 4,040.
This commit is contained in:
maj00r
2026-09-17 17:24:25 +02:00
parent d2a5f284e3
commit 951a54cde7
6 changed files with 349 additions and 126 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
rendering/opengl33renderer.cpp
View File

@@ -2885,11 +2885,9 @@ void opengl33_renderer::Render_terrain()
[](auto const &Left, auto const &Right) { return Left->gridstep() < Right->gridstep(); });
for (std::size_t index = 0; index < m_terrain.size(); ++index)
{
// every field sits a little under the triangles, so where a baked heightfield and
// the geometry it was baked from are both drawn, the geometry is what shows. a
// coarser field is sunk further still, far enough that rounding cannot let it
// through, and little enough to be invisible at the spacing it is drawn at
m_terrain[index]->depthbias(index == 0 ? 0.02f : 0.1f);
// every field loses the depth test to geometry lying on it, and a coarser field
// to a finer one
m_terrain[index]->depthrank(static_cast<unsigned>(index));
}
}

6
rendering/terrainclipmap.cpp
View File

@@ -393,7 +393,7 @@ terrain_clipmap::render( glm::dvec3 const &Viewpoint, visibility const &Visible
m_instances[ tile.level ].insert(
m_instances[ tile.level ].end(),
{ static_cast<float>( tile.x * tilesize - Viewpoint.x ),
static_cast<float>( -Viewpoint.y ) - m_depthbias,
static_cast<float>( -Viewpoint.y ),
static_cast<float>( tile.z * tilesize - Viewpoint.z ),
static_cast<float>( tile.slot ),
static_cast<float>( tile.x * tilesize ),
@@ -403,6 +403,9 @@ terrain_clipmap::render( glm::dvec3 const &Viewpoint, visibility const &Visible
m_shader->bind();
m_vao->bind();
// the depth buffer is reversed - nearer is greater - so pushing back is a negative offset
::glEnable( GL_POLYGON_OFFSET_FILL );
::glPolygonOffset( -1.f - 2.f * static_cast<float>( m_depthrank ), -2.f - 4.f * static_cast<float>( m_depthrank ) );
::glUniform1f( m_uniforms.heightbias, m_reader.height( 0 ) );
::glUniform1f( m_uniforms.heightscale, m_reader.height( 1 ) - m_reader.height( 0 ) );
@@ -462,6 +465,7 @@ terrain_clipmap::render( glm::dvec3 const &Viewpoint, visibility const &Visible
}
m_stats.drawn = drawn;
::glDisable( GL_POLYGON_OFFSET_FILL );
gl::buffer::unbind( gl::buffer::TEXTURE_BUFFER );
::glActiveTexture( GL_TEXTURE0 );
opengl_texture::reset_unit_cache();

9
rendering/terrainclipmap.cppm
View File

@@ -88,9 +88,10 @@ public:
// metres between samples at the finest level; a scenery's fields are drawn from the
// finest to the coarsest so that the depth test keeps the better surface
float gridstep() const { return m_reader.gridstep(); }
// sinks this field by the given amount, so a coarse field cannot win fragments from
// a finer one describing the same ground
void depthbias( float const Metres ) { m_depthbias = Metres; }
// pushes this field back in depth by the given rank, so that geometry lying on it wins the
// depth test, and a coarser field loses to a finer one describing the same ground. the
// offset is in depth, scaled by slope, so it holds at any distance and on any incline
void depthrank( unsigned const Rank ) { m_depthrank = Rank; }
terrain_statistics const &stats() const { return m_stats; }
std::string const &path() const { return m_path; }
@@ -173,7 +174,7 @@ private:
std::size_t m_reportedresidency { 0 };
bool m_reported { false };
float m_range { 6000.f };
float m_depthbias { 0.f };
unsigned m_depthrank { 0 };
// tiles allowed to stay resident; at 129 samples a tile costs about 50 kB of texture
std::size_t m_budget { 2048 };
// tiles put on the gpu per frame. a tile is about 50 kB of texture, so this is not about

4
scene/heightfieldformat.h
View File

@@ -38,7 +38,9 @@ inline constexpr std::uint32_t version { 5 };
// 2 - every triangle as drawn; a triangle stays drawn unless the heightfield shows it, and a
// triangle bordering a hole always stays drawn
// 3 - as 2, with a hole checked for diagonally and at the coarser levels too
inline constexpr std::uint32_t selection_rules { 3 };
// 4 - the heightfield holds only the triangles it shows, and their supports
// 5 - no triangle refused for its size; each goes to a field with a step suited to it
inline constexpr std::uint32_t selection_rules { 5 };
// height value reserved to mean "this sample carries no terrain"; a cooked map is a
// ribbon inside a much larger rectangle, so most edge tiles are partly empty

141
scene/terraincooker.h
View File

@@ -112,13 +112,15 @@ public:
static constexpr std::uint32_t tilestride { heightfield::samples_at_level( tilesamples, 0 ) };
// mip chain down to a 5 sample side, which is where a tile stops being worth its entry
static constexpr std::uint32_t miplevels { heightfield::level_count( tilesamples, 5 ) };
// triangles larger than this are not terrain: the water plane spans the whole map
static constexpr double maxedge { 200.0 };
// two surfaces closer than this are the same ground, one laid over the other, rather
// than a deck carried above it. a kerb or a slab stands centimetres proud; a viaduct
// stands metres
static constexpr double surfaceepsilon { 0.5 };
// whether nothing has been laid, so there is nothing to write
bool
empty() const { return m_tiles.empty(); }
// stop printing the cooking report. the tool wants the tables; the engine, baking on
// a first run, wants one line in its own log instead
void
@@ -144,23 +146,19 @@ public:
m_triangleclaimed.resize( size, 0u );
m_trianglekept.resize( size, 0u );
m_trianglerejected.resize( size, 0u );
m_trianglelaid.resize( size, 0u );
}
m_trianglelaid[ Triangle ] = 1u;
}
vertex const points[ 3 ] { A, B, C };
for( auto index { 0u }; index < 3u; ++index ) {
if( planar_length( points[ index ], points[ ( index + 1 ) % 3 ] ) > maxedge ) {
++m_oversized;
reject();
return;
}
}
// barycentric setup in the horizontal plane; a triangle standing on its edge
// covers no samples and would divide by zero
// barycentric setup in the horizontal plane. a triangle standing on its edge covers
// no samples and would divide by zero; it is a wall, a tree or a fence, never ground,
// and it is the only kind of triangle refused outright - a long one is laid like any
// other, however large
auto const area { ( B.x - A.x ) * ( C.z - A.z ) - ( C.x - A.x ) * ( B.z - A.z ) };
if( std::abs( area ) < 1e-9 ) { ++m_degenerate; reject(); return; }
if( std::abs( area ) < 1e-9 ) { ++m_degenerate; reject( refusal::upright ); return; }
auto const inversearea { 1.0 / area };
// a palette slot is taken only by a material that reaches this far, so triangles
// standing on their edge do not fill the palette
@@ -207,8 +205,9 @@ public:
}
// whether the heightfield shows this triangle well enough for it not to be drawn on its
// own: it was laid at all, it reached at least one sample, and it still holds nearly
// every sample it wanted once everything else has been laid. valid after finish()
// own: it was laid at all, it reached at least one sample, it still holds nearly every
// sample it wanted once everything else has been laid, and the ground around it is drawn.
// valid after settle()
bool
represented( std::uint32_t const Triangle ) const {
if( Triangle >= m_triangleclaimed.size() ) { return false; }
@@ -217,12 +216,51 @@ public:
if( claimed == 0 ) { return false; }
return static_cast<double>( m_trianglekept[ Triangle ] ) >= representedshare * static_cast<double>( claimed ); }
// marks in Support every triangle that owns a corner sample of a cell around a sample of a
// represented triangle. those samples have to stay in the heightfield for the represented
// triangles to be drawn whole, even though their own triangles are drawn as geometry.
// valid after settle()
void
finish( std::filesystem::path const &Path, std::filesystem::path const &Previewpath ) {
mark_support( std::vector<std::uint8_t> &Support ) const {
for( auto const & [ key, entry ] : m_tiles ) {
for( std::uint32_t row = 0; row < tilesamples; ++row ) {
for( std::uint32_t column = 0; column < tilesamples; ++column ) {
auto const owner { entry.owner[ row * tilestride + column ] };
if( false == represented( owner ) ) { continue; }
auto const globalcolumn { static_cast<std::int64_t>( entry.x ) * tilesamples + column };
auto const globalrow { static_cast<std::int64_t>( entry.z ) * tilesamples + row };
for_each_corner( globalcolumn, globalrow, [ & ]( std::int64_t const Cornercolumn, std::int64_t const Cornerrow ) {
auto const corner { owner_at( Cornercolumn, Cornerrow ) };
if( corner < Support.size() ) { Support[ corner ] = 1u; } } );
}
}
} }
if( true == m_tiles.empty() ) {
report( " nothing to cook\n" );
return; }
// why the triangles the heightfield does not show are not shown, counted by settle()
struct refusals {
std::size_t upright { 0 }; // standing on its edge: a wall, a tree, a fence
std::size_t unreached { 0 }; // smaller than the gap between samples
std::size_t outvoted { 0 }; // lost too many samples to other surfaces
std::size_t bordering { 0 }; // next to ground the heightfield does not draw
};
refusals const &refused() const { return m_refused; }
// why one laid triangle is not shown: it counts one in the member of Refusals naming the
// reason, or in none of them when it is shown. valid after settle()
void
refusal_of( std::uint32_t const Triangle, refusals &Refusals ) const {
if( ( Triangle >= m_trianglelaid.size() ) || ( m_trianglelaid[ Triangle ] == 0 ) ) { return; }
auto const reason { m_trianglerejected[ Triangle ] };
if( reason == refusal::upright ) { ++Refusals.upright; }
else if( m_triangleclaimed[ Triangle ] == 0 ) { ++Refusals.unreached; }
else if( static_cast<double>( m_trianglekept[ Triangle ] ) < representedshare * static_cast<double>( m_triangleclaimed[ Triangle ] ) ) { ++Refusals.outvoted; }
else if( reason == refusal::bordering ) { ++Refusals.bordering; } }
// works out, once every triangle has been laid, which of them the heightfield shows
void
settle() {
if( true == m_settled ) { return; }
m_settled = true;
// every sample counted once, in the tile that owns it rather than in the
// neighbours carrying a copy of the shared edge
@@ -239,12 +277,30 @@ public:
auto const globalrow { static_cast<std::int64_t>( entry.z ) * tilesamples + row };
if( m_trianglerejected[ owner ] != 0 ) { continue; }
if( false == drawn_around( globalcolumn, globalrow ) ) {
m_trianglerejected[ owner ] = 1u;
m_trianglerejected[ owner ] = refusal::bordering;
}
}
}
}
// a caller spreading its triangles over several cookers numbers them across all of
// them; only the ones laid here are counted, which refusal_of sees to
m_refused = {};
for( std::size_t triangle = 0; triangle < m_triangleclaimed.size(); ++triangle ) {
refusal_of( static_cast<std::uint32_t>( triangle ), m_refused );
}
}
// writes the heightfield, and a greyscale preview of it when a path for one is given
void
finish( std::filesystem::path const &Path, std::filesystem::path const &Previewpath ) {
if( true == m_tiles.empty() ) {
report( " nothing to cook\n" );
return; }
settle();
auto const scale { std::max( 1.0, m_maxheight - m_minheight ) / heightfield::heightrange };
std::vector<tile const *> ordered;
@@ -579,8 +635,7 @@ private:
m_materials[ index ].c_str(), material_scale( index ), 100.0 * scale_spread( index ),
( index < m_scalecounts.size() ) && ( m_scalecounts[ index ] > 0 ) ? "" : " (assumed)" );
}
report( " oversized skipped %12zu (edges over %.0f m, the water plane)\n", m_oversized, maxedge );
report( " degenerate skipped %12zu\n", m_degenerate );
report( " upright skipped %12zu (standing on an edge: walls, trees, fences)\n", m_degenerate );
report( " stacked samples %12zu (surfaces more than %.2f m apart)\n", m_stacked, conflictepsilon );
if( true == m_compress ) {
report( " payload %12.1f MB from %.1f MB (%.2fx, zstd level %d)\n",
@@ -687,7 +742,7 @@ private:
std::size_t operator()( std::string_view const Name ) const { return std::hash<std::string_view>{}( Name ); } };
std::unordered_map<std::string, std::uint8_t, name_hash, std::equal_to<>> m_materialindex;
double m_minheight { 1e30 }, m_maxheight { -1e30 };
std::size_t m_covered { 0 }, m_stacked { 0 }, m_oversized { 0 }, m_degenerate { 0 };
std::size_t m_covered { 0 }, m_stacked { 0 }, m_degenerate { 0 };
std::size_t m_materialoverflow { 0 };
// whether the sample at a grid position carries ground, wherever it is stored. the last
@@ -713,6 +768,16 @@ private:
// hole further out. corners are checked on every cell touching the sample, diagonals too
bool
drawn_around( std::int64_t const Column, std::int64_t const Row ) const {
auto drawn { true };
for_each_corner( Column, Row, [ & ]( std::int64_t const Cornercolumn, std::int64_t const Cornerrow ) {
if( false == covered_at( Cornercolumn, Cornerrow ) ) { drawn = false; } } );
return drawn; }
// calls Visit with every sample that is a corner of a cell touching the given sample, at
// every level a triangle node of the terrain is likely to be seen at
template <typename Visit_>
void
for_each_corner( std::int64_t const Column, std::int64_t const Row, Visit_ &&Visit ) const {
for( std::uint32_t level = 0; level < checkedlevels; ++level ) {
auto const stride { static_cast<std::int64_t>( 1 ) << level };
auto const basecolumn { heightfield::floor_div( Column, stride ) * stride };
@@ -722,15 +787,32 @@ private:
auto const firstrow { baserow == Row ? baserow - stride : baserow };
for( auto row { firstrow }; row <= baserow + stride; row += stride ) {
for( auto column { firstcolumn }; column <= basecolumn + stride; column += stride ) {
if( false == covered_at( column, row ) ) { return false; }
Visit( column, row );
}
}
}
return true; }
} }
// the triangle a sample came from, or notriangle
std::uint32_t
owner_at( std::int64_t const Column, std::int64_t const Row ) const {
auto const tilex { static_cast<std::int32_t>( heightfield::floor_div( Column, tilesamples ) ) };
auto const tilez { static_cast<std::int32_t>( heightfield::floor_div( Row, tilesamples ) ) };
auto const lookup { m_tiles.find( heightfield::tile_key( tilex, tilez ) ) };
if( lookup == m_tiles.end() ) { return notriangle; }
auto const column { static_cast<std::size_t>( Column - static_cast<std::int64_t>( tilex ) * tilesamples ) };
auto const row { static_cast<std::size_t>( Row - static_cast<std::int64_t>( tilez ) * tilesamples ) };
return lookup->second.owner[ row * tilestride + column ]; }
// why a triangle was refused, as held per triangle
enum refusal : std::uint8_t {
accepted = 0,
upright = 1,
bordering = 2
};
void
reject() {
if( m_current != notriangle ) { m_trianglerejected[ m_current ] = 1u; } }
reject( refusal const Reason ) {
if( m_current != notriangle ) { m_trianglerejected[ m_current ] = Reason; } }
// levels up to 2048 m, the farthest a triangle node of the terrain is normally drawn
static constexpr std::uint32_t checkedlevels { 3 };
@@ -743,8 +825,11 @@ private:
static constexpr double representedshare { 0.9 };
std::uint32_t m_current { notriangle };
bool m_settled { false };
refusals m_refused;
std::vector<std::uint32_t> m_triangleclaimed, m_trianglekept;
std::vector<std::uint8_t> m_trianglerejected;
std::vector<std::uint8_t> m_trianglelaid;
std::vector<std::size_t> m_materialclaimed, m_materialmissed;
struct place { double x { 0.0 }, z { 0.0 }; bool seen { false }; };
std::vector<place> m_materialwhere;

307
simulation/terrainbake.cpp
View File

@@ -8,11 +8,15 @@ http://mozilla.org/MPL/2.0/.
*/
module;
#include <algorithm>
#include <array>
#include <cmath>
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <filesystem>
#include <memory>
#include <fstream>
#include <string>
#include <string_view>
@@ -33,11 +37,24 @@ namespace simulation::terrainbake {
namespace {
// triangles are buffered until there are enough vertices to say what the grid step is, and
// only then rasterised. the step has to be known before the first sample is placed, and it
// is a property of the whole terrain rather than of the first triangle that turns up
// vertices kept from the start of the scenery to tell the grid step from
constexpr std::size_t samplegoal { 200000 };
// Terrain comes in two kinds: modelled by hand with triangles hundreds of metres long, or taken
// from an elevation model with small triangles along the line. One grid step for both either
// loses the detail or samples a field of large triangles every metre. So a bake writes several
// heightfields at steps a factor apart. A triangle's own field is the coarsest that still puts
// samplesperedge samples along its longest edge, and it is laid into that field and every
// coarser one: the coarsest field then holds all the ground, as the layers of a level of
// detail scheme do, and a triangle at the edge of a finer field - next to ground that field
// does not hold - can still be replaced by a coarser one lying under it. Where two fields
// hold the same ground the finer one wins the depth test. A flat triangle sampled coarsely is
// the same plane; what a coarser step gives up is the exact line of the creases between
// triangles, which a hand modelled terrain does not have to spare anyway.
constexpr std::uint32_t fieldcount { 3 };
constexpr double fieldstride { 4.0 };
constexpr double samplesperedge { 16.0 };
// the node table written beside a baked heightfield
constexpr char tablemagic[ 8 ] { 'E', 'U', '0', '7', 'H', 'F', 'N', 'D' };
// 2: nodes keyed by file, line, include parameters and placement rather than by their text
@@ -65,9 +82,11 @@ struct file_stamp {
std::int64_t time { 0 };
};
struct buffered {
terrain::vertex v[ 3 ];
std::uint32_t node; // the node it came from, which holds its material
// one triangle as it waits in the spill file for the passes: its corners in world space with
// their texture coordinates, the node it came from, which holds its material, and its number
struct spilled {
double corners[ 15 ];
std::uint32_t node;
std::uint32_t id;
};
@@ -76,18 +95,21 @@ struct baked_node {
std::uint64_t key;
std::uint32_t first;
std::uint32_t count;
// kept once per node rather than once per buffered triangle
// kept once per node rather than once per spilled triangle
std::string material;
};
struct {
// baking
// baking. triangles go to a spill file as the scenery is parsed; the heightfield is made
// from it once everything is in
bool collecting { false };
bool configured { false };
terrain::cooker cooker;
std::vector<buffered> buffer;
std::ofstream spill;
std::string spillpath;
std::vector<terrain::vertex> sample;
std::vector<baked_node> nodes;
std::uint32_t triangles { 0 };
// per triangle, once baked: whether the heightfield shows it
std::vector<std::uint8_t> shown;
std::chrono::steady_clock::time_point started;
// planning
std::unordered_map<std::uint64_t, table_record> table;
@@ -97,10 +119,12 @@ struct {
std::unordered_map<std::string, file_stamp> files;
} state;
// where a field is written: the finest beside the scenery as <scenario>.ehf, coarser ones as
// <scenario>.<field>.ehf
std::string
heightfield_path( std::string const &Sceneryfile ) {
heightfield_path( std::string const &Sceneryfile, std::uint32_t const Field = 0 ) {
return scenery_sidecar( Sceneryfile, ".ehf" );
return scenery_sidecar( Sceneryfile, Field == 0 ? std::string( ".ehf" ) : "." + std::to_string( Field ) + ".ehf" );
}
std::string
@@ -148,7 +172,7 @@ write_table( std::string const &Path ) {
auto &entry { shown[ node.key ] };
entry.resize( node.count, false );
for( std::uint32_t index = 0; index < node.count; ++index ) {
if( true == state.cooker.represented( node.first + index ) ) { entry[ index ] = true; }
if( state.shown[ node.first + index ] != 0 ) { entry[ index ] = true; }
}
}
@@ -196,41 +220,62 @@ write_table( std::string const &Path ) {
+ " triangle nodes are shown by the heightfield, " + std::to_string( full ) + " of them entirely" );
}
void
rasterize( buffered const &Triangle ) {
// the field a triangle belongs to, given the finest step
std::uint32_t
field_of( terrain::vertex const &A, terrain::vertex const &B, terrain::vertex const &C, double const Basestep ) {
state.cooker.rasterize(
Triangle.v[ 0 ], Triangle.v[ 1 ], Triangle.v[ 2 ], state.nodes[ Triangle.node ].material, Triangle.id );
auto const longest { std::max( { terrain::planar_length( A, B ), terrain::planar_length( B, C ), terrain::planar_length( C, A ) } ) };
std::uint32_t field { 0 };
auto step { Basestep };
while( ( field + 1 < fieldcount ) && ( longest / ( step * fieldstride ) >= samplesperedge ) ) {
step *= fieldstride;
++field;
}
return field;
}
// works the grid step out of what has been buffered, then puts the buffer through
void
configure_and_drain() {
using fields = std::array<std::unique_ptr<terrain::cooker>, fieldcount>;
std::vector<terrain::vertex> sample;
sample.reserve( state.buffer.size() * 3 );
for( auto const &triangle : state.buffer ) {
sample.emplace_back( triangle.v[ 0 ] );
sample.emplace_back( triangle.v[ 1 ] );
sample.emplace_back( triangle.v[ 2 ] );
// one pass over the spill file: lays every triangle into fresh cookers for its own field and
// the coarser ones, wherever Keep has the bit of that field set
fields
lay( std::string const &Spillpath, double const Basestep, std::vector<std::uint8_t> const &Keep,
std::vector<std::uint8_t> *Ownfield = nullptr ) {
fields result;
auto step { Basestep };
for( auto &cooker : result ) {
cooker = std::make_unique<terrain::cooker>();
if( false == loadprofile::enabled() ) {
// the per-material survival table is what a bad bake is diagnosed from, so it is
// printed when the load is being measured anyway
cooker->quiet();
}
cooker->configure( step, true );
step *= fieldstride;
}
auto const step { terrain::detect_step( sample ) };
WriteLog( "Terrain bake: grid step " + std::to_string( step ) + " m, from " + std::to_string( sample.size() ) + " sampled vertices" );
if( false == loadprofile::enabled() ) {
// the per-material survival table is what a bad bake is diagnosed from, so it is
// printed when the load is being measured anyway
state.cooker.quiet();
std::ifstream input( Spillpath, std::ios::binary );
std::vector<spilled> chunk( 4096 );
while( input ) {
input.read( reinterpret_cast<char *>( chunk.data() ), static_cast<std::streamsize>( chunk.size() * sizeof( spilled ) ) );
auto const count { static_cast<std::size_t>( input.gcount() ) / sizeof( spilled ) };
for( std::size_t index = 0; index < count; ++index ) {
auto const &triangle { chunk[ index ] };
if( Keep[ triangle.id ] == 0 ) { continue; }
auto const corner = [ &triangle ]( std::size_t const Corner ) {
auto const *values { triangle.corners + Corner * 5 };
return terrain::vertex { values[ 0 ], values[ 1 ], values[ 2 ], values[ 3 ], values[ 4 ] }; };
auto const a { corner( 0 ) }, b { corner( 1 ) }, c { corner( 2 ) };
auto const own { field_of( a, b, c, Basestep ) };
if( Ownfield != nullptr ) { ( *Ownfield )[ triangle.id ] = static_cast<std::uint8_t>( own ); }
for( auto field { own }; field < fieldcount; ++field ) {
if( ( Keep[ triangle.id ] & ( 1u << field ) ) == 0 ) { continue; }
result[ field ]->rasterize( a, b, c, state.nodes[ triangle.node ].material, triangle.id );
}
}
}
state.cooker.configure( step, true );
state.configured = true;
for( auto const &triangle : state.buffer ) {
rasterize( triangle );
}
state.buffer.clear();
state.buffer.shrink_to_fit();
return result;
}
} // anonymous namespace
@@ -298,26 +343,45 @@ begin( std::string const &Sceneryfile ) {
auto const path { heightfield_path( Sceneryfile ) };
std::error_code error;
if( true == FileExists( path ) ) {
// a baked file is a cache: one this build cannot read, one cooked under older rules
// about what counts as terrain, or one whose node table is missing, is thrown away
// and baked again rather than used
heightfield::reader probe;
if( ( true == probe.open( path ) )
&& ( probe.selection() == heightfield::selection_rules )
&& ( true == load_table( table_path( path ) ) ) ) {
Global.terrain_heightfields.push_back( path );
WriteLog( "Terrain bake: using \"" + path + "\", baked earlier, " + std::to_string( state.table.size() ) + " triangle nodes replaced by it" );
if( true == FileExists( table_path( path ) ) ) {
// a bake is a cache: one this build cannot read, one cooked under older rules about
// what counts as terrain, or one missing a part, is thrown away and baked again
// rather than used
std::vector<std::string> baked;
auto usable { true == load_table( table_path( path ) ) };
for( std::uint32_t field = 0; ( field < fieldcount ) && ( true == usable ); ++field ) {
auto const fieldpath { heightfield_path( Sceneryfile, field ) };
if( false == FileExists( fieldpath ) ) { continue; }
heightfield::reader probe;
usable = ( true == probe.open( fieldpath ) ) && ( probe.selection() == heightfield::selection_rules );
baked.push_back( fieldpath );
}
if( ( true == usable ) && ( false == baked.empty() ) ) {
for( auto const &fieldpath : baked ) {
Global.terrain_heightfields.push_back( fieldpath );
}
WriteLog(
"Terrain bake: using " + std::to_string( baked.size() ) + " heightfields baked earlier, "
+ std::to_string( state.table.size() ) + " triangle nodes replaced by them" );
return;
}
WriteLog( "Terrain bake: \"" + path + "\" was baked under different rules, baking it again" );
probe.close();
WriteLog( "Terrain bake: the heightfields beside this scenery were baked under different rules, baking them again" );
state.table.clear();
state.masks.clear();
std::filesystem::remove( path, error );
for( std::uint32_t field = 0; field < fieldcount; ++field ) {
std::filesystem::remove( heightfield_path( Sceneryfile, field ), error );
}
std::filesystem::remove( table_path( path ), error );
}
// the triangles wait on disk beside the heightfield rather than in memory: a large scenery
// holds millions of them, and the bake reads them several times
state.spillpath = path + ".bake";
state.spill.open( state.spillpath, std::ios::binary | std::ios::trunc );
if( false == state.spill.good() ) {
ErrorLog( "Terrain bake: cannot write \"" + state.spillpath + "\", not baking" );
return;
}
WriteLog( "Terrain bake: no heightfield for this scenery, baking one. This load will be slow; the next will not" );
state.collecting = true;
state.started = std::chrono::steady_clock::now();
@@ -395,23 +459,27 @@ add( std::uint64_t const Key, std::vector<world_vertex> const &Vertices, std::st
auto const node { static_cast<std::uint32_t>( state.nodes.size() ) };
state.nodes.push_back( { Key, state.triangles, count, std::string { Material } } );
for( std::uint32_t triangle = 0; triangle < count; ++triangle ) {
auto const vertex = [ & ]( std::size_t const Index ) {
auto const &source { Vertices[ triangle * 3u + Index ] };
return terrain::vertex{ source.position.x, source.position.y, source.position.z, source.texture.x, source.texture.y }; };
buffered entry { { vertex( 0 ), vertex( 1 ), vertex( 2 ) }, node, state.triangles + triangle };
if( true == state.configured ) {
rasterize( entry );
continue;
spilled triangle {};
triangle.node = node;
for( std::uint32_t index = 0; index < count; ++index ) {
for( std::size_t corner = 0; corner < 3; ++corner ) {
auto const &source { Vertices[ index * 3u + corner ] };
auto *values { triangle.corners + corner * 5 };
values[ 0 ] = source.position.x;
values[ 1 ] = source.position.y;
values[ 2 ] = source.position.z;
values[ 3 ] = source.texture.x;
values[ 4 ] = source.texture.y;
// the grid step is a property of the whole terrain; the first vertices that turn up
// are enough to tell it
if( state.sample.size() < samplegoal ) {
state.sample.push_back( { values[ 0 ], values[ 1 ], values[ 2 ], values[ 3 ], values[ 4 ] } );
}
}
state.buffer.emplace_back( entry );
triangle.id = state.triangles + index;
state.spill.write( reinterpret_cast<char const *>( &triangle ), sizeof( triangle ) );
}
state.triangles += count;
if( ( false == state.configured ) && ( state.buffer.size() * 3 >= samplegoal ) ) {
configure_and_drain();
}
}
void
@@ -427,37 +495,102 @@ finish( std::string const &Sceneryfile ) {
if( false == Global.terrain_heightfields.empty() ) {
// the scenery named its own heightfield part way through, and that one wins
WriteLog( "Terrain bake: the scenery names its own heightfield, nothing baked" );
state.spill.close();
std::error_code error;
std::filesystem::remove( state.spillpath, error );
state = {};
return;
}
if( false == state.configured ) {
// a scenery with less terrain than the sample goal never triggered the drain
if( true == state.buffer.empty() ) {
WriteLog( "Terrain bake: the scenery holds no triangles, nothing baked" );
state.collecting = false;
return;
}
configure_and_drain();
state.spill.close();
auto const cleanup = []() {
std::error_code error;
std::filesystem::remove( state.spillpath, error );
state = {};
};
if( state.triangles == 0 ) {
WriteLog( "Terrain bake: the scenery holds no triangles, nothing baked" );
cleanup();
return;
}
auto const path { heightfield_path( Sceneryfile ) };
state.cooker.finish( path, {} );
write_table( table_path( path ) );
auto const step { terrain::detect_step( state.sample ) };
WriteLog( "Terrain bake: grid step " + std::to_string( step ) + " m, from " + std::to_string( state.sample.size() ) + " sampled vertices" );
// The heightfield should hold the triangles it replaces and nothing else: anything it
// holds that is also drawn as geometry is drawn twice, and a roof or a bridge deck left in
// it bends the ground. But a replaced triangle is only drawn whole if every cell around it
// is, and those cells have corners owned by neighbours that are not replaced - the edge of
// the ground, a slope too steep, a patch too small. So the first pass lays everything and
// decides what is replaced; the second lays the replaced triangles together with the
// neighbours that support them, and that is what gets written. The replaced triangles
// stay replaced in it: their supports are all there, and they have fewer rivals than
// before. The supports are drawn both ways, and the depth offset lets the geometry win.
std::vector<std::uint8_t> ownfield( state.triangles, 0u );
auto first { lay( state.spillpath, step, std::vector<std::uint8_t>( state.triangles, 0xffu ), &ownfield ) };
// per triangle, the fields it goes into on the second pass: those that show it, and those
// where it supports a triangle they show
std::vector<std::uint8_t> keep( state.triangles, 0u );
state.shown.assign( state.triangles, 0u );
for( std::uint32_t field = 0; field < fieldcount; ++field ) {
first[ field ]->settle();
for( std::uint32_t id = 0; id < state.triangles; ++id ) {
if( true == first[ field ]->represented( id ) ) {
state.shown[ id ] = 1u;
keep[ id ] |= static_cast<std::uint8_t>( 1u << field );
}
}
}
// a triangle no field shows, counted by the reason its own field gives
terrain::cooker::refusals refused;
for( std::uint32_t id = 0; id < state.triangles; ++id ) {
if( state.shown[ id ] == 0 ) { first[ ownfield[ id ] ]->refusal_of( id, refused ); }
}
WriteLog(
"Terrain bake: not replaced - " + std::to_string( refused.upright ) + " standing on edge, "
+ std::to_string( refused.unreached ) + " too small, " + std::to_string( refused.outvoted ) + " under other surfaces, "
+ std::to_string( refused.bordering ) + " next to a hole" );
std::vector<std::uint8_t> support( state.triangles, 0u );
for( std::uint32_t field = 0; field < fieldcount; ++field ) {
std::fill( support.begin(), support.end(), 0u );
first[ field ]->mark_support( support );
first[ field ].reset();
for( std::uint32_t id = 0; id < state.triangles; ++id ) {
if( support[ id ] != 0 ) { keep[ id ] |= static_cast<std::uint8_t>( 1u << field ); }
}
}
auto const shown { static_cast<std::size_t>( std::count( state.shown.begin(), state.shown.end(), 1u ) ) };
std::size_t supports { 0 };
for( std::uint32_t id = 0; id < state.triangles; ++id ) {
if( ( keep[ id ] != 0 ) && ( state.shown[ id ] == 0 ) ) { ++supports; }
}
auto const second { lay( state.spillpath, step, keep ) };
std::string written;
std::error_code error;
auto const bytes { std::filesystem::file_size( path, error ) };
std::uintmax_t bytes { 0 };
for( std::uint32_t field = 0; field < fieldcount; ++field ) {
auto const path { heightfield_path( Sceneryfile, field ) };
std::filesystem::remove( path, error );
if( true == second[ field ]->empty() ) { continue; }
second[ field ]->finish( path, {} );
auto const size { std::filesystem::file_size( path, error ) };
bytes += ( error ? 0 : size );
written +=
( written.empty() ? "" : ", " ) + ( "\"" + path + "\" at " )
+ std::to_string( step * std::pow( fieldstride, field ) ) + " m";
}
write_table( table_path( heightfield_path( Sceneryfile ) ) );
auto const seconds {
std::chrono::duration<double>( std::chrono::steady_clock::now() - state.started ).count() };
WriteLog(
"Terrain bake: wrote \"" + path + "\" from " + std::to_string( state.triangles )
+ " triangles in " + std::to_string( seconds ) + " s"
+ ( error ? std::string() : ", " + std::to_string( bytes / 1048576 ) + " MB" ) );
"Terrain bake: wrote " + written + "; " + std::to_string( shown ) + " of " + std::to_string( state.triangles )
+ " triangles replaced and " + std::to_string( supports ) + " more supporting them, "
+ std::to_string( seconds ) + " s, " + std::to_string( bytes / 1048576 ) + " MB" );
// the triangles of this load were all drawn; the table applies from the next one
state.collecting = false;
state.buffer.clear();
state.nodes.clear();
cleanup();
}
} // namespace simulation::terrainbake