17
0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-10-31 15:49:06 +01:00

Cook the ground into a quadtree of mesh tiles, and keep the mesh the scenery drew

The ground in a scenery is already an irregular mesh, so the cook keeps it: it cuts the
triangles along the tile grid, welds what coincides, and builds each coarser level by
collapsing edges. Nothing is resampled, nothing is interpolated, and a hole the scenery left
stays a hole. What was here before threw the triangles away, kept their vertices and built a
fresh Delaunay triangulation over them, which filled every hole, merged the two heights of
every retaining wall into one, and - because it never erased the super-triangle - emitted
indices three too large together with triangles reaching out to the artificial corners.

Tiles form a quadtree, as Cesium Quantized Mesh intends. A distant tile is a large one, so the
horizon costs tens of tiles rather than the thousands a fixed grid needed, and the level to
draw follows from the tile's own geometric error against what the screen can resolve - which is
what the format carries that field for. Every vertex on a tile's border survives into all of
its coarser levels, and so does every edge between two of them, so two neighbours drawn at
different levels share their border exactly: there is nothing to stitch and nothing to morph.

The file layout is the format's, with two deliberate departures, both written down in
quantizedmeshformat.h: the geodetic header fields carry scenery metres, there being no
geographic reference to convert from, and the material a vertex belongs to rides in an
extension of ours. Texture coordinates are not stored at all - ground tiles in world space, so
the coordinate follows from the position and from how many metres one repeat of the material
covers, measured off the scenery once and kept in the terrain's table.

Tiles live in one archive beside the scenery, an entry each, compressed on its own. Baltyk
cooks to 546 kB against 2.2 MB of heightfields, quark to 361 kB against 9.6 MB, tomaszewo to
375 kB against 9.9 MB.

Drawing is one call a frame: every resident tile lives in the same vertex and index buffer,
with its vertices measured from the middle of the terrain so that the whole map fits in floats,
and a frame is one glMultiDrawElementsBaseVertex over the tiles the walk chose and the frustum
kept. On baltyk that settles at 291 tiles resident, 16 MB of mesh, 85 tiles and 6k triangles
drawn.

What counts as ground is what the scenery says, not what its triangles look like. A node under
origin, rotate or scale is an object standing on the ground; a material with an alpha channel is
a tree, a bush, a fence or a catenary mast. Everything else is laid whichever way it faces, so
the side of an embankment or a cutting goes in with the rest instead of leaving a hole along
every bank. Whatever is left out is recorded per node, and the scenery goes on drawing it.

A cook is a cache: it is used again only when the archive, the note of which nodes went into it
and the manifest of what the scenery was made from all agree. A manifest listing a file the
scenery names but does not have used to fail that test for ever, since an absent file never
matches - quark has such an include, and nothing beside it could ever be reused.

tools/terraincook cooks the same way outside the engine, and meshtest beside it checks what the
cook promises: a tile reads back as written, neighbouring tiles agree on their border to the
millimetre, a hole is not filled in, a wall is laid exactly once, and a material's tiling comes
out of the source. seamcheck.py goes: it tested the morphing constants of a heightfield nothing
reads any more.
This commit is contained in:
maj00r
2026-09-20 08:04:46 +02:00
parent 975e62df95
commit b877bc0772
36 changed files with 3356 additions and 3614 deletions

179
scene/pakformat.h Normal file
View File

@@ -0,0 +1,179 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#pragma once
// One file holding a scenery's cooked terrain tiles.
//
// A terrain is tens of thousands of small files, which is the one thing a filesystem is worst at:
// opening them costs more than reading them. They go into an archive instead, each entry
// compressed on its own so that a tile can be read without touching the rest, and the index is
// written last so that cooking never has to hold the tiles in memory.
#include <cstdint>
#include <cstring>
#include <fstream>
#include <string>
#include <unordered_map>
#include <vector>
#include <zstd.h>
namespace pak {
inline constexpr char magic[ 8 ] { 'E', 'U', '0', '7', 'P', 'A', 'K', '\0' };
inline constexpr std::uint32_t version { 2 };
inline constexpr std::size_t namelength { 64 };
#pragma pack( push, 1 )
struct file_header {
char magic[ 8 ];
std::uint32_t version;
std::uint32_t entrycount;
std::uint64_t indexoffset;
};
struct index_entry {
char name[ namelength ];
std::uint64_t offset;
std::uint64_t stored; // bytes in the file
std::uint64_t plain; // bytes once decompressed; equal to stored when kept as it is
};
#pragma pack( pop )
// Writes entries straight through to disk as they are added.
class writer {
public:
bool open( std::string const &Path ) {
m_file.open( Path, std::ios::binary | std::ios::trunc );
if( false == m_file.good() ) { return false; }
file_header header {};
m_file.write( reinterpret_cast<char const *>( &header ), sizeof( header ) );
m_at = sizeof( header );
return m_file.good();
}
bool add( std::string const &Name, std::vector<std::uint8_t> const &Data ) {
if( ( false == m_file.good() ) || ( Name.size() >= namelength ) ) { return false; }
index_entry entry {};
std::memcpy( entry.name, Name.c_str(), Name.size() );
entry.offset = m_at;
entry.plain = Data.size();
m_packed.resize( ZSTD_compressBound( Data.size() ) );
auto const packed {
ZSTD_compress( m_packed.data(), m_packed.size(), Data.data(), Data.size(), 3 ) };
auto const *bytes { reinterpret_cast<char const *>( Data.data() ) };
auto size { Data.size() };
if( ( 0 == ZSTD_isError( packed ) ) && ( packed < Data.size() ) ) {
bytes = reinterpret_cast<char const *>( m_packed.data() );
size = packed;
}
entry.stored = size;
m_file.write( bytes, static_cast<std::streamsize>( size ) );
m_at += size;
m_index.push_back( entry );
return m_file.good();
}
bool close() {
if( false == m_file.good() ) { return false; }
file_header header {};
std::memcpy( header.magic, magic, sizeof( magic ) );
header.version = version;
header.entrycount = static_cast<std::uint32_t>( m_index.size() );
header.indexoffset = m_at;
for( auto const &entry : m_index ) {
m_file.write( reinterpret_cast<char const *>( &entry ), sizeof( entry ) );
}
m_file.seekp( 0 );
m_file.write( reinterpret_cast<char const *>( &header ), sizeof( header ) );
auto const good { m_file.good() };
m_file.close();
return good;
}
std::size_t entrycount() const { return m_index.size(); }
std::uint64_t bytes() const { return m_at + m_index.size() * sizeof( index_entry ); }
private:
std::ofstream m_file;
std::uint64_t m_at { 0 };
std::vector<index_entry> m_index;
std::vector<std::uint8_t> m_packed;
};
// Reads entries back. One reader is used from one thread at a time: it keeps a handle and seeks.
class reader {
public:
bool open( std::string const &Path ) {
close();
m_file.open( Path, std::ios::binary );
if( false == m_file.good() ) { return false; }
file_header header {};
m_file.read( reinterpret_cast<char *>( &header ), sizeof( header ) );
if( false == m_file.good() ) { return false; }
if( 0 != std::memcmp( header.magic, magic, sizeof( magic ) ) ) { return false; }
if( header.version != version ) { return false; }
m_file.seekg( static_cast<std::streamoff>( header.indexoffset ) );
for( std::uint32_t entry = 0; entry < header.entrycount; ++entry ) {
index_entry record {};
m_file.read( reinterpret_cast<char *>( &record ), sizeof( record ) );
if( false == m_file.good() ) { return false; }
record.name[ namelength - 1 ] = '\0';
m_index.emplace( record.name, record );
}
return true;
}
void close() {
m_index.clear();
if( true == m_file.is_open() ) { m_file.close(); }
m_file.clear();
}
bool exists( std::string const &Name ) const { return m_index.contains( Name ); }
std::size_t entrycount() const { return m_index.size(); }
std::vector<std::string> names() const {
std::vector<std::string> names;
names.reserve( m_index.size() );
for( auto const &[ name, entry ] : m_index ) { names.push_back( name ); }
return names;
}
bool read( std::string const &Name, std::vector<std::uint8_t> &Out ) {
auto const found { m_index.find( Name ) };
if( found == m_index.end() ) { return false; }
auto const &entry { found->second };
m_stored.resize( entry.stored );
m_file.seekg( static_cast<std::streamoff>( entry.offset ) );
m_file.read( reinterpret_cast<char *>( m_stored.data() ), static_cast<std::streamsize>( entry.stored ) );
if( false == m_file.good() ) { m_file.clear(); return false; }
if( entry.stored == entry.plain ) {
Out.swap( m_stored );
return true;
}
Out.resize( entry.plain );
auto const plain {
ZSTD_decompress( Out.data(), Out.size(), m_stored.data(), m_stored.size() ) };
return ( 0 == ZSTD_isError( plain ) ) && ( plain == entry.plain );
}
private:
std::ifstream m_file;
std::unordered_map<std::string, index_entry> m_index;
std::vector<std::uint8_t> m_stored;
};
} // namespace pak

View File

@@ -0,0 +1,981 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#include "scene/quantizedmeshcooker.h"
#include <algorithm>
#include <array>
#include <cmath>
#include <cstdio>
#include <cstring>
#include <filesystem>
#include <limits>
#include <map>
#include <queue>
namespace quantizedmesh {
namespace {
// a triangle with less surface than this is nothing at all, whichever way it faces, and is the
// only thing the cook refuses to lay. Walls - the sides of an embankment, a cutting, an abutment -
// are ground as much as anything else: nothing about a mesh says it has to be a height field, and
// leaving them out is what opens holes along every bank
constexpr double nullarea { 1e-6 };
// vertices nearer than this to each other, to a tile border or to a plane are the same thing.
// a tenth of a millimetre is below anything the game can show and far above what the arithmetic
// of cutting a triangle can be out by
constexpr double weldepsilon { 1e-4 };
// the deepest quadtree worth cooking: 4096 tiles along a side of the finest level
constexpr std::uint32_t maxlevels { 13 };
// The finest tile is this many typical triangle edges across, and never outside these bounds. A tile
// has to be large against the triangles in it: a triangle crossing a border is cut, and the pieces
// cost vertices neither side had. It also has to stay small enough to be worth loading and culling
// on its own, which is what the bounds are for.
constexpr double tileedges { 16.0 };
constexpr double smallesttileside { 128.0 };
constexpr double largesttileside { 1024.0 };
// how many edge lengths are kept to find the typical one. the first hundred thousand triangles say
// as much about a scenery as all of them
constexpr std::size_t edgesample { 100000 };
// a tile is never collapsed below this, however far away it is drawn: past a point the triangles
// saved are not worth the ground losing its shape
constexpr std::size_t smallesttile { 64 };
// A level may move the ground by this share of its tile's side, and no further. A level is used from
// about twice its tile's side away, where a share this small is well under a pixel; letting it go
// further would buy triangles that no distance is far enough to use. Ground already so coarse that a
// level cannot be thinned within it simply comes out nearly a copy of the level below, which is the
// right answer: there was nothing there to save.
constexpr double errorshare { 128.0 };
// the area of a triangle in space, whichever way it faces
double
surface_area( cooker::corner const &A, cooker::corner const &B, cooker::corner const &C ) {
auto const nx { ( B.y - A.y ) * ( C.z - A.z ) - ( B.z - A.z ) * ( C.y - A.y ) };
auto const ny { ( B.z - A.z ) * ( C.x - A.x ) - ( B.x - A.x ) * ( C.z - A.z ) };
auto const nz { ( B.x - A.x ) * ( C.y - A.y ) - ( B.y - A.y ) * ( C.x - A.x ) };
return 0.5 * std::sqrt( nx * nx + ny * ny + nz * nz );
}
// a triangle standing upright has no extent in the plan at all. it still belongs to the ground,
// but it has to be told apart: the tile it goes to cannot be decided by cutting it in two
double
planar_extent( cooker::corner const &A, cooker::corner const &B, cooker::corner const &C ) {
return std::max(
std::max( { A.x, B.x, C.x } ) - std::min( { A.x, B.x, C.x } ),
std::max( { A.z, B.z, C.z } ) - std::min( { A.z, B.z, C.z } ) );
}
// The error of collapsing a vertex away, as Garland and Heckbert measure it: the sum of the
// squared distances to the planes of the triangles that met there. Kept as the ten distinct
// entries of the symmetric matrix.
struct quadric {
double m[ 10 ] {};
void add_plane( double const A, double const B, double const C, double const D, double const Weight ) {
m[ 0 ] += Weight * A * A; m[ 1 ] += Weight * A * B; m[ 2 ] += Weight * A * C; m[ 3 ] += Weight * A * D;
m[ 4 ] += Weight * B * B; m[ 5 ] += Weight * B * C; m[ 6 ] += Weight * B * D;
m[ 7 ] += Weight * C * C; m[ 8 ] += Weight * C * D;
m[ 9 ] += Weight * D * D;
}
void operator+=( quadric const &Other ) {
for( std::size_t index = 0; index < 10; ++index ) { m[ index ] += Other.m[ index ]; }
}
double at( double const X, double const Y, double const Z ) const {
return m[ 0 ] * X * X + 2.0 * m[ 1 ] * X * Y + 2.0 * m[ 2 ] * X * Z + 2.0 * m[ 3 ] * X
+ m[ 4 ] * Y * Y + 2.0 * m[ 5 ] * Y * Z + 2.0 * m[ 6 ] * Y
+ m[ 7 ] * Z * Z + 2.0 * m[ 8 ] * Z
+ m[ 9 ];
}
};
// which sides of a tile's square a point lies on, one bit each
std::uint8_t
sides_of( double const X, double const Z, double const West, double const North, double const Side ) {
std::uint8_t sides { 0 };
if( std::abs( X - West ) < weldepsilon ) { sides |= 1; }
if( std::abs( X - ( West + Side ) ) < weldepsilon ) { sides |= 2; }
if( std::abs( Z - North ) < weldepsilon ) { sides |= 4; }
if( std::abs( Z - ( North + Side ) ) < weldepsilon ){ sides |= 8; }
return sides;
}
// a vertex of a polygon being cut along the tile grid
struct clipvertex {
double x, y, z;
};
clipvertex
between( clipvertex const &From, clipvertex const &To, double const Share ) {
return {
From.x + ( To.x - From.x ) * Share,
From.y + ( To.y - From.y ) * Share,
From.z + ( To.z - From.z ) * Share };
}
// Sutherland and Hodgman against one side of the tile's square. Axis 0 is x and 1 is z; the
// coordinate cut against is set to the bound itself rather than to what the division gives, so
// that the two tiles sharing that border end up with a vertex in exactly the same place.
void
clip_side( std::vector<clipvertex> &Polygon, int const Axis, double const Bound, bool const Keepabove ) {
if( Polygon.empty() ) { return; }
auto const coordinate = [ Axis ]( clipvertex const &Vertex ) {
return ( Axis == 0 ) ? Vertex.x : Vertex.z; };
auto const inside = [ & ]( clipvertex const &Vertex ) {
return Keepabove ? ( coordinate( Vertex ) >= Bound - weldepsilon )
: ( coordinate( Vertex ) <= Bound + weldepsilon ); };
std::vector<clipvertex> kept;
kept.reserve( Polygon.size() + 2 );
for( std::size_t index = 0; index < Polygon.size(); ++index ) {
auto const &here { Polygon[ index ] };
auto const &next { Polygon[ ( index + 1 ) % Polygon.size() ] };
auto const hereinside { inside( here ) };
if( true == hereinside ) { kept.push_back( here ); }
if( hereinside == inside( next ) ) { continue; }
auto const span { coordinate( next ) - coordinate( here ) };
if( std::abs( span ) < weldepsilon ) { continue; }
auto crossing { between( here, next, ( Bound - coordinate( here ) ) / span ) };
if( Axis == 0 ) { crossing.x = Bound; } else { crossing.z = Bound; }
kept.push_back( crossing );
}
Polygon.swap( kept );
}
} // anonymous namespace
cooker::~cooker() {
if( true == m_scratch.is_open() ) { m_scratch.close(); }
if( false == m_scratchpath.empty() ) {
std::error_code error;
std::filesystem::remove( m_scratchpath, error );
}
}
bool
cooker::begin( std::string const &Scratchpath, std::size_t const Tilebudget ) {
m_scratchpath = Scratchpath;
m_tilebudget = std::max<std::size_t>( 256, Tilebudget );
m_scratch.open( m_scratchpath, std::ios::binary | std::ios::in | std::ios::out | std::ios::trunc );
return m_scratch.good();
}
std::uint16_t
cooker::material_index( std::string_view const Material ) {
auto const found { m_materiallookup.find( std::string { Material } ) };
if( found != m_materiallookup.end() ) { return found->second; }
auto const index { static_cast<std::uint16_t>( m_materials.size() ) };
m_materials.emplace_back( Material );
m_materiallookup.emplace( std::string { Material }, index );
return index;
}
// The scenery's texture coordinates are not kept: ground textures tile in world space, so a
// coordinate follows from where a vertex is. What has to be kept is the scale the scenery drew the
// material at, and that is measured here - the ratio of world distance to texture distance along
// a triangle's edges. Edges too short, or with no texture movement along them, say nothing.
void
cooker::measure_repeat( std::uint16_t const Material, corner const ( &Corners )[ 3 ] ) {
if( Material >= m_repeatcounts.size() ) {
m_repeatsums.resize( Material + 1, 0.0 );
m_repeatsquares.resize( Material + 1, 0.0 );
m_repeatcounts.resize( Material + 1, 0 );
}
for( std::size_t index = 0; index < 3; ++index ) {
auto const &from { Corners[ index ] };
auto const &to { Corners[ ( index + 1 ) % 3 ] };
auto const world {
std::sqrt( ( to.x - from.x ) * ( to.x - from.x ) + ( to.z - from.z ) * ( to.z - from.z ) ) };
auto const texture {
std::sqrt( ( to.u - from.u ) * ( to.u - from.u ) + ( to.v - from.v ) * ( to.v - from.v ) ) };
if( ( world < 0.5 ) || ( texture < 1e-4 ) ) { continue; }
auto const ratio { world / texture };
m_repeatsums[ Material ] += ratio;
m_repeatsquares[ Material ] += ratio * ratio;
++m_repeatcounts[ Material ];
}
}
void
cooker::add( corner const &A, corner const &B, corner const &C, std::string_view const Material ) {
auto const id { static_cast<std::uint32_t>( m_added ) };
++m_added;
if( ( surface_area( A, B, C ) < nullarea ) || ( planar_extent( A, B, C ) < weldepsilon ) ) {
// no surface, or standing on a single point in the plan: nothing to lay, and the scenery
// goes on drawing it itself
m_refusals.push_back( id );
return;
}
corner const corners[ 3 ] { A, B, C };
if( m_edges.size() < edgesample ) {
for( std::size_t index = 0; index < 3; ++index ) {
auto const &from { corners[ index ] };
auto const &to { corners[ ( index + 1 ) % 3 ] };
auto const length {
std::sqrt( ( to.x - from.x ) * ( to.x - from.x ) + ( to.z - from.z ) * ( to.z - from.z ) ) };
if( length > 0.0 ) { m_edges.push_back( length ); }
}
}
spilled record {};
record.material = material_index( Material );
measure_repeat( record.material, corners );
for( std::size_t index = 0; index < 3; ++index ) {
record.corners[ index * 3 + 0 ] = corners[ index ].x;
record.corners[ index * 3 + 1 ] = corners[ index ].y;
record.corners[ index * 3 + 2 ] = corners[ index ].z;
}
m_scratch.write( reinterpret_cast<char const *>( &record ), sizeof( record ) );
for( auto const &vertex : corners ) {
if( false == m_anything ) {
m_anything = true;
m_minx = m_maxx = vertex.x;
m_minz = m_maxz = vertex.z;
m_lowest = m_highest = vertex.y;
continue;
}
m_minx = std::min( m_minx, vertex.x ); m_maxx = std::max( m_maxx, vertex.x );
m_minz = std::min( m_minz, vertex.z ); m_maxz = std::max( m_maxz, vertex.z );
m_lowest = std::min( m_lowest, vertex.y ); m_highest = std::max( m_highest, vertex.y );
}
}
// The root square starts on a multiple of the finest tile side and doubles until it holds
// everything, so that where a tile's borders fall follows from the terrain alone and a second
// cook of the same scenery puts them in the same place.
void
cooker::measure() {
// the middle edge length of the sample, and the tile that suits it, rounded to a power of two so
// that the same scenery always comes out the same way
if( false == m_edges.empty() ) {
auto const middle { m_edges.begin() + m_edges.size() / 2 };
std::nth_element( m_edges.begin(), middle, m_edges.end() );
auto const wanted { std::clamp( tileedges * *middle, smallesttileside, largesttileside ) };
m_finesttile = std::exp2( std::round( std::log2( wanted ) ) );
m_finesttile = std::clamp( m_finesttile, smallesttileside, largesttileside );
m_edges.clear();
m_edges.shrink_to_fit();
}
m_west = std::floor( m_minx / m_finesttile ) * m_finesttile;
m_north = std::floor( m_minz / m_finesttile ) * m_finesttile;
auto const needed { std::max( m_maxx - m_west, m_maxz - m_north ) };
m_side = m_finesttile;
m_levels = 1;
while( ( m_side < needed ) && ( m_levels < maxlevels ) ) {
m_side *= 2.0;
++m_levels;
}
// a height range of zero would make every height quantize to nothing
if( m_highest - m_lowest < 1.0 ) { m_highest = m_lowest + 1.0; }
}
void
cooker::square( tile_address const &Tile, double &West, double &North, double &Side ) const {
Side = m_side / static_cast<double>( 1u << Tile.level );
West = m_west + Tile.x * Side;
North = m_north + Tile.z * Side;
}
bool
cooker::finish( sink const &Sink, terrain_table &Table ) {
std::sort( m_refusals.begin(), m_refusals.end() );
m_report.refused = m_refusals.size();
if( ( false == m_anything ) || ( false == m_scratch.good() ) ) {
return false;
}
m_scratch.flush();
measure();
// which of the spilled triangles touch which tile of the finest level. a triangle can span
// several, and each of them cuts its own piece out of it
auto const finest { m_levels - 1 };
auto const acrossfinest { static_cast<std::int32_t>( 1u << finest ) };
auto const records { m_scratch.tellp() / static_cast<std::streamoff>( sizeof( spilled ) ) };
m_scratch.seekg( 0 );
for( std::streamoff index = 0; index < records; ++index ) {
spilled record {};
m_scratch.read( reinterpret_cast<char *>( &record ), sizeof( record ) );
if( false == m_scratch.good() ) { return false; }
auto minx { record.corners[ 0 ] }, maxx { minx };
auto minz { record.corners[ 2 ] }, maxz { minz };
for( std::size_t at = 0; at < 3; ++at ) {
minx = std::min( minx, record.corners[ at * 3 + 0 ] ); maxx = std::max( maxx, record.corners[ at * 3 + 0 ] );
minz = std::min( minz, record.corners[ at * 3 + 2 ] ); maxz = std::max( maxz, record.corners[ at * 3 + 2 ] );
}
auto const first = [ this ]( double const Value, double const Origin ) {
return static_cast<std::int32_t>( std::floor( ( Value - Origin ) / m_finesttile ) ); };
auto const fromx { std::clamp( first( minx, m_west ), 0, acrossfinest - 1 ) };
auto const tox { std::clamp( first( maxx, m_west ), 0, acrossfinest - 1 ) };
auto const fromz { std::clamp( first( minz, m_north ), 0, acrossfinest - 1 ) };
auto const toz { std::clamp( first( maxz, m_north ), 0, acrossfinest - 1 ) };
for( auto x = fromx; x <= tox; ++x ) {
for( auto z = fromz; z <= toz; ++z ) {
m_placement.emplace_back(
tile_key( { finest, x, z } ), static_cast<std::uint32_t>( index ) );
}
}
}
std::sort( m_placement.begin(), m_placement.end() );
m_errors.assign( m_levels, 0.0 );
cook( { 0, 0, 0 }, Sink );
m_report.levels = m_levels;
Table.rules = cook_rules;
Table.west = m_west;
Table.north = m_north;
Table.side = m_side;
Table.lowest = m_lowest;
Table.highest = m_highest;
Table.levels = m_levels;
Table.errors.assign( m_errors.begin(), m_errors.end() );
Table.materials.clear();
for( std::size_t index = 0; index < m_materials.size(); ++index ) {
// a material the measurement never caught falls back to something plausible rather than
// to zero, which would divide by nothing in the shader
auto const caught { ( index < m_repeatcounts.size() ) && ( m_repeatcounts[ index ] > 0 ) };
auto const repeat {
caught ? m_repeatsums[ index ] / static_cast<double>( m_repeatcounts[ index ] ) : 4.0 };
Table.materials.push_back( { m_materials[ index ], static_cast<float>( repeat ) } );
}
m_scratch.close();
std::error_code error;
std::filesystem::remove( m_scratchpath, error );
m_scratchpath.clear();
return true;
}
cooker::mesh
cooker::cook( tile_address const &Tile, sink const &Sink ) {
auto result {
( Tile.level + 1 == m_levels ) ? lay( Tile ) : merge( Tile, Sink ) };
if( true == result.empty() ) { return result; }
if( Tile.level + 1 < m_levels ) {
// The finest level is the ground as the scenery drew it and is left alone. A level above it
// holds what four tiles below it held, and is collapsed back to what one of them held: the
// cost of a level is then a quarter of the level below, and the whole pyramid costs a third
// more than the finest alone. A count fixed in advance cannot do that - a tile that never
// reaches it would be copied rather than simplified - so it only serves as a ceiling.
double west { 0.0 }, north { 0.0 }, side { 0.0 };
square( Tile, west, north, side );
auto const merged { result.indices.size() / 3 };
simplify( result, std::clamp( merged / 4, smallesttile, m_tilebudget ), side / errorshare );
}
m_errors[ Tile.level ] = std::max( m_errors[ Tile.level ], result.error );
std::vector<std::uint8_t> bytes;
encode( Tile, result, bytes );
Sink( Tile, bytes );
++m_report.tiles;
m_report.triangles += result.indices.size() / 3;
m_report.vertices += result.vertices.size();
m_report.bytes += bytes.size();
if( ( false == m_quiet ) && ( m_report.tiles % 200 == 0 ) ) {
std::printf( "\rCooking terrain: %zu tiles", m_report.tiles );
std::fflush( stdout );
}
return result;
}
namespace {
// Welds vertices that land in the same place and belong to the same material. Everything the cut
// produces is either a vertex of the source or a point on a tile border worked out the same way
// from both sides, so the tolerance only has to cover the last bits of the arithmetic.
class welder {
public:
explicit welder( std::vector<cooker::meshvertex> &Vertices ) : m_vertices( Vertices ) {}
std::uint32_t operator()( cooker::meshvertex const &Vertex ) {
auto const key { make_key( Vertex ) };
auto const found { m_known.find( key ) };
if( found != m_known.end() ) { return found->second; }
auto const index { static_cast<std::uint32_t>( m_vertices.size() ) };
m_vertices.push_back( Vertex );
m_known.emplace( key, index );
return index;
}
private:
struct key {
std::int64_t x, y, z;
std::uint16_t material;
bool operator==( key const & ) const = default;
};
struct keyhash {
std::size_t operator()( key const &Key ) const noexcept {
std::size_t hash { 1469598103934665603ull };
auto const mix = [ &hash ]( std::int64_t const Value ) {
hash ^= static_cast<std::size_t>( Value );
hash *= 1099511628211ull; };
mix( Key.x ); mix( Key.y ); mix( Key.z );
mix( Key.material );
return hash;
}
};
static key make_key( cooker::meshvertex const &Vertex ) {
auto const grid = []( double const Value ) {
return static_cast<std::int64_t>( std::llround( Value / weldepsilon ) ); };
return { grid( Vertex.x ), grid( Vertex.y ), grid( Vertex.z ), Vertex.material };
}
std::vector<cooker::meshvertex> &m_vertices;
std::unordered_map<key, std::uint32_t, keyhash> m_known;
};
} // anonymous namespace
cooker::mesh
cooker::lay( tile_address const &Tile ) {
mesh result;
double west { 0.0 }, north { 0.0 }, side { 0.0 };
square( Tile, west, north, side );
auto const wanted { tile_key( Tile ) };
auto const from {
std::lower_bound( m_placement.begin(), m_placement.end(),
std::pair<std::int64_t, std::uint32_t> { wanted, 0 } ) };
auto const to {
std::upper_bound( m_placement.begin(), m_placement.end(),
std::pair<std::int64_t, std::uint32_t> { wanted, std::numeric_limits<std::uint32_t>::max() } ) };
if( from == to ) { return result; }
welder weld { result.vertices };
std::vector<clipvertex> polygon;
for( auto entry = from; entry != to; ++entry ) {
m_scratch.seekg( static_cast<std::streamoff>( entry->second ) * sizeof( spilled ) );
spilled record {};
m_scratch.read( reinterpret_cast<char *>( &record ), sizeof( record ) );
if( false == m_scratch.good() ) { m_scratch.clear(); continue; }
polygon.clear();
for( std::size_t at = 0; at < 3; ++at ) {
polygon.push_back(
{ record.corners[ at * 3 + 0 ], record.corners[ at * 3 + 1 ], record.corners[ at * 3 + 2 ] } );
}
clip_side( polygon, 0, west, true );
clip_side( polygon, 0, west + side, false );
clip_side( polygon, 1, north, true );
clip_side( polygon, 1, north + side, false );
if( polygon.size() < 3 ) { continue; }
// A wall lying along a tile border is inside both tiles: the cut keeps it whole on either
// side, since it has nothing to cut. Whichever border it sits on, the tile past it is the
// one that keeps it, so that it is laid exactly once.
{
auto onborderonly { false };
for( int axis = 0; axis < 2; ++axis ) {
auto const at = [ axis ]( clipvertex const &Vertex ) { return axis == 0 ? Vertex.x : Vertex.z; };
auto low { at( polygon.front() ) }, high { low };
for( auto const &vertex : polygon ) {
low = std::min( low, at( vertex ) );
high = std::max( high, at( vertex ) );
}
auto const upper { ( axis == 0 ? west + side : north + side ) };
if( ( high - low < weldepsilon ) && ( std::abs( low - upper ) < weldepsilon ) ) {
onborderonly = true;
}
}
if( true == onborderonly ) { continue; }
}
auto const place = [ & ]( clipvertex const &Vertex ) {
return weld( { Vertex.x, Vertex.y, Vertex.z, record.material,
sides_of( Vertex.x, Vertex.z, west, north, side ) } ); };
auto const first { place( polygon[ 0 ] ) };
for( std::size_t corner = 1; corner + 1 < polygon.size(); ++corner ) {
// the cut keeps the source winding, so fanning the polygon keeps it too
auto const second { place( polygon[ corner ] ) };
auto const third { place( polygon[ corner + 1 ] ) };
if( ( first == second ) || ( second == third ) || ( third == first ) ) { continue; }
result.indices.push_back( first );
result.indices.push_back( second );
result.indices.push_back( third );
}
}
return result;
}
cooker::mesh
cooker::merge( tile_address const &Tile, sink const &Sink ) {
mesh result;
double west { 0.0 }, north { 0.0 }, side { 0.0 };
square( Tile, west, north, side );
welder weld { result.vertices };
for( std::int32_t stepz = 0; stepz < 2; ++stepz ) {
for( std::int32_t stepx = 0; stepx < 2; ++stepx ) {
auto const child {
cook( { Tile.level + 1, Tile.x * 2 + stepx, Tile.z * 2 + stepz }, Sink ) };
if( true == child.empty() ) { continue; }
result.error = std::max( result.error, child.error );
// the borders between the four are no longer borders: only the outer edge of this
// tile has to stay where it is
std::vector<std::uint32_t> here;
here.reserve( child.vertices.size() );
for( auto vertex : child.vertices ) {
vertex.sides = sides_of( vertex.x, vertex.z, west, north, side );
here.push_back( weld( vertex ) );
}
for( auto const index : child.indices ) {
result.indices.push_back( here[ index ] );
}
}
}
return result;
}
// Edge collapse, cheapest first. A vertex is only ever collapsed onto one of its neighbours -
// never onto a new point between them - so a vertex that has to stay put, because a neighbouring
// tile shares it, simply is never the one that goes. That is the whole of the crack handling:
// levels meet because the border is literally the same set of vertices in both.
void
cooker::simplify( mesh &Mesh, std::size_t const Budget, double const Allowed ) {
auto triangles { Mesh.indices.size() / 3 };
if( triangles <= Budget ) { return; }
auto &vertices { Mesh.vertices };
// The quadrics are built and evaluated about the middle of the mesh rather than about the
// scenery's origin. A plane's own constant is minus the normal times a point on it, so in world
// coordinates it runs to tens or hundreds of thousands, and the quadric's value - a small
// distance squared - comes out as the difference of two such numbers. There is no precision left
// in that. Measured from the middle of the tile the constants are the size of the tile.
std::vector<std::array<double, 3>> local( vertices.size() );
{
std::array<double, 3> middle { 0.0, 0.0, 0.0 };
for( auto const &vertex : vertices ) {
middle[ 0 ] += vertex.x; middle[ 1 ] += vertex.y; middle[ 2 ] += vertex.z;
}
for( auto &value : middle ) { value /= static_cast<double>( vertices.size() ); }
for( std::size_t index = 0; index < vertices.size(); ++index ) {
local[ index ] = {
vertices[ index ].x - middle[ 0 ],
vertices[ index ].y - middle[ 1 ],
vertices[ index ].z - middle[ 2 ] };
}
}
std::vector<quadric> quadrics( vertices.size() );
// the worst the ground has been moved at each vertex by a collapse into it, in metres. what a
// level is off by against the level below it; merging four tiles already carried their own
// figures up, so the levels are not counted twice
std::vector<double> moved( vertices.size(), 0.0 );
std::vector<std::vector<std::uint32_t>> incident( vertices.size() );
std::vector<bool> gone( triangles, false );
std::vector<std::uint32_t> version( vertices.size(), 0 );
std::vector<bool> removed( vertices.size(), false );
auto const cornerof = [ &Mesh ]( std::size_t const Triangle, std::size_t const Corner ) -> std::uint32_t & {
return Mesh.indices[ Triangle * 3 + Corner ]; };
for( std::size_t triangle = 0; triangle < triangles; ++triangle ) {
auto const &a { local[ cornerof( triangle, 0 ) ] };
auto const &b { local[ cornerof( triangle, 1 ) ] };
auto const &c { local[ cornerof( triangle, 2 ) ] };
double nx { ( b[ 1 ] - a[ 1 ] ) * ( c[ 2 ] - a[ 2 ] ) - ( b[ 2 ] - a[ 2 ] ) * ( c[ 1 ] - a[ 1 ] ) };
double ny { ( b[ 2 ] - a[ 2 ] ) * ( c[ 0 ] - a[ 0 ] ) - ( b[ 0 ] - a[ 0 ] ) * ( c[ 2 ] - a[ 2 ] ) };
double nz { ( b[ 0 ] - a[ 0 ] ) * ( c[ 1 ] - a[ 1 ] ) - ( b[ 1 ] - a[ 1 ] ) * ( c[ 0 ] - a[ 0 ] ) };
auto const length { std::sqrt( nx * nx + ny * ny + nz * nz ) };
if( length < 1e-12 ) { continue; }
auto const area { 0.5 * length };
nx /= length; ny /= length; nz /= length;
auto const offset { -( nx * a[ 0 ] + ny * a[ 1 ] + nz * a[ 2 ] ) };
for( std::size_t corner = 0; corner < 3; ++corner ) {
auto const index { cornerof( triangle, corner ) };
quadrics[ index ].add_plane( nx, ny, nz, offset, area );
incident[ index ].push_back( static_cast<std::uint32_t>( triangle ) );
}
}
// Where the ground ends - the outline of a hole the scenery left, or the edge of the terrain -
// only one triangle meets the edge. Nothing in the quadrics above holds such an edge in
// place, so a plane standing on it, weighted heavily, is added to both its ends: the outline
// then survives into the coarse levels instead of being eaten away.
{
std::map<std::pair<std::uint32_t, std::uint32_t>, std::size_t> uses;
for( std::size_t triangle = 0; triangle < triangles; ++triangle ) {
for( std::size_t corner = 0; corner < 3; ++corner ) {
auto const from { cornerof( triangle, corner ) };
auto const to { cornerof( triangle, ( corner + 1 ) % 3 ) };
++uses[ { std::min( from, to ), std::max( from, to ) } ];
}
}
for( std::size_t triangle = 0; triangle < triangles; ++triangle ) {
auto const &a { local[ cornerof( triangle, 0 ) ] };
auto const &b { local[ cornerof( triangle, 1 ) ] };
auto const &c { local[ cornerof( triangle, 2 ) ] };
double nx { ( b[ 1 ] - a[ 1 ] ) * ( c[ 2 ] - a[ 2 ] ) - ( b[ 2 ] - a[ 2 ] ) * ( c[ 1 ] - a[ 1 ] ) };
double ny { ( b[ 2 ] - a[ 2 ] ) * ( c[ 0 ] - a[ 0 ] ) - ( b[ 0 ] - a[ 0 ] ) * ( c[ 2 ] - a[ 2 ] ) };
double nz { ( b[ 0 ] - a[ 0 ] ) * ( c[ 1 ] - a[ 1 ] ) - ( b[ 1 ] - a[ 1 ] ) * ( c[ 0 ] - a[ 0 ] ) };
auto const facelength { std::sqrt( nx * nx + ny * ny + nz * nz ) };
if( facelength < 1e-12 ) { continue; }
nx /= facelength; ny /= facelength; nz /= facelength;
for( std::size_t corner = 0; corner < 3; ++corner ) {
auto const from { cornerof( triangle, corner ) };
auto const to { cornerof( triangle, ( corner + 1 ) % 3 ) };
auto const use { uses.find( { std::min( from, to ), std::max( from, to ) } ) };
if( ( use == uses.end() ) || ( use->second != 1 ) ) { continue; }
auto const &head { local[ from ] };
auto const &tail { local[ to ] };
double ex { tail[ 0 ] - head[ 0 ] }, ey { tail[ 1 ] - head[ 1 ] }, ez { tail[ 2 ] - head[ 2 ] };
auto const edgelength { std::sqrt( ex * ex + ey * ey + ez * ez ) };
if( edgelength < 1e-9 ) { continue; }
ex /= edgelength; ey /= edgelength; ez /= edgelength;
// perpendicular to both the edge and the face it belongs to
auto const px { ey * nz - ez * ny };
auto const py { ez * nx - ex * nz };
auto const pz { ex * ny - ey * nx };
auto const offset { -( px * head[ 0 ] + py * head[ 1 ] + pz * head[ 2 ] ) };
auto const weight { 100.0 * edgelength * edgelength };
quadrics[ from ].add_plane( px, py, pz, offset, weight );
quadrics[ to ].add_plane( px, py, pz, offset, weight );
}
}
}
struct candidate {
double cost;
std::uint32_t from, to;
std::uint32_t fromversion, toversion;
bool operator<( candidate const &Other ) const { return cost > Other.cost; }
};
std::priority_queue<candidate> queue;
// the cost of taking From away and letting To stand for both. only what the collapse leaves
// behind is measured; where To sits does not change, so its own quadric is already paid for
auto const cost_of = [ & ]( std::uint32_t const From, std::uint32_t const To ) {
quadric together { quadrics[ From ] };
together += quadrics[ To ];
return std::max( 0.0, together.at( local[ To ][ 0 ], local[ To ][ 1 ], local[ To ][ 2 ] ) ); };
// How far the ground moves where From used to be, if From were collapsed into To: the height of
// whatever would then lie over that spot, against the height that is there now. Worked out before
// the collapse rather than after it, so that a collapse which moves the ground too far is passed
// over instead of being taken and regretted. The cost above orders the candidates; this is the
// one that says whether a candidate may be taken at all, because it is a distance and the cost is
// not.
auto const drop_of = [ & ]( std::uint32_t const From, std::uint32_t const To ) {
auto const &was { local[ From ] };
for( auto const triangle : incident[ From ] ) {
if( true == gone[ triangle ] ) { continue; }
std::array<std::uint32_t, 3> corners {
cornerof( triangle, 0 ), cornerof( triangle, 1 ), cornerof( triangle, 2 ) };
// a triangle holding both ends of the edge disappears with it, and covers nothing after
if( std::find( corners.begin(), corners.end(), To ) != corners.end() ) { continue; }
std::array<std::array<double, 3> const *, 3> after {
&local[ corners[ 0 ] ], &local[ corners[ 1 ] ], &local[ corners[ 2 ] ] };
for( std::size_t corner = 0; corner < 3; ++corner ) {
if( corners[ corner ] == From ) { after[ corner ] = &local[ To ]; }
}
auto const &a { *after[ 0 ] }; auto const &b { *after[ 1 ] }; auto const &c { *after[ 2 ] };
auto const twice {
( b[ 0 ] - a[ 0 ] ) * ( c[ 2 ] - a[ 2 ] ) - ( c[ 0 ] - a[ 0 ] ) * ( b[ 2 ] - a[ 2 ] ) };
if( std::abs( twice ) < 1e-12 ) { continue; }
auto const first {
( ( b[ 0 ] - was[ 0 ] ) * ( c[ 2 ] - was[ 2 ] ) - ( c[ 0 ] - was[ 0 ] ) * ( b[ 2 ] - was[ 2 ] ) ) / twice };
auto const second {
( ( c[ 0 ] - was[ 0 ] ) * ( a[ 2 ] - was[ 2 ] ) - ( a[ 0 ] - was[ 0 ] ) * ( c[ 2 ] - was[ 2 ] ) ) / twice };
auto const third { 1.0 - first - second };
if( ( first < -1e-9 ) || ( second < -1e-9 ) || ( third < -1e-9 ) ) { continue; }
return std::abs( first * a[ 1 ] + second * b[ 1 ] + third * c[ 1 ] - was[ 1 ] );
}
// nothing would lie over that spot: the edge of the ground, or a wall, whose triangles cover
// nothing in the plan at all. what changes there is the height where the vertex went
return std::abs( was[ 1 ] - local[ To ][ 1 ] ); };
auto const allowed = [ & ]( std::uint32_t const From, std::uint32_t const To ) {
if( ( true == removed[ From ] ) || ( true == removed[ To ] ) ) { return false; }
// a vertex a neighbouring tile shares stays where it is
if( true == vertices[ From ].locked() ) { return false; }
// and a material boundary stays where it is too, or the ground would change colour by
// level
if( vertices[ From ].material != vertices[ To ].material ) { return false; }
// nothing that turns a triangle over or squashes it flat
for( auto const triangle : incident[ From ] ) {
if( true == gone[ triangle ] ) { continue; }
std::array<std::uint32_t, 3> corners {
cornerof( triangle, 0 ), cornerof( triangle, 1 ), cornerof( triangle, 2 ) };
if( std::find( corners.begin(), corners.end(), To ) != corners.end() ) {
// this triangle goes when the edge does. it may not be the only one holding a
// piece of the tile's border up: From is not on a border, so the only edge of it
// that can be one is between To and the third corner
for( auto const third : corners ) {
if( ( third == From ) || ( third == To ) ) { continue; }
if( ( vertices[ To ].sides & vertices[ third ].sides ) != 0 ) { return false; }
}
continue;
}
std::array<std::array<double, 3> const *, 3> before {
&local[ corners[ 0 ] ], &local[ corners[ 1 ] ], &local[ corners[ 2 ] ] };
std::array<std::array<double, 3> const *, 3> after { before };
for( std::size_t corner = 0; corner < 3; ++corner ) {
if( corners[ corner ] == From ) { after[ corner ] = &local[ To ]; }
}
// the normal in space, so that this holds for a wall as well as for open ground
auto const normal = []( std::array<std::array<double, 3> const *, 3> const &Corners ) {
auto const &a { *Corners[ 0 ] }; auto const &b { *Corners[ 1 ] }; auto const &c { *Corners[ 2 ] };
return std::array<double, 3> {
( b[ 1 ] - a[ 1 ] ) * ( c[ 2 ] - a[ 2 ] ) - ( b[ 2 ] - a[ 2 ] ) * ( c[ 1 ] - a[ 1 ] ),
( b[ 2 ] - a[ 2 ] ) * ( c[ 0 ] - a[ 0 ] ) - ( b[ 0 ] - a[ 0 ] ) * ( c[ 2 ] - a[ 2 ] ),
( b[ 0 ] - a[ 0 ] ) * ( c[ 1 ] - a[ 1 ] ) - ( b[ 1 ] - a[ 1 ] ) * ( c[ 0 ] - a[ 0 ] ) }; };
auto const was { normal( before ) };
auto const now { normal( after ) };
auto const length {
std::sqrt( now[ 0 ] * now[ 0 ] + now[ 1 ] * now[ 1 ] + now[ 2 ] * now[ 2 ] ) };
// nothing that squashes a triangle flat, and nothing that turns one over
if( 0.5 * length < nullarea ) { return false; }
if( was[ 0 ] * now[ 0 ] + was[ 1 ] * now[ 1 ] + was[ 2 ] * now[ 2 ] <= 0.0 ) { return false; }
}
// and nothing that moves the ground further than this level is allowed to
if( drop_of( From, To ) > Allowed ) { return false; }
return true; };
auto const offer = [ & ]( std::uint32_t const Vertex ) {
for( auto const triangle : incident[ Vertex ] ) {
if( true == gone[ triangle ] ) { continue; }
for( std::size_t corner = 0; corner < 3; ++corner ) {
auto const other { cornerof( triangle, corner ) };
if( other == Vertex ) { continue; }
for( auto const [ from, to ] : { std::pair { Vertex, other }, std::pair { other, Vertex } } ) {
if( false == allowed( from, to ) ) { continue; }
queue.push( { cost_of( from, to ), from, to, version[ from ], version[ to ] } );
}
}
} };
for( std::uint32_t vertex = 0; vertex < vertices.size(); ++vertex ) { offer( vertex ); }
while( ( triangles > Budget ) && ( false == queue.empty() ) ) {
auto const best { queue.top() };
queue.pop();
if( ( version[ best.from ] != best.fromversion ) || ( version[ best.to ] != best.toversion ) ) {
continue; // one of the two has moved on since this was worked out
}
if( false == allowed( best.from, best.to ) ) { continue; }
auto const drop { drop_of( best.from, best.to ) };
moved[ best.to ] = std::max( { moved[ best.to ], moved[ best.from ], drop } );
Mesh.error = std::max( Mesh.error, drop );
removed[ best.from ] = true;
quadrics[ best.to ] += quadrics[ best.from ];
for( auto const triangle : incident[ best.from ] ) {
if( true == gone[ triangle ] ) { continue; }
auto wasthere { false };
for( std::size_t corner = 0; corner < 3; ++corner ) {
if( cornerof( triangle, corner ) == best.to ) { wasthere = true; }
}
if( true == wasthere ) {
// the triangle had both ends of the edge: it has no area left
gone[ triangle ] = true;
--triangles;
continue;
}
for( std::size_t corner = 0; corner < 3; ++corner ) {
if( cornerof( triangle, corner ) == best.from ) { cornerof( triangle, corner ) = best.to; }
}
incident[ best.to ].push_back( triangle );
}
incident[ best.from ].clear();
++version[ best.from ];
++version[ best.to ];
offer( best.to );
}
// drop what went, and close up the numbering
std::vector<std::uint32_t> indices;
indices.reserve( triangles * 3 );
for( std::size_t triangle = 0; triangle < gone.size(); ++triangle ) {
if( true == gone[ triangle ] ) { continue; }
for( std::size_t corner = 0; corner < 3; ++corner ) {
indices.push_back( cornerof( triangle, corner ) );
}
}
std::vector<std::uint32_t> renumber( vertices.size(), std::numeric_limits<std::uint32_t>::max() );
std::vector<meshvertex> kept;
kept.reserve( vertices.size() );
for( auto &index : indices ) {
if( renumber[ index ] == std::numeric_limits<std::uint32_t>::max() ) {
renumber[ index ] = static_cast<std::uint32_t>( kept.size() );
kept.push_back( vertices[ index ] );
}
index = renumber[ index ];
}
Mesh.vertices.swap( kept );
Mesh.indices.swap( indices );
}
namespace {
template <typename Type_>
void
append( std::vector<std::uint8_t> &Out, Type_ const &Value ) {
auto const *bytes { reinterpret_cast<std::uint8_t const *>( &Value ) };
Out.insert( Out.end(), bytes, bytes + sizeof( Type_ ) );
}
template <typename Index_>
void
append_indices( std::vector<std::uint8_t> &Out, std::vector<std::uint32_t> const &Indices ) {
std::vector<Index_> narrow( Indices.begin(), Indices.end() );
std::vector<Index_> codes;
highwater_encode( narrow, codes );
for( auto const code : codes ) { append( Out, code ); }
}
template <typename Index_>
void
append_edge( std::vector<std::uint8_t> &Out, std::vector<std::uint32_t> const &Edge ) {
append( Out, static_cast<std::uint32_t>( Edge.size() ) );
for( auto const index : Edge ) { append( Out, static_cast<Index_>( index ) ); }
}
} // anonymous namespace
// The layout is the format's: the header, then the vertices as zigzag deltas of their quantized
// coordinates, then the triangles high-water-mark encoded, then the four lists of border
// vertices, then extensions. Texture coordinates and materials are ours, and ride in one.
void
cooker::encode( tile_address const &Tile, mesh const &Mesh, std::vector<std::uint8_t> &Out ) const {
double west { 0.0 }, north { 0.0 }, side { 0.0 };
square( Tile, west, north, side );
// The format's indices are coded against the highest one used so far, which only works when a
// vertex is numbered the first time a triangle reaches for it. So the vertices are put in the
// order the triangles ask for them; anything no triangle asks for is left out.
std::vector<std::uint32_t> order;
std::vector<std::uint32_t> indices;
{
constexpr std::uint32_t unplaced { std::numeric_limits<std::uint32_t>::max() };
std::vector<std::uint32_t> placed( Mesh.vertices.size(), unplaced );
order.reserve( Mesh.vertices.size() );
indices.reserve( Mesh.indices.size() );
for( auto const index : Mesh.indices ) {
if( placed[ index ] == unplaced ) {
placed[ index ] = static_cast<std::uint32_t>( order.size() );
order.push_back( index );
}
indices.push_back( placed[ index ] );
}
}
auto lowest { Mesh.vertices[ order.front() ].y }, highest { lowest };
for( auto const index : order ) {
lowest = std::min( lowest, Mesh.vertices[ index ].y );
highest = std::max( highest, Mesh.vertices[ index ].y );
}
// heights are quantized against this tile's own range, as the format says. two tiles sharing
// a border therefore round a shared vertex a fraction of a millimetre differently, which is
// what the format's skirts are for elsewhere and is below anything visible here
auto const heightspan { std::max( 1e-3, highest - lowest ) };
file_header header {};
header.centre_x = west + side * 0.5;
header.centre_y = ( lowest + highest ) * 0.5;
header.centre_z = north + side * 0.5;
header.lowest = static_cast<float>( lowest );
header.highest = static_cast<float>( highest );
header.sphere_x = header.centre_x;
header.sphere_y = header.centre_y;
header.sphere_z = header.centre_z;
header.sphere_radius = std::sqrt( 0.5 * side * side + 0.25 * heightspan * heightspan );
header.horizon_x = header.horizon_y = header.horizon_z = 0.0;
Out.clear();
Out.reserve( order.size() * 8 + indices.size() * 2 + sizeof( header ) + 64 );
append( Out, header );
append( Out, static_cast<std::uint32_t>( order.size() ) );
std::vector<std::uint16_t> quantx( order.size() );
std::vector<std::uint16_t> quanty( order.size() );
std::vector<std::uint16_t> quantz( order.size() );
for( std::size_t index = 0; index < order.size(); ++index ) {
auto const &vertex { Mesh.vertices[ order[ index ] ] };
quantx[ index ] = quantize( vertex.x, west, side );
quanty[ index ] = quantize( vertex.y, lowest, heightspan );
quantz[ index ] = quantize( vertex.z, north, side );
}
// the deltas between one vertex and the next, so the run is small numbers
auto const deltas = [ &Out ]( std::vector<std::uint16_t> const &Values ) {
std::int32_t previous { 0 };
for( auto const value : Values ) {
append( Out, zigzag_encode( static_cast<std::int32_t>( value ) - previous ) );
previous = static_cast<std::int32_t>( value );
} };
deltas( quantx );
deltas( quantz );
deltas( quanty );
auto const wide { order.size() > 65536 };
auto const indexsize { wide ? 4u : 2u };
// the format wants the indices to start on a boundary of their own size
while( ( Out.size() % indexsize ) != 0 ) { Out.push_back( 0 ); }
append( Out, static_cast<std::uint32_t>( indices.size() / 3 ) );
if( true == wide ) { append_indices<std::uint32_t>( Out, indices ); }
else { append_indices<std::uint16_t>( Out, indices ); }
// the border vertices, which the format expects to be told about by the quantized coordinate
// they share
std::vector<std::uint32_t> edges[ 4 ];
for( std::size_t index = 0; index < order.size(); ++index ) {
if( quantx[ index ] == 0 ) { edges[ 0 ].push_back( static_cast<std::uint32_t>( index ) ); }
if( quantz[ index ] == 0 ) { edges[ 1 ].push_back( static_cast<std::uint32_t>( index ) ); }
if( quantx[ index ] == quantum_max ) { edges[ 2 ].push_back( static_cast<std::uint32_t>( index ) ); }
if( quantz[ index ] == quantum_max ) { edges[ 3 ].push_back( static_cast<std::uint32_t>( index ) ); }
}
for( auto const &edge : edges ) {
if( true == wide ) { append_edge<std::uint32_t>( Out, edge ); }
else { append_edge<std::uint16_t>( Out, edge ); }
}
// ours: which ground material each vertex belongs to. the format has nowhere for it
append( Out, static_cast<std::uint8_t>( ext_ground ) );
append( Out, static_cast<std::uint32_t>( order.size() * sizeof( std::uint16_t ) ) );
for( auto const index : order ) {
append( Out, Mesh.vertices[ index ].material );
}
}
} // namespace quantizedmesh

176
scene/quantizedmeshcooker.h Normal file
View File

@@ -0,0 +1,176 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#pragma once
// Cuts a scenery's ground into Cesium Quantized Mesh tiles.
//
// The source is already a mesh, so this does not triangulate anything: it cuts the triangles it
// is given along the tile grid, welds what coincides, and builds each coarser level by taking
// the four tiles below it and collapsing edges until the tile costs what one tile should. What
// it never does is invent ground - a hole in the scenery stays a hole - or move a vertex that
// sits on a tile's border, which is what lets neighbours drawn at different levels meet.
//
// Triangles standing upright have no ground in them, and are refused rather than laid: they are
// walls, and the scenery goes on drawing them itself. refusals() says which.
//
// Shared by the engine's first-run bake and the offline tool, so the two cannot drift apart.
// Millions of triangles do not fit in memory, so what is added is spilled to a scratch file and
// read back a tile at a time.
#include <cstdint>
#include <fstream>
#include <functional>
#include <string>
#include <string_view>
#include <unordered_map>
#include <vector>
#include "scene/quantizedmeshformat.h"
namespace quantizedmesh {
class cooker {
public:
// a corner of a source triangle, in world space, with the texture coordinates the scenery
// gave it
struct corner {
double x, y, z;
float u, v;
};
// one finished tile, handed over as the bytes of its file
using sink = std::function<void( tile_address const &, std::vector<std::uint8_t> const & )>;
cooker() = default;
~cooker();
cooker( cooker const & ) = delete;
cooker &operator=( cooker const & ) = delete;
// Tilebudget is the most triangles a tile of any level may hold. The side of a tile at the
// finest level is not given: it is worked out from how large the triangles turn out to be, since
// a tile much smaller than them would spend most of its triangles on the cuts along its own
// borders. Opens the scratch file; returns false when it cannot be written
bool begin( std::string const &Scratchpath, std::size_t Tilebudget = 8192 );
// the side of a tile at the finest level, once finish() has worked it out
double finesttile() const { return m_finesttile; }
// stops talking to standard output, for the engine
void quiet() { m_quiet = true; }
// lays one triangle of ground. the id counts from zero in the order they are added, and is
// what refusals() speaks of
void add( corner const &A, corner const &B, corner const &C, std::string_view Material );
std::size_t added() const { return m_added; }
// cuts, welds, simplifies and hands every tile to the sink, then fills the table. the
// scratch file is read several times over and removed afterwards
bool finish( sink const &Sink, terrain_table &Table );
// triangles that carry no ground, by the id add() gave them, in ascending order. only
// meaningful once finish() has run
std::vector<std::uint32_t> const &refusals() const { return m_refusals; }
// what the cook did, for the log
struct report {
std::size_t triangles { 0 }; // laid, after cutting along the grid
std::size_t refused { 0 };
std::size_t tiles { 0 };
std::size_t vertices { 0 };
std::size_t bytes { 0 };
std::uint32_t levels { 1 };
};
report const &result() const { return m_report; }
// a vertex of a tile's mesh while it is being worked on, in metres
struct meshvertex {
double x, y, z;
std::uint16_t material;
// which of the tile's four sides it sits on, one bit each. a vertex on a side is shared
// with the tile beyond it, so it survives into every level; and an edge whose two ends
// share a side is a piece of the border itself, and survives too
std::uint8_t sides { 0 };
bool locked() const { return sides != 0; }
};
struct mesh {
std::vector<meshvertex> vertices;
std::vector<std::uint32_t> indices;
double error { 0.0 }; // metres this mesh departs from the finest one
bool empty() const { return indices.empty(); }
};
private:
// a triangle as it waits in the scratch file. fixed size, so a tile's triangles can be read
// back by their place in it. only the corners' positions matter by then: the texture
// coordinates have already been turned into the material's tiling
struct spilled {
double corners[ 9 ];
std::uint16_t material;
std::uint16_t padding[ 3 ];
};
std::uint16_t material_index( std::string_view Material );
// how many metres of ground one repeat of a material covers, from the texture coordinates
// the scenery drew it with: the ratio of world distance to texture distance along an edge
void measure_repeat( std::uint16_t Material, corner const ( &Corners )[ 3 ] );
// the square the root tile covers, from what was added
void measure();
// cooks a tile and everything under it, depth first, and hands each to the sink. returns the
// tile's own mesh, which its parent merges
mesh cook( tile_address const &Tile, sink const &Sink );
// the finest level: the triangles of the scratch file that fall on this tile, cut to it
mesh lay( tile_address const &Tile );
// the four tiles below this one, welded into one mesh
mesh merge( tile_address const &Tile, sink const &Sink );
// collapses edges until the mesh fits the budget, or until the ground would move further than
// Allowed, whichever comes first. locked vertices are left where they are
void simplify( mesh &Mesh, std::size_t Budget, double Allowed );
// the bytes of a tile's file
void encode( tile_address const &Tile, mesh const &Mesh, std::vector<std::uint8_t> &Out ) const;
// where a tile's square lies
void square( tile_address const &Tile, double &West, double &North, double &Side ) const;
std::string m_scratchpath;
std::fstream m_scratch;
std::size_t m_added { 0 };
bool m_quiet { false };
double m_finesttile { 256.0 };
std::size_t m_tilebudget { 8192 };
// a sample of how long a triangle's edges are in the plan, which decides the finest tile
std::vector<double> m_edges;
std::vector<std::string> m_materials;
std::unordered_map<std::string, std::uint16_t> m_materiallookup;
std::vector<double> m_repeatsums;
std::vector<double> m_repeatsquares;
std::vector<std::size_t> m_repeatcounts;
// extent and height range of everything added
double m_minx { 0.0 }, m_maxx { 0.0 };
double m_minz { 0.0 }, m_maxz { 0.0 };
double m_lowest { 0.0 }, m_highest { 0.0 };
bool m_anything { false };
// the root square and the depth under it
double m_west { 0.0 }, m_north { 0.0 }, m_side { 0.0 };
std::uint32_t m_levels { 1 };
// which triangles of the scratch file touch which finest tile, sorted by tile. built once,
// walked as the finest level is laid
std::vector<std::pair<std::int64_t, std::uint32_t>> m_placement;
std::vector<std::uint32_t> m_refusals;
// the worst error any tile of each level came out with
std::vector<double> m_errors;
report m_report;
};
} // namespace quantizedmesh

View File

@@ -9,155 +9,191 @@ http://mozilla.org/MPL/2.0/.
#pragma once
// Cesium Quantized Mesh format - TIN-based terrain tiles with LOD.
// Replaces regular heightfield with adaptive triangulation.
// Cesium Quantized Mesh, as the terrain tiles are written.
// https://github.com/CesiumGS/quantized-mesh
//
// Format: https://github.com/CesiumGS/quantized-mesh
//
// File layout:
// header (88 bytes)
// vertex data (positions, uvs, indices)
// edge indices (for seamless LOD stitching)
// extensions (optional metadata)
// The ground in a MaSzyna scenery is already an irregular mesh: fine along the track where it
// came from a survey, coarse in the open where it was drawn by hand. The cook keeps that mesh
// rather than resampling it - it cuts it into tiles, welds the vertices and builds coarser
// levels of it - so there is nothing to interpolate, nothing to fill in where the ground has
// holes, and no grid step to pick.
//
// Advantages over heightfield:
// - Variable density: fine detail where needed (track corridor), coarse elsewhere
// - ~50% fewer vertices for same visual quality on mixed-detail terrain
// - Built-in LOD with proper edge constraints (no T-junctions)
// - Industry-standard format, tooling available
// Tiles form a quadtree, as the format intends: one tile at the root covers the whole terrain,
// and each level splits every tile into four. A distant tile is therefore a large one, and the
// horizon costs tens of tiles rather than thousands. Every vertex on a tile's border survives
// into all of its coarser levels, so two neighbours drawn at different levels share their
// border exactly, and nothing has to be stitched or morphed at draw time.
//
// Disadvantages:
// - More complex to generate (requires constrained triangulation)
// - Slightly larger per-tile overhead (index buffer)
// Where this departs from the specification, and why:
//
// - the header's fields are geodetic (ECEF centre, bounding sphere, horizon occlusion point).
// A scenery has a local metric frame and no geographic reference to convert from, so those
// fields carry scenery metres. The geometry reads back correctly anywhere; a Cesium client
// would place the tile wrongly on the globe, which is a trade for something we do not want.
// - the format has no texture coordinates and no materials: its u and v are the vertex's
// horizontal position within the tile. A vertex of ours carries which ground material it
// belongs to, in extension ext_ground below, which is what the extension mechanism is for.
// Texture coordinates are not stored at all: ground textures tile in world space, so the
// coordinate follows from where the vertex is and how many metres one repeat of its material
// covers, which is measured from the scenery once and kept in the terrain's table.
// - tiles are stored inside the scenery's archive, one zstd-compressed entry each, in place of
// the gzip the format expects from an HTTP layer. The bytes of a tile are the format's.
#include <algorithm>
#include <cstdint>
#include <cmath>
#include <cstring>
#include <string>
#include <vector>
namespace quantizedmesh {
// "EU07QMSH" - distinguishes from other formats
inline constexpr char magic[ 8 ] { 'E', 'U', '0', '7', 'Q', 'M', 'S', 'H' };
inline constexpr std::uint32_t version { 1 };
// u, v and height are quantized to this, not to 65535: the format says so
inline constexpr std::uint16_t quantum_max { 32767 };
// what the cook counted as ground, and how it cut and simplified it. tiles cooked under other
// rules are baked again rather than read
inline constexpr std::uint32_t cook_rules { 2 };
#pragma pack( push, 1 )
// the format's header, unchanged. 88 bytes
struct file_header {
char magic[ 8 ];
std::uint32_t version;
// Tile bounds in world space (metres)
double center_x, center_y, center_z; // tile centre
double min_x, min_y, min_z; // bounding box min
double max_x, max_y, max_z; // bounding box max
// Quantization ranges - vertices stored as uint16, decoded to this range
double horizont_error; // maximum geometric error for this LOD
std::uint32_t vertex_count;
std::uint32_t triangle_count;
// Offsets into file (all after this header)
std::uint32_t vertex_data_offset; // packed uint16[3] positions
std::uint32_t uv_data_offset; // packed uint16[2] texture coords
std::uint32_t index_data_offset; // uint16 indices (or uint32 if >65535 verts)
std::uint32_t edge_indices_offset; // seamless stitching data
std::uint32_t extension_offset; // optional metadata
std::uint16_t index_size; // 2 (uint16) or 4 (uint32)
std::uint16_t edge_count_north;
std::uint16_t edge_count_south;
std::uint16_t edge_count_west;
std::uint16_t edge_count_east;
std::uint16_t reserved; // padding for future use
double centre_x, centre_y, centre_z;
float lowest, highest; // the tile's height range, in metres
double sphere_x, sphere_y, sphere_z; // bounding sphere, for culling
double sphere_radius;
double horizon_x, horizon_y, horizon_z; // horizon occlusion point; unused here
};
static_assert( sizeof( file_header ) == 132 ); // Actual size with current layout
// Edge indices for seamless LOD transitions
// Each edge stores indices of vertices lying on that tile boundary, in order
// Parent LOD uses these to stitch to child tiles without gaps
struct edge_indices {
std::uint16_t count;
std::uint16_t indices[]; // variable length
};
// Optional extensions (material splat, metadata, etc.)
struct extension_header {
std::uint32_t extension_id;
std::uint32_t extension_size;
};
// Extension IDs
enum extension_id : std::uint32_t {
ext_material_splat = 1, // RGBA8 per-vertex material weights
ext_metadata = 2, // JSON metadata
ext_meshlet = 3, // Meshlet data for mesh shading
};
static_assert( sizeof( file_header ) == 88 );
#pragma pack( pop )
// Helper functions
// extension ids. 1, 2 and 4 are taken by the format for normals, the water mask and metadata
enum extension_id : std::uint8_t {
ext_normals = 1,
ext_watermask = 2,
ext_metadata = 4,
// ours: one uint16 per vertex, its material's place in the terrain's table
ext_ground = 64,
};
// Quantize position to uint16
inline std::uint16_t quantize( double value, double min, double max ) {
if( max <= min ) return 0;
double const normalized = std::clamp( ( value - min ) / ( max - min ), 0.0, 1.0 );
return static_cast<std::uint16_t>( normalized * 65535.0 + 0.5 );
// zigzag, as the format uses it on the deltas between consecutive vertices
inline std::uint16_t
zigzag_encode( std::int32_t const Value ) {
return static_cast<std::uint16_t>( ( Value << 1 ) ^ ( Value >> 31 ) );
}
// Dequantize uint16 to position
inline double dequantize( std::uint16_t encoded, double min, double max ) {
double const normalized = static_cast<double>( encoded ) / 65535.0;
return min + normalized * ( max - min );
inline std::int32_t
zigzag_decode( std::uint16_t const Stored ) {
return ( Stored >> 1 ) ^ -static_cast<std::int32_t>( Stored & 1 );
}
// ZigZag encode for delta-compression of indices
inline std::uint32_t zigzag_encode( std::int32_t value ) {
return ( value << 1 ) ^ ( value >> 31 );
// a quantized coordinate back to metres, against the range it was quantized in
inline double
dequantize( std::uint16_t const Stored, double const Low, double const Span ) {
return Low + ( static_cast<double>( Stored ) / quantum_max ) * Span;
}
inline std::int32_t zigzag_decode( std::uint32_t value ) {
return ( value >> 1 ) ^ -static_cast<std::int32_t>( value & 1 );
inline std::uint16_t
quantize( double const Value, double const Low, double const Span ) {
if( Span <= 0.0 ) { return 0; }
auto const share { std::clamp( ( Value - Low ) / Span, 0.0, 1.0 ) };
return static_cast<std::uint16_t>( share * quantum_max + 0.5 );
}
// Octahedral encoding for normals (optional, saves space)
inline void encode_oct( float nx, float ny, float nz, std::int16_t &out_x, std::int16_t &out_y ) {
float const l1norm = std::abs(nx) + std::abs(ny) + std::abs(nz);
float px = nx / l1norm;
float py = ny / l1norm;
if( nz < 0.0f ) {
float const old_x = px;
px = ( 1.0f - std::abs(py) ) * ( old_x >= 0.0f ? 1.0f : -1.0f );
py = ( 1.0f - std::abs(old_x) ) * ( py >= 0.0f ? 1.0f : -1.0f );
// Indices are stored high-water-mark encoded: a code of zero means the next index never used
// before, and anything else counts back from the highest index used so far. The run of codes is
// small numbers whatever the mesh looks like, which is what makes the tile compress.
template <typename Index_>
void
highwater_encode( std::vector<Index_> const &Indices, std::vector<Index_> &Out ) {
Out.clear();
Out.reserve( Indices.size() );
std::int64_t highest { 0 };
for( auto const index : Indices ) {
Out.push_back( static_cast<Index_>( highest - static_cast<std::int64_t>( index ) ) );
if( static_cast<std::int64_t>( index ) == highest ) { ++highest; }
}
out_x = static_cast<std::int16_t>( std::clamp( px * 32767.0f, -32767.0f, 32767.0f ) );
out_y = static_cast<std::int16_t>( std::clamp( py * 32767.0f, -32767.0f, 32767.0f ) );
}
inline void decode_oct( std::int16_t enc_x, std::int16_t enc_y, float &nx, float &ny, float &nz ) {
float px = static_cast<float>( enc_x ) / 32767.0f;
float py = static_cast<float>( enc_y ) / 32767.0f;
nz = 1.0f - std::abs(px) - std::abs(py);
if( nz < 0.0f ) {
float const old_x = px;
px = ( 1.0f - std::abs(py) ) * ( old_x >= 0.0f ? 1.0f : -1.0f );
py = ( 1.0f - std::abs(old_x) ) * ( py >= 0.0f ? 1.0f : -1.0f );
// returns false when a code points past what has been used, which a corrupted tile can do
template <typename Index_>
bool
highwater_decode( std::vector<Index_> const &Codes, std::vector<std::uint32_t> &Out ) {
Out.clear();
Out.reserve( Codes.size() );
std::int64_t highest { 0 };
for( auto const code : Codes ) {
auto const index { highest - static_cast<std::int64_t>( code ) };
if( ( index < 0 ) || ( index > highest ) ) { return false; }
Out.push_back( static_cast<std::uint32_t>( index ) );
if( index == highest ) { ++highest; }
}
float const len = std::sqrt( px*px + py*py + nz*nz );
nx = px / len;
ny = py / len;
nz = nz / len;
return true;
}
// Tile key - same as heightfield for compatibility
// A tile's place in the quadtree. Level 0 is the root, covering the whole terrain; each level
// splits every tile into four. X grows east, Z grows with the scenery's z, so a tile's square
// follows from the terrain's extent and the level alone.
struct tile_address {
std::uint32_t level { 0 };
std::int32_t x { 0 }, z { 0 };
bool operator==( tile_address const & ) const = default;
};
constexpr std::int64_t
tile_key( std::int32_t const X, std::int32_t const Z ) {
return ( static_cast<std::int64_t>( X ) << 32 ) ^ static_cast<std::uint32_t>( Z );
tile_key( tile_address const &Tile ) {
// a level fits in six bits for any terrain worth cooking, and the coordinates cannot exceed
// the level, so this is unique
return ( static_cast<std::int64_t>( Tile.level ) << 58 )
^ ( static_cast<std::int64_t>( Tile.x ) << 29 )
^ static_cast<std::int64_t>( Tile.z );
}
// where a tile's file sits in the archive, following the format's own layout of level, column
// and row
inline std::string
tile_name( tile_address const &Tile ) {
return std::to_string( Tile.level ) + "/" + std::to_string( Tile.x ) + "/"
+ std::to_string( Tile.z ) + ".terrain";
}
// the terrain's extent, height range, depth and material table are the same for every tile, so
// they are written once beside them, under this name. the format keeps the same thing in
// layer.json
inline constexpr char const *tablename { "terrain.table" };
// what that file says
struct terrain_table {
std::uint32_t rules { 0 };
// the square the root tile covers. always square, so that a tile at any level is too
double west { 0.0 }, north { 0.0 }, side { 0.0 };
double lowest { 0.0 }, highest { 0.0 };
// levels cooked, root included: addresses run from 0 to levels - 1
std::uint32_t levels { 1 };
// metres the ground of a tile at each level departs from the finest one. what decides which
// level to draw: a level is good enough while its error covers few enough pixels
std::vector<float> errors;
// one ground texture: its name, and how many metres of ground one repeat of it covers, as
// measured from the texture coordinates the scenery drew it with
struct material {
std::string name;
float repeat { 4.f };
};
// in the order the vertices' material indices count
std::vector<material> materials;
// metres along the side of a tile at this level
double tilesize( std::uint32_t const Level ) const {
return side / static_cast<double>( 1u << Level );
}
float error( std::uint32_t const Level ) const {
return Level < errors.size() ? errors[ Level ] : 0.f;
}
};
} // namespace quantizedmesh

View File

@@ -9,215 +9,369 @@ http://mozilla.org/MPL/2.0/.
#pragma once
// Reads Cesium Quantized Mesh tiles - replaces heightfield reader entirely.
// Used by terrain_tile_loader to stream TIN tiles from disk.
// Reads cooked terrain tiles back, from a scenery's archive or from a directory of loose files.
// A directory is what a tile cooked by hand or by another tool arrives as; the archive is what
// the bake writes.
//
// Everything a tile says about itself is checked before it is believed: a truncated or mangled
// tile is refused rather than handed on, because what comes out of here goes straight into a
// vertex buffer and an index buffer.
#include <cstdint>
#include <cstdio>
#include <cstring>
#include <filesystem>
#include <fstream>
#include <string>
#include <unordered_set>
#include <vector>
#include <memory>
#include "scene/pakformat.h"
#include "scene/quantizedmeshformat.h"
namespace quantizedmesh {
// Reader for cooked TIN terrain tiles (.qm files)
// Single-threaded, meant to be used from terrain_tile_loader worker thread
// one vertex as the renderer wants it: measured from the terrain's origin, so that the whole
// terrain fits in floats and every tile of it can go into one vertex buffer. there are no texture
// coordinates - the ground tiles in world space, so the shader works them out from the position
#pragma pack( push, 1 )
struct render_vertex {
float x, y, z;
std::uint16_t material;
std::uint16_t padding;
};
#pragma pack( pop )
static_assert( sizeof( render_vertex ) == 16 );
class reader {
public:
struct tile_data {
tile_address tile;
std::vector<render_vertex> vertices;
std::vector<std::uint32_t> indices;
// what the vertices are measured from, in scenery metres
double originx { 0.0 }, originy { 0.0 }, originz { 0.0 };
// for culling, in scenery metres
double centrex { 0.0 }, centrey { 0.0 }, centrez { 0.0 };
double radius { 0.0 };
float lowest { 0.f }, highest { 0.f };
};
reader() = default;
~reader() { close(); }
reader( reader const & ) = delete;
reader &operator=( reader const & ) = delete;
// Open tile directory (contains terrain_X_Z_lodN.qm files)
bool open( std::string const &Directory );
// an archive or a directory. reads the terrain's table, without which nothing can be placed
bool open( std::string const &Path );
void close();
// Read single LOD of a tile
struct tile_data {
// Dequantized positions
std::vector<double> positions_x;
std::vector<double> positions_y;
std::vector<double> positions_z;
// Texture coordinates
std::vector<float> uvs_u;
std::vector<float> uvs_v;
// Triangle indices
std::vector<std::uint32_t> indices;
// Material per vertex (index into material table)
std::vector<std::uint8_t> materials;
// Edge indices for stitching
std::vector<std::uint16_t> north_edge;
std::vector<std::uint16_t> south_edge;
std::vector<std::uint16_t> west_edge;
std::vector<std::uint16_t> east_edge;
// Tile bounds (for positioning)
double center_x, center_y, center_z;
double min_x, min_y, min_z;
double max_x, max_y, max_z;
double geometric_error;
};
bool read_tile( std::int32_t X, std::int32_t Z, std::uint32_t LOD, tile_data &Out );
// Check if tile exists
bool contains( std::int32_t X, std::int32_t Z, std::uint32_t LOD ) const;
std::string const &directory() const { return m_directory; }
bool ready() const { return m_ready; }
terrain_table const &table() const { return m_table; }
std::string const &path() const { return m_path; }
// Origin is what the vertices come back measured from: one point for the whole terrain, so
// that every tile can go into the same vertex buffer and be drawn in one call
bool read_tile( tile_address const &Tile, tile_data &Out,
double Originx, double Originy, double Originz );
// the tiles the source holds, by tile_key. a tile the cook found nothing for is not there
std::unordered_set<std::int64_t> const &present() const { return m_present; }
bool contains( tile_address const &Tile ) const { return m_present.contains( tile_key( Tile ) ); }
private:
std::string tile_path( std::int32_t X, std::int32_t Z, std::uint32_t LOD ) const;
std::string m_directory;
bool entry( std::string const &Name, std::vector<std::uint8_t> &Out );
bool read_table();
void find_tiles();
std::string m_path;
bool m_archive { false };
pak::reader m_pak;
terrain_table m_table;
std::unordered_set<std::int64_t> m_present;
bool m_ready { false };
std::vector<std::uint8_t> m_buffer;
};
// Implementation
// Implementation.
inline bool
reader::open( std::string const &Path ) {
inline bool reader::open( std::string const &Directory ) {
close();
m_directory = Directory;
return true; // Lazy - tiles opened on demand
}
inline void reader::close() {
m_directory.clear();
}
inline std::string reader::tile_path( std::int32_t X, std::int32_t Z, std::uint32_t LOD ) const {
char filename[ 256 ];
std::snprintf( filename, sizeof( filename ), "terrain_%d_%d_lod%u.qm", X, Z, LOD );
return m_directory + "/" + filename;
}
inline bool reader::contains( std::int32_t X, std::int32_t Z, std::uint32_t LOD ) const {
auto const path = tile_path( X, Z, LOD );
FILE *f = std::fopen( path.c_str(), "rb" );
if( !f ) return false;
std::fclose( f );
m_path = Path;
std::error_code error;
if( true == std::filesystem::is_directory( Path, error ) ) {
m_archive = false;
}
else if( true == std::filesystem::is_regular_file( Path, error ) ) {
m_archive = true;
if( false == m_pak.open( Path ) ) { close(); return false; }
}
else {
close();
return false;
}
if( false == read_table() ) { close(); return false; }
find_tiles();
m_ready = true;
return true;
}
inline bool reader::read_tile( std::int32_t X, std::int32_t Z, std::uint32_t LOD, tile_data &Out ) {
auto const path = tile_path( X, Z, LOD );
FILE *file = std::fopen( path.c_str(), "rb" );
if( !file ) return false;
// Read header
file_header header;
if( std::fread( &header, sizeof( header ), 1, file ) != 1 ) {
std::fclose( file );
return false;
inline void
reader::close() {
m_pak.close();
m_table = {};
m_present.clear();
m_path.clear();
m_archive = false;
m_ready = false;
}
inline bool
reader::entry( std::string const &Name, std::vector<std::uint8_t> &Out ) {
if( true == m_archive ) { return m_pak.read( Name, Out ); }
std::ifstream file( std::filesystem::path( m_path ) / Name, std::ios::binary );
if( false == file.good() ) { return false; }
file.seekg( 0, std::ios::end );
auto const size { static_cast<std::size_t>( file.tellg() ) };
file.seekg( 0, std::ios::beg );
Out.resize( size );
if( size > 0 ) {
file.read( reinterpret_cast<char *>( Out.data() ), static_cast<std::streamsize>( size ) );
}
// Verify magic
if( std::memcmp( header.magic, magic, sizeof( magic ) ) != 0 ) {
std::fclose( file );
return false;
}
if( header.version != version ) {
std::fclose( file );
return false;
}
// Store bounds
Out.center_x = header.center_x;
Out.center_y = header.center_y;
Out.center_z = header.center_z;
Out.min_x = header.min_x;
Out.min_y = header.min_y;
Out.min_z = header.min_z;
Out.max_x = header.max_x;
Out.max_y = header.max_y;
Out.max_z = header.max_z;
Out.geometric_error = header.horizont_error;
// Read quantized positions
std::fseek( file, header.vertex_data_offset, SEEK_SET );
std::vector<std::uint16_t> quantized_x( header.vertex_count );
std::vector<std::uint16_t> quantized_y( header.vertex_count );
std::vector<std::uint16_t> quantized_z( header.vertex_count );
for( std::uint32_t i = 0; i < header.vertex_count; ++i ) {
std::fread( &quantized_x[ i ], sizeof( std::uint16_t ), 1, file );
std::fread( &quantized_y[ i ], sizeof( std::uint16_t ), 1, file );
std::fread( &quantized_z[ i ], sizeof( std::uint16_t ), 1, file );
}
// Dequantize positions
Out.positions_x.resize( header.vertex_count );
Out.positions_y.resize( header.vertex_count );
Out.positions_z.resize( header.vertex_count );
for( std::uint32_t i = 0; i < header.vertex_count; ++i ) {
Out.positions_x[ i ] = dequantize( quantized_x[ i ], header.min_x, header.max_x );
Out.positions_y[ i ] = dequantize( quantized_y[ i ], header.min_y, header.max_y );
Out.positions_z[ i ] = dequantize( quantized_z[ i ], header.min_z, header.max_z );
}
// Read UVs
std::fseek( file, header.uv_data_offset, SEEK_SET );
Out.uvs_u.resize( header.vertex_count );
Out.uvs_v.resize( header.vertex_count );
for( std::uint32_t i = 0; i < header.vertex_count; ++i ) {
std::uint16_t u, v;
std::fread( &u, sizeof( std::uint16_t ), 1, file );
std::fread( &v, sizeof( std::uint16_t ), 1, file );
Out.uvs_u[ i ] = static_cast<float>( dequantize( u, 0.0, 1.0 ) );
Out.uvs_v[ i ] = static_cast<float>( dequantize( v, 0.0, 1.0 ) );
}
// Read indices
std::fseek( file, header.index_data_offset, SEEK_SET );
Out.indices.resize( header.triangle_count * 3 );
if( header.index_size == 2 ) {
for( std::uint32_t i = 0; i < header.triangle_count * 3; ++i ) {
std::uint16_t idx;
std::fread( &idx, sizeof( std::uint16_t ), 1, file );
Out.indices[ i ] = idx;
return static_cast<std::size_t>( file.gcount() ) == size;
}
// the table is text: a terrain outlives the build that cooked it, and a line of it should be
// readable without a tool
inline bool
reader::read_table() {
std::vector<std::uint8_t> bytes;
if( false == entry( tablename, bytes ) ) { return false; }
std::string const text( bytes.begin(), bytes.end() );
auto found { false };
std::size_t at { 0 };
while( at < text.size() ) {
auto const end { std::min( text.find( '\n', at ), text.size() ) };
std::string const line { text.substr( at, end - at ) };
at = end + 1;
if( line.empty() ) { continue; }
auto const space { line.find( ' ' ) };
std::string const what { line.substr( 0, space ) };
std::string const rest { space == std::string::npos ? std::string {} : line.substr( space + 1 ) };
if( what == "eu07-terrain" ) { found = true; }
else if( what == "rules" ) { m_table.rules = static_cast<std::uint32_t>( std::strtoul( rest.c_str(), nullptr, 10 ) ); }
else if( what == "levels" ) { m_table.levels = static_cast<std::uint32_t>( std::strtoul( rest.c_str(), nullptr, 10 ) ); }
else if( what == "square" ) {
char const *cursor { rest.c_str() };
m_table.west = std::strtod( cursor, const_cast<char **>( &cursor ) );
m_table.north = std::strtod( cursor, const_cast<char **>( &cursor ) );
m_table.side = std::strtod( cursor, const_cast<char **>( &cursor ) );
}
else if( what == "height" ) {
char const *cursor { rest.c_str() };
m_table.lowest = std::strtod( cursor, const_cast<char **>( &cursor ) );
m_table.highest = std::strtod( cursor, const_cast<char **>( &cursor ) );
}
else if( what == "material" ) {
// name first, then how many metres one repeat covers. a name can hold spaces, so the
// number is taken off the end
auto const gap { rest.rfind( ' ' ) };
if( gap == std::string::npos ) { m_table.materials.push_back( { rest, 4.f } ); }
else {
m_table.materials.push_back(
{ rest.substr( 0, gap ), static_cast<float>( std::strtod( rest.c_str() + gap + 1, nullptr ) ) } );
}
}
else if( what == "error" ) {
char const *cursor { rest.c_str() };
auto const level { std::strtoul( cursor, const_cast<char **>( &cursor ), 10 ) };
auto const metres { std::strtod( cursor, const_cast<char **>( &cursor ) ) };
if( m_table.errors.size() <= level ) { m_table.errors.resize( level + 1, 0.f ); }
m_table.errors[ level ] = static_cast<float>( metres );
}
} else {
std::fread( Out.indices.data(), sizeof( std::uint32_t ), header.triangle_count * 3, file );
}
// Read edge indices
std::fseek( file, header.edge_indices_offset, SEEK_SET );
Out.north_edge.resize( header.edge_count_north );
Out.south_edge.resize( header.edge_count_south );
Out.west_edge.resize( header.edge_count_west );
Out.east_edge.resize( header.edge_count_east );
if( header.edge_count_north > 0 )
std::fread( Out.north_edge.data(), sizeof( std::uint16_t ), header.edge_count_north, file );
if( header.edge_count_south > 0 )
std::fread( Out.south_edge.data(), sizeof( std::uint16_t ), header.edge_count_south, file );
if( header.edge_count_west > 0 )
std::fread( Out.west_edge.data(), sizeof( std::uint16_t ), header.edge_count_west, file );
if( header.edge_count_east > 0 )
std::fread( Out.east_edge.data(), sizeof( std::uint16_t ), header.edge_count_east, file );
// Materials - for now, all zeros (placeholder)
Out.materials.resize( header.vertex_count, 0 );
std::fclose( file );
return found
&& ( m_table.rules == cook_rules )
&& ( m_table.side > 0.0 )
&& ( m_table.levels >= 1 );
}
// Which tiles there are is worked out once. Asking the filesystem, or even the archive's index,
// for a tile that is not there is the commonest thing the renderer does - most of a quadtree is
// empty - and it must not cost anything.
inline void
reader::find_tiles() {
auto const remember = [ this ]( std::string const &Name ) {
if( false == Name.ends_with( ".terrain" ) ) { return; }
tile_address tile {};
if( 3 != std::sscanf( Name.c_str(), "%u/%d/%d.terrain", &tile.level, &tile.x, &tile.z ) ) { return; }
if( Name != tile_name( tile ) ) { return; }
m_present.insert( tile_key( tile ) ); };
if( true == m_archive ) {
for( auto const &name : m_pak.names() ) { remember( name ); }
return;
}
std::error_code error;
auto const base { std::filesystem::path( m_path ) };
for( auto const &found : std::filesystem::recursive_directory_iterator( base, error ) ) {
if( false == found.is_regular_file( error ) ) { continue; }
auto const relative { std::filesystem::relative( found.path(), base, error ) };
if( error ) { continue; }
remember( relative.generic_string() );
}
}
inline bool
reader::read_tile( tile_address const &Tile, tile_data &Out,
double const Originx, double const Originy, double const Originz ) {
if( false == m_ready ) { return false; }
if( Tile.level >= m_table.levels ) { return false; }
if( false == entry( tile_name( Tile ), m_buffer ) ) { return false; }
auto const *cursor { m_buffer.data() };
auto const *end { cursor + m_buffer.size() };
auto const take = [ &cursor, end ]( void *Into, std::size_t const Bytes ) {
if( static_cast<std::size_t>( end - cursor ) < Bytes ) { return false; }
std::memcpy( Into, cursor, Bytes );
cursor += Bytes;
return true; };
file_header header {};
if( false == take( &header, sizeof( header ) ) ) { return false; }
std::uint32_t vertexcount { 0 };
if( false == take( &vertexcount, sizeof( vertexcount ) ) ) { return false; }
if( ( vertexcount < 3 ) || ( vertexcount > ( 1u << 24 ) ) ) { return false; }
// the three runs of zigzag deltas: x, z, then height
std::vector<std::uint16_t> quantized[ 3 ];
for( auto &run : quantized ) {
run.resize( vertexcount );
std::int32_t value { 0 };
for( auto &stored : run ) {
std::uint16_t delta { 0 };
if( false == take( &delta, sizeof( delta ) ) ) { return false; }
value += zigzag_decode( delta );
if( ( value < 0 ) || ( value > quantum_max ) ) { return false; }
stored = static_cast<std::uint16_t>( value );
}
}
auto const wide { vertexcount > 65536 };
auto const indexsize { wide ? 4u : 2u };
while( ( static_cast<std::size_t>( cursor - m_buffer.data() ) % indexsize ) != 0 ) {
if( cursor >= end ) { return false; }
++cursor;
}
std::uint32_t trianglecount { 0 };
if( false == take( &trianglecount, sizeof( trianglecount ) ) ) { return false; }
if( ( trianglecount == 0 ) || ( trianglecount > ( 1u << 24 ) ) ) { return false; }
auto const codes { static_cast<std::size_t>( trianglecount ) * 3 };
auto decoded { false };
if( true == wide ) {
std::vector<std::uint32_t> raw( codes );
for( auto &code : raw ) { if( false == take( &code, sizeof( code ) ) ) { return false; } }
decoded = highwater_decode( raw, Out.indices );
}
else {
std::vector<std::uint16_t> raw( codes );
for( auto &code : raw ) { if( false == take( &code, sizeof( code ) ) ) { return false; } }
decoded = highwater_decode( raw, Out.indices );
}
if( false == decoded ) { return false; }
for( auto const index : Out.indices ) {
if( index >= vertexcount ) { return false; }
}
// the four lists of border vertices. nothing here needs them - a tile's border survives into
// every level of it, so neighbours meet whatever level they are drawn at - but they are part
// of the format and are stepped over
for( int edge = 0; edge < 4; ++edge ) {
std::uint32_t count { 0 };
if( false == take( &count, sizeof( count ) ) ) { return false; }
if( count > vertexcount ) { return false; }
if( static_cast<std::size_t>( end - cursor ) < count * indexsize ) { return false; }
cursor += count * indexsize;
}
// which material each vertex belongs to, from the extension the cook writes
std::vector<std::uint16_t> ground;
while( cursor < end ) {
std::uint8_t id { 0 };
std::uint32_t length { 0 };
if( false == take( &id, sizeof( id ) ) ) { return false; }
if( false == take( &length, sizeof( length ) ) ) { return false; }
if( static_cast<std::size_t>( end - cursor ) < length ) { return false; }
if( ( id == ext_ground ) && ( length == vertexcount * sizeof( std::uint16_t ) ) ) {
ground.resize( vertexcount );
std::memcpy( ground.data(), cursor, length );
}
cursor += length;
}
if( ground.size() != vertexcount ) { return false; }
auto const side { m_table.tilesize( Tile.level ) };
auto const west { m_table.west + Tile.x * side };
auto const north { m_table.north + Tile.z * side };
auto const heightspan { std::max( 1e-3, static_cast<double>( header.highest ) - header.lowest ) };
Out.tile = Tile;
Out.originx = Originx;
Out.originy = Originy;
Out.originz = Originz;
Out.centrex = header.sphere_x;
Out.centrey = header.sphere_y;
Out.centrez = header.sphere_z;
Out.radius = header.sphere_radius;
Out.lowest = header.lowest;
Out.highest = header.highest;
Out.vertices.resize( vertexcount );
for( std::uint32_t index = 0; index < vertexcount; ++index ) {
auto &vertex { Out.vertices[ index ] };
vertex.x = static_cast<float>( dequantize( quantized[ 0 ][ index ], west, side ) - Out.originx );
vertex.y = static_cast<float>( dequantize( quantized[ 2 ][ index ], header.lowest, heightspan ) - Out.originy );
vertex.z = static_cast<float>( dequantize( quantized[ 1 ][ index ], north, side ) - Out.originz );
vertex.material = ground[ index ];
vertex.padding = 0;
}
return true;
}
// writes the table the reader expects, so that the cook and the reader cannot disagree about it
inline std::vector<std::uint8_t>
write_table( terrain_table const &Table ) {
auto const number = []( double const Value ) {
char text[ 64 ];
std::snprintf( text, sizeof( text ), "%.6f", Value );
return std::string { text }; };
std::string text { "eu07-terrain 1\n" };
text += "rules " + std::to_string( Table.rules ) + "\n";
text += "square " + number( Table.west ) + " " + number( Table.north ) + " " + number( Table.side ) + "\n";
text += "height " + number( Table.lowest ) + " " + number( Table.highest ) + "\n";
text += "levels " + std::to_string( Table.levels ) + "\n";
for( std::size_t level = 0; level < Table.errors.size(); ++level ) {
text += "error " + std::to_string( level ) + " " + number( Table.errors[ level ] ) + "\n";
}
for( auto const &material : Table.materials ) {
text += "material " + material.name + " " + number( material.repeat ) + "\n";
}
return { text.begin(), text.end() };
}
} // namespace quantizedmesh

View File

@@ -1,579 +0,0 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#include "scene/terraincooker_tin.h"
#include "scene/terraincooker.h"
#include <algorithm>
#include <cassert>
#include <cmath>
#include <fstream>
#include <iostream>
#include <queue>
#include <unordered_set>
namespace terrain {
namespace {
// Hash for int64 tile key
struct tile_key_hash {
std::size_t operator()( std::int64_t key ) const noexcept {
return static_cast<std::size_t>( key ^ ( key >> 32 ) );
}
};
// Calculate tile coordinates from world position
std::int32_t tile_coord( double world_pos, double tile_size ) {
return static_cast<std::int32_t>( std::floor( world_pos / tile_size ) );
}
std::int64_t make_tile_key( std::int32_t x, std::int32_t z ) {
return ( static_cast<std::int64_t>( x ) << 32 ) | static_cast<std::uint32_t>( z );
}
} // anonymous namespace
void tin_cooker::configure(
double TileSize,
std::uint32_t LodLevels,
double ErrorMultiplier
) {
m_tilesize = TileSize;
m_lodlevels = std::max( 1u, LodLevels );
m_errormultiplier = std::max( 1.1, ErrorMultiplier );
}
void tin_cooker::rasterize(
vertex const &A,
vertex const &B,
vertex const &C,
std::string_view Material
) {
++m_triangles_in;
// Find centroid to determine tile
double const cx = ( A.x + B.x + C.x ) / 3.0;
double const cz = ( A.z + B.z + C.z ) / 3.0;
std::int32_t const tile_x = tile_coord( cx, m_tilesize );
std::int32_t const tile_z = tile_coord( cz, m_tilesize );
std::int64_t const key = make_tile_key( tile_x, tile_z );
auto &tile = m_tiles[ key ];
tile.triangles.push_back( { { A, B, C }, std::string( Material ) } );
tile.dirty = true;
}
void tin_cooker::add_constraint( vertex const &A, vertex const &B ) {
// Add constraint to all tiles it intersects
double const minx = std::min( A.x, B.x );
double const maxx = std::max( A.x, B.x );
double const minz = std::min( A.z, B.z );
double const maxz = std::max( A.z, B.z );
std::int32_t const tile_x0 = tile_coord( minx, m_tilesize );
std::int32_t const tile_z0 = tile_coord( minz, m_tilesize );
std::int32_t const tile_x1 = tile_coord( maxx, m_tilesize );
std::int32_t const tile_z1 = tile_coord( maxz, m_tilesize );
for( std::int32_t tx = tile_x0; tx <= tile_x1; ++tx ) {
for( std::int32_t tz = tile_z0; tz <= tile_z1; ++tz ) {
std::int64_t const key = make_tile_key( tx, tz );
auto &tile = m_tiles[ key ];
tile.constraints.push_back( { A, B } );
tile.dirty = true;
}
}
}
std::vector<tin_tile> tin_cooker::finish() {
std::vector<tin_tile> result;
result.reserve( m_tiles.size() );
std::size_t total_tiles = m_tiles.size();
std::size_t processed = 0;
for( auto &[ key, accumulator ] : m_tiles ) {
if( !accumulator.dirty || accumulator.triangles.empty() ) {
continue;
}
++processed;
if( processed % 100 == 0 || processed == total_tiles ) {
std::printf( "\rTriangulating tiles: %zu/%zu", processed, total_tiles );
std::fflush( stdout );
}
std::int32_t const tile_x = static_cast<std::int32_t>( key >> 32 );
std::int32_t const tile_z = static_cast<std::int32_t>( key & 0xffffffff );
tin_tile tile;
tile.x = tile_x;
tile.z = tile_z;
// Tile bounds
tile.minx = tile_x * m_tilesize;
tile.minz = tile_z * m_tilesize;
tile.maxx = tile.minx + m_tilesize;
tile.maxz = tile.minz + m_tilesize;
tile.miny = std::numeric_limits<double>::max();
tile.maxy = std::numeric_limits<double>::lowest();
// Collect unique vertices from triangles
std::vector<vertex> points;
points.reserve( accumulator.triangles.size() * 3 );
for( auto const &tri : accumulator.triangles ) {
for( auto const &v : tri.v ) {
points.push_back( v );
tile.miny = std::min( tile.miny, v.y );
tile.maxy = std::max( tile.maxy, v.y );
}
}
// Build material table
for( auto const &tri : accumulator.triangles ) {
get_material_index( tile, tri.material );
}
// Generate LODs from finest to coarsest
double current_error = 0.1; // Start with very fine detail
for( std::uint32_t lod = 0; lod < m_lodlevels; ++lod ) {
tin_lod level;
if( lod == 0 ) {
// Finest LOD: triangulate all points with constraints
level = triangulate_lod( points, accumulator.constraints, current_error );
} else {
// Coarser LODs: simplify previous level
level = simplify_lod( tile.lods.back(), current_error );
}
level.geometric_error = current_error;
extract_edges( level, tile.minx, tile.minz, tile.maxx, tile.maxz );
tile.lods.push_back( std::move( level ) );
m_triangles_out += level.indices.size() / 3;
m_vertices_out += level.vertices.size();
current_error *= m_errormultiplier;
}
result.push_back( std::move( tile ) );
}
std::printf( "\n" );
std::printf( "Triangulation complete:\n" );
std::printf( " Input triangles: %zu\n", m_triangles_in );
std::printf( " Output triangles: %zu\n", m_triangles_out );
std::printf( " Output vertices: %zu\n", m_vertices_out );
std::printf( " Reduction: %.1f%%\n",
m_triangles_in > 0 ? 100.0 * ( 1.0 - static_cast<double>( m_triangles_out ) / m_triangles_in ) : 0.0 );
return result;
}
tin_lod tin_cooker::triangulate_lod(
std::vector<vertex> const &Points,
std::vector<tin_constraint> const &Constraints,
double TargetError
) {
tin_lod result;
if( Points.empty() ) {
return result;
}
// Convert points to CDT format, removing duplicates
std::vector<CDT::V2d<double>> cdt_points;
std::vector<CDT::Edge> cdt_edges;
cdt_points.reserve( Points.size() );
std::unordered_map<std::uint64_t, std::uint32_t> point_map;
auto hash_point = []( vertex const &v ) -> std::uint64_t {
// Hash to 1cm precision
std::uint32_t const x = static_cast<std::uint32_t>( std::round( v.x * 100.0 ) );
std::uint32_t const z = static_cast<std::uint32_t>( std::round( v.z * 100.0 ) );
return ( static_cast<std::uint64_t>( x ) << 32 ) | z;
};
for( auto const &v : Points ) {
std::uint64_t const h = hash_point( v );
auto const [ it, inserted ] = point_map.try_emplace( h, static_cast<std::uint32_t>( cdt_points.size() ) );
if( inserted ) {
cdt_points.push_back( to_cdt_point( v ) );
result.vertices.push_back( v );
}
}
// Add constraint edges
for( auto const &constraint : Constraints ) {
std::uint64_t const h0 = hash_point( constraint.a );
std::uint64_t const h1 = hash_point( constraint.b );
auto it0 = point_map.find( h0 );
auto it1 = point_map.find( h1 );
if( it0 != point_map.end() && it1 != point_map.end() && it0->second != it1->second ) {
cdt_edges.push_back( CDT::Edge( it0->second, it1->second ) );
}
}
// Run CDT
CDT::Triangulation<double> cdt;
cdt.insertVertices( cdt_points );
if( !cdt_edges.empty() ) {
cdt.insertEdges( cdt_edges );
cdt.eraseOuterTrianglesAndHoles();
}
// Extract triangles
result.indices.reserve( cdt.triangles.size() * 3 );
for( auto const &tri : cdt.triangles ) {
result.indices.push_back( tri.vertices[ 0 ] );
result.indices.push_back( tri.vertices[ 1 ] );
result.indices.push_back( tri.vertices[ 2 ] );
}
// Assign material indices (simplified - use most common material for now)
result.materials.resize( result.vertices.size(), 0 );
return result;
}
tin_lod tin_cooker::simplify_lod(
tin_lod const &FineLod,
double TargetError
) {
// Edge collapse simplification
// For now: simple vertex removal based on geometric error
tin_lod result = FineLod; // Start with copy
if( FineLod.vertices.size() < 100 ) {
// Too small to simplify meaningfully
return result;
}
// Build adjacency information
std::unordered_map<std::uint32_t, std::vector<std::uint32_t>> adjacency;
for( std::size_t i = 0; i < FineLod.indices.size(); i += 3 ) {
std::uint32_t const i0 = FineLod.indices[ i + 0 ];
std::uint32_t const i1 = FineLod.indices[ i + 1 ];
std::uint32_t const i2 = FineLod.indices[ i + 2 ];
adjacency[ i0 ].push_back( i1 );
adjacency[ i0 ].push_back( i2 );
adjacency[ i1 ].push_back( i0 );
adjacency[ i1 ].push_back( i2 );
adjacency[ i2 ].push_back( i0 );
adjacency[ i2 ].push_back( i1 );
}
// Calculate error for each vertex
struct vertex_error {
std::uint32_t index;
double error;
bool operator>( vertex_error const &other ) const {
return error > other.error; // Min-heap
}
};
std::priority_queue<vertex_error, std::vector<vertex_error>, std::greater<vertex_error>> error_queue;
std::unordered_set<std::uint32_t> removed;
for( std::uint32_t i = 0; i < FineLod.vertices.size(); ++i ) {
auto it = adjacency.find( i );
if( it == adjacency.end() || it->second.empty() ) {
continue;
}
std::vector<vertex> neighbours;
for( auto const j : it->second ) {
neighbours.push_back( FineLod.vertices[ j ] );
}
double const error = calculate_error( FineLod.vertices[ i ], neighbours );
if( error < TargetError ) {
error_queue.push( { i, error } );
}
}
// Remove vertices with low error (target: remove ~30%)
std::size_t const target_removals = FineLod.vertices.size() * 3 / 10;
while( !error_queue.empty() && removed.size() < target_removals ) {
auto const top = error_queue.top();
error_queue.pop();
if( removed.count( top.index ) > 0 ) {
continue;
}
removed.insert( top.index );
}
// Rebuild mesh without removed vertices
std::unordered_map<std::uint32_t, std::uint32_t> index_remap;
std::vector<vertex> new_vertices;
std::vector<std::uint8_t> new_materials;
for( std::uint32_t i = 0; i < FineLod.vertices.size(); ++i ) {
if( removed.count( i ) == 0 ) {
index_remap[ i ] = static_cast<std::uint32_t>( new_vertices.size() );
new_vertices.push_back( FineLod.vertices[ i ] );
if( i < FineLod.materials.size() ) {
new_materials.push_back( FineLod.materials[ i ] );
}
}
}
result.vertices = std::move( new_vertices );
result.materials = std::move( new_materials );
// Re-triangulate (simple approach: use CDT again)
std::vector<CDT::V2d<double>> cdt_points;
cdt_points.reserve( result.vertices.size() );
for( auto const &v : result.vertices ) {
cdt_points.push_back( to_cdt_point( v ) );
}
CDT::Triangulation<double> cdt;
cdt.insertVertices( cdt_points );
result.indices.clear();
result.indices.reserve( cdt.triangles.size() * 3 );
for( auto const &tri : cdt.triangles ) {
result.indices.push_back( tri.vertices[ 0 ] );
result.indices.push_back( tri.vertices[ 1 ] );
result.indices.push_back( tri.vertices[ 2 ] );
}
return result;
}
double tin_cooker::calculate_error(
vertex const &V,
std::vector<vertex> const &Neighbours
) {
if( Neighbours.size() < 3 ) {
return std::numeric_limits<double>::max();
}
// Calculate maximum distance to plane fitted through neighbours
// Simplified: just use average height difference
double sum_y = 0.0;
for( auto const &n : Neighbours ) {
sum_y += n.y;
}
double const avg_y = sum_y / Neighbours.size();
return std::abs( V.y - avg_y );
}
void tin_cooker::extract_edges(
tin_lod &Lod,
double TileMinX,
double TileMinZ,
double TileMaxX,
double TileMaxZ
) {
constexpr double edge_epsilon = 0.01;
Lod.north_edge.clear();
Lod.south_edge.clear();
Lod.west_edge.clear();
Lod.east_edge.clear();
for( std::uint32_t i = 0; i < Lod.vertices.size(); ++i ) {
auto const &v = Lod.vertices[ i ];
bool on_edge = false;
if( std::abs( v.x - TileMinX ) < edge_epsilon ) {
Lod.west_edge.push_back( static_cast<std::uint16_t>( i ) );
on_edge = true;
}
if( std::abs( v.x - TileMaxX ) < edge_epsilon ) {
Lod.east_edge.push_back( static_cast<std::uint16_t>( i ) );
on_edge = true;
}
if( std::abs( v.z - TileMinZ ) < edge_epsilon ) {
Lod.south_edge.push_back( static_cast<std::uint16_t>( i ) );
on_edge = true;
}
if( std::abs( v.z - TileMaxZ ) < edge_epsilon ) {
Lod.north_edge.push_back( static_cast<std::uint16_t>( i ) );
on_edge = true;
}
}
// Sort edges by position (west/east by Z, north/south by X)
auto sort_west_east = [ &Lod ]( std::uint16_t a, std::uint16_t b ) {
return Lod.vertices[ a ].z < Lod.vertices[ b ].z;
};
auto sort_north_south = [ &Lod ]( std::uint16_t a, std::uint16_t b ) {
return Lod.vertices[ a ].x < Lod.vertices[ b ].x;
};
std::sort( Lod.west_edge.begin(), Lod.west_edge.end(), sort_west_east );
std::sort( Lod.east_edge.begin(), Lod.east_edge.end(), sort_west_east );
std::sort( Lod.north_edge.begin(), Lod.north_edge.end(), sort_north_south );
std::sort( Lod.south_edge.begin(), Lod.south_edge.end(), sort_north_south );
}
std::uint8_t tin_cooker::get_material_index( tin_tile &Tile, std::string const &Material ) {
auto it = Tile.material_lookup.find( Material );
if( it != Tile.material_lookup.end() ) {
return it->second;
}
std::uint8_t const index = static_cast<std::uint8_t>( Tile.materials.size() );
Tile.materials.push_back( Material );
Tile.material_lookup[ Material ] = index;
return index;
}
bool tin_cooker::write_tile(
tin_tile const &Tile,
std::uint32_t LodIndex,
std::string const &Path,
bool Compress
) {
if( LodIndex >= Tile.lods.size() ) {
return false;
}
auto const &lod = Tile.lods[ LodIndex ];
// Calculate bounding box centre
double const cx = ( Tile.minx + Tile.maxx ) * 0.5;
double const cy = ( Tile.miny + Tile.maxy ) * 0.5;
double const cz = ( Tile.minz + Tile.maxz ) * 0.5;
// Build header
quantizedmesh::file_header header{};
std::memcpy( header.magic, quantizedmesh::magic, sizeof( header.magic ) );
header.version = quantizedmesh::version;
header.center_x = cx;
header.center_y = cy;
header.center_z = cz;
header.min_x = Tile.minx;
header.min_y = Tile.miny;
header.min_z = Tile.minz;
header.max_x = Tile.maxx;
header.max_y = Tile.maxy;
header.max_z = Tile.maxz;
header.horizont_error = lod.geometric_error;
header.vertex_count = static_cast<std::uint32_t>( lod.vertices.size() );
header.triangle_count = static_cast<std::uint32_t>( lod.indices.size() / 3 );
header.index_size = lod.vertices.size() > 65535 ? 4 : 2;
header.edge_count_north = static_cast<std::uint16_t>( lod.north_edge.size() );
header.edge_count_south = static_cast<std::uint16_t>( lod.south_edge.size() );
header.edge_count_west = static_cast<std::uint16_t>( lod.west_edge.size() );
header.edge_count_east = static_cast<std::uint16_t>( lod.east_edge.size() );
// Calculate offsets
std::uint32_t offset = sizeof( header );
header.vertex_data_offset = offset;
offset += header.vertex_count * 3 * sizeof( std::uint16_t );
header.uv_data_offset = offset;
offset += header.vertex_count * 2 * sizeof( std::uint16_t );
header.index_data_offset = offset;
offset += header.triangle_count * 3 * header.index_size;
header.edge_indices_offset = offset;
std::uint32_t const edge_data_size =
( header.edge_count_north + header.edge_count_south +
header.edge_count_west + header.edge_count_east ) * sizeof( std::uint16_t );
offset += edge_data_size;
header.extension_offset = offset;
// Write file
std::ofstream file( Path, std::ios::binary );
if( !file.good() ) {
return false;
}
file.write( reinterpret_cast<char const *>( &header ), sizeof( header ) );
// Write quantized vertex positions
for( auto const &v : lod.vertices ) {
std::uint16_t const qx = quantizedmesh::quantize( v.x, Tile.minx, Tile.maxx );
std::uint16_t const qy = quantizedmesh::quantize( v.y, Tile.miny, Tile.maxy );
std::uint16_t const qz = quantizedmesh::quantize( v.z, Tile.minz, Tile.maxz );
file.write( reinterpret_cast<char const *>( &qx ), sizeof( qx ) );
file.write( reinterpret_cast<char const *>( &qy ), sizeof( qy ) );
file.write( reinterpret_cast<char const *>( &qz ), sizeof( qz ) );
}
// Write UV coordinates (from texture coords in vertex)
for( auto const &v : lod.vertices ) {
std::uint16_t const u = quantizedmesh::quantize( v.u, 0.0, 1.0 );
std::uint16_t const uv = quantizedmesh::quantize( v.v, 0.0, 1.0 );
file.write( reinterpret_cast<char const *>( &u ), sizeof( u ) );
file.write( reinterpret_cast<char const *>( &uv ), sizeof( uv ) );
}
// Write indices
for( auto const idx : lod.indices ) {
if( header.index_size == 2 ) {
std::uint16_t const idx16 = static_cast<std::uint16_t>( idx );
file.write( reinterpret_cast<char const *>( &idx16 ), sizeof( idx16 ) );
} else {
file.write( reinterpret_cast<char const *>( &idx ), sizeof( idx ) );
}
}
// Write edge indices
for( auto const idx : lod.north_edge ) {
file.write( reinterpret_cast<char const *>( &idx ), sizeof( idx ) );
}
for( auto const idx : lod.south_edge ) {
file.write( reinterpret_cast<char const *>( &idx ), sizeof( idx ) );
}
for( auto const idx : lod.west_edge ) {
file.write( reinterpret_cast<char const *>( &idx ), sizeof( idx ) );
}
for( auto const idx : lod.east_edge ) {
file.write( reinterpret_cast<char const *>( &idx ), sizeof( idx ) );
}
file.close();
return file.good();
}
} // namespace terrain

View File

@@ -1,194 +0,0 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#pragma once
// Triangulates terrain into TIN with multiple LODs using Constrained Delaunay.
// Input: triangle soup from scenery (same as heightfield cooker)
// Output: quantized mesh tiles with proper edge constraints for seamless LOD
#include <algorithm>
#include <cmath>
#include <cstdint>
#include <limits>
#include <memory>
#include <string>
#include <string_view>
#include <unordered_map>
#include <vector>
#include <CDT.h>
#include "scene/quantizedmeshformat.h"
#include "scene/terraincooker.h" // For terrain::vertex definition
namespace terrain {
// Triangle with material
struct tin_triangle {
vertex v[ 3 ];
std::string material;
};
// Constraint edge (e.g. track corridor boundary)
struct tin_constraint {
vertex a, b;
};
// Single LOD level of a TIN tile
struct tin_lod {
std::vector<vertex> vertices;
std::vector<std::uint32_t> indices;
std::vector<std::uint8_t> materials; // Per-vertex material indices
// Edge vertices for seamless stitching (indices into vertices array)
std::vector<std::uint16_t> north_edge;
std::vector<std::uint16_t> south_edge;
std::vector<std::uint16_t> west_edge;
std::vector<std::uint16_t> east_edge;
double geometric_error; // Maximum error vs. finer LOD
};
// Complete TIN tile with all LODs
struct tin_tile {
std::int32_t x, z; // Tile coordinates
std::vector<tin_lod> lods; // Finest to coarsest
// Bounding box
double minx, miny, minz;
double maxx, maxy, maxz;
// Material table (shared across LODs)
std::vector<std::string> materials;
std::unordered_map<std::string, std::uint8_t> material_lookup;
};
class tin_cooker {
public:
tin_cooker() = default;
// Configure tile size and LOD parameters
void configure(
double TileSize, // Metres per tile side
std::uint32_t LodLevels = 4, // Number of LOD levels
double ErrorMultiplier = 2.0 // Geometric error growth per LOD
);
// Accumulate terrain triangles (same interface as heightfield cooker)
void rasterize(
vertex const &A,
vertex const &B,
vertex const &C,
std::string_view Material
);
// Add constraint edges (track corridors, roads, water boundaries)
void add_constraint( vertex const &A, vertex const &B );
// Generate all tiles with LODs
std::vector<tin_tile> finish();
// Export single tile to Quantized Mesh format
static bool write_tile(
tin_tile const &Tile,
std::uint32_t LodIndex,
std::string const &Path,
bool Compress = true
);
private:
struct tile_accumulator {
std::vector<tin_triangle> triangles;
std::vector<tin_constraint> constraints;
bool dirty { false };
};
// Build single LOD level using CDT
tin_lod triangulate_lod(
std::vector<vertex> const &Points,
std::vector<tin_constraint> const &Constraints,
double TargetError
);
// Simplify LOD by removing vertices with low geometric error
tin_lod simplify_lod(
tin_lod const &FineLod,
double TargetError
);
// Calculate geometric error of removing a vertex
double calculate_error(
vertex const &V,
std::vector<vertex> const &Neighbours
);
// Extract edge vertices for stitching
void extract_edges( tin_lod &Lod, double TileMinX, double TileMinZ, double TileMaxX, double TileMaxZ );
// Find material index, adding to table if new
std::uint8_t get_material_index( tin_tile &Tile, std::string const &Material );
double m_tilesize { 256.0 };
std::uint32_t m_lodlevels { 4 };
double m_errormultiplier { 2.0 };
std::unordered_map<std::int64_t, tile_accumulator> m_tiles;
// Statistics
std::size_t m_triangles_in { 0 };
std::size_t m_triangles_out { 0 };
std::size_t m_vertices_out { 0 };
};
// Helper: convert CDT point to our vertex
inline vertex from_cdt_point( CDT::V2d<double> const &P, double Y = 0.0 ) {
return vertex{ P.x, Y, P.y };
}
// Helper: convert vertex to CDT point
inline CDT::V2d<double> to_cdt_point( vertex const &V ) {
return CDT::V2d<double>::make( V.x, V.z );
}
// Helper: calculate distance from point to line segment
inline double point_to_segment_distance(
vertex const &Point,
vertex const &SegmentA,
vertex const &SegmentB
) {
double const dx = SegmentB.x - SegmentA.x;
double const dz = SegmentB.z - SegmentA.z;
double const len_sq = dx * dx + dz * dz;
if( len_sq < 1e-10 ) {
// Degenerate segment
double const pdx = Point.x - SegmentA.x;
double const pdz = Point.z - SegmentA.z;
return std::sqrt( pdx * pdx + pdz * pdz );
}
// Project point onto line
double const t = std::clamp(
( ( Point.x - SegmentA.x ) * dx + ( Point.z - SegmentA.z ) * dz ) / len_sq,
0.0, 1.0
);
double const proj_x = SegmentA.x + t * dx;
double const proj_z = SegmentA.z + t * dz;
double const pdx = Point.x - proj_x;
double const pdy = Point.y - ( SegmentA.y + t * ( SegmentB.y - SegmentA.y ) );
double const pdz = Point.z - proj_z;
return std::sqrt( pdx * pdx + pdy * pdy + pdz * pdz );
}
} // namespace terrain