17
0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-10-17 13:19:20 +02:00
Files
maszyna/utilities/utilities.cppm
maj00r 2660d14565 Stream the scenery: a baked terrain heightfield, a logic container, and prefabs
Today's work on the binary scene format plan (binary-scene-format-plan.md), from the
first measurement to a terrain that bakes itself on first run and streams from a
thread. The step-by-step history is kept on backup/scene-streaming-format-presquash-0916.

Measuring first. -loadprofile times a scenery load by stage and by node kind and reports
node counts and resident memory. On l204 terrain is 95% of the bytes and 99.8% of the
nodes; on quarkmce2007 vehicles, not models, dominate what is left. The terrain turned
out to be a heightfield in all but name, a ribbon covering 12% of its bounding box.

The heightfield. scene/heightfieldformat.h is the .ehf layout: Morton-ordered tiles
of 128 samples with one shared edge sample, a point-sampled mip chain, zstd per tile,
a nodata sentinel, a material palette with a measured tiling per material, and a stamp
of the rules that decided what counted as terrain. scene/heightfieldreader.h reads it
for the tool, the renderer and the loader alike, and scene/terraincooker.h writes it,
shared by the terraincook tool and the engine so a first-run bake is the file the tool
would have made. The cooker rasterises every triangle where it is drawn (through
scene::place_in_world, the placement the region applies too), keeps an upper surface
laid within half a metre over another and the lower of two further apart, and then
asks each triangle whether the heightfield shows it: laid, reaching a sample, keeping
nearly all it claimed, and with the ground around it drawn at every level out to 2 km.

Baking on first run. simulation/terrainbake bakes <scenario>.ehf the first time a
scenery is opened, choosing the grid step from the first 200,000 vertices, and on later
loads leaves out the triangles its node table says the heightfield shows. Everything
else - trees, grass tufts, gantries, bridge decks - is drawn as written, so a bad bake
can at worst draw something twice, and the heightfield sits 2 cm under the triangles so
the original wins there. Nodes are keyed by file, line, include parameters and
placement; an edited file changes its keys and those nodes are drawn again. A bake under
older rules, or without its table, is thrown away and redone. A launcher copy named
$scenario shares the scenario's files. -nobaketerrain shows the triangles as written.
A scenery can still name hand-cooked fields, several at different steps.

Drawing and streaming. rendering/terrainclipmap draws resident tiles at the mip level
their distance earns, deriving the grid from gl_VertexID and texturing in world space
with the scenery's own materials. rendering/terraintileloader reads and decodes tiles on
a thread of its own; the renderer rescans what is in range only after moving a quarter
of a tile, asks for what is missing nearest first, uploads at most 32 tiles a frame,
drops arrivals the camera has moved past, and keeps drawing a tile at the level it has
until the right one arrives. The terrain has its own range, terrainrange in the ini and
a slider in the debug panel, 20 km by default and independent of basedrawrange; the far
plane of an ordinary view is pushed out to it. The debug panel shows the range, tiles in
view, on the gpu, wanted and queued, and how many sit at each level.

Events. The event queue moved out of the events into a binary heap with a deterministic
tie-break, and an event is handed its activator instead of storing it, so basic_event
is a definition and nothing else.

The logic container. scene/logicformat.h and scene/logiccontainer write a scenery's
events, their targets and sibling chains as flat, deterministic arrays with a schema
hash that refuses a mismatched build. simulation/logiccook cooks it from the loaded
state whenever it is missing, stale or unreadable, reads it straight back to check it,
and resolves each target to a position in the registry its event kind searches.
-cooklogic forces a cook, -verifylogic only checks. Nothing reads it at load yet.

Prefabs. scene/prefab describes a signal by the aspects it can show - lights, speeds,
animation channels, a sound - instead of pasting an .inc per variant, and expands an
instance to ordinary scenery text. Tried on a light signal and a shape signal.

Screenshots without a person present: -shot x,y,z[,yaw[,pitch]], -shotsettle, -shotstay.
2026-09-16 23:18:09 +02:00

467 lines
13 KiB
C++

/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
/*rozne takie duperele do operacji na stringach w paszczalu, pewnie w delfi sa lepsze*/
/*konwersja zmiennych na stringi, funkcje matematyczne, logiczne, lancuchowe, I/O etc*/
module;
#include <array>
#include <deque>
#include <ostream>
#include <unordered_map>
#include <cstddef>
#include <limits>
#include <sstream>
#include <string>
#include <type_traits>
#include <utility>
#include <vector>
#include "utilities/utilities_macros.h"
#include <algorithm>
#include <chrono>
#include <cmath>
#include <condition_variable>
#include <filesystem>
#include <mutex>
export module eu07.utilities.utilities;
import eu07.glm;
import eu07.utilities.parser;
export {
template <typename T> T sign(T x)
{
return x < static_cast<T>(0) ? static_cast<T>(-1) : x > static_cast<T>(0) ? static_cast<T>(1) : static_cast<T>(0);
}
template <typename T> constexpr T sq(T v)
{
return v * v;
}
// returns the per-user config path for `filename` (platform-specific dir),
// or an empty path if the home/appdata env var is unavailable
std::filesystem::path user_config_path(const std::string &filename);
// TODO: Shouldn't this be in globals?
namespace paths
{
inline constexpr const char *scenery = "scenery/";
inline constexpr const char *textures = "textures/";
inline constexpr const char *models = "models/";
inline constexpr const char *dynamic = "dynamic/";
inline constexpr const char *sounds = "sounds/";
inline constexpr const char *data = "data/";
}
extern bool DebugModeFlag;
extern bool FreeFlyModeFlag;
extern bool EditorModeFlag;
extern bool DebugCameraFlag;
extern bool DebugTractionFlag;
/*funkcje matematyczne*/
inline double Sign(double x)
{
return x >= 0 ? 1.0 : -1.0;
}
inline long Round(double const f)
{
return (long)(f + 0.5);
// return lround(f);
}
double Random(double a, double b);
int Random(int min, int max);
std::string generate_uuid_v4();
double LocalRandom(double a, double b);
inline double Random()
{
return Random(0.0, 1.0);
}
inline double Random(double b)
{
return Random(0.0, b);
}
inline double LocalRandom()
{
return LocalRandom(0.0, 1.0);
}
inline double LocalRandom(double b)
{
return LocalRandom(0.0, b);
}
inline double BorlandTime()
{
auto timesinceepoch = std::time(nullptr);
return timesinceepoch / (24.0 * 60 * 60);
/*
// std alternative
auto timesinceepoch = std::chrono::system_clock::now().time_since_epoch();
return std::chrono::duration_cast<std::chrono::seconds>( timesinceepoch ).count() / (24.0 * 60 * 60);
*/
}
std::string Now();
double CompareTime(double t1h, double t1m, double t2h, double t2m);
/*funkcje logiczne*/
inline bool TestFlag(int const Flag, int const Value)
{
return (Flag & Value) == Value;
}
inline bool TestFlagAny(int const Flag, int const Value)
{
return (Flag & Value) != 0;
}
bool SetFlag(int &Flag, int const Value);
bool ClearFlag(int &Flag, int const Value);
bool FuzzyLogic(double Test, double Threshold, double Probability);
/*jesli Test>Threshold to losowanie*/
bool FuzzyLogicAI(double Test, double Threshold, double Probability);
/*to samo ale zawsze niezaleznie od DebugFlag*/
/*operacje na stringach*/
std::vector<std::string> Split(std::string_view s, char delim);
std::pair<std::string, int> split_string_and_number(std::string const &Key);
std::string to_string(int Value, int width);
std::string to_string(double Value, int precision);
std::string to_string(double Value, int precision, int width);
std::string to_hex_str(int const Value, int const width = 4);
std::string to_minutes_str(float const Minutes, bool const Leadingzero, int const Width);
template <std::same_as<bool> T> // Without this line this function can be used with other types implicit casted to boolean which may create hard to debug bugs.
inline std::string to_string(T Value)
{
return Value == true ? "true" : "false";
}
template <typename Type_, glm::precision Precision_ = glm::defaultp> std::string to_string(glm::tvec3<Type_, Precision_> const &Value)
{
return to_string(Value.x, 2) + ", " + to_string(Value.y, 2) + ", " + to_string(Value.z, 2);
}
template <typename Type_, glm::precision Precision_ = glm::defaultp> std::string to_string(glm::tvec4<Type_, Precision_> const &Value, int const Width = 2)
{
return to_string(Value.x, Width) + ", " + to_string(Value.y, Width) + ", " + to_string(Value.z, Width) + ", " + to_string(Value.w, Width);
}
int stol_def(const std::string &str, const int &DefaultValue);
std::string ToLower(std::string const &text);
std::string ToUpper(std::string const &text);
// replaces polish letters with basic ascii
void win1250_to_ascii(std::string &Input);
// TODO: unify with win1250_to_ascii()
std::string Bezogonkow(std::string Input, bool const Underscorestospaces = false);
std::string win1250_to_utf8(const std::string &input);
inline std::string extract_value(std::string const &Key, std::string const &Input)
{
// NOTE, HACK: the leading space allows to uniformly look for " variable=" substring
std::string const input{" " + Input};
std::string value;
auto lookup = input.find(" " + Key + "=");
if (lookup != std::string::npos)
{
value = input.substr(input.find_first_not_of(' ', lookup + Key.size() + 2));
lookup = value.find(' ');
if (lookup != std::string::npos)
{
// trim everything past the value
value.erase(lookup);
}
}
return value;
}
template <typename Type_> bool extract_value(Type_ &Variable, std::string const &Key, std::string const &Input, std::string const &Default)
{
auto value = extract_value(Key, Input);
if (false == value.empty())
{
// set the specified variable to retrieved value
std::stringstream converter;
converter << value;
converter >> Variable;
return true; // located the variable
}
else
{
// set the variable to provided default value
if (false == Default.empty())
{
std::stringstream converter;
converter << Default;
converter >> Variable;
}
return false; // couldn't locate the variable in provided input
}
}
template <> bool extract_value(bool &Variable, std::string const &Key, std::string const &Input, std::string const &Default);
bool FileExists(std::string const &Filename);
std::pair<std::string, std::string> FileExists(std::vector<std::string> const &Names, std::vector<std::string> const &Extensions);
// returns time of last modification for specified file
std::time_t last_modified(std::string const &Filename);
// potentially erases file extension from provided file name. returns: true if extension was removed, false otherwise
bool erase_extension(std::string &Filename);
// where a scenery file lives: a bare name is looked up in the scenery directory
std::string scenery_file(std::string const &Sceneryfile);
// a file cooked for a scenery and kept beside it, with the given extension in place of the
// scenery's. named after the scenario rather than the $-prefixed copy of it the launcher
// writes on every start, so the launcher and a direct start share one
std::string scenery_sidecar(std::string const &Sceneryfile, std::string const &Extension);
// potentially erase leading slashes from provided file path
void erase_leading_slashes(std::string &Filename);
// potentially replaces backward slashes in provided file path with unix-compatible forward slashes
void replace_slashes(std::string &Filename);
// returns potential path part from provided file name
std::string substr_path(std::string const &Filename);
// returns common prefix of two provided strings
std::ptrdiff_t len_common_prefix(std::string_view a, std::string_view b);
// returns true if provided string contains another provided string
bool contains(std::string_view const String, std::string_view Substring);
bool contains(std::string_view const String, char Character);
// TODO: Ideally unique_ptr should be used instead of this (not safe) inline functions
template <typename T> inline void SafeDelete(T* &Pointer)
{
delete Pointer;
Pointer = nullptr;
}
template <typename T> inline void SafeDeleteArray(T *&Pointer)
{
delete[] Pointer;
Pointer = nullptr;
}
template <typename T> bool is_equal(T const &Left, T const &Right, T const Epsilon = T(1e-5))
{
if (Epsilon != T(0))
return glm::epsilonEqual(Left, Right, Epsilon);
return Left == Right;
}
// Tolerant clamp. Unlike std::clamp, this does NOT invoke undefined behaviour when
// the bounds are inverted (Lo > Hi) - it normalizes them instead. This restores the
// pre-refactor behaviour of the old custom clamp() for code paths where the bounds are
// computed at runtime and can legitimately cross (e.g. AI acceleration/braking limits,
// physics jolt calculations), where std::clamp's UB produced wrong results.
template <typename Type_>
constexpr Type_ safe_clamp( Type_ const Value, Type_ const Lo, Type_ const Hi )
{
return ( Hi < Lo )
? std::clamp( Value, Hi, Lo )
: std::clamp( Value, Lo, Hi );
}
// keeps the provided value in specified range 0-Range, as if the range was circular buffer
template <typename T> T clamp_circular(T Value, T const Range = T(360))
{
if constexpr (std::is_floating_point_v<T>)
{
Value = std::fmod(Value, Range);
}
else
{
Value %= Range;
}
if (Value < T(0))
Value += Range;
return Value;
}
// rounds down provided value to nearest power of two
template <typename T> T clamp_power_of_two(T Value, T const Min = T(1), T const Max = T(16384))
{
if (Value < Min)
return Min;
T p2 = std::bit_floor(Value);
if (p2 > Max)
return Max;
return p2;
}
template <typename Type_> Type_ quantize(Type_ const Value, Type_ const Step)
{
return Step * std::round(Value / Step);
}
template <typename T> T min_speed(T const Left, T const Right)
{
constexpr T none = T(-1);
if (Left == none)
return Right;
if (Right == none)
return Left;
return std::min(Left, Right);
}
template <typename T> T smoothInterpolate(T const &First, T const &Second, double Factor)
{
// Apply smoothing (ease-in-out quadratic)
Factor = Factor * Factor * (3 - 2 * Factor);
return First + (Second - First) * Factor;
}
// tests whether provided points form a degenerate triangle
template <typename VecType_> bool degenerate(VecType_ const &Vertex1, VecType_ const &Vertex2, VecType_ const &Vertex3)
{
// degenerate( A, B, C, minarea ) = ( ( B - A ).cross( C - A ) ).lengthSquared() < ( 4.0f * minarea * minarea );
return glm::length2(glm::cross(Vertex2 - Vertex1, Vertex3 - Vertex1)) == 0.0;
}
// calculates bounding box for provided set of points
template <class Iterator_, class VecType_> void bounding_box(VecType_ &Mincorner, VecType_ &Maxcorner, Iterator_ First, Iterator_ Last)
{
Mincorner = VecType_(std::numeric_limits<typename VecType_::value_type>::max());
Maxcorner = VecType_(std::numeric_limits<typename VecType_::value_type>::lowest());
std::for_each(First, Last,
[&](typename Iterator_::value_type &point)
{
Mincorner = glm::min(Mincorner, VecType_{point});
Maxcorner = glm::max(Maxcorner, VecType_{point});
});
}
// finds point on specified segment closest to specified point in 3d space. returns: point on segment as value in range 0-1 where 0 = start and 1 = end of the segment
template <typename VecType_> typename VecType_::value_type nearest_segment_point(VecType_ const &Segmentstart, VecType_ const &Segmentend, VecType_ const &Point)
{
auto const v = Segmentend - Segmentstart;
auto const w = Point - Segmentstart;
auto const c1 = glm::dot(w, v);
if (c1 <= 0.0)
{
return 0.0;
}
auto const c2 = glm::dot(v, v);
if (c2 <= c1)
{
return 1.0;
}
return c1 / c2;
}
glm::dvec3 LoadPoint(cParser &Input);
// extracts a group of tokens from provided data stream
std::string deserialize_random_set(cParser &Input, char const *Break = "\n\r\t ;");
// extracts a group of <key, value> pairs from provided data stream
// NOTE: expects no more than single pair per line
template <typename MapType_> void deserialize_map(MapType_ &Map, cParser &Input)
{
while (Input.ok() && !Input.eof())
{
auto const key{Input.getToken<typename MapType_::key_type>(false)};
auto const value{Input.getToken<typename MapType_::mapped_type>(false, "\n")};
Map.emplace(key, value);
}
}
namespace threading
{
// simple POD pairing of a data item and a mutex
// NOTE: doesn't do any locking itself, it's merely for cleaner argument arrangement and passing
template <typename Type_> struct lockable
{
Type_ data;
std::mutex mutex;
};
// basic wrapper simplifying use of std::condition_variable for most typical cases.
// has its own mutex and secondary variable to ignore spurious wakeups
class condition_variable
{
public:
// methods
void wait()
{
std::unique_lock<std::mutex> lock(m_mutex);
m_condition.wait(lock, [this]() { return m_spurious == false; });
}
template <class Rep_, class Period_> void wait_for(const std::chrono::duration<Rep_, Period_> &Time)
{
std::unique_lock<std::mutex> lock(m_mutex);
m_condition.wait_for(lock, Time, [this]() { return m_spurious == false; });
}
void notify_one()
{
spurious(false);
m_condition.notify_one();
}
void notify_all()
{
spurious(false);
m_condition.notify_all();
}
void spurious(bool const Spurious)
{
std::lock_guard<std::mutex> lock(m_mutex);
m_spurious = Spurious;
}
private:
// members
mutable std::mutex m_mutex;
std::condition_variable m_condition;
bool m_spurious{true};
};
} // namespace threading
//---------------------------------------------------------------------------
} // export