Draw terrain one call per level, from texture arrays, culled to the view

Each resident tile had a height texture and a material texture of its own, uploaded with
glTexImage2D, and was drawn once per material it held with every other material's
fragments discarded, all of them whether in view or not. Thousands of draws a frame.

Now the tiles of a level live in one pair of texture arrays (terrain_tile_pool), a layer
per tile. A pool starts at sixteen layers and doubles when full, copying what it holds on
the gpu, so a level takes only the memory its tiles need; uploads go through a pixel
buffer orphaned for each tile. The ground textures of every material are gathered into
one sRGB array with mipmaps (terrain_ground) - drawn into it layer by layer by a small
shader once the engine has each texture ready, since the scenery's textures are
compressed and cannot be blitted - with a table of placeholder colours and measured
tiling beside it.

Each frame the tiles in range are tested against the frustum by bounding sphere and
gathered per level into a buffer texture of instance data, and each level is drawn in
one instanced call. The fragment shader reads the four samples around it, drops the cell
when any of them carries no ground - exactly the rule the bake already assumed - and
blends the ground textures of their materials by distance, so material boundaries
follow the samples smoothly instead of stepping along triangle edges.

State the copy pass changes is captured and put back, and the texture unit cache is
reset after raw binds, so the rest of the renderer finds things as it left them.

On baltyk: all 30 materials land in the array, 663 of the 1120 tiles in range are drawn
after culling, and a frame is at most six terrain draws. The residency line in the log
now says how many tiles in range were drawn.

rendering/terrainclipmap.cppm

@@ -10,13 +10,13 @@ http://mozilla.org/MPL/2.0/.
 module;
 #include <array>
 #include <cstdint>
+#include <functional>
 #include <memory>
 #include <optional>
 #include <string>
 #include <unordered_map>
 #include <vector>
 
-#include "global_include/interfaces/ITexture_macros.h"
 #include "scene/heightfieldreader.h"
 
 export module eu07.rendering.terrainclipmap;
@@ -26,26 +26,34 @@ import eu07.gl.vao;
 import eu07.glm;
 import eu07.rendering.terraintileloader;
 import eu07.rendering.terrainstatus;
-import eu07.model.material;
-import eu07.global_include.interfaces.itexture;
+import eu07.rendering.terraintilepool;
+import eu07.rendering.terrainground;
 
 export {
 
 // Draws the cooked terrain heightfield.
 //
-// Not a toroidal clipmap, despite what the plan called it. The measurements say the
-// terrain is a ribbon: 390 km2 of surface inside a 51 x 46 km rectangle, twelve percent
-// of the tile bounding box occupied. Clipmap rings sized for that rectangle would be
-// mostly empty, so this draws the cooked tiles themselves, each at the mip level its
-// distance earns. The tiles were laid out for exactly this - sorted by morton code, one
-// shared sample of overlap so neighbouring levels meet without skirts.
+// Not a toroidal clipmap, despite what the plan called it. The measurements say the terrain is
+// a ribbon: 390 km2 of surface inside a 51 x 46 km rectangle, twelve percent of the tile
+// bounding box occupied. Clipmap rings sized for that rectangle would be mostly empty, so this
+// draws the cooked tiles themselves, each at the mip level its distance earns (CDLOD), morphing
+// into the next level as it nears the end of its range.
 //
-// The class owns its gl resources and the file handle, and knows nothing about the
-// scenery, the camera class or the rest of the renderer: it is given a viewpoint and a
-// pair of matrices, and draws.
+// Tiles live on the gpu in one texture array per level (terrain_tile_pool), and the ground
+// textures of every material in another (terrain_ground). Each frame the tiles in view are
+// culled against the frustum and gathered per level into a buffer of instance data, and each
+// level is drawn in one instanced call; the fragment shader picks and blends the materials of
+// the four samples around it.
+//
+// The class owns its gl resources and the file handle, and knows nothing about the scenery,
+// the camera class or the rest of the renderer: it is given a viewpoint and a visibility test,
+// and draws.
 class terrain_clipmap {
 
 public:
+    // whether a sphere, in world space, can be seen: centre and radius
+    using visibility = std::function<bool( glm::dvec3 const &, float )>;
+
     terrain_clipmap() = default;
     ~terrain_clipmap();
     terrain_clipmap( terrain_clipmap const & ) = delete;
@@ -62,11 +70,11 @@ public:
     // and decoding happen on the loader's thread, and a tile waiting for a finer level keeps
     // being drawn at the level it has
     void update( glm::dvec3 const &Viewpoint );
-    // draws the resident tiles. the caller is expected to have the scene and model
-    // uniform blocks already set for a camera-relative pass, the way the rest of the
-    // renderer does: tile positions reach the shader relative to the viewpoint, so
-    // nothing here needs a matrix of its own
-    void render( glm::dvec3 const &Viewpoint );
+    // draws the resident tiles in range that the test says can be seen. the caller is
+    // expected to have the scene and model uniform blocks already set for a camera-relative
+    // pass, the way the rest of the renderer does: tile positions reach the shader relative
+    // to the viewpoint, so nothing here needs a matrix of its own
+    void render( glm::dvec3 const &Viewpoint, visibility const &Visible );
 
     // how far from the viewpoint terrain is drawn, in metres
     void range( float const Range ) { m_range = Range; }
@@ -91,19 +99,19 @@ public:
     std::size_t backlog() const { return m_loader.backlog(); }
 
 private:
-    // one tile resident on the gpu. heights are unsigned so that the no-data value stays
-    // exactly what the cooker wrote instead of being normalised into a float
+    // one tile resident on the gpu: a layer in its level's pool
     struct resident_tile {
         std::int32_t x { 0 }, z { 0 };
-        std::uint32_t level { 0 };      // mip level currently uploaded
-        std::uint32_t side { 0 };       // samples per side at that level
-        std::uint32_t heighttexture { 0 };
-        std::uint32_t materialtexture { 0 };
+        std::uint32_t level { 0 };
+        std::uint32_t slot { 0 };
         float minheight { 0.f }, maxheight { 0.f };
         std::uint64_t lastused { 0 };
-        // materials appearing in this tile. a tile is drawn once per entry, since each
-        // needs its own texture bound, and most tiles carry only one or two
-        std::vector<std::uint8_t> materials;
+    };
+
+    // one tile within range, and the level its distance earns
+    struct in_range {
+        std::int64_t key;
+        std::uint32_t level;
     };
 
     // mip level a tile at the given distance is drawn at
@@ -113,11 +121,8 @@ private:
     void scan( glm::dvec3 const &Viewpoint );
     // puts a tile the loader has read on the gpu, replacing whatever level it had before
     void upload( terrain_tile_loader::payload const &Tile );
-    void retire( resident_tile &Tile );
+    void retire( resident_tile const &Tile );
     void build_indices();
-    void build_palette();
-    // resolves the cooked material names to the engine's materials, once
-    void resolve_materials();
     void resolve_uniforms();
     void report_residency();
 
@@ -129,11 +134,6 @@ private:
     terrain_tile_loader m_loader;
     std::vector<terrain_tile_loader::request> m_wanted;
     std::vector<terrain_tile_loader::payload> m_arrived;
-    // one tile within range, and the level its distance earns
-    struct in_range {
-        std::int64_t key;
-        std::uint32_t level;
-    };
     // what the last scan found in range, and the level wanted for each tile, to tell a useful
     // arrival from one the camera has already moved past
     std::vector<in_range> m_inrange;
@@ -145,29 +145,28 @@ private:
     double m_finestwanted { 0.0 };
     double m_finest { 0.0 };
     std::unordered_map<std::int64_t, resident_tile> m_tiles;
-    // one index buffer per mip level, over a grid of that level's sample count. vertices
-    // are computed in the shader from gl_VertexID, so there is no vertex buffer at all.
-    // held by pointer because a gl::buffer owns a name it cannot hand to a copy
+
+    // per level: the tiles' storage, the index buffer over that level's grid, and the
+    // instances gathered for the draw. vertices are computed in the shader from gl_VertexID,
+    // so there is no vertex buffer at all. held by pointer because the gl wrappers own names
+    // they cannot hand to a copy
+    std::vector<std::unique_ptr<terrain_tile_pool>> m_pools;
     std::vector<std::unique_ptr<gl::buffer>> m_indices;
     std::vector<std::uint32_t> m_indexcounts;
+    std::vector<std::vector<float>> m_instances;
+    // instance data for the draw being made, read by the vertex shader as a buffer texture
+    std::optional<gl::buffer> m_instancebuffer;
+    std::uint32_t m_instancetexture { 0 };
+    terrain_ground m_ground;
     std::optional<gl::vao> m_vao;
     std::optional<gl::program> m_shader;
     // resolved once at link time; -1 for a uniform the compiler dropped, which gl ignores
     struct uniform_locations {
-        int tileorigin { -1 }, samplestep { -1 };
-        int heightbias { -1 }, heightscale { -1 }, nodata { -1 }, side { -1 }, morph { -1 };
-        int heights { -1 }, materials { -1 }, palette { -1 };
-        int groundtexture { -1 }, groundscale { -1 }, drawnmaterial { -1 }, tileworld { -1 };
-        int hastexture { -1 }, fallback { -1 };
+        int samplestep { -1 }, side { -1 }, morph { -1 };
+        int heightbias { -1 }, heightscale { -1 }, nodata { -1 };
+        int heights { -1 }, materials { -1 }, instances { -1 };
+        int ground { -1 }, materialtable { -1 };
     } m_uniforms;
-    std::vector<glm::vec3> m_palette;
-    // one entry per cooked material: the engine texture to bind and how much ground one
-    // repeat of it covers
-    struct material_binding {
-        texture_handle texture { null_handle };
-        float scale { 4.f };
-    };
-    std::vector<material_binding> m_materials;
 
     terrain_statistics m_stats;
     std::uint64_t m_frame { 0 };