Cull the instances of the shadow maps of headlights and lamps on the GPU

docs/opengl46-renderer.md

@@ -265,8 +265,9 @@ Many models of the scenery (trees, bushes, poles, fences: models without lights
 batch per model and level of detail. Which of them are drawn, and at which level of detail, can be worked out on the
 processor (as before) or on the graphics card. On the graphics card the processor no longer goes through the models one by
 one, so with dense vegetation it is much less loaded. The main view, the three cascades of the sun shadows and the faces of
-the environment map (reflections) are drawn this way, and so are the sleepers of the tracks with a `sleepermodel` in the main
+the environment map (reflections), the shadow maps of headlights and the scenery of the shadow maps of scenery lamps are drawn
-view; the shadow maps of lamps and headlights, the cab and other views go as before.
+this way, and so are the sleepers of the tracks with a `sleepermodel` in the main view; the shadow map of the cab lamp, the
+cab and other views go as before.
 
 | Entry | Default | Meaning |
 |---|---|---|
@@ -277,6 +278,8 @@ view; the shadow maps of lamps and headlights, the cab and other views go as bef
 | `gfx.gpucull.multidraw yes/no` | `yes` | with `gfx.gpucull yes`: parts of models with the same material (and texture variant) in the same model are drawn with one call (multi draw), see below |
 | `gfx.gpucull.shadows yes/no` | `yes` | with `gfx.gpucull yes` or `compare`: the models of the cascades of the sun shadows too (each cascade with its own camera and the distances from the real camera, as before; models none of whose materials cast shadows are left out at once) |
 | `gfx.gpucull.reflections yes/no` | `yes` | with `gfx.gpucull yes` or `compare`: the models of the faces of the environment map too (its own range of models, as before) |
+| `gfx.gpucull.shadows.headlights yes/no` | `yes` | with `gfx.gpucull yes` or `compare`: the models of the shadow maps of headlights too (each map tested against the beam it is drawn with, to its reach, the distances from the real camera, as before) |
+| `gfx.gpucull.shadows.lamps yes/no` | `yes` | with `gfx.gpucull yes` or `compare`: the models of the scenery part of the shadow maps of scenery lamps too (each map tested against the reach of its lamp, as before); the part with the vehicles has no models of the scenery |
 | `gfx.gpucull.sleepers yes/no` | `yes` | with `gfx.gpucull yes` or `compare`: the sleepers of the tracks (`sleepermodel`) of the main view too, each one tested against the view and given its level of detail by its distance, as before (the tracks within `sleeperdistance` are chosen as before); they also take the occlusion test, the depth prepass and the two phases of the models |
 | `gfx.depthprepass.scene yes/no` | `no` | the depth of the whole opaque scene of the main view (terrain, tracks, buildings, models, vehicles, the cab) is drawn before its colour, see below |
 | `gfx.depthprepass.replay yes/no` | `yes` | the scene is gone through once for its depth and its colour: what the colour pass draws is recorded and both are drawn from the record, see below. `no`: twice, as before |
@@ -405,7 +408,11 @@ editor goes through the first `dynamiclights` headlights, the reflections throug
   differ. With `gfx.gpucull.shadows` and `gfx.gpucull.reflections` the same rows (without the occlusion test and the
   prepass) follow for each cascade, `gpu cull shadows 1:` to `3:` (1 the nearest), and for the environment map,
   `gpu cull reflections, last face:` (the last face of the map drawn, as the map is drawn now and then), with the timings
-  `instance cull shadows 1:` … and `instance cull reflections, last face:`. With `gfx.gpucull.sleepers` the row
+  `instance cull shadows 1:` … and `instance cull reflections, last face:`. With `gfx.gpucull.shadows.headlights` and
+  `gfx.gpucull.shadows.lamps` they follow for each map of headlights drawn so far, `gpu cull headlight shadows 1:` … (by the
+  slot of the map), and for the scenery of the maps of lamps, `gpu cull lamp shadows, scenery 1:` and `2:` (the first and
+  the second drawn in a frame), each with the numbers of its last drawing and the timings `instance cull headlight shadows 1:`
+  … and `instance cull lamp shadows, scenery 1:` …. With `gfx.gpucull.sleepers` the row
   `gpu cull:` is followed by how many sleepers of how many tracks were queued (they are counted among the models).
 - With `gfx.depthprepass.scene yes` Shift+F12 shows the `scene depth:` row (draws of the depth pass and its processor time)
   and the timing `scene depth:` (its GPU time). With `gfx.depthprepass.replay yes` the row tells how many of the recorded

rendering/opengl46/opengl46instancecull.h

@@ -43,9 +43,10 @@ class TAnimModel;
 // of them, and the ones drawn second, each copy command_count() commands long (draw_command_offset())
 // with Global.gfx_gpucull 2 the cpu still draws, and the result is only compared with its own: the counts per entry are
 // read back a few frames later and set against the counts Render_Instanced() reported for the same frame
-// each render pass culled on the gpu (the main view, the cascades of the sun shadows, the faces of the environment map) has a
+// each render pass culled on the gpu (the main view, the cascades of the sun shadows, the faces of the environment map, the
-// culling of its own, with its own buffers, while the records and the programs are shared by all of them: the records of a
+// shadow maps of headlights and the scenery parts of those of lamps) has a culling of its own, with its own buffers, while the
-// bucket go to the gpu once and serve every pass which queues the bucket
+// records and the programs are shared by all of them: the records of a bucket go to the gpu once and serve every pass which
+// queues the bucket
 // the sleepers of tracks (TTrack::m_sleeper_local_transforms) are culled the same way, each track with them a bucket of its
 // own, and drawn together with the instances; their groups are kept apart from those of the model instances
 class opengl46_instance_cull

rendering/opengl46/opengl46localshadows.cpp

@@ -122,6 +122,24 @@ void opengl46_local_shadows::init()
 	}
 	glProgramUniform1f(*m_program, NEAR_LOCATION, SHADOW_NEAR);
 	glProgramUniform1f(*m_alpha_program, NEAR_LOCATION, SHADOW_NEAR);
+	// the variants for the multi draws of the instances drawn from the gpu culling (the scenery parts of the maps of lamps); the
+	// fragment stages read nothing of the model block
+	try
+	{
+		gl::shader multidrawvertex("shadowcube.vert", "#define MODEL_DRAWS 1\n");
+		gl::shader fragment("shadowmap.frag");
+		gl::shader alphafragment("alphashadowmap.frag");
+		m_multidraw_program = std::make_unique<gl::program>(std::vector<std::reference_wrapper<const gl::shader>>{multidrawvertex, geometry, fragment});
+		m_alpha_multidraw_program = std::make_unique<gl::program>(std::vector<std::reference_wrapper<const gl::shader>>{multidrawvertex, geometry, alphafragment});
+		glProgramUniform1f(*m_multidraw_program, NEAR_LOCATION, SHADOW_NEAR);
+		glProgramUniform1f(*m_alpha_multidraw_program, NEAR_LOCATION, SHADOW_NEAR);
+	}
+	catch (gl::shader_exception const &e)
+	{
+		ErrorLog("multi draws of the lamp shadow maps not available: " + std::string(e.what()));
+		m_multidraw_program.reset();
+		m_alpha_multidraw_program.reset();
+	}
 	set_layer(CAB_SLOT * 6);
 
 	// a shadow sampler with no depth texture to read from is undefined, also where the shading doesn't read it
@@ -143,6 +161,10 @@ void opengl46_local_shadows::set_layer(int const Layer)
 {
 	glProgramUniform1i(*m_program, LAYER_LOCATION, Layer);
 	glProgramUniform1i(*m_alpha_program, LAYER_LOCATION, Layer);
+	if (m_multidraw_program)
+		glProgramUniform1i(*m_multidraw_program, LAYER_LOCATION, Layer);
+	if (m_alpha_multidraw_program)
+		glProgramUniform1i(*m_alpha_multidraw_program, LAYER_LOCATION, Layer);
 }
 
 opengl46_local_shadows::level_setup opengl46_local_shadows::setup(int const Level)

rendering/opengl46/opengl46localshadows.h

@@ -61,6 +61,12 @@ class opengl46_local_shadows
 	{
 		return Alpha ? *m_alpha_program : *m_program;
 	}
+	// the same for the multi draws of the instances drawn from the gpu culling, which read the model block of a draw by its
+	// number (MODEL_DRAWS). nullptr when they couldn't be made: the draws are made one by one with program() then
+	gl::program *multidraw_program(bool const Alpha)
+	{
+		return Alpha ? m_alpha_multidraw_program.get() : m_multidraw_program.get();
+	}
 
 	// the cab lamp
 
@@ -216,6 +222,8 @@ class opengl46_local_shadows
 
 	std::unique_ptr<gl::program> m_program;
 	std::unique_ptr<gl::program> m_alpha_program;
+	std::unique_ptr<gl::program> m_multidraw_program;
+	std::unique_ptr<gl::program> m_alpha_multidraw_program;
 	GLuint m_empty{0}; // stand-in: a cube map array of 1x1 faces with the farthest depth
 	GLuint m_emptyheadlights{0}; // the same, a 2d array of one 1x1 layer
 	int m_level{-1}; // level the maps were made for

rendering/opengl46/opengl46renderer.cpp

@@ -816,6 +816,7 @@ void opengl46_renderer::SwapBuffers()
     if( Global.gfx_gpucull_reflections && m_reflectioncull.ran() ) {
         m_debugtimestext += m_reflectioncull.info_times( "instance cull reflections, last face" );
     }
+    m_debugtimestext += Light_Shadow_Cull_Info( true );
     if( Global.gfx_depthprepass_scene || m_contactshadowslast ) {
         if( auto const scenedepthtime = m_scenedepthtimer.result() )
             m_debugtimestext += " scene depth: " + to_string( static_cast<double>( *scenedepthtime ) / 1000000.0, 3 ) + " ms\n";
@@ -916,6 +917,7 @@ void opengl46_renderer::SwapBuffers()
         m_reflectioncull.collect();
         m_debugstatstext += m_reflectioncull.info( "gpu cull reflections, last face", false );
     }
+    m_debugstatstext += Light_Shadow_Cull_Info( false );
     if( auto const cullstats = ( m_viewports.empty() || !( Global.gfx_lights_clustered || Global.gfx_lights_clusterheatmap ) ? std::nullopt : m_viewports.front()->clusters->stats() ) ) {
         m_debugstatstext +=
             "\ncluster lists: " + std::to_string( cullstats->index_count )
@@ -1020,6 +1022,8 @@ void opengl46_renderer::draw_debug_ui()
         {
             ImGui::Checkbox("the same for the cascades of the sun shadows (gfx.gpucull.shadows)", &Global.gfx_gpucull_shadows);
             ImGui::Checkbox("the same for the environment map, when it's drawn next (gfx.gpucull.reflections)", &Global.gfx_gpucull_reflections);
+            ImGui::Checkbox("the same for the shadow maps of headlights (gfx.gpucull.shadows.headlights)", &Global.gfx_gpucull_shadows_headlights);
+            ImGui::Checkbox("the same for the scenery of the shadow maps of lamps, when it's drawn next (gfx.gpucull.shadows.lamps)", &Global.gfx_gpucull_shadows_lamps);
             ImGui::Checkbox("the same for the sleepers of the tracks of the view (gfx.gpucull.sleepers)", &Global.gfx_gpucull_sleepers);
         }
         ImGui::Checkbox("draw the depth of the whole scene before its colour (gfx.depthprepass.scene)", &Global.gfx_depthprepass_scene);
@@ -1037,6 +1041,7 @@ void opengl46_renderer::draw_debug_ui()
         }
         if (Global.gfx_gpucull_reflections && m_reflectioncull.ran())
             info += m_reflectioncull.info("gpu cull reflections, last face", false);
+        info += Light_Shadow_Cull_Info(false);
         if (!info.empty())
             ImGui::TextUnformatted(info.c_str() + 1);
     }
@@ -2951,8 +2956,8 @@ void opengl46_renderer::Render(scene::basic_region *Region)
 		bucket.second.clear();
 	}
 	// the gpu culling runs along the first color pass of the main viewport of a frame, and the passes of the cascades of the
-	// sun shadows and of the faces of the environment map drawn for it, each with a culling of its own. its result is drawn
+	// sun shadows, of the faces of the environment map and of the maps of headlights and lamps drawn for it, each with a culling
-	// (Global.gfx_gpucull 1), or compared with the one of the cpu, which draws (2)
+	// of its own. its result is drawn (Global.gfx_gpucull 1), or compared with the one of the cpu, which draws (2)
 	m_instancecullpass = false;
 	m_passcull = nullptr;
 	if( ( Global.gfx_gpucull != 0 )
@@ -2963,12 +2968,18 @@ void opengl46_renderer::Render(scene::basic_region *Region)
 			m_instancecullframe = m_framestamp;
 			m_passcull = &m_instancecull;
 		}
+		else if( ( m_renderpass.draw_mode == rendermode::shadows )
+		      && ( m_lightshadowcull != nullptr ) ) {
+			// the map of a set of headlights or the scenery part of the map of a lamp, which picked its culling
+			m_passcull = m_lightshadowcull;
+			m_shadowcasters.clear();
+		}
 		else if( ( m_renderpass.draw_mode == rendermode::shadows )
 		      && ( Global.gfx_gpucull_shadows )
 		      && ( m_shadowcascade >= 0 )
 		      && ( m_shadowcascade < static_cast<int>( m_shadowcull.size() ) )
 		      && ( false == m_cubeshadows ) ) {
-			// the cascades of the sun; the maps of lamps and headlights go as before
+			// the cascades of the sun
 			m_passcull = &m_shadowcull[ m_shadowcascade ];
 			m_shadowcasters.clear();
 		}
@@ -4429,7 +4440,9 @@ opengl46_renderer::indirect_counts opengl46_renderer::Render_Instances_Indirect_
 		auto const texture { Indirect_Shadow_Texture( draw.material ) };
 		auto *multidrawprogram {
 			m_indirectmultidraw ?
-				( texture != null_handle ? m_alpha_shadow_multidraw_shader.get() : m_shadow_multidraw_shader.get() ) :
+				( m_cubeshadows ?
+					m_localshadows.multidraw_program( texture != null_handle ) :
+					( texture != null_handle ? m_alpha_shadow_multidraw_shader.get() : m_shadow_multidraw_shader.get() ) ) :
 				nullptr };
 		if( multidrawprogram != nullptr ) {
 			// the draws which follow in the same bank and of the same primitive, with the same texture for the alpha test
@@ -8057,6 +8070,7 @@ void opengl46_renderer::Render_local_shadows()
 
 void opengl46_renderer::Render_lamp_shadows()
 {
+	m_lampscenerydrawn = 0;
 	auto const &lamps{m_localshadows.lamps()};
 	if (lamps.empty() || (false == Global.gfx_lights_clustered))
 		return;
@@ -8157,7 +8171,10 @@ void opengl46_renderer::Render_lamp_shadow(std::size_t const Slot, bool const Ve
 	{
 		m_cubescenery = true;
 		m_cubeskipinstance = static_cast<TAnimModel const *>(slot.owner);
+		// the instances culled on the gpu, each scenery part drawn in a frame with a culling of its own
+		m_lightshadowcull = Light_Shadow_Cull(m_lampcull, m_lampscenerydrawn++, Global.gfx_gpucull_shadows_lamps);
 		Render(simulation::Region);
+		m_lightshadowcull = nullptr;
 		m_cubeskipinstance = nullptr;
 		m_cubescenery = false;
 	}
@@ -8307,7 +8324,10 @@ void opengl46_renderer::Render_headlight_shadow(std::size_t const Slot, light_ar
 
 	m_localshadows.begin_headlight(Slot, projector, rotation, projection, Light.direction, m_framestamp);
 	m_shadowskipvehicle = Light.owner;
+	// the instances culled on the gpu, each slot with a culling of its own
+	m_lightshadowcull = Light_Shadow_Cull(m_headlightcull, Slot, Global.gfx_gpucull_shadows_headlights);
 	Render(simulation::Region);
+	m_lightshadowcull = nullptr;
 	if ((Cab != nullptr) && (Cab != Light.owner) && (glm::length(Cab->GetPosition() - projector) <= reach + Cab->radius()))
 	{
 		Render_cab(Cab, 0.0f, false);
@@ -8320,6 +8340,51 @@ void opengl46_renderer::Render_headlight_shadow(std::size_t const Slot, light_ar
 	glPolygonMode(GL_FRONT_AND_BACK, Global.bWireFrame ? GL_LINE : GL_FILL);
 }
 
+opengl46_instance_cull *opengl46_renderer::Light_Shadow_Cull(std::vector<std::unique_ptr<opengl46_instance_cull>> &Culls, std::size_t const Index, bool const Enabled)
+{
+	// the same conditions as those of the other passes in Render(scene::basic_region *)
+	if ((Global.gfx_gpucull == 0) || (false == Enabled) || (m_current_viewport == nullptr) || (false == m_current_viewport->main))
+		return nullptr;
+	if (Culls.size() <= Index)
+		Culls.resize(Index + 1);
+	if (!Culls[Index])
+	{
+		// the buffers of its own, the records and the programs of the main view
+		Culls[Index] = std::make_unique<opengl46_instance_cull>();
+		Culls[Index]->init(&m_instancecull);
+	}
+	return Culls[Index].get();
+}
+
+std::string opengl46_renderer::Light_Shadow_Cull_Info(bool const Times)
+{
+	std::string text;
+	if (Global.gfx_gpucull == 0)
+		return text;
+	// the maps are drawn now and then, the numbers are of the last drawing of each
+	auto const add = [&](std::vector<std::unique_ptr<opengl46_instance_cull>> &Culls, bool const Enabled, std::string const &Label) {
+		if (false == Enabled)
+			return;
+		for (std::size_t idx = 0; idx < Culls.size(); ++idx)
+		{
+			if (!Culls[idx] || (false == Culls[idx]->ran()))
+				continue;
+			if (Times)
+			{
+				text += Culls[idx]->info_times("instance cull " + Label + " " + std::to_string(idx + 1));
+			}
+			else
+			{
+				Culls[idx]->collect();
+				text += Culls[idx]->info("gpu cull " + Label + " " + std::to_string(idx + 1), false);
+			}
+		}
+	};
+	add(m_headlightcull, Global.gfx_gpucull_shadows_headlights, "headlight shadows");
+	add(m_lampcull, Global.gfx_gpucull_shadows_lamps, "lamp shadows, scenery");
+	return text;
+}
+
 void opengl46_renderer::Update_Clusters(viewport_config &vp, glm::ivec2 const &Targetsize)
 {
 	glDebug("Update_Clusters");

rendering/opengl46/opengl46renderer.h

@@ -315,6 +315,11 @@ class opengl46_renderer : public gfx_renderer {
 	void Render_headlight_shadows();
 	// the map of the headlights in the slot: the scene within reach ahead of them (and Cab, the occupied cab), without their vehicle
 	void Render_headlight_shadow(std::size_t const Slot, light_array::light_record const &Light, TDynamicObject *Cab);
+	// the culling of one of the maps of lamps and headlights (Global.gfx_gpucull_shadows_headlights, _lamps), the one at Index of
+	// Culls, made when it's first needed; nullptr when the instances of the map go through the cpu
+	opengl46_instance_cull *Light_Shadow_Cull(std::vector<std::unique_ptr<opengl46_instance_cull>> &Culls, std::size_t const Index, bool const Enabled);
+	// the rows of these cullings for the debug panel: their timings (Times) or their counts
+	std::string Light_Shadow_Cull_Info(bool const Times);
 	// the projector of the beam of a set of headlights: a narrow frustum along the vehicle, from behind it, its near plane at the
 	// end of the vehicle. the beam texture is projected with it (Update_Lights()) and the shadow map of the headlights drawn with it.
 	// Position in world space, Rotation from world axes to its axes; Near is the distance of the near plane from the projector
@@ -355,9 +360,10 @@ class opengl46_renderer : public gfx_renderer {
 	// (Global.gfx_gpucull_occlusion_instances) the instances seen in the previous frame are drawn first (their depth, or
 	// their colour without the prepass), the pyramid is made again and the other instances are culled against it
 	void Render_Instances_Indirect();
-	// the same for the passes culled on the gpu other than the main view (Global.gfx_gpucull_shadows, _reflections): the
+	// the same for the passes culled on the gpu other than the main view (Global.gfx_gpucull_shadows, _reflections,
-	// cascades of the sun shadows, drawn with the shadow programs, and the faces of the environment map, drawn with the
+	// _shadows_headlights, _shadows_lamps): the cascades of the sun shadows and the maps of headlights and lamps, drawn with the
-	// programs of the materials like the main view. no occlusion test, no depth prepass, one phase
+	// shadow programs (the cube ones for the lamps), and the faces of the environment map, drawn with the programs of the
+	// materials like the main view. no occlusion test, no depth prepass, one phase
 	void Render_Instances_Indirect_Pass();
 	// the walk of the submodel trees of the groups culled by Cull, once per lod band, which records their draws
 	// (m_indirectdraws, m_indirectcommands), puts them in order for multi draws and has Cull fill in their commands
@@ -593,6 +599,12 @@ class opengl46_renderer : public gfx_renderer {
 	// _reflections), with records and programs shared with m_instancecull
 	std::array<opengl46_instance_cull, 3> m_shadowcull;
 	opengl46_instance_cull m_reflectioncull;
+	// the cullings of the maps of headlights, by the slot of the map, and of the scenery parts of the maps of lamps, by their order
+	// among those drawn in a frame: the passes of a frame don't share the buffers of a culling. made when first needed
+	std::vector<std::unique_ptr<opengl46_instance_cull>> m_headlightcull;
+	std::vector<std::unique_ptr<opengl46_instance_cull>> m_lampcull;
+	opengl46_instance_cull *m_lightshadowcull { nullptr }; // the one of the map of headlights or of a lamp being drawn
+	std::size_t m_lampscenerydrawn { 0 }; // scenery parts of the maps of lamps drawn in this frame so far
 	opengl46_instance_cull *m_passcull { nullptr }; // the one the current pass feeds
 	opengl46_instance_cull *m_indirectcull { &m_instancecull }; // the one the recorded draws (m_indirectdraws) were culled by
 	int m_shadowcascade { -1 }; // the region is drawn for this cascade of the sun shadows

utilities/Globals.cpp

@@ -1464,6 +1464,16 @@ global_settings::ConfigParse_gfx( cParser &Parser, std::string_view const Token
         Parser.getTokens(1);
         Parser >> gfx_gpucull_reflections;
     }
+    else if (Token == "gfx.gpucull.shadows.headlights")
+    {
+        Parser.getTokens(1);
+        Parser >> gfx_gpucull_shadows_headlights;
+    }
+    else if (Token == "gfx.gpucull.shadows.lamps")
+    {
+        Parser.getTokens(1);
+        Parser >> gfx_gpucull_shadows_lamps;
+    }
     else if (Token == "gfx.gpucull.sleepers")
     {
         Parser.getTokens(1);
@@ -1972,6 +1982,8 @@ global_settings::export_as_text( std::ostream &Output ) const {
     export_as_text( Output, "gfx.gpucull.multidraw", gfx_gpucull_multidraw );
     export_as_text( Output, "gfx.gpucull.shadows", gfx_gpucull_shadows );
     export_as_text( Output, "gfx.gpucull.reflections", gfx_gpucull_reflections );
+    export_as_text( Output, "gfx.gpucull.shadows.headlights", gfx_gpucull_shadows_headlights );
+    export_as_text( Output, "gfx.gpucull.shadows.lamps", gfx_gpucull_shadows_lamps );
     export_as_text( Output, "gfx.gpucull.sleepers", gfx_gpucull_sleepers );
     export_as_text( Output, "gfx.depthprepass.scene", gfx_depthprepass_scene );
     export_as_text( Output, "gfx.depthprepass.replay", gfx_depthprepass_replay );

utilities/Globals.h

@@ -298,6 +298,10 @@ struct global_settings {
     // culled in compute passes as well (drawn from their result, or compared), not only those of the main view
     bool gfx_gpucull_shadows = true;
     bool gfx_gpucull_reflections = true;
+    // with gfx_gpucull 1 or 2: the same for the maps of the headlights of vehicles and for the scenery parts of the maps of
+    // scenery lamps, each map with a culling of its own
+    bool gfx_gpucull_shadows_headlights = true;
+    bool gfx_gpucull_shadows_lamps = true;
     // with gfx_gpucull 1 or 2: the sleepers of the tracks (sleepermodel) of the main view are culled with the instances as well,
     // and with 1 drawn from the result
     bool gfx_gpucull_sleepers = true;