18
0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-11-20 16:14:58 +01:00
Files
maszyna/tests/unit/COVERAGE.md
jerrrrycho cce66422a3 docs: coverage of the rest of the core library, and what its tests found
The lines run of the files besides the physics, from the Coverage run of #325, what of the parser, the helpers and the PLC stays untested and what nothing in the game calls, and the behaviour the new tests recorded which may deserve a look.
2026-11-11 22:40:03 +01:00

23 KiB

Coverage of the simulation core by the unit tests

These tests characterise the physics of the vehicles (McZapkie/Mover.cpp, hamulce.cpp, Oerlikon_ESt.cpp, friction.cpp). They exist to show that refactors of those files, such as splitting TractionForce or RunCommand, leave the behaviour as it was. This page lists:

  • what the tests reach;
  • what nothing in the game ever runs;
  • what only data no game vehicle has would run;
  • what is left to test;
  • the behaviour the tests found that may deserve a look.

How it was measured

Coverage was measured on Windows with MSVC's SanitizerCoverage (/fsanitize-coverage=trace-pc-guard) on the unit test binary. The counts are basic blocks, which comes close to branch coverage: both sides of a condition are blocks of their own. The tool is local and not part of the build, by decision (2026-10). Any block-coverage tool run over eu07_unit_tests gives comparable numbers.

File Blocks run, at the start Now
Mover.cpp 8.9% 80.7% (3323 of 4118)
hamulce.cpp 30% 76.3% (649 of 851)
Oerlikon_ESt.cpp 58% 87.4% (118 of 135)
friction.cpp 9% 90.9% (10 of 11)

The aim is branch coverage with the rest explained below, not 100%.

The numbers above are from 2026-11-01. Since then the Coverage workflow (.github/workflows/coverage.yml) measures every pull request and every push to master: a GCC build of the unit tests with gcov, lines, functions and branches per source file of the core library. The totals and the per-file table are in the summary of the workflow run, and the HTML report with the lines marked is in its unit-test-coverage artifact. Its counts are source lines and gcov's branches, so they differ somewhat from the blocks in the table. The report isn't sent to SonarQube Cloud: the project's quality gate would then require 80% coverage of the new code of every pull request.

Code nothing in the game runs

Nothing creates these objects or calls these functions anywhere in the game, which was checked against the whole source tree. A refactor can delete them, or leave them untested.

  • Brake valves that never get created:
    • TESt, TESt3 and TESt3AL2, including Init, PLC and SetLP. The valve names ESt3, ESt3AL2 and ESt4 all create TNESt3.
    • TESt4R::GetPF, overridden by TLSt and never called by it.
    • TESt::CheckState, CVs, BVs, GetPF and EStParams, and TNESt3::EStParams.
  • TFV4a (GetPF, Init): the FV4a handle of the vehicle definitions creates TFV4aM.
  • Base implementations always overridden: TFricMat::GetFC (every material, the default P10 too, has its own) and TPrzekladnik::Update (every relay has its own).
  • Functions without callers:
    • In hamulce.cpp: PF_old, TBrake::GetBCF, TDriverHandle::OvrldButton, TReservoir::pa and TBrakeCyl::pa.
    • In Mover.cpp: BrakeVP, PipeRatio, RealPipeRatio, IncBrakeLevel, DecBrakeLevel, IncBrakePress, DecBrakePress, MainCtrlActualPowerPos and GetExternalCommand.
  • Inverter failures of the induction motor drives (inverter::Failure_Drive, Failure_Const, Activate): nothing ever sets them. The cab only shows them. The branches reading them in TractionForce can't be reached.

Code only data no game vehicle has would run

Where it was worth it, a test fits the missing parameter to a real vehicle or builds a definition with it (definition::set()). Those cases are marked "tested" below. The rest is reachable only by a vehicle definition the game data doesn't have:

  • Tested, fitted or handmade:
    • The valves CV1, CV1_R, CV1_L_TR, the W, K and Hik variants, SW, NESt3 and M483 (brake cycle: the valves the game knows…).
    • An Oerlikon valve named with -ED or AL2 without ESt3 (its relays TPrzekED and TPrzekCiagly).
    • The brake materials FR513, FR510, P10yBg and P10yBgu (brake cycle: every material…).
    • The handles test, D2, Westinghouse, MHZ_EN57 and St113 fitted to an EU07.
    • A CompressorList, a PmaxList and a Headlights section.
    • The rear cab's compressor limits (MinCP_B, MaxCP_B).
    • The controller of the electrodynamic brake apart from the local brake (SplitEDPneumaticBrake).
  • Not tested:
    • The retarder's clutch (R_IsClutch, R_ClutchSpeed) and the retarder worked by the local brake (R_WithIndividual).
    • The automatic relays of type 2 (RelayType=2).
    • The variant of CheckLocomotiveParameters for a definition whose RList doesn't match its powered axles.

Reachable, not tested yet

These are run by the game, by vehicles it has or by the cab, the AI or the scenery, but no test reaches them yet. Most of them are the far sides of conditions inside functions the tests do run.

  • Entry points from outside the core:
    • TBrake::ForceEmptiness and its overrides: a scenario's vehicle set up with empty brakes (DynObj.cpp).
    • TBrake::GetSoundFlag: the sounds of the brakes.
    • LoadFIZ and reload_FIZ: the file loading around LoadFIZ_Text, which the tests call directly.
    • TSecuritySystem::set_cabsignal_lock.
    • TMHZ_EN57::GetEP: an MHZ_EN57 handle in the EP mode of the brake.
  • Inside the functions the tests run (blocks run of the function's total):
    • TractionForce (305 of 373): the inverter failures above. The rest are mostly halves of conditions:
      • the SST table of the series-excited diesel-electrics;
      • the speed control's brake intervention with the air under BrakeInterventionVel;
      • a diesel-electric's generator tripping on overcurrent;
      • the ffBrakeList of the 22WEc, whose electrodynamic brake no test applies.
    • AutoRelayCheck (87 of 119): the ET22's group switch under some sequences, the relays at reverse with BackwardsBranchesAllowed, and the shunting of the EZTs.
    • update_doors (134 of 160), PantographsCheck (44 of 64) and PowerCouplersCheck (70 of 88): door modes and pantograph valve kinds only some game vehicles have, and the 24 V circuit across couplers.
    • RelayReset (29 of 43) and MainSwitchCheck (16 of 27): relays tripping in states the drive cycles don't reach.
    • UpdatePipePressure (108 of 130): the pressure switch lock of an electric unit's pantographs, which the EW58's automatic valve keeps away. Also the EP mode of the EN57's handle and the LockPipe thresholds (LPOn, LPOff), which no vehicle sets.
    • TCV1L_TR::GetPF (9 of 20), TKE::GetPF (28 of 37) and the MHZ handles: settings (G with the CV1L_TR, the KE's anti-slip pressure ASBP) that no real vehicle has together.

The rest of the core library

The core library (eu07_core) and the test binary hold more than the physics: the parser of the game's text files, the helpers of utilities.cpp, the PLC of a vehicle, the parts of the vehicles and of the simulation moved out of the big classes to be tested, and the hosts of the script screens. The sections above don't cover them. Their numbers are lines run, from the report of the Coverage workflow.

The tables show the run before the tests of the parser, the helpers and the PLC (test_parser.cpp, test_utilities.cpp, test_ladder_logic.cpp) were written. What those tests add is measured by the Coverage run of the pull request which brings them, and shown in the summary of that run.

The run measured: pull request #325 on 2026-11-11 (its commit 33cedc45 merged into master 10c7527a), GCC 13.3, unoptimised, gcov lines. All 49 files of the game in the report together: lines 84.4% (10305 of 12204), functions 85.0% (927 of 1090), branches 62.3% (10325 of 16578).

File Lines run Functions run
McZapkie/couplerphysics.cpp 93.6% (147 of 157) 20 of 21
scripting/ladderlogic.cpp 4.1% (9 of 217) 2 of 11
scripting/ladderlogic.h 0% (0 of 25) 0 of 4
scripting/luaprogram.cpp 74.7% (334 of 447) 50 of 66
scripting/scriptscreendrawing.cpp 93.8% (135 of 144) 19 of 19
scripting/scriptscreenkernels.cpp 97.7% (129 of 132) 11 of 11
scripting/scriptscreenrules.cpp 100% (50 of 50) 7 of 7
scripting/scriptscreens.cpp 82.8% (216 of 261) 28 of 33
scripting/wasmprogram.cpp 54.0% (157 of 291) 35 of 49
simulation/simulationtestrules.cpp 97.4% (75 of 77) 10 of 10
utilities/parser.cpp 40.0% (170 of 425) 18 of 41
utilities/parser.h 97.8% (44 of 45) 11 of 15
utilities/utilities.cpp 28.2% (82 of 291) 15 of 49
utilities/utilities.h 100% (41 of 41) 12 of 12
utilities/framepacer.h 100% (17 of 17) 2 of 2
vehicle/blackbox.cpp 100% (331 of 331) 41 of 41
vehicle/enginestart.cpp 96.2% (50 of 52) 6 of 6
vehicle/handover.cpp 89.2% (33 of 37) 5 of 5
vehicle/horn.cpp 100% (5 of 5) 2 of 2
vehicle/linebreaker.cpp 100% (6 of 6) 1 of 1
vehicle/smoke.cpp 100% (12 of 12) 2 of 2
vehicle/gaugescale.h 97.9% (47 of 48) 5 of 5
world/TrackUneven.h 100% (111 of 111) 23 of 25

Notes to the table:

  • The small headers with the logic of a single part are all run in full: vehicle/doorwings.h, gaugeduplicates.h, gaugepositions.h, holdcoillever.h, lightsprogrammer.h, mainctrlact.h, mirrors.h, scripting/scriptscreendrawing.h, scriptscreenkernels.h, scriptscreenprogram.h, scriptscreens.h, utilities/dictionary.h and vehicle/blackbox.h.
  • vehicle/brakesetting.cpp and vehicle/odometer.cpp (the brake setting of the wagons, the odometer store) joined the library on master after that run. They have tests (test_brake_setting.cpp, test_odometer.cpp) and no numbers yet.
  • The script hosts (scriptscreens.cpp, luaprogram.cpp, wasmprogram.cpp) aren't in the library. The test binary compiles them itself, with the real runtimes.
  • scene/scenelayers.h, scene/scenenode.h and scene/scenenodegroups.h are in the report with a few lines each: the parser reports the files it includes to the scenery, which the tests replace with stubs (engine_stubs.cpp).

The parser, the helpers and the PLC

test_parser.cpp, test_utilities.cpp and test_ladder_logic.cpp record how utilities/parser.cpp, utilities/utilities.cpp and scripting/ladderlogic.cpp behave now, oddities included, so that a later change of them can be checked. The lines the measured run left out, by what it takes to run them:

File Not run Through the public interface With files of the test Left: the system or the game state Nothing in the game calls it
utilities/parser.cpp 255 lines 131 105 2 17
utilities/utilities.cpp 209 lines 154 19 19 17
scripting/ladderlogic.cpp 208 lines 169 37 0 2

The tests are written for the first two columns, and for what the game doesn't call as far as it can be called from outside. The files of a test are made in the build folder of the tests (scratch.h) and removed with it.

What stays untested, and why:

  • parser.cpp:
    • The scenery layers of a scenery opened for editing are stubs in the tests. The parser's calls of them are run (sceneryLayers), but only the templates used can be checked. The include of a terrain file left out for the binary terrain marks its layer as binary (2 lines): the stub has no layer to mark, so that isn't run.
    • The messages of the log (Failed to open file, Bad include) are run, and not checked: the log of the tests is a stub.
    • A buffer type which is neither a file nor a text (the default: of the constructor) can't be given without an invalid value of the type.
  • utilities.cpp:
    • Now(), user_config_path(), user_cache_path() and true_random_seed() depend on the clock, the environment and the entropy source of the system. The tests call them and check the shape of the result. The branches for a system without the variables or without an entropy source aren't run, and those of the other systems aren't compiled.
    • FileCased() looks through the working folder for a name given without a folder, and through the folder of the file elsewhere; on Windows it only asks whether the file is there. The search is run on Linux only.
    • Bezogonkow() gives other letters with MSVC than with GCC and Clang, see the findings. The test takes either, told by the entry of the table the difference starts at, so each build checks the half it is.
  • ladderlogic.cpp: the two returns marked "not reachable" after the switches over the kind of an element.

Code nothing in the game runs (checked against the whole source tree):

  • The PLC as a whole. A vehicle has one (TMoverParameters::m_plc) and runs it in every step (m_plc.update()), but nothing calls load(), so its program is always empty; nothing calls find_or_insert(), input() or output() either, so no file of the game data can be a program for it. Of ladderlogic.cpp the game runs the 9 lines of an update without timers and without operations. The tests run the rest with programs made up for them.
  • In parser.cpp: countTokens() with its count(), and addCommentStyle(). No caller gives the separators as a null pointer either, which updateCharClasses() allows for.
  • In utilities.cpp: generate_uuid_v4(), FuzzyLogicAI() and last_modified(). In utilities.h: BorlandTime().

Cheap to add next: the Lua host (luaprogram.cpp, 113 lines not run) is mostly functions of the scripts' API no test script calls yet: fill, fill_rect, the aligned texts, draw_image, copy, sprites, time, log, a command from a screen outside of the cab. A script of a few lines each runs them, with the folders test_script_screen_host.cpp already makes. The WebAssembly host (wasmprogram.cpp, 134 lines) has the same functions behind imports, and needs a module encoded by hand for each group of them.

What the tests found

The tests record the behaviour as it is. The following points may deserve a look by someone who knows how the real vehicles behave.

  • Random(10) - Random(10) (Mover.cpp, the noise of the control pipe pressure CntrlPipePress): the order the two calls were evaluated in isn't specified. MSVC Release, MSVC Debug and GCC gave different traces, a control pipe pressure mirrored around 5 bar, which failed 32 tests on the Linux CI. Fixed: the draws are sequenced in the order of MSVC Release, which the golden traces were recorded with.
  • TCommand::Location isn't initialised in a new vehicle. The game always sets it through PutCommand(), but a test calling RunCommand() directly got different results run to run.
  • CompressorList is int[4][9] (MOVER.h), though its factors are fractions: 0.9 is read as 0. No game vehicle has the list yet.
  • A train whose pipe leaks under 2.75 bar before its start-up can't be released:
    • Where this happens: an FV4a handle and an LSt valve, for example 4 minutes standing with wagons, waiting for a hot engine.
    • The cause: the pipe gets cut off the main reservoir (lock_old in UpdatePipePressure). The AI presses its unlock button only for the lock set by LPOn and LPOff (control_main_pipe).
    • The same after every emergency stop of such a train, by the alarm chain or the radio-stop: reset, it stays with its pipe empty until the releaser is held (b_rls), which the test of the emergency stops does.
  • EW58:
    • Its electrodynamic brake comes in only by the stages of the ep brake (TUHEX), never by the brake pipe alone.
    • It needs 110 V, from the converter of the middle car only.
  • ET42:
    • At the high current setting the controller stops at the last series position, and the train crawls.
    • From the low setting's top speed (over 100 km/h) the dynamic brake's current trips the overload relay at once.
  • EU47: its local brake sets the flag of the electrodynamic brake, which keeps the train brake's air out, but the motors don't brake. The local brake's air does it.
  • EP09: the flag of the automatic electrodynamic brake stays on at a standstill while the cylinders hold less than 2.1 bar, and it flicks on while a high application is released.
  • Diesel engines with a second water circuit (SU45, ST44) started at 88 degrees: they don't start until that circuit, over its limit (74 or 75 degrees), cools. Off, it cools slowly.
  • Brake cylinders over the vehicle's highest pressure (found by the invariants, see the README):
    • The K valve of the SM03 and the EU04 fills the cylinders to the pressure of the auxiliary reservoir in a full service or emergency application: 4.27 bar of the SM03's MaxBP 3.5, 3.19 bar of the EU04's 2.8.
    • The ESt3d_PZZ valve of the SN61 does so in an emergency: 4.6 bar of its MaxBP 3.8.
    • The EU07's LSt overshoots a little, to 4.0 bar of its 3.8.
  • Diesel engines driven over their top speed: a lower gear taken at speed drives the engine through the wheels over its nmax, the SM03's by 10%, the WMB10's by 25%. The cutoff (nmax_cutoff) only takes the fuel off.
  • The ELF's C car's main reservoir over its governor: the cars share the supply pipe, the A car's governor stops at 9 bar, the C car's at 8.5, so the C car's reservoir reaches 8.64 bar.
  • Wagons thrown about as a train starts with full physics of the couplers: the slack taken up throws them at up to 5.7 g, and back within a step.
  • Nothing limits the speed: a light train at full power runs well past its Vmax (the 754 past 160 km/h, against a Vmax of 100).
  • Vehicles that don't do what they should, as driven by the tests:
    • An EN57 doesn't get ready (main reservoir).
    • An EN57AKL alone doesn't charge its pipe.
    • The TEM2 barely moves, and the SM31 crawls at 0.3 km/h at full power, its motors taking 444 A.
    • The draisine can't start from rest (the pump's force is under the resistance of a standing draisine).
    • The Elf and the Impuls, their doors opened with the steps permitted, close the doors as they start off but keep the steps out, which keeps them from moving.
    • The ED72 holds an ep application with its handle at running, and with the ep brake switched off too; only the ep release position lets it off. The EW58 releases at running.
    • The road vehicle (MAN) is only slowed down by a minute of full service.
  • A train run with five physics steps in a frame: an EU07 with wagons takes no current, its controller at full power, and rolls at a few km/h. The test steps the vehicles the way TDynamicObject does (FastComputeMovement for the steps before the last one), so the game may do the same.
  • Game data:
    • The 214M's file has TurboPos and MotorParamTable twice, and the EN57AL's Engine twice. The loader reads each in turn, so the later one overrides what both set.
    • A wagon set up braked keeps its hand brake on until someone releases it. The AI does that for the whole train (AutoRewident), so a test has to as well.
  • The parser (parser.cpp):
    • Read line by line ("\n\r" as the separators), a traction include left out because the traction isn't loaded takes the following lines with it, up to the next end: the directive is cut out of its line, but the skipping reads on in the file.
    • A traction file is told by tr/ or tra/ anywhere in its name, so with the traction off an include from any folder ending in "tr" (centr/…) is left out too.
    • Read line by line, every line starting with the letters include is taken for the directive (included in price includes the file d), also when the parser is told not to expand includes (expandIncludes = false).
    • Empty quotes ("") give no token: the next token is read in their place, so everything after a value meant to be empty moves by one.
    • The text in quotes keeps its case when the caller asks for lower case. That may be meant.
    • An include of an empty file is logged as a file which can't be opened (Bad include: can't open file, seen in the log of the test, which the test itself doesn't check).
    • A text which ends inside a comment or inside quotes leaves the parser failed (ok() false), not just at its end; an empty text fails with its first read, through the check for a byte order mark.
    • deserialize_map() (utilities.h, the sound overrides) therefore reads an empty key from an empty file, and one from a file ending with a line break; in a file with Windows line ends its values keep the carriage return. The game's sound_overrides.txt is a single line without a line break.
    • The separators given as a null pointer are taken as none by the tokenizer, but passed on to strcmp for a token of 7 characters or more starting with i, which crashes (tried by hand, not kept as a test). No caller gives a null pointer.
  • The helpers (utilities.cpp, utilities.h):
    • Bezogonkow(), which takes the marks off the letters for the timetable and the on-screen texts: its table has a ± in the source, a UTF-8 file. MSVC makes one byte of it, GCC and Clang two, which moves every entry from 0xB2 on by one. With MSVC the 18 polish letters come out as ACELNOSZZacelnoszz; with GCC as ALELDNSZZ,le,dnszl: Ć as L, Ń as D, Ó as N, ą and ł as a comma, ć as l, ń as d, ó as n, ż as l, only Ę and ę by luck as themselves. The Linux and macOS builds of the game show the names of the stations that way. The test checks whichever table the build has (the GCC one was tried with the table alone, compiled by GCC 6.3).
    • to_minutes_str() reads past its table of raised digits for minutes with a fraction of 0.99 or more, and before it for negative minutes, which CompareTime() can give it for a timetable whose times go back. Tried by hand, both ended in std::bad_alloc, the text read from outside of the table being no text; not kept as a test. A width under 2 throws std::length_error.
    • stol_def() gives its default only for an empty text. A text which isn't a number gives 0 (the flags of an event, the log setting).
    • split_string_and_number() throws for a name with a hyphen which no digit follows (a-b), and for a number too big. The gamepad bindings and the PLC use it on names from files.
    • extract_value() throws std::out_of_range when the key is the last thing in the text with nothing after its =. With a space after the =, the next key=value is taken for the value.
    • seed_of() takes a word starting with digits for the number in front (12abc is 12), so different words can stand for the same seed.
    • erase_extension() cuts at the dot of a folder when the file has none (dir.d/file becomes dir).
    • Round() adds a half and cuts the fraction, which rounds the negative numbers up: -1.6 becomes -1.
  • The PLC (ladderlogic.cpp):
    • Nothing loads a program into it, see above.
    • and and or combine the states bit by bit, and out passes a number on as it is: two elements which are both on, with the states 2 and 1, give 0.
    • A counter is done until the first cycle of the program gives it its limit.
    • end is read as "no operation": the lines after it are loaded and run.
    • A name with a hyphen (door-open) throws out of load(), through split_string_and_number().
    • The kind of an element is told by its name: t, ton or timer. and c or counter. before the number, so a variable can't be named t1 or c2, nor t or c alone: an input named c is a counter, on from the start.
    • The line of an operation is split with getTokens(3, "\t "), which gives the separators in the place of the lower case flag. The result is the same, the default separators being those and the semicolon.
    • An update longer than 32.7 s doesn't fit the milliseconds a timer counts in (short). Tried by hand: a timer of 1 s wasn't on after an update of 40 s, nor a second later. Not kept as a test, the conversion being undefined.