* feat(blackbox): a recorder of the run, the cab, the signals and the stations to a file With the physics logs on (and the new eu07.ini setting blackbox, on by default), the simulator writes a black box of the session to physicslog/*.blackbox: the scenery map (tracks, signals, W4 stop points with their heading, W6 horn points), the timetables of the user's train, and at 10 Hz the cab of the vehicle the user sits in (speed, controllers, brakes, pressures, lights with the pattern the run asks for, the movement kind), the AI's reference for the same place (target speed, next limit, braking distance), the consist, the cameras and the vehicles around. Changes go in as they happen: switches, signal aspects, commands, signals passed with their aspect, stations stopped at, passed or missed, horn requests, the autopilot switched on or off, couplings and vehicles removed. The driving aid's next speed limit moved from the UI panel to TController::next_speed_limit(), so the panel and the recorder read the same. The format is described in blackbox_doc.md; the writer and the reader of the records have unit tests. * fix(blackbox): the aspect a signal had when passed, the lamps the ends have, vehicles taken off - A passed signal is written with the aspect the train had of it, kept in the speed table, not the one its memory cell shows when the pass is noticed: by then the train itself on the track behind has often set it to stop, and most passes were written as at stop. - The cab record gains the lamps the vehicles at the head and the tail have at their outer ends (136 bytes), so a reader can check the light pattern against them as the driver does (no upper headlight, the auxiliary lamps, the rear-end signals only). - A vehicle taken off the scenery and still pointed at by a coupler or the camera gets no new number; the camera lets go of it. - The timetable written before a station goes under the vehicle the user is in now, not the one of the last look. - The file buffer is given once the file is open: MSVC passed over one given before. - The doc tells the AI fields before the consist is started and when entering a vehicle is a driver change. * fix(blackbox): the stop of an automatic block signal passed is written as a stop The speed table keeps the stop of an automatic block signal (SetVelocity 0 x) as the speed allowed after it, with spStopOnSBL; a pass of it was written as that speed with no flag. It is written as 0 with the flag of an automatic block stop again. * feat(blackbox): the doors, the brakes of the whole train, a diesel's engine and the rest of the cab The cab record grows from 136 to 190 bytes, so that a reader can check everything the driver did: - the doors: the permits on each side, the departure signal, the steps, the door lock and whether the doors are locked; - the brakes: the equalizing reservoir of the driver's valve, the main reservoir pipe, the highest brake cylinder pressure of the train, the hand brake, the releaser, the setting (G/P/R/Mg) and the mode (PS/PN/EP/MED); - what drives the vehicle (electric, induction, diesel, diesel-electric, steam, none), its power, a diesel's oil pressure, water, oil and engine temperatures, overheating, fuel and oil pumps, clutch and torque converter, the motor fans; - the cruise control and its speed, the integrated driving handle of a unit, the radio channel, the horn's tones. New device bits use the upper half of the 32-bit field; the new fields come at the end of the record, so older readers still read it. * fix(blackbox): the driver's valve field is what its gauge shows, the radio channel has no none Handle->GetCP() is the equalizing reservoir of most valves but the time reservoir of the FV4a, empty but after an overcharge: the field is named and documented as what the valve shows. With no driver the radio channel reads as the first one, not 0. --------- Co-authored-by: maj00r <maj00r@host>
Unit tests
C++ tests of the simulation logic: set up the conditions, run the code, check the outcome. They run
as part of every build of the game, in CI and locally alike, and from the IDE. They need no game
data and no window: the test vehicles are JSON definitions in data/, loaded from memory by the
game's own vehicle loader.
The code under test is eu07_core, the library of the simulation parts the game links as well
(physics, vehicle definition loader, ...): built once, with the game's settings, and tested as the
game uses it. The python library gets linked along with the game's settings, but the tests don't
use it and run without it.
COVERAGE.md says how much of the simulation core the tests reach, what's left and why, and what they found.
Running
- With the build: building the game (
WITH_TESTS, on by default) runs them at the end, once the game is linked, and printsUnit tests: 11/11 passed (80 checks). A failing test prints the full report and fails the build; the game binary gets removed, so the next build links it again and reruns them. They also run when only the tests, a definition indata/orrun_tests.cmakechanged (Ninja and Makefile generators).-DWITH_TESTS=OFFbuilds the game without them. - From the IDE: run or debug the
eu07_unit_teststarget. CLion recognises doctest and lists the test cases, so a single one can be run or debugged from the gutter. - By hand:
ctest --test-dir <build dir> --build-config Release, or the binary itself, which takes doctest options, e.g.eu07_unit_tests -tc="brakes*".EU07_UNIT_TESTS_LOG=1shows the engine log, quiet otherwise.
Writing a test
Add a test_<topic>.cpp here, it's picked up on its own.
#include "stdafx.h"
#include "doctest.h"
#include "vehicle.h"
TEST_CASE("brakes: service braking stops a heavier wagon too") {
// conditions: the wagon from data/wagon.json, heavier, running at 40 km/h
test::vehicle wagon{test::definition::load("wagon").set("Param", "M", 60000), 40.0 /* km/h */};
// action: full service application, as a coupled locomotive would make it
wagon.hold_brake_pipe(3.5); // bar
wagon.run(40.0 /* s */, test::track{0.15 /* friction */, 0.0 /* gradient */});
// outcome
CHECK(std::abs(wagon.speed()) < 0.1);
}
definition::load("<name>")readsdata/<name>.json, with the sections and keys of the game's vehicle definitions;set()andremove()change it for the test. Table sections give their rows as"rows": [[...], ...], keys starting with_are comments. A section the loader doesn't know is an error (the table of them is infiz.cpp); a misspelt key isn't caught, so check in the test that the value the test is about made it to the vehicle. Keep the definitions physically sensible: some parts need keys others don't (the ESt brake valves needBrake.Size), and a test on a half-defined vehicle tests the definition rather than the simulation.definition::fiz("<dir>/<name>")is one of the game's own vehicles,data/fiz/<dir>/<name>.json, in the same form, converted from the game's.fizwith the files it includes (the list and how to add one:data/fiz/README.md). The conversion keeps what the game's loader reads of the file and checks it reads the same of the converted one (fiz.h), so the vehicle is the game's.- The random numbers of the simulation are reseeded before every test case, so a test gives the
same result alone or among the others. Loading a vehicle draws some too (the scatter of its brake
pipe pressure), so a test making vehicles in a loop calls
test::reseed_random()before each. - Logic which lives in classes too big to set up in a test (the AI driver, the cab) goes into free
functions on the vehicle state the class calls, and the test calls them the same way: see
vehicle/handover.h, whichTController::TakeControl()uses, andtest_handover.cpp. vehiclecreates the vehicle the wayTDynamicObject::Init()does and steps it the way the simulation loop does, on a straight track of the given friction, gradient, catenary voltage and physics iterations a frame. With a cab (vehicle{definition, 0.0, "name", 1}) it has a driver.consist(consist.h) couples vehicles into a train and steps them together, coupler forces, air, control and power passing between them. The physics reaches a neighbour through itsTDynamicObject, for itsMoverParametersonly; the tests have a stand-in of it with just that.drive.hhas the driver's part, made of the AI's steps:start_up()(battery, pumps, pantographs, line breaker, converter, compressor),drive()(running, acknowledging the vigilance),release_brakes(),drive_cycle()(a whole run from start-up to a stop).scenario(scenario.h) puts a train, its driver and a golden trace for each vehicle together, for a test of a feature:act()marks an event in the traces,wait()runs the train.pair(vehicle.h) puts two vehicles on that track facing each other with the given ends and runs them together, as the simulation runs the vehicles of a scene: the coupler and buffer forces, collisions and automatic coupling come from the physics (test_couplers.cpp).- Check behaviour (speeds, pressures, states the player sees) rather than internal flags, and make
the test check its own preconditions (
REQUIREthat the braking skidded, before checking what a skid leads to).
Golden traces
A golden trace pins down how the simulation behaves now, for changes which mustn't alter it, like
restructuring a function: the values of some channels of a vehicle over a run, compared with the
ones recorded from the code before, in data/golden/<name>.csv. A difference fails the test, naming
the channel, the time and both rows.
The samples and their key figures aren't kept in the repository (tens of megabytes, and they only mean something against the code before the change): record them before the change, and compare after it.
git switch master
EU07_UNIT_TESTS_UPDATE_GOLDEN=1 eu07_unit_tests # records data/golden/**/*.csv, ignored by git
git switch my-refactor
EU07_UNIT_TESTS_GOLDEN_EXACT=1 eu07_unit_tests # compares every sample, bit for bit
The recorded files stay through switching branches. Without them, a trace checks the invariants and
its test's own CHECKs, which is what the CI does: every test states the behaviour it's about in
CHECKs, a scenario of a feature what the feature does (the doors open, the train stops), a run of
every real vehicle what holds for each of them (it brakes harder by every step, it runs off and stops).
test::trace trace{"eu07/notch up", test::channels::motion() + test::channels::traction()};
eu07.record(trace); // samples every 10th step
eu07->MainSwitch(true);
trace.mark("main switch"); // shows in the event column of the row
eu07.run(30.0);
trace.check();
EU07_UNIT_TESTS_UPDATE_GOLDEN=1records the samples and the key figures, for a new test or a change which alters the behaviour on purpose; the diff of the key figures shows what it altered, review it like code.- Without
EU07_UNIT_TESTS_GOLDEN_EXACT=1, samples recorded are compared within a relative 1e-9, as GCC and glibc's maths functions can differ from MSVC's in the last bits, and a trace without them passes. With it they're compared exactly, and a trace without them or its key figures fails, so a run of the refactoring check can't pass by having nothing to compare. - Keep the runs short (tens of seconds): the simulation feeds back on itself, and a last bit grows.
- A trace with no assertion about what the behaviour should be is a characterisation, bugs included;
put the behaviour the test is about into
CHECKs besides.
Key figures
Next to its samples, every trace records the figures a person reads (figures.h), for each part of
the run between two events, in data/golden/<name>.figures: the duration, the speeds, the distance,
the time to stop, the brake pipe's lowest pressure and the time to get back to 4.9 bar, the peaks of
the brake cylinders, the motor current and the tractive force.
[brake]
duration = 60.00 s
speed at the start = 118.5 km/h
distance = 944.3 m
stopped after = 45.97 s
brake cylinders peak = 3.997 bar
They're compared within 0.5% or the last decimal written, loose enough for any compiler, and the differences are told by part and figure:
key figures of "traction/4e_v1/4e" differ:
[brake] distance: 944.3 m now, 980.0 m recorded
EU07_UNIT_TESTS_UPDATE_GOLDEN=1 records them with the samples. After a change which alters the
behaviour, their comparison says in a few lines what the thousands of changed samples mean. A trace
sampled by hand, at times of its own (the commands and switches tests), records none.
Invariants
Every trace also checks, on every step of its vehicle and not only on the recorded ones, the rules
which hold for any vehicle whatever a test does (invariants.h): no NaN, the brake cylinders under
the vehicle's highest pressure, the pipes under the highest governor setting, no motor current with
the line breaker open, no tractive force without power, the distance counted as the speed runs, no
acceleration over 5 g, and a few more. They compare nothing with a recorded file, so they hold on
any compiler, and they still check a behaviour recorded anew on purpose makes sense. A broken rule
fails the test, naming the time, the last event and the values:
invariant "no motor current with the line breaker open" broken in traction/et42 fuse/0 at 63.200 s,
after "dynamic brake": Mains=0 Im=-412.5 Itot=-412.5
Where a test makes a vehicle break a rule (a collision), or a game vehicle breaks one as it is, the
list of excuses in invariants.cpp names the trace, the rule and the reason; the second kind goes
into the findings of COVERAGE.md too.
Code to test, and when the tests fail to link
A source joins the tested code by moving from SOURCES to CORE_SOURCES in the main
CMakeLists.txt; the game links it from the library from then on. The library reaches for a little
of the rest of the engine, which engine_stubs.cpp stands in for in the tests (logs, settings,
scenery layers). If a change makes it use something new, either move the real source into the
library too, when it stands on its own, or add a stub to engine_stubs.cpp.