mirror of
https://github.com/MaSzyna-EU07/maszyna.git
synced 2026-11-20 19:44:58 +01:00
The tokens, comments, quotes, parameters and include directives of cParser, its numbers and its place in the text, as they behave now, with the files of the tests made in a folder of the build (scratch.h). The helpers of utilities.h which read from a parser are here too.
183 lines
11 KiB
Markdown
183 lines
11 KiB
Markdown
# Unit tests
|
|
|
|
C++ tests of the simulation logic: set up the conditions, run the code, check the outcome. They run
|
|
as part of every build of the game, in CI and locally alike, and from the IDE. They need no game
|
|
data and no window: the test vehicles are JSON definitions in `data/`, loaded from memory by the
|
|
game's own vehicle loader.
|
|
|
|
The code under test is `eu07_core`, the library of the simulation parts the game links as well
|
|
(physics, vehicle definition loader, ...): built once, with the game's settings, and tested as the
|
|
game uses it. The python library gets linked along with the game's settings, but the tests don't
|
|
use it and run without it.
|
|
|
|
[COVERAGE.md](COVERAGE.md) says how much of the simulation core the tests reach, what's left and why, and
|
|
what they found.
|
|
|
|
## Running
|
|
|
|
- **With the build**: building the game (`WITH_TESTS`, on by default) runs them at the end, once the
|
|
game is linked, and prints `Unit tests: 11/11 passed (80 checks)`. A failing test prints the full
|
|
report and fails the build; the game binary gets removed, so the next build links it again and
|
|
reruns them. They also run when only the tests, a definition in `data/` or `run_tests.cmake`
|
|
changed (Ninja and Makefile generators). `-DWITH_TESTS=OFF` builds the game without them.
|
|
- **From the IDE**: run or debug the `eu07_unit_tests` target. CLion recognises doctest and lists the
|
|
test cases, so a single one can be run or debugged from the gutter.
|
|
- **By hand**: `ctest --test-dir <build dir> --build-config Release`, or the binary itself, which takes
|
|
[doctest options](https://github.com/doctest/doctest/blob/master/doc/markdown/commandline.md), e.g.
|
|
`eu07_unit_tests -tc="brakes*"`. `EU07_UNIT_TESTS_LOG=1` shows the engine log, quiet otherwise.
|
|
|
|
## Writing a test
|
|
|
|
Add a `test_<topic>.cpp` here, it's picked up on its own.
|
|
|
|
```cpp
|
|
#include "stdafx.h"
|
|
#include "doctest.h"
|
|
#include "vehicle.h"
|
|
|
|
TEST_CASE("brakes: service braking stops a heavier wagon too") {
|
|
// conditions: the wagon from data/wagon.json, heavier, running at 40 km/h
|
|
test::vehicle wagon{test::definition::load("wagon").set("Param", "M", 60000), 40.0 /* km/h */};
|
|
// action: full service application, as a coupled locomotive would make it
|
|
wagon.hold_brake_pipe(3.5); // bar
|
|
wagon.run(40.0 /* s */, test::track{0.15 /* friction */, 0.0 /* gradient */});
|
|
// outcome
|
|
CHECK(std::abs(wagon.speed()) < 0.1);
|
|
}
|
|
```
|
|
|
|
- `definition::load("<name>")` reads `data/<name>.json`, with the sections and keys of the game's
|
|
vehicle definitions; `set()` and `remove()` change it for the test. Table sections give their
|
|
rows as `"rows": [[...], ...]`, keys starting with `_` are comments. A section the loader doesn't
|
|
know is an error (the table of them is in `fiz.cpp`); a misspelt key isn't caught, so check
|
|
in the test that the value the test is about made it to the vehicle. Keep the definitions
|
|
physically sensible: some parts need keys others don't (the ESt brake valves need `Brake.Size`),
|
|
and a test on a half-defined vehicle tests the definition rather than the simulation.
|
|
- `definition::fiz("<dir>/<name>")` is one of the game's own vehicles, `data/fiz/<dir>/<name>.json`,
|
|
in the same form, converted from the game's `.fiz` with the files it includes (the list and how to
|
|
add one: `data/fiz/README.md`). The conversion keeps what the game's loader reads of the file and
|
|
checks it reads the same of the converted one (`fiz.h`), so the vehicle is the game's.
|
|
- The random numbers of the simulation are reseeded before every test case, so a test gives the
|
|
same result alone or among the others. Loading a vehicle draws some too (the scatter of its brake
|
|
pipe pressure), so a test making vehicles in a loop calls `test::reseed_random()` before each.
|
|
- Logic which lives in classes too big to set up in a test (the AI driver, the cab) goes into free
|
|
functions on the vehicle state the class calls, and the test calls them the same way: see
|
|
`vehicle/handover.h`, which `TController::TakeControl()` uses, and `test_handover.cpp`.
|
|
- `vehicle` creates the vehicle the way `TDynamicObject::Init()` does and steps it the way the
|
|
simulation loop does, on a straight track of the given friction, gradient, catenary voltage and
|
|
physics iterations a frame. With a cab (`vehicle{definition, 0.0, "name", 1}`) it has a driver.
|
|
- `consist` (`consist.h`) couples vehicles into a train and steps them together, coupler forces,
|
|
air, control and power passing between them. The physics reaches a neighbour through its
|
|
`TDynamicObject`, for its `MoverParameters` only; the tests have a stand-in of it with just that.
|
|
- `drive.h` has the driver's part, made of the AI's steps: `start_up()` (battery, pumps,
|
|
pantographs, line breaker, converter, compressor), `drive()` (running, acknowledging the
|
|
vigilance), `release_brakes()`, `drive_cycle()` (a whole run from start-up to a stop).
|
|
- `scenario` (`scenario.h`) puts a train, its driver and a golden trace for each vehicle together,
|
|
for a test of a feature: `act()` marks an event in the traces, `wait()` runs the train.
|
|
- `pair` (`vehicle.h`) puts two vehicles on that track facing each other with the given ends and
|
|
runs them together, as the simulation runs the vehicles of a scene: the coupler and buffer
|
|
forces, collisions and automatic coupling come from the physics (`test_couplers.cpp`).
|
|
- A test which reads files makes them in a `test::scratch_folder` (`scratch.h`): a folder of its own
|
|
in the build folder of the tests, removed with the object. The files are written byte for byte, so
|
|
the line ends are the ones the test gives (`test_parser.cpp`, `test_ladder_logic.cpp`).
|
|
- Check behaviour (speeds, pressures, states the player sees) rather than internal flags, and make
|
|
the test check its own preconditions (`REQUIRE` that the braking skidded, before checking what a
|
|
skid leads to).
|
|
|
|
## Golden traces
|
|
|
|
A golden trace pins down how the simulation behaves now, for changes which mustn't alter it, like
|
|
restructuring a function: the values of some channels of a vehicle over a run, compared with the
|
|
ones recorded from the code before, in `data/golden/<name>.csv`. A difference fails the test, naming
|
|
the channel, the time and both rows.
|
|
|
|
The samples and their key figures aren't kept in the repository (tens of megabytes, and they only mean
|
|
something against the code before the change): record them before the change, and compare after it.
|
|
|
|
```
|
|
git switch master
|
|
EU07_UNIT_TESTS_UPDATE_GOLDEN=1 eu07_unit_tests # records data/golden/**/*.csv, ignored by git
|
|
git switch my-refactor
|
|
EU07_UNIT_TESTS_GOLDEN_EXACT=1 eu07_unit_tests # compares every sample, bit for bit
|
|
```
|
|
|
|
The recorded files stay through switching branches. Without them, a trace checks the invariants and
|
|
its test's own `CHECK`s, which is what the CI does: every test states the behaviour it's about in
|
|
`CHECK`s, a scenario of a feature what the feature does (the doors open, the train stops), a run of
|
|
every real vehicle what holds for each of them (it brakes harder by every step, it runs off and stops).
|
|
|
|
```cpp
|
|
test::trace trace{"eu07/notch up", test::channels::motion() + test::channels::traction()};
|
|
eu07.record(trace); // samples every 10th step
|
|
eu07->MainSwitch(true);
|
|
trace.mark("main switch"); // shows in the event column of the row
|
|
eu07.run(30.0);
|
|
trace.check();
|
|
```
|
|
|
|
- `EU07_UNIT_TESTS_UPDATE_GOLDEN=1` records the samples and the key figures, for a new test or a
|
|
change which alters the behaviour on purpose; the diff of the key figures shows what it altered,
|
|
review it like code.
|
|
- Without `EU07_UNIT_TESTS_GOLDEN_EXACT=1`, samples recorded are compared within a relative 1e-9, as
|
|
GCC and glibc's maths functions can differ from MSVC's in the last bits, and a trace without them
|
|
passes. With it they're compared exactly, and a trace without them or its key figures fails, so a
|
|
run of the refactoring check can't pass by having nothing to compare.
|
|
- Keep the runs short (tens of seconds): the simulation feeds back on itself, and a last bit grows.
|
|
- A trace with no assertion about what the behaviour should be is a characterisation, bugs included;
|
|
put the behaviour the test is about into `CHECK`s besides.
|
|
|
|
### Key figures
|
|
|
|
Next to its samples, every trace records the figures a person reads (`figures.h`), for each part of
|
|
the run between two events, in `data/golden/<name>.figures`: the duration, the speeds, the distance,
|
|
the time to stop, the brake pipe's lowest pressure and the time to get back to 4.9 bar, the peaks of
|
|
the brake cylinders, the motor current and the tractive force.
|
|
|
|
```
|
|
[brake]
|
|
duration = 60.00 s
|
|
speed at the start = 118.5 km/h
|
|
distance = 944.3 m
|
|
stopped after = 45.97 s
|
|
brake cylinders peak = 3.997 bar
|
|
```
|
|
|
|
They're compared within 0.5% or the last decimal written, loose enough for any compiler, and the
|
|
differences are told by part and figure:
|
|
|
|
```
|
|
key figures of "traction/4e_v1/4e" differ:
|
|
[brake] distance: 944.3 m now, 980.0 m recorded
|
|
```
|
|
|
|
`EU07_UNIT_TESTS_UPDATE_GOLDEN=1` records them with the samples. After a change which alters the
|
|
behaviour, their comparison says in a few lines what the thousands of changed samples mean. A trace
|
|
sampled by hand, at times of its own (the commands and switches tests), records none.
|
|
|
|
### Invariants
|
|
|
|
Every trace also checks, on every step of its vehicle and not only on the recorded ones, the rules
|
|
which hold for any vehicle whatever a test does (`invariants.h`): no NaN, the brake cylinders under
|
|
the vehicle's highest pressure, the pipes under the highest governor setting, no motor current with
|
|
the line breaker open, no tractive force without power, the distance counted as the speed runs, no
|
|
acceleration over 5 g, and a few more. They compare nothing with a recorded file, so they hold on
|
|
any compiler, and they still check a behaviour recorded anew on purpose makes sense. A broken rule
|
|
fails the test, naming the time, the last event and the values:
|
|
|
|
```
|
|
invariant "no motor current with the line breaker open" broken in traction/et42 fuse/0 at 63.200 s,
|
|
after "dynamic brake": Mains=0 Im=-412.5 Itot=-412.5
|
|
```
|
|
|
|
Where a test makes a vehicle break a rule (a collision), or a game vehicle breaks one as it is, the
|
|
list of excuses in `invariants.cpp` names the trace, the rule and the reason; the second kind goes
|
|
into the findings of [COVERAGE.md](COVERAGE.md) too.
|
|
|
|
## Code to test, and when the tests fail to link
|
|
|
|
A source joins the tested code by moving from `SOURCES` to `CORE_SOURCES` in the main
|
|
`CMakeLists.txt`; the game links it from the library from then on. The library reaches for a little
|
|
of the rest of the engine, which `engine_stubs.cpp` stands in for in the tests (logs, settings,
|
|
scenery layers). If a change makes it use something new, either move the real source into the
|
|
library too, when it stands on its own, or add a stub to `engine_stubs.cpp`.
|