Both waited for every vehicle of the scenery to stand still, so on a scenery
with AI traffic a test could pass only when the traffic happened to stand too,
and the standstill end hardly ever came.
A test scenery names the train under test with test.vehicle <name> in its
config block (once for each train, if several): the pass and the standstill wait
for the consists of these vehicles as they are at the moment, so a train the
tested one couples to counts with it. A name the scenery doesn't have ends the
test with the error code. Without test.vehicle every vehicle counts, as before.
A signal passed at danger counts for every train either way.
run.py puts the case's lead vehicle into test.vehicle. test_mode_standstill_traffic
has the train under test standing on track 2 of Stacja1 while AI traffic leaves
track 1 on the timetable: it ends on its 100 s standstill, where counting every
vehicle runs into the 300 s timeout. Physicslogs of the other trains of a case
(<something>_<vehicle>) are moved out of the game dir too.
A test run that hung, in the loading or anywhere else, waited until whoever
started it killed it. A watchdog thread now ends a run still going test.wall_timeout
seconds of real time after the start of the simulator, loading included: it says
`test: timeout ... still running after N s of real time` and quits with the
timeout code, whatever the main thread is doing, with no dialog. A test that has
ended but whose simulator doesn't close ends with its own result.
The limit is test.wall_timeout (settings, the scenery's config block) or
--test-wall-timeout, which wins. Not given, it is test.timeout + 600 s: ten
minutes for the loading, and a simulation that runs at least as fast as real
time is never cut short. test.timeout 0 means no watchdog unless the limit is
given; test.wall_timeout 0 turns it off.
run.py passes the case's timeout as --test-wall-timeout and kills a simulator
only 30 s after it. A timeout from the watchdog counts as wall_timeout there, so
a run that hung doesn't pass a case expecting the simulation time to run out;
test_mode_wall_timeout ends while loading on a limit of 2 s.
The config block of a scenery is read like the settings files, after the test
mode settings were applied, so a scenery with soundenabled, multiplayer,
execonexit, feedbackmode or the like in it got them back in a test run. The test
mode settings now are applied again after each config block of the scenery;
apply_test_mode_settings() moved to simulationtest.cpp for that.
The command line options of the test mode and the end conditions checked after
each frame are now plain functions in simulation/simulationtestrules.cpp, which
needs nothing of the running simulation: parse_test_command_line() and
evaluate_frame(). The file is part of the core library, so the unit tests check
them (tests/unit/test_test_mode.cpp): the --test-timeout value, values of other
options not taken for options, a pass that waits for the stop, the standstill,
the timeout. test_manager only gathers what a frame brought and acts on the
verdict. No change in behaviour.
- `--test` is looked for before the rest of the command line is read. An
unknown option before it (e.g. `-s x.scn --test-timout 90 --test`) used
to end the run with exit code 0, which a caller reads as a pass.
- `--test-timeout` without a value, or with one that isn't a number of
seconds, fails the run instead of leaving the default or, through atof,
turning the timeout off.
- These, a missing -s, a failed start and an exception caught in main()
end the test through test_manager::conclude(), which writes the usual
`test: error (exit code 5) ... : <reason>` line to stdout and the log.
Before, `eu07 --test` alone exited with 5 and nothing anywhere: the
error went to the log queue and the process ended before the log
thread wrote it. In test mode main() now stops the log thread and
waits for it before _Exit().
Outside the test mode nothing changes: conclude() does nothing there.
- TController::TableCheck() calls TSpeedPos::Update() as it did before the
test mode: the check for a signal passed at danger is a call of its own
after it (CheckSignalPassedAtDanger), so the update lines nest no deeper
than on master (S134). A switch is a track, so a signal's update never
takes the switch-change branch and the check runs in the same place as
before; the physicslogs of all 15 scenario cases are byte for byte the
same.
- null_imgui_renderer is final (S1235), to_string(test_result) uses
`using enum` (S6177), init_result_cell() is const (S5817),
ConfigParseTest() takes a std::string_view (S6009) and the test settings
struct is declared apart from its member (S3646).
The config key from eu07/maszyna#541. By default (yes) a pass reported to
__test.result waits until every vehicle has stood still for 1 s; with no it
ends the test at once. The key is listed with the other test.* settings in the
log and in the test start line.
Scenario tests: new case test_mode_pass_while_moving reports the pass from
event2 of tor1k as the train runs onto it and checks the run ended with the
train still moving (new check ends_moving). The README lists the test.* keys
with their defaults and the input the test mode ignores.
The test manager is reached through simulation::test() instead of a global,
the result cell is owned by a unique_ptr until the memory table takes it, the
cell value is compared with a margin, strings are built with std::format and
the --test settings moved to apply_test_mode_settings(). The driver's signal
check is folded into UpdateSpeedPosition(), which updates a speed table entry
and reports a signal at stop the head of the train just passed. The test.*
keys are listed with the other settings at the start of the log.
eu07 --test -s <scenery> [--test-timeout <s>] runs the scenery in ghostview with
the null renderer (models loaded, no textures), a hidden window, no sound,
network or python, and in fixed steps of 0.03 s of simulation time per frame
(three 10 ms physics steps) as fast as it can. Without -seed it uses a fixed
seed, so two runs give byte-identical physicslogs.
The scenery reports its result to the __test.result memory cell, which the
simulator always creates: value 1 passes once every vehicle has stood still for
1 s, value 0 fails at once. The run ends with exit code
0 pass, 1 fail,
2 a train passed a signal at stop (SetVelocity 0 0, as seen in its driver's
speed table; SetVelocity 0 x on automatic block doesn't count),
3 timeout (test.timeout, default 3600 s of simulation time),
4 standstill (every vehicle still for test.standstill s, default 300, with no
result; a pass with test.pass_on_standstill yes),
5 error (no -s, the scenery didn't load or has no vehicles, an exception).
The test.* keys go in the scenery's config block; test.fail_if_stop_overran no
only logs a signal passed at danger. Outside the test mode a signal passed at
danger is logged.
The null renderer gets a ui backend which draws nothing, hands out the null
texture and material instead of throwing, and ends the ui frame. A GL context
is still made for the hidden window, without asking for a version, as parts
of the ui create textures with gl calls of their own. No crash dialog or
message box in test mode; the window code no longer needs a monitor.
