The checks of en57_2000_automatic_coupler_couples said only that nothing
collided nor derailed, so the case passed just as well with the units
buffering, bouncing apart or never meeting.
The simulator writes nothing to the log when automatic couplers catch,
but the black box of the run (physicslog/<scenery>_<date>.blackbox, written
along with the physicslogs the runner turns on) gets a coupling record with
the two vehicles. run.py reads the vehicle and the coupling records of it
and gets a check on them, couples: the two named vehicles couple after the
start and are still coupled at the end. It fails when the black box has no
coupling of that pair, when they came apart again, when they were coupled
from the start, and when the run left no black box.
The case names the vehicles that meet: the last car of the standing unit
and the first of the rolling one. The black box is moved to out/<case>/
with the other files of the run.
Re-lands #288, which was merged into feat/exe-test-mode after that
branch had already gone to master, so master never got it.
Two EN57-2000 units on td.scn, one standing, the other rolling into it
without a driver. At 50 km/h their automatic couplers mustn't catch: the
log has to show the collision and the derailment. At 4 km/h they couple
and nothing collides. Nothing reports a result, so both cases end on the
60 s test timeout.
run.py gets log_contains / log_lacks checks on the run's log.txt, and an
"ini" case key for extra eu07.ini lines (here crashdamage yes). The log
checks fail when log.txt is missing or empty: with no log text log_lacks
would pass without checking anything.
main() tells a test run (--test) apart before anything runs. Only a test run
catches every exception and installs a std::terminate handler, both ending the
run with the error code and a test: line; any other run keeps master's
exception handling, so on Linux an uncaught runtime_error aborts at the throw
site again.
A test run doesn't start the Discord RPC thread: it published the test
scenery as the player's presence and held the closing up to 5 s.
run.py drops a case scenery's .sbt cache after a run cut short (wall_timeout,
error, a crash, killed by run.py), which could have been writing it.
The -edit branch of the test command line parser had no effect (a token that
isn't an option is ignored either way) and is gone; its test checks what can
be seen instead.
Both waited for every vehicle of the scenery to stand still, so on a scenery
with AI traffic a test could pass only when the traffic happened to stand too,
and the standstill end hardly ever came.
A test scenery names the train under test with test.vehicle <name> in its
config block (once for each train, if several): the pass and the standstill wait
for the consists of these vehicles as they are at the moment, so a train the
tested one couples to counts with it. A name the scenery doesn't have ends the
test with the error code. Without test.vehicle every vehicle counts, as before.
A signal passed at danger counts for every train either way.
run.py puts the case's lead vehicle into test.vehicle. test_mode_standstill_traffic
has the train under test standing on track 2 of Stacja1 while AI traffic leaves
track 1 on the timetable: it ends on its 100 s standstill, where counting every
vehicle runs into the 300 s timeout. Physicslogs of the other trains of a case
(<something>_<vehicle>) are moved out of the game dir too.
A test run that hung, in the loading or anywhere else, waited until whoever
started it killed it. A watchdog thread now ends a run still going test.wall_timeout
seconds of real time after the start of the simulator, loading included: it says
`test: timeout ... still running after N s of real time` and quits with the
timeout code, whatever the main thread is doing, with no dialog. A test that has
ended but whose simulator doesn't close ends with its own result.
The limit is test.wall_timeout (settings, the scenery's config block) or
--test-wall-timeout, which wins. Not given, it is test.timeout + 600 s: ten
minutes for the loading, and a simulation that runs at least as fast as real
time is never cut short. test.timeout 0 means no watchdog unless the limit is
given; test.wall_timeout 0 turns it off.
run.py passes the case's timeout as --test-wall-timeout and kills a simulator
only 30 s after it. A timeout from the watchdog counts as wall_timeout there, so
a run that hung doesn't pass a case expecting the simulation time to run out;
test_mode_wall_timeout ends while loading on a limit of 2 s.
The simulator reads eu07.ini from %APPDATA%/MaSzyna (~/.config/MaSzyna on Linux)
when there is one, in place of the game folder's. A test run then depended on
whose machine it ran on. With --test only the game folder's eu07.ini is read,
then the files given with -ini. The test mode options are taken from the command
line before the settings for that; --test-timeout still wins over test.timeout
of any settings file.
1 is what a process killed with TerminateProcess (and the plain failure of most
programs) ends with, 3 what abort() ends with on Windows, so a fail or a timeout
couldn't be told from a run that was killed or blew up. The codes other than
pass now are 10 fail, 11 signal passed at danger, 12 timeout, 13 standstill and
14 error: none of them a code a process gets on its own, neither the shells'
126-127 nor 128 + a signal number on Linux. pass stays 0.
Przystanek on td.scn is a technical stop of the td/rozklad timetable (pt): the AI
doesn't wait there for the departure time but sets off as soon as it counts the
train as standing at the W4, below 1 km/h. How long the speed then stays under
0.5 km/h is only how fast the brake releases. With the AI braking of #264 the
trains creep up to the W4 at about 10 km/h on a light application and stood there
for 1-5 s; the SN61 turns back at 0.36 km/h without stopping. The hold of 5 s
(2 s for the SN61) failed for EP09, SN61 and Vectron and had well under a second
to spare for others.
The cases now check that the train comes down to 1 km/h, the AI's own standstill
at a W4, 1380-1420 m from the start; every vehicle stops at 1397-1401 m, before
and after #264. `stops` takes min_distance and max_distance for that.
The config key from eu07/maszyna#541. By default (yes) a pass reported to
__test.result waits until every vehicle has stood still for 1 s; with no it
ends the test at once. The key is listed with the other test.* settings in the
log and in the test start line.
Scenario tests: new case test_mode_pass_while_moving reports the pass from
event2 of tor1k as the train runs onto it and checks the run ended with the
train still moving (new check ends_moving). The README lists the test.* keys
with their defaults and the input the test mode ignores.
run.py starts eu07 with --test (and --test-timeout <duration> for cases that
record a fixed time) and compares the result from the exit code with the
case's expect key, before the physicslog checks. The 14 cases take about 30 s
with -j 4 instead of minutes. --watch keeps the old windowed run.
New case keys: expect, test_config (lines of a config block), track_events
(lines added to a track of the base scenery, e.g. event0 test_pass). New cases
show each end of the test mode: test_mode_pass, _signal_passed_at_danger,
_timeout and _standstill. The slippery rail cases slide past the red signal,
which now ends a test, so they set test.fail_if_stop_overran no.
README documents the test mode for scenery authors.
- physicslog rows keep all columns (text ones as None), checks can read
the wheelslip flag and the new vehicle speed column
- slips: the lead vehicle's wheels slip at some point (makes sure a
slippery rail case does what it's meant to)
- slip_clears: once stopped, the wheels stop slipping within N seconds
- holds_still: once stopped, the vehicle doesn't creep, by vehicle speed
- td_red_signal_moving.scn: the red signal case starting already moving
- eu07_slippery_red_signal / eu07_slippery_push_red_signal: skid on
braking for the red signal with track friction 0.03; the pushing one
fails without the fix for eu07/maszyna#479 (stays skidding, creeps)
- sn61_td_timetable: stop hold 5 -> 2 s, the railcar arrives 5-7 s
before its departure time, so 5 s made the case flaky
- eu07_td_red_signal: no route set, the train has to stop before the exit
signal and stay there (new checks stays_stopped and distance; distance
is integrated from the speed, the physicslog has no position)
- en57/ep09/et22/sm42/sn61/vectron/eu07_td_timetable share one fragment
template, td_timetable.scn; a case sets vars.consist and the lead vehicle
is named {{vehicle}} = autotest_<case>
- -j N runs cases in parallel: each case has its own physicslog and, with
multiplelogs, its own log files, which are moved to out/<case>/
- a run stops at once on Bad vehicle / Bad trainset instead of waiting for
the timeout, and fails after 120 s without physicslog rows
- the .sbt geometry cache of a test scenery is dropped when the base
scenery is newer
All 8 cases pass in about a minute with -j 4.
eu07_td_timetable now takes about 20 s instead of 4 min. Its speed profile
at 20 and 60 matched a real-time run within a few km/h and seconds; the
physicslog keeps its 0.05 s rows. The runner polls the log every 0.25 s so
it does not overshoot the case duration by a minute of simulation.
tests/scenarios/run.py builds a test scenery from a game install's scenery
(own trainsets removed, case fragment added), runs eu07 in ghostview with
physicslog on, and checks what the vehicle did: starts, max speed, brakes,
stops. First case: EU07 on td.scn driven by AI on the td/rozklad timetable,
with the Stacja1 exit route set by an event launcher instead of shift+4.
eu07 gets a -ini <file> option: an extra settings file read after eu07.ini,
so a test run can switch to a window and turn on physicslog without touching
the user's config.
Local only: scenery and vehicle data are not in the repo.