18
0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-11-19 22:44:59 +01:00

test(scenarios): the coupler control case checks that the units coupled

The checks of en57_2000_automatic_coupler_couples said only that nothing
collided nor derailed, so the case passed just as well with the units
buffering, bouncing apart or never meeting.

The simulator writes nothing to the log when automatic couplers catch,
but the black box of the run (physicslog/<scenery>_<date>.blackbox, written
along with the physicslogs the runner turns on) gets a coupling record with
the two vehicles. run.py reads the vehicle and the coupling records of it
and gets a check on them, couples: the two named vehicles couple after the
start and are still coupled at the end. It fails when the black box has no
coupling of that pair, when they came apart again, when they were coupled
from the start, and when the run left no black box.

The case names the vehicles that meet: the last car of the standing unit
and the first of the rolling one. The black box is moved to out/<case>/
with the other files of the run.
This commit is contained in:
jerrrrycho
2026-11-11 21:25:19 +01:00
parent 85119cc110
commit 75c8ac6149
3 changed files with 146 additions and 7 deletions

View File

@@ -142,8 +142,8 @@ Python 3.10+, no extra packages. Options:
watching what a case does.
Exit code is 0 when every case passed. Logs of each run (`log.txt`, `errors.txt`,
`stdout.txt`, the physicslogs of the vehicle and of other trains of the case) are moved to
`tests/scenarios/out/<case>/`.
`stdout.txt`, the physicslogs of the vehicle and of other trains of the case, the black box as
`autotest_<case>.blackbox`) are moved to `tests/scenarios/out/<case>/`.
## How it works
@@ -157,14 +157,15 @@ For each case the runner:
2. starts the simulator with `--test -s autotest_<case>.scn -ini out/<case>/autotest.ini
--test-wall-timeout <timeout>`, plus `--test-timeout <duration>` when the case gives a
`duration`; the extra ini turns on
`physicslog` and writes the logs to `logs/log (autotest_<case>) <date>.txt` rather than the
`physicslog` (and with it the black box of the run, `physicslog/autotest_<case>_<date>.blackbox`)
and writes the logs to `logs/log (autotest_<case>) <date>.txt` rather than the
shared `log.txt` (`multiplelogs`),
3. waits until the simulator quits by itself. It gives up early when the log reports
`Bad vehicle` / `Bad trainset`, or when the scenario has loaded and no physicslog appeared.
After the case's `timeout` of wall-clock time the simulator ends itself with the timeout code;
one that doesn't is killed 30 s later,
4. compares the result from the exit code with the case's `expect`, then runs the checks on
the physicslog.
the physicslog, the log and the black box.
The lead vehicle is named `autotest_<case>`, so every case has its own physicslog and log
files and cases can run in parallel. The `autotest_*` sceneries (and the simulator's `.sbt`
@@ -224,7 +225,10 @@ The `en57_2000_automatic_coupler_*` cases (issue #86) share `td_automatic_couple
EN57-2000 unit without a driver runs at `vars.velocity` km/h into a standing one, from
`35.6 - vars.offset` m behind it, with `crashdamage yes` in `ini`. At 50 km/h the automatic
couplers mustn't catch and the log has to show the collision and the derailment; at 4 km/h they
couple and neither shows up. Nothing reports a result, so both end on the 60 s test timeout.
couple and neither shows up: the black box has to have the last car of the standing unit,
`{{vehicle}}_rb`, and the first of the rolling one, `{{vehicle}}_hit_ra`, coupled (the simulator
writes nothing to the log when couplers catch). Nothing reports a result, so both end on the 60 s
test timeout.
Checks (speeds in km/h, times in seconds of physicslog time, distances in metres). Most use
the physicslog Velocity column, which is the wheel speed: it stays at 0 while the wheels are
@@ -254,7 +258,16 @@ the row instead, logged by builds from this branch on.
- `log_contains` `pattern`: a line of the run's `log.txt` matches the regex, e.g. the
`Bad driving: ... collided` message of a collision.
- `log_lacks` `pattern`: no line of the run's `log.txt` matches the regex.
- `couples` `vehicle`, `other`: the black box of the run has the two vehicles coupled to each
other (the coupler itself, whatever else of the hoses and the cables) at some time after the
start, and still coupled at the end. The vehicles go by their node names, with `{{vehicle}}`
filled in as in the fragment, e.g. `{{vehicle}}_rb`. Fails when the two were coupled from the
start, came apart again, or the run left no black box. The black box is looked at once a
second, so the time given is that of the first look after the couplers caught.
New check types go in `CHECKS` in `run.py`; each gets the physicslog rows and the check's
parameters and returns `(passed, detail)`. Checks on the log go in `LOG_CHECKS` and get the
text of `log.txt` instead of the rows.
text of `log.txt` instead of the rows. Checks on the black box (`blackbox_doc.md` in the root of
the repo; the simulator writes it along with the physicslogs) go in `BLACKBOX_CHECKS` and get
what `read_blackbox()` read of it: the names of the vehicles and the coupling records. A check
on another kind of record has to add it there.

View File

@@ -1,5 +1,5 @@
{
"description": "Issue #86, the control case: an EN57-2000 unit without a driver rolls at 4 km/h into a standing one on td.scn. Their automatic couplers catch, nothing collides nor derails; nothing reports a result, so the run ends on the 60 s test timeout.",
"description": "Issue #86, the control case: an EN57-2000 unit without a driver rolls at 4 km/h into a standing one on td.scn. Their automatic couplers catch: the black box of the run has the last car of the standing unit and the first of the rolling one coupled. Nothing collides nor derails; nothing reports a result, so the run ends on the 60 s test timeout.",
"base_scenery": "td.scn",
"ini": ["crashdamage yes"],
"fragment": "td_automatic_coupler_hit.scn",
@@ -11,6 +11,7 @@
"duration": 60,
"timeout": 300,
"checks": [
{ "type": "couples", "vehicle": "{{vehicle}}_rb", "other": "{{vehicle}}_hit_ra" },
{ "type": "log_lacks", "pattern": "collided" },
{ "type": "log_lacks", "pattern": "derailed" }
]

View File

@@ -31,6 +31,7 @@ import json
import os
import re
import shutil
import struct
import subprocess
import sys
import time
@@ -230,6 +231,81 @@ def read_text(path: Path | None) -> str:
return ""
# the black box of a run, see blackbox_doc.md: written along with the physicslogs, to
# physicslog/<scenery>_<date>.blackbox. A header, then records: a byte of the kind, the length, the content
BLACKBOX_MAGIC = b"EU07BBX\0"
BLACKBOX_HEADER = 12 # the magic, the version of the format, two bytes reserved
BLACKBOX_VEHICLE = 0x12 # a vehicle: its number and its name
BLACKBOX_COUPLING = 0x37 # a coupler of a vehicle coupled, uncoupled, or its hoses and cables changed
COUPLING_COUPLER = 0x01 # of the bits telling what is coupled: the coupler itself
@dataclass
class Coupling:
"""A coupling record: what a coupler of a vehicle is coupled to from `time` on."""
time: float
vehicle: int # vehicles by their numbers, 0 for none
side: int # 0 the front coupler, 1 the rear one
other: int
other_side: int
flags: int # what is coupled, 0 nothing
previous_flags: int
previous_other: int
@dataclass
class Blackbox:
vehicles: dict[int, str] = field(default_factory=dict) # names by number
couplings: list[Coupling] = field(default_factory=list) # in the order written
def find_blackbox(game_dir: Path, scenery: str, since: float) -> Path | None:
"""The black box of one run: the simulator names it after the scenery and the time it started."""
stem = re.sub(r"[^0-9A-Za-z_-]", "_", scenery.rsplit(".", 1)[0])
# the whole name, so a case whose name another one starts with doesn't take that one's file
name = re.compile(re.escape(stem) + r"_\d{4}-\d\d-\d\d_\d\d-\d\d-\d\d\.blackbox")
files = [p for p in (game_dir / "physicslog").glob(f"{stem}_*.blackbox")
if name.fullmatch(p.name) and p.stat().st_mtime >= since]
return max(files, key=lambda p: p.stat().st_mtime, default=None)
def read_blackbox(path: Path | None) -> Blackbox | None:
"""The vehicles and the coupling records of a black box; None when there is none or it can't be read."""
try:
data = path.read_bytes() if path else b""
except FileNotFoundError:
return None
if data[:len(BLACKBOX_MAGIC)] != BLACKBOX_MAGIC:
return None
box = Blackbox()
at = BLACKBOX_HEADER
try:
while at + 5 <= len(data):
kind, length = struct.unpack_from("<BI", data, at)
content = data[at + 5: at + 5 + length]
if len(content) < length:
break # a record still being written when the run ended
at += 5 + length
if kind == BLACKBOX_VEHICLE:
number, = struct.unpack_from("<I", content)
# a text: its length in 7 bit groups, low first, the high bit set on all but the last
size, shift, start = 0, 0, 4
while True:
byte = content[start]
start += 1
size |= (byte & 0x7F) << shift
shift += 7
if byte < 0x80:
break
box.vehicles[number] = content[start: start + size].decode("utf-8", errors="replace")
elif kind == BLACKBOX_COUPLING:
# the position of the coupler follows
box.couplings.append(Coupling(*struct.unpack_from("<dIBIBBBI", content)))
except (struct.error, IndexError):
return None # a record shorter than its kind is
return box
# load problems that leave the case without its vehicle; no point waiting for the timeout
FATAL_LOG_LINES = ("Bad vehicle:", "Bad trainset:")
# wall seconds from start to the first physicslog row before giving up. "Scenario loading
@@ -248,6 +324,7 @@ class RunResult:
failure: str = "" # why the run was cut short
wall_seconds: float = 0.0
logs: dict[str, Path] = field(default_factory=dict)
blackbox: Path | None = None
def run_simulator(exe: Path, game_dir: Path, scenery: str, ini: Path, physicslog: Path,
@@ -313,6 +390,7 @@ def run_simulator(exe: Path, game_dir: Path, scenery: str, ini: Path, physicslog
stdout.close()
result.rows = read_physicslog(physicslog)
result.logs = find_logs(game_dir, scenery, since)
result.blackbox = find_blackbox(game_dir, scenery, since)
return result
@@ -499,6 +577,40 @@ LOG_CHECKS = {
}
def check_couples(box, vehicle, other):
"""The two vehicles couple in the run and stay coupled to its end; coupled from the start doesn't count."""
numbers = {name: number for number, name in box.vehicles.items()}
unknown = [name for name in (vehicle, other) if name not in numbers]
if unknown:
return False, f"the black box has no vehicle {' nor '.join(unknown)}"
pair = {numbers[vehicle], numbers[other]}
# of the two vehicles of a coupling the one with the lower number tells it, each change of that coupler
# under the same vehicle and side: the record which coupled the two, for as long as they stay coupled
made, parted = {}, None
for c in box.couplings:
coupler = (c.vehicle, c.side)
if {c.vehicle, c.other} == pair and c.flags & COUPLING_COUPLER:
made.setdefault(coupler, c)
elif coupler in made:
parted = (made.pop(coupler), c)
if not made:
if parted:
return False, f"{vehicle} and {other} coupled at t={parted[0].time:.1f}s but came apart at t={parted[1].time:.1f}s"
return False, f"none of the {len(box.couplings)} coupling record(s) of the black box couples {vehicle} and {other}"
c = min(made.values(), key=lambda c: c.time)
if c.time <= 0:
return False, f"{vehicle} and {other} were coupled when the recording started"
ends = ("front", "rear")
return True, (f"at t={c.time:.1f}s the {ends[c.side]} coupler of {box.vehicles[c.vehicle]} coupled to the "
f"{ends[c.other_side]} one of {box.vehicles[c.other]}, coupling 0x{c.flags:02X}")
# checks on the black box of the run rather than on the physicslog rows
BLACKBOX_CHECKS = {
"couples": check_couples,
}
# ---------------------------------------------------------------- main
@@ -560,6 +672,9 @@ def run_case(name: str, exe: Path, game_dir: Path, timespeed: float, watch: bool
# other trains of the case with a driver, named <something>_<vehicle>, log too
for other in physicslog.parent.glob(f"*_{vehicle}.dat"):
shutil.move(other, out / other.name)
# the black box is named after the case scenery too
if result.blackbox:
result.blackbox = Path(shutil.move(result.blackbox, out / f"{vehicle}.blackbox"))
ok = True
sim_t = result.rows[-1][COL_TIME] if result.rows else 0.0
@@ -592,6 +707,7 @@ def run_case(name: str, exe: Path, game_dir: Path, timespeed: float, watch: bool
return False
log = read_text(out / "log.txt")
box = read_blackbox(result.blackbox)
for check in case.get("checks", []):
params = {k: v for k, v in check.items() if k != "type"}
if check["type"] in LOG_CHECKS:
@@ -600,6 +716,15 @@ def run_case(name: str, exe: Path, game_dir: Path, timespeed: float, watch: bool
passed, detail = LOG_CHECKS[check["type"]](log, **params)
else:
passed, detail = False, "log.txt is missing or empty"
elif check["type"] in BLACKBOX_CHECKS:
if box is not None:
# vehicles are named as in the fragment
params = {k: v.replace("{{vehicle}}", vehicle) if isinstance(v, str) else v for k, v in params.items()}
passed, detail = BLACKBOX_CHECKS[check["type"]](box, **params)
elif result.blackbox:
passed, detail = False, f"{result.blackbox.name} is not a black box, or is damaged"
else:
passed, detail = False, "the run left no black box (physicslog/*.blackbox); the build is too old, or has blackbox no"
else:
passed, detail = CHECKS[check["type"]](result.rows, **params)
say(f" {'PASS' if passed else 'FAIL'} {check['type']}: {detail}")