* fix: a failed start says why in the log and ends with an error code
When the window could not be made, the log only said "Bad init: failed to
create glfw window", without the reason glfw knows (no OpenGL driver, no
pixel format that fits, no EGL). Sometimes it did not even say that: the
log thread writes every 50 ms, and main() ended the process with
std::_Exit(0) without stopping it, so a start that failed within that time
left a log that ends at "// startup".
- glfw's errors go to the log and errors.txt through glfwSetErrorCallback,
set before glfwInit(); each distinct message once.
- after a failed start (or an exception caught in main) the log thread is
stopped and waited for, as the test runs already do, so nothing logged
last is lost.
- a failed start ends with exit code 1 instead of 0, so the starter can
tell it from a closing. A start that worked ends as before.
* fix: glfw's errors told during the start only, a closed crash dialog is no failed start
After review of the previous commit:
- the glfw error callback is taken off at the end of init(). Once the
simulator runs, glfw would only repeat what the platform can't do
(glfwSetCursorPos on Wayland, on every mouse-look) or what a scenery
asks for in vain (launcher key codes below GLFW_KEY_SPACE, checked on
every update), which would put new lines in errors.txt of existing
sceneries and cost a format and a lock per call.
- closing the crash-report window ends the simulator as before, but
init() now returns 1 for it (above 0: closed by the player), and only a
result below 0 gives exit code 1.
- a start that throws from init() (bad_alloc, or a runtime_error on
Windows) ends with exit code 1 too; an exception after a start that
worked ends with 0 as before.
* feat: a small crash dump by default, the full one with "crashdump.full yes"
The dump written by the crash handler (Windows, WITHDUMPGEN) held all the
memory of the simulator (MiniDumpWithFullMemory), so it was as large as
the memory the simulator took: gigabytes on a large scenery, written to
the game folder at the moment of the crash, on machines with 8 GB of RAM
too.
By default it now holds the stacks with the memory they point to, the
threads, the modules, the handles, the memory map and the modules' data
segments (MiniDumpWithDataSegs in place of MiniDumpWithFullMemory, the
rest of the flags as before). The globals, Global among them, stay
readable. "crashdump.full yes" in eu07.ini writes the full dump as before.
Measured on td.scn with a ghostview camera (1.2 GB committed): the small
dump 44 MB in 2 s, the full one 1.6 GB in 19 s.
The key is new and optional: eu07.ini files without it read as before,
and the setting is listed in the log at start.
* fix: the small crash dump keeps the process and thread blocks
MiniDumpWithProcessThreadData adds the PEB and the TEBs, a few KB: the
command line (the scenery and the vehicle), the environment and the last
error of each thread, which the small dump otherwise lacks, since the heap
behind Global's longer strings is not in it.
* fix: an exception that ends the simulator says what it was in the log
After review of #425:
- the exceptions main() catches are logged ("Critical error: ..."): on
Windows the runtime_error that only showed a message box, and on the
other systems any std::exception, which used to end the simulator
through std::terminate with no word in the log. On Windows the others
are still left to CrashHandler, whose dump shows where they were
thrown.
- every exception caught there ends with exit code 1, in the start or
later, as a crash; it used to be 0 when caught, the abort code when not.
- closing the crash-report window logs "Crash report window closed,
closing" instead of the error "Failed to run crash gui!", which stays
for a result below 0.
* feat: the stack of a crash on Linux in the log and on the standard error
On Linux no dump is written (the crash handler is Windows only, crashpad
is off), so a crash left a log that just stopped.
A handler of SIGSEGV, SIGBUS, SIGFPE, SIGILL and SIGABRT (installed at
the start of main) now writes, with write() only:
Crash: signal 11 (SIGSEGV) at address 0x...
Version: EU07 (cmake), <hash> (...)
Stack (addr2line -f -C -e <the eu07 file of this version> <offset>):
<backtrace_symbols_fd>
to the end of the log file (its whole path, set when the log opens it)
and to the standard error, which StarterNG shows first. The Linux builds
carry their symbols (GENERATE_PDB adds -g, nothing strips them), so
addr2line on the eu07 file of the same version gives the files and lines.
Then the signal ends the process as before (SA_RESETHAND, raise): the
same exit status, and a core dump where the system makes one. The
handler runs on an alternate stack, so an overflow of the main thread's
stack is reported too; backtrace() is warmed up at install so it doesn't
allocate in the handler, and alarm(5) ends a handler that hangs (a crash
in the allocator or the loader). WAMR is built without its hardware bound
checks, so it installs no SIGSEGV handler of its own.
Elsewhere crash_signals does nothing. A unit test (Linux only) crashes a
child process and checks the report on its standard error and in the log,
and that the child still ended by SIGSEGV.
* fix: the crash report can't be cut by a second crash signal or end with SIGALRM
After review:
- the five crash signals are blocked while a report is written, so a
second one waits rather than starting a second report in its middle
(a fault that comes while blocked still ends the process).
- a report that hangs past alarm(5) ends the process by the signal of
the crash, not by SIGALRM: a SIGALRM handler, set only in the crash
handler, raises the crash signal and unblocks it.
- the child of the unit test calls alarm(10) first, so a child stuck
before the crash can't hang the build.
---------
Co-authored-by: maj00r <maj00r@host>
A failed write put its lines back, ahead of those since, so a file held by
another program for a while gets its header and every line once it can be
written; up to 4 MB kept per log. Stopping is joined once, and not from the
writer's own thread. main stops the logs after an error it caught too, and the
test watchdog flushes with a time limit, so a disk that doesn't answer doesn't
keep the run from ending.
"physicslog yes" kept the .dat of every vehicle with a driver open for the whole run and flushed it twenty
times a second from the simulation's thread: some 580 open files on Tomaszewo. The lines are kept now and a
thread of their own writes them every two seconds, each file opened only for the moment it is written to.
The files, their lines and their text mode stay as they were.
- The black box, with the physics logs off, writes the tapes of the user's vehicles through the same
writer as it goes, not at the end. A log begins when the user is first in the vehicle and goes on
while the user looks around outside, until another vehicle is taken; back in it, the log goes on after
the gap. The vehicle's own driver keeps the log only if it is the primary one, as a passive one's clock
stands still; otherwise the driver of its train does.
- What is kept is written when the simulator closes, after the black box, at the end of a test run, and
when the test watchdog ends a run. A crash loses the last two seconds.
- On Tomaszewo with the black box alone, the one tape grows every two seconds. With "physicslog yes", the
581 logs grow and the simulator holds as many handles as without them; writing them takes the thread
about 1.9 s, which the closing waits for once.
With "physicslog yes" every driver keeps its .dat open, and a large scenery (Tomaszewo: some 500 vehicles,
cars of the roads included) took all 512 streams the MSVC runtime gives: the next fopen failed and the
simulator stopped at "cannot read shader: shaders/map.vert" on opening the map. The limit is raised to
8192 at the start (the soft limit of open files up to the hard one on Linux), and a shader file that
doesn't open is told as such, with the system's reason, rather than as one that can't be read.
main() tells a test run (--test) apart before anything runs. Only a test run
catches every exception and installs a std::terminate handler, both ending the
run with the error code and a test: line; any other run keeps master's
exception handling, so on Linux an uncaught runtime_error aborts at the throw
site again.
A test run doesn't start the Discord RPC thread: it published the test
scenery as the player's presence and held the closing up to 5 s.
run.py drops a case scenery's .sbt cache after a run cut short (wall_timeout,
error, a crash, killed by run.py), which could have been writing it.
The -edit branch of the test command line parser had no effect (a token that
isn't an option is ignored either way) and is gone; its test checks what can
be seen instead.
The runtime_error the simulator throws on a fatal problem was caught on Windows
only. On Linux a test run aborted on it, with the code of abort() and no
`test:` line. It is caught there too in test mode and ends the test with the
error code and the reason; outside a test it is thrown on as before.
The crash handler of WITHDUMPGEN builds (on by default) shows a message
box before it writes the dump. It also gets the C++ exceptions nothing
catches, since it replaces the CRT's unhandled exception filter. In test
mode nobody is there to close the box, so a crashing test waited for it
until whatever started it gave up (run.py after the case's timeout,
600 s by default). The dump is still written, the exit code is the
exception's.
- `--test` is looked for before the rest of the command line is read. An
unknown option before it (e.g. `-s x.scn --test-timout 90 --test`) used
to end the run with exit code 0, which a caller reads as a pass.
- `--test-timeout` without a value, or with one that isn't a number of
seconds, fails the run instead of leaving the default or, through atof,
turning the timeout off.
- These, a missing -s, a failed start and an exception caught in main()
end the test through test_manager::conclude(), which writes the usual
`test: error (exit code 5) ... : <reason>` line to stdout and the log.
Before, `eu07 --test` alone exited with 5 and nothing anywhere: the
error went to the log queue and the process ended before the log
thread wrote it. In test mode main() now stops the log thread and
waits for it before _Exit().
Outside the test mode nothing changes: conclude() does nothing there.
The test manager is reached through simulation::test() instead of a global,
the result cell is owned by a unique_ptr until the memory table takes it, the
cell value is compared with a margin, strings are built with std::format and
the --test settings moved to apply_test_mode_settings(). The driver's signal
check is folded into UpdateSpeedPosition(), which updates a speed table entry
and reports a signal at stop the head of the train just passed. The test.*
keys are listed with the other settings at the start of the log.
eu07 --test -s <scenery> [--test-timeout <s>] runs the scenery in ghostview with
the null renderer (models loaded, no textures), a hidden window, no sound,
network or python, and in fixed steps of 0.03 s of simulation time per frame
(three 10 ms physics steps) as fast as it can. Without -seed it uses a fixed
seed, so two runs give byte-identical physicslogs.
The scenery reports its result to the __test.result memory cell, which the
simulator always creates: value 1 passes once every vehicle has stood still for
1 s, value 0 fails at once. The run ends with exit code
0 pass, 1 fail,
2 a train passed a signal at stop (SetVelocity 0 0, as seen in its driver's
speed table; SetVelocity 0 x on automatic block doesn't count),
3 timeout (test.timeout, default 3600 s of simulation time),
4 standstill (every vehicle still for test.standstill s, default 300, with no
result; a pass with test.pass_on_standstill yes),
5 error (no -s, the scenery didn't load or has no vehicles, an exception).
The test.* keys go in the scenery's config block; test.fail_if_stop_overran no
only logs a signal passed at danger. Outside the test mode a signal passed at
danger is logged.
The null renderer gets a ui backend which draws nothing, hands out the null
texture and material instead of throwing, and ends the ui frame. A GL context
is still made for the hidden window, without asking for a version, as parts
of the ui create textures with gl calls of their own. No crash dialog or
message box in test mode; the window code no longer needs a monitor.