18
0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-11-21 05:04:58 +01:00

fix: a failed start says why and ends with an error code; a small crash dump by default (#425)

* fix: a failed start says why in the log and ends with an error code

When the window could not be made, the log only said "Bad init: failed to
create glfw window", without the reason glfw knows (no OpenGL driver, no
pixel format that fits, no EGL). Sometimes it did not even say that: the
log thread writes every 50 ms, and main() ended the process with
std::_Exit(0) without stopping it, so a start that failed within that time
left a log that ends at "// startup".

- glfw's errors go to the log and errors.txt through glfwSetErrorCallback,
  set before glfwInit(); each distinct message once.
- after a failed start (or an exception caught in main) the log thread is
  stopped and waited for, as the test runs already do, so nothing logged
  last is lost.
- a failed start ends with exit code 1 instead of 0, so the starter can
  tell it from a closing. A start that worked ends as before.

* fix: glfw's errors told during the start only, a closed crash dialog is no failed start

After review of the previous commit:

- the glfw error callback is taken off at the end of init(). Once the
  simulator runs, glfw would only repeat what the platform can't do
  (glfwSetCursorPos on Wayland, on every mouse-look) or what a scenery
  asks for in vain (launcher key codes below GLFW_KEY_SPACE, checked on
  every update), which would put new lines in errors.txt of existing
  sceneries and cost a format and a lock per call.
- closing the crash-report window ends the simulator as before, but
  init() now returns 1 for it (above 0: closed by the player), and only a
  result below 0 gives exit code 1.
- a start that throws from init() (bad_alloc, or a runtime_error on
  Windows) ends with exit code 1 too; an exception after a start that
  worked ends with 0 as before.

* feat: a small crash dump by default, the full one with "crashdump.full yes"

The dump written by the crash handler (Windows, WITHDUMPGEN) held all the
memory of the simulator (MiniDumpWithFullMemory), so it was as large as
the memory the simulator took: gigabytes on a large scenery, written to
the game folder at the moment of the crash, on machines with 8 GB of RAM
too.

By default it now holds the stacks with the memory they point to, the
threads, the modules, the handles, the memory map and the modules' data
segments (MiniDumpWithDataSegs in place of MiniDumpWithFullMemory, the
rest of the flags as before). The globals, Global among them, stay
readable. "crashdump.full yes" in eu07.ini writes the full dump as before.

Measured on td.scn with a ghostview camera (1.2 GB committed): the small
dump 44 MB in 2 s, the full one 1.6 GB in 19 s.

The key is new and optional: eu07.ini files without it read as before,
and the setting is listed in the log at start.

* fix: the small crash dump keeps the process and thread blocks

MiniDumpWithProcessThreadData adds the PEB and the TEBs, a few KB: the
command line (the scenery and the vehicle), the environment and the last
error of each thread, which the small dump otherwise lacks, since the heap
behind Global's longer strings is not in it.

* fix: an exception that ends the simulator says what it was in the log

After review of #425:

- the exceptions main() catches are logged ("Critical error: ..."): on
  Windows the runtime_error that only showed a message box, and on the
  other systems any std::exception, which used to end the simulator
  through std::terminate with no word in the log. On Windows the others
  are still left to CrashHandler, whose dump shows where they were
  thrown.
- every exception caught there ends with exit code 1, in the start or
  later, as a crash; it used to be 0 when caught, the abort code when not.
- closing the crash-report window logs "Crash report window closed,
  closing" instead of the error "Failed to run crash gui!", which stays
  for a result below 0.

* feat: the stack of a crash on Linux in the log and on the standard error

On Linux no dump is written (the crash handler is Windows only, crashpad
is off), so a crash left a log that just stopped.

A handler of SIGSEGV, SIGBUS, SIGFPE, SIGILL and SIGABRT (installed at
the start of main) now writes, with write() only:

  Crash: signal 11 (SIGSEGV) at address 0x...
  Version: EU07 (cmake), <hash> (...)
  Stack (addr2line -f -C -e <the eu07 file of this version> <offset>):
  <backtrace_symbols_fd>

to the end of the log file (its whole path, set when the log opens it)
and to the standard error, which StarterNG shows first. The Linux builds
carry their symbols (GENERATE_PDB adds -g, nothing strips them), so
addr2line on the eu07 file of the same version gives the files and lines.

Then the signal ends the process as before (SA_RESETHAND, raise): the
same exit status, and a core dump where the system makes one. The
handler runs on an alternate stack, so an overflow of the main thread's
stack is reported too; backtrace() is warmed up at install so it doesn't
allocate in the handler, and alarm(5) ends a handler that hangs (a crash
in the allocator or the loader). WAMR is built without its hardware bound
checks, so it installs no SIGSEGV handler of its own.

Elsewhere crash_signals does nothing. A unit test (Linux only) crashes a
child process and checks the report on its standard error and in the log,
and that the child still ended by SIGSEGV.

* fix: the crash report can't be cut by a second crash signal or end with SIGALRM

After review:
- the five crash signals are blocked while a report is written, so a
  second one waits rather than starting a second report in its middle
  (a fault that comes while blocked still ends the process).
- a report that hangs past alarm(5) ends the process by the signal of
  the crash, not by SIGALRM: a SIGALRM handler, set only in the crash
  handler, raises the crash signal and unblocks it.
- the child of the unit test calls alarm(10) first, so a child stuck
  before the crash can't hang the build.

---------

Co-authored-by: maj00r <maj00r@host>
This commit is contained in:
maj00r
2026-11-20 03:53:42 +01:00
committed by GitHub
parent ba372e9830
commit 201cae07e1
10 changed files with 444 additions and 8 deletions

View File

@@ -1294,6 +1294,12 @@ bool global_settings::ConfigParseDebug(cParser& Parser, const std::string& token
return true;
}
if (token == "crashdump.full")
{
ParseOne(Parser, CrashDumpFull);
return true;
}
if (token == "showsystemconsole")
{
ParseOne(Parser, ShowSystemConsole);
@@ -1753,6 +1759,7 @@ global_settings::export_as_text( std::ostream &Output ) const {
export_as_text( Output, "fullphysics", FullPhysics );
export_as_text( Output, "debuglog", iWriteLogEnabled );
export_as_text( Output, "multiplelogs", MultipleLogs );
export_as_text( Output, "crashdump.full", CrashDumpFull );
export_as_text( Output, "showsystemconsole", ShowSystemConsole );
export_as_text( Output, "logs.filter", DisabledLogTypes );
Output

View File

@@ -113,6 +113,10 @@ struct global_settings {
bool priorityLoadText3D{false}; // ladowanie T3D priorytetowo
int iWriteLogEnabled{ 3 }; // maska bitowa: 1-zapis do pliku, 2-okienko, 4-nazwy torów
bool MultipleLogs{ false };
// the dump written when the simulator crashes (Windows): a small one by default, with the stacks, the threads, the
// modules, the memory the stacks point to and the modules' data. "crashdump.full yes" writes all the memory of the
// simulator instead, as large as the memory it took
bool CrashDumpFull{ false };
// the black box of the session (blackbox_doc.md): "blackbox yes" writes it whatever the physics logs, "blackbox no"
// never, and with no entry it is written along with the physics logs
std::optional<bool> BlackBox;

View File

@@ -9,6 +9,7 @@ http://mozilla.org/MPL/2.0/.
#include "stdafx.h"
#include "utilities/Logs.h"
#include "utilities/crashsignals.h"
#include <format>
@@ -115,6 +116,8 @@ void LogService()
{
std::string filename = Global.MultipleLogs ? "logs/log (" + filename_scenery() + ") " + filename_date() + ".txt" : "log.txt";
output.open(filename, std::ios::trunc);
// the stack of a crash on Linux goes to the end of this file
crash_signals::set_log_path(filename);
}
output << msg << "\n";
output.flush();

234
utilities/crashsignals.cpp Normal file
View File

@@ -0,0 +1,234 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#include "stdafx.h"
#include "utilities/crashsignals.h"
#if defined(__linux__)
#include <algorithm>
#include <array>
#include <atomic>
#include <cstddef>
#include <cstdint>
#include <filesystem>
#include <string>
#include <system_error>
#include <execinfo.h>
#include <fcntl.h>
#include <signal.h>
#include <unistd.h>
namespace {
// the handler may only call what is safe in a signal: no allocation, no locks, no streams. so what it needs is
// kept in fixed buffers, filled in beforehand, and written with write()
constexpr std::array crash_signal_list{SIGSEGV, SIGBUS, SIGFPE, SIGILL, SIGABRT};
// a handler that hangs (a crash inside the allocator or the loader, which backtrace() may need) ends this long after
constexpr unsigned int crash_report_timeout_seconds{5};
// a text of fixed size, filled outside the handler and read in it
template <std::size_t Size> struct fixed_text {
std::array<char, Size> text{};
std::atomic<bool> ready{false};
void set(std::string_view const Text) {
ready = false;
auto const length{std::min(Text.size(), text.size() - 1)};
std::copy_n(Text.data(), length, text.data());
text[length] = '\0';
ready = true;
}
char const *get() const { return ready ? text.data() : nullptr; }
};
fixed_text<1024> &log_path() {
static fixed_text<1024> path;
return path;
}
fixed_text<256> &version_text() {
static fixed_text<256> version;
return version;
}
// the stack the handler runs on, so an overflow of the main thread's own stack is reported too
std::array<char, 64 * 1024> &alternate_stack() {
static std::array<char, 64 * 1024> stack{};
return stack;
}
void write_all(int const File, std::string_view Text) {
while (false == Text.empty()) {
auto const written{::write(File, Text.data(), Text.size())};
if (written <= 0)
return;
Text.remove_prefix(static_cast<std::size_t>(written));
}
}
void write_hex(int const File, std::uintptr_t const Value) {
constexpr std::size_t digits{sizeof(Value) * 2};
constexpr std::string_view hexdigits{"0123456789abcdef"};
std::array<char, digits + 2> text{'0', 'x'};
for (std::size_t digit = 0; digit < digits; ++digit)
text[2 + digit] = hexdigits[(Value >> (4 * (digits - 1 - digit))) & 0xF];
write_all(File, {text.data(), text.size()});
}
void write_decimal(int const File, int const Value) {
std::array<char, 16> text{};
auto position{text.size()};
auto rest{static_cast<unsigned int>(Value < 0 ? -Value : Value)};
do {
text[--position] = static_cast<char>('0' + rest % 10);
rest /= 10;
} while (rest != 0 && position > 1);
if (Value < 0)
text[--position] = '-';
write_all(File, {text.data() + position, text.size() - position});
}
std::string_view signal_name(int const Signal) {
switch (Signal) {
case SIGSEGV:
return "SIGSEGV";
case SIGBUS:
return "SIGBUS";
case SIGFPE:
return "SIGFPE";
case SIGILL:
return "SIGILL";
case SIGABRT:
return "SIGABRT";
default:
return "?";
}
}
// the signal being reported, for a report that hangs
std::atomic<int> &crash_signal() {
static std::atomic<int> signal{0};
return signal;
}
// a report that hangs is given up: the process ends by the signal of the crash, as it would have without the handler
void handle_report_timeout(int) {
auto const crashsignal{crash_signal().load()};
raise(crashsignal);
// it is blocked while its own handler runs, which this one interrupted
sigset_t unblocked;
sigemptyset(&unblocked);
sigaddset(&unblocked, crashsignal);
sigprocmask(SIG_UNBLOCK, &unblocked, nullptr);
}
struct crash_stack {
std::array<void *, 64> frames{};
int count{0};
};
void write_report(int const File, int const Signal, siginfo_t const *Info, crash_stack const &Stack) {
write_all(File, "\nCrash: signal ");
write_decimal(File, Signal);
write_all(File, " (");
write_all(File, signal_name(Signal));
write_all(File, ")");
// the address of the fault; an abort has none
if (Signal != SIGABRT && Info != nullptr) {
write_all(File, " at address ");
write_hex(File, reinterpret_cast<std::uintptr_t>(Info->si_addr));
}
write_all(File, "\n");
if (auto const *version{version_text().get()}) {
write_all(File, "Version: ");
write_all(File, version);
write_all(File, "\n");
}
write_all(File, "Stack (addr2line -f -C -e <the eu07 file of this version> <the offset after the +>):\n");
backtrace_symbols_fd(Stack.frames.data(), Stack.count, File);
}
void handle_crash_signal(int const Signal, siginfo_t *Info, void *) {
crash_signal() = Signal;
struct sigaction timeout{};
timeout.sa_handler = handle_report_timeout;
sigemptyset(&timeout.sa_mask);
sigaction(SIGALRM, &timeout, nullptr);
alarm(crash_report_timeout_seconds);
crash_stack stack;
stack.count = backtrace(stack.frames.data(), static_cast<int>(stack.frames.size()));
if (auto const *path{log_path().get()}) {
if (auto const log{::open(path, O_WRONLY | O_APPEND | O_CLOEXEC)}; log >= 0) {
write_report(log, Signal, Info, stack);
::close(log);
}
}
write_report(STDERR_FILENO, Signal, Info, stack);
// the handler was taken off as it was called (SA_RESETHAND): the signal does what it always did
raise(Signal);
}
} // namespace
namespace crash_signals {
void install(std::string_view const Version) {
static std::atomic<bool> installed{false};
if (installed.exchange(true))
return;
version_text().set(Version);
// backtrace() loads what it needs on its first call, which allocates: done here, not in the handler
std::array<void *, 1> warmup{};
backtrace(warmup.data(), static_cast<int>(warmup.size()));
stack_t alternate{};
alternate.ss_sp = alternate_stack().data();
alternate.ss_size = alternate_stack().size();
sigaltstack(&alternate, nullptr);
struct sigaction action{};
action.sa_sigaction = handle_crash_signal;
action.sa_flags = SA_SIGINFO | SA_ONSTACK | SA_RESETHAND;
// another crash signal of the same thread waits until the report is written, rather than starting a second one in
// its middle (a fault that comes while it is blocked still ends the process). a crash of another thread at the
// same moment may still write its report alongside
sigemptyset(&action.sa_mask);
for (auto const crashsignal : crash_signal_list)
sigaddset(&action.sa_mask, crashsignal);
for (auto const crashsignal : crash_signal_list)
sigaction(crashsignal, &action, nullptr);
}
void set_log_path(std::string_view const Path) {
// whole, so the report finds the file wherever the simulator stands then
std::error_code error;
auto const path{std::filesystem::absolute(std::filesystem::path{Path}, error)};
log_path().set(error ? std::string{Path} : path.string());
}
} // namespace crash_signals
#else
namespace crash_signals {
void install(std::string_view) {}
void set_log_path(std::string_view) {}
} // namespace crash_signals
#endif

25
utilities/crashsignals.h Normal file
View File

@@ -0,0 +1,25 @@
/*
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
*/
#pragma once
#include <string_view>
// the stack of a crash on Linux, where no dump is written: a signal of a fault (SIGSEGV, SIGBUS, SIGFPE, SIGILL) or
// SIGABRT writes the stack of the thread that took it to the log and to the standard error, which the starter shows,
// and then ends the simulator as the signal always did (a core dump where the system makes one). the addresses are
// read with addr2line on the eu07 file of the same version, which carries its symbols. elsewhere it does nothing
namespace crash_signals {
// sets the handlers up, once. Version goes into the report, to tell which eu07 file the addresses belong to
void install(std::string_view Version);
// the log file the report is added to, once the log has opened it
void set_log_path(std::string_view Path);
} // namespace crash_signals