mirror of
https://github.com/MaSzyna-EU07/maszyna.git
synced 2026-11-05 08:44:32 +01:00
09b1c2de10
utilities.cpp starts with a byte order mark, so the compiler reads it as utf-8. The game's targets don't get /utf-8, though: set_target_properties(... COMPILE_OPTIONS ...) in the MSVC block of CMakeLists.txt replaces the options add_compile_options gave them. So the string literals went into the program in the code page of the system, windows-1250 on a polish one, and the map of win1250_to_utf8 turned each letter into itself. What read windows-1250 through it got no utf-8: the names, descriptions and categories of the sceneries and the names of the textures in the launcher, and the descriptions of the templates in the editor. The editor writes such a description back through utf8_to_win1250, which makes a question mark of each byte that doesn't start a utf-8 sequence, so the letters it had read unchanged didn't come back. The utf-8 sequences are written as bytes now, the way utf8_to_win1250 has them, so they don't depend on the options. The options stay as they are: with /utf-8 other literals of the files with a byte order mark would change, the "±" of the table of Bezogonkow would take two bytes and move the letters after it. The unit tests check the letters and that the text given is valid utf-8. Co-authored-by: maj00r <maj00r@host>
27 lines
1.1 KiB
C++
27 lines
1.1 KiB
C++
#include "stdafx.h"
|
|
#include "doctest.h"
|
|
#include "utilities/utilities.h"
|
|
|
|
// the polish letters, in windows-1250 and in utf-8; written as bytes, the build reads the sources in any code page
|
|
namespace
|
|
{
|
|
std::string const kWin1250{"\xA5\xC6\xCA\xA3\xD1\xD3\x8C\x8F\xAF\xB9\xE6\xEA\xB3\xF1\xF3\x9C\x9F\xBF"};
|
|
std::string const kUtf8{"\xC4\x84\xC4\x86\xC4\x98\xC5\x81\xC5\x83\xC3\x93\xC5\x9A\xC5\xB9\xC5\xBB"
|
|
"\xC4\x85\xC4\x87\xC4\x99\xC5\x82\xC5\x84\xC3\xB3\xC5\x9B\xC5\xBA\xC5\xBC"};
|
|
} // namespace
|
|
|
|
TEST_CASE("win1250: the polish letters of a scenery file read as utf-8") {
|
|
CHECK(win1250_to_utf8("Bia\xB3" "a Podlaska") == "Bia\xC5\x82" "a Podlaska");
|
|
CHECK(win1250_to_utf8(kWin1250) == kUtf8);
|
|
// the rest of the text stays as it is
|
|
CHECK(win1250_to_utf8("tor 1, km 0.178") == "tor 1, km 0.178");
|
|
}
|
|
|
|
TEST_CASE("win1250: what win1250_to_utf8 gives is valid utf-8") {
|
|
std::u32string decoded;
|
|
CHECK_FALSE(utf8_to_utf32(kWin1250, decoded));
|
|
REQUIRE(utf8_to_utf32(win1250_to_utf8(kWin1250), decoded));
|
|
CHECK(decoded.size() == kWin1250.size());
|
|
CHECK(decoded[12] == U'\x0142');
|
|
}
|