eu07/maszyna
17
0
Fork 0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-11-05 10:59:02 +01:00
Code Issues 156 Wiki Activity

fix(utilities): win1250_to_utf8 turns the polish letters into utf-8, not into themselves (#252)

Browse Source 
utilities.cpp starts with a byte order mark, so the compiler reads it as utf-8. The game's targets don't get
/utf-8, though: set_target_properties(... COMPILE_OPTIONS ...) in the MSVC block of CMakeLists.txt replaces the
options add_compile_options gave them. So the string literals went into the program in the code page of the
system, windows-1250 on a polish one, and the map of win1250_to_utf8 turned each letter into itself.

What read windows-1250 through it got no utf-8: the names, descriptions and categories of the sceneries and the
names of the textures in the launcher, and the descriptions of the templates in the editor. The editor writes
such a description back through utf8_to_win1250, which makes a question mark of each byte that doesn't start
a utf-8 sequence, so the letters it had read unchanged didn't come back.

The utf-8 sequences are written as bytes now, the way utf8_to_win1250 has them, so they don't depend on the
options. The options stay as they are: with /utf-8 other literals of the files with a byte order mark would
change, the "±" of the table of Bezogonkow would take two bytes and move the letters after it.

The unit tests check the letters and that the text given is valid utf-8.

Co-authored-by: maj00r <maj00r@host>
This commit is contained in:
maj00r
2026-11-05 03:12:05 +01:00
committed by GitHub
parent b7ff8bab73
commit 09b1c2de10
2 changed files with 31 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
utilities/utilities.cpp
View File

@@ -317,8 +317,11 @@ void win1250_to_ascii(std::string &Input)
std::string win1250_to_utf8(const std::string &Input)
{
std::unordered_map<char, std::string> const charmap{{165, "Ą"}, {198, "Ć"}, {202, "Ę"}, {163, "Ł"}, {209, "Ń"}, {211, "Ó"}, {140, "Ś"}, {143, "Ź"}, {175, "Ż"},
{185, "ą"}, {230, "ć"}, {234, "ę"}, {179, "ł"}, {241, "ń"}, {243, "ó"}, {156, "ś"}, {159, "ź"}, {191, "ż"}};
// the utf-8 sequences written as bytes: the file has a byte order mark and the build doesn't give the compiler /utf-8, so letters
// written as letters would end up in the program in windows-1250, the same as the input
static std::unordered_map<char, std::string> const charmap{{'\xA5', "\xC4\x84"}, {'\xC6', "\xC4\x86"}, {'\xCA', "\xC4\x98"}, {'\xA3', "\xC5\x81"}, {'\xD1', "\xC5\x83"}, {'\xD3', "\xC3\x93"},
{'\x8C', "\xC5\x9A"}, {'\x8F', "\xC5\xB9"}, {'\xAF', "\xC5\xBB"}, {'\xB9', "\xC4\x85"}, {'\xE6', "\xC4\x87"}, {'\xEA', "\xC4\x99"},
{'\xB3', "\xC5\x82"}, {'\xF1', "\xC5\x84"}, {'\xF3', "\xC3\xB3"}, {'\x9C', "\xC5\x9B"}, {'\x9F', "\xC5\xBA"}, {'\xBF', "\xC5\xBC"}};
std::string output;
std::unordered_map<char, std::string>::const_iterator lookup;
for (auto &input : Input)