eu07/maszyna
16
0
Fork 0
mirror of https://github.com/MaSzyna-EU07/maszyna.git synced 2026-07-19 23:49:18 +02:00
Code Issues 123 Wiki Activity

Add headless parallel eu7v2 scenario bake with streaming and PLCE placements

Browse Source 
Enable --eu7v2-bake from the main binary: parallel module pool, bounded-RAM
spool flush, streaming terrain triangles, flat include/model parsing, and
eu7v2 emit/load with optional verify. Large placement .scm files emit lean
PLCE records and bake referenced .inc modules separately for reuse.

- CLI: --eu7v2-bake, --eu7v2-verify, --eu7v2-mem-limit-gb, --eu7v2-threads,
  --eu7v2-max-parse; wire max_threads through to the bake parser
- eu7v2 v2 records: PLCE placements, runtime emitter/loader, batch verify
- Parallel bake pool with session cache; drop heavy-serial parse gate in spool
  mode; parse concurrency matches thread count
- Streaming terrain: batched parallel parse+bake, scan/bake pipeline, shape
  spool with persistent buffered I/O and flush-before-read
- Parallel flat-file streaming for models/includes; pack/model spool for
  low-memory incremental flush
- Optional 50 GB private-bytes guard during headless bake

Braniewo_szeroki: 160 modules, verify PASS, ~34s bake (nmt100 ~17s vs ~190s
serial baseline).

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
maj00r
2026-06-17 21:15:36 +02:00
parent 9574d2051b
commit beacc00932
40 changed files with 9327 additions and 193 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
utilities/parser.cpp
View File

@@ -235,34 +235,40 @@ std::string cParser::readTokenFromStream(bool ToLower, const char *Break)
std::string token;
token.reserve(64);
const auto breakTable = makeBreakTable(Break);
char c = 0;
// rebuild the separator lookup table only when the break set actually changes;
// callers overwhelmingly pass the same string literal, so this stays cached.
if (Break != m_breakTableKey) {
m_breakTable = makeBreakTable(Break);
m_breakTableKey = Break;
}
const auto &breakTable = m_breakTable;
int ci = 0;
while ((ci = mStream->get()) != EOF) {
char c = static_cast<char>(ci);
if (c == '\n') {
++mLine;
}
while (token.empty() && mStream->peek() != EOF) {
while (mStream->peek() != EOF) { // idk why but with mStream->get(c) not all cars are loaded
c = static_cast<char>(mStream->get());
if (c == '\n') {
++mLine;
}
const unsigned char uc = static_cast<unsigned char>(c);
if (breakTable[uc]) {
// separator ends token (or continues skipping if token empty)
if (!token.empty())
break;
continue;
}
const unsigned char uc = static_cast<unsigned char>(c);
if (breakTable[uc]) {
// separator ends token (or continues skipping if token empty)
if (!token.empty())
break;
continue;
}
if (ToLower) c = toLowerChar(c);
token.push_back(c);
if (ToLower) c = toLowerChar(c);
token.push_back(c);
if (findQuotes(token)) {
continue; // glue quoted content
}
if (skipComments && trimComments(token)) {
break; // don't glue tokens separated by comment
}
if (findQuotes(token)) {
continue; // glue quoted content
}
if (skipComments && trimComments(token)) {
// comment stripped: return the token if we already have one,
// otherwise keep scanning for the next real token
if (!token.empty())
break;
}
}