Architecture

Architecture

Project Structure

icarus-pets/
├── mount_editor.py          # Main application — launches GUI, orchestrates tabs
├── ue4_parser.py            # Binary → Python: reads UE4 serialized properties
├── ue4_serializer.py        # Python → Binary: writes properties back to byte arrays
├── talent_data.py           # Game data: genetics, lineages, talent trees (auto-generated)
├── variation_data.py        # Game data: phenotype variations per creature (auto-generated)
├── bestiary_data.json       # Intermediate bestiary data (77 creatures)
├── VERSION                  # Version file (semver)
├── README.md
├── build.py                 # Build script for Windows .exe
├── mount_editor.spec        # PyInstaller spec file
├── .gitignore
├── .gitlab-ci.yml           # CI/CD pipeline configuration
│
├── editor/                  # GUI module package
│   ├── __init__.py
│   ├── mount_model.py       # Data model wrapping parsed props with a clean API
│   ├── overview_tab.py      # Tab: identity, lineage, vital stats, talent summary
│   ├── genetics_tab.py      # Tab: 7 genetic stats with spinboxes
│   ├── talents_tab.py       # Tab: full talent tree with rank controls
│   ├── advanced_tab.py      # Tab: raw property tree for power users
│   ├── bug_report.py        # In-app bug reporting dialog + GitLab API
│   └── tooltip.py           # Reusable hover tooltip widget
├── tests/                   # Test files
│   ├── sample_mounts/       # Bundled test fixtures (gitignore exception)
│   │   ├── EN-US/Mounts.json   # UTF-8 encoded, 6 mounts
│   │   └── FR-FR/Mounts.json   # UTF-16 LE encoded, 8 mounts
│   ├── run_all.py           # Unified test runner: auto-discovers all tests
│   ├── test_roundtrip.py    # Roundtrip test: parse → serialize → compare
│   ├── test_ue4_parser.py   # UE4 binary parser tests (28 tests)
│   ├── test_ue4_serializer.py # UE4 binary serializer tests (28 tests)
│   ├── test_mount_model.py  # MountModel API tests (28 tests)
│   ├── test_species_swap.py # Species swap core tests (13 tests)
│   ├── test_species_swap_edge.py # Species swap edge cases (22 tests)
│   ├── test_save_backup.py  # Save/backup/restore tests (14 tests)
│   ├── test_packaging.py    # Frozen-app paths & version tests (13 tests)
│   ├── test_talent_data.py  # Talent data integrity tests (35 tests)
│   ├── test_variation.py    # Phenotype variation tests (36 tests)
│   └── test_refresh.py      # Refresh tool tests (7 tests)
├── scripts/                 # Extraction & generation scripts
│   ├── pak_extract.py       # PAK file reader: lists and extracts zlib chunks
│   ├── pak_talent_extract.py # Talent extractor: parses D_Talents from pak chunks
│   ├── generate_talent_data.py  # Code generator: JSON → talent_data.py module
│   ├── extract_bestiary.py  # Bestiary extractor: parses creature names & lore from pak
│   ├── add_bestiary_to_talent_data.py  # Injects BESTIARY_DATA into talent_data.py
│   ├── refresh_talent_data.py  # One-command talent data refresh tool
│   ├── pak_variation_extract.py # Variation extractor: parses IcarusMount DataTable
│   ├── generate_variation_data.py # Code generator: JSON → variation_data.py module
│   ├── phenotype_analysis.py   # Phenotype variation analysis script
│   ├── check_test_changes.py   # CI: warns when source changes lack test changes
│   └── mount-bin.py         # CLI utility for analyzing binary data
├── docs/                    # Research & reference docs
│   ├── mount_talent_reference.md  # Human-readable talent reference
│   └── species_swap_research.md   # Research notes on species field mapping
│
├── wiki/                    # Wiki repo (separate git)
├── pak-files/               # Extracted pak data (gitignored)
├── build/                   # Build output (gitignored)
└── CGitLab-Runner/          # Runner config

Data Flow

Mounts.json
    │
    ▼
┌─────────────────────┐
│  JSON parsing        │  json.load() → list of mount entries
│  (mount_editor.py)   │  Each entry has RecorderBlob.BinaryData
└─────────┬───────────┘
          │
          ▼
┌─────────────────────┐
│  UE4 Binary Parser   │  BinaryReader + parse_properties()
│  (ue4_parser.py)     │  Byte array → list of property dicts
└─────────┬───────────┘
          │
          ▼
┌─────────────────────┐
│  Mount Model         │  MountModel wraps props with .name, .genetics,
│  (mount_model.py)    │  .talents, .health, etc. — clean getter/setter API
└─────────┬───────────┘
          │
          ▼
┌─────────────────────┐
│  GUI Tabs            │  Each tab calls model.load() to populate widgets
│  (overview, genetics,│  User edits values via spinboxes, dropdowns, etc.
│   talents, advanced) │  Each tab calls model.save() to push changes back
└─────────┬───────────┘
          │
          ▼
┌─────────────────────┐
│  UE4 Serializer      │  props_to_binary_array()
│  (ue4_serializer.py) │  Property dicts → byte array (JSON-compatible int list)
└─────────┬───────────┘
          │
          ▼
    Mounts_edited.json     (new file — original never modified)

Pak Extraction Pipeline

Talent data is extracted directly from the game's pak files rather than manually entered:

data.pak (Icarus game file)
    │
    ▼
┌─────────────────────┐
│  pak_extract.py      │  Finds zlib streams, decompresses chunks
│                      │  Saves as pak-files/extracted_*.bin
└─────────┬───────────┘
          │
          ▼
┌─────────────────────┐
│  pak_talent_extract  │  Concatenates sequential chunks (offset 1420000–1570000)
│  .py                 │  Parses D_Talents JSON: Name, DisplayName, Description,
│                      │  TalentTree, Rewards (stat + values per rank)
└─────────┬───────────┘  → pak_talents_extracted.json (519 entries, 30 trees)
          │
          ▼
┌─────────────────────┐
│  generate_talent_   │  Maps pak tree names to editor types
│  data.py            │  Generates Python module with all definitions
└─────────┬───────────┘
          │
          ▼
    talent_data.py         (26 types, 457 rankable talents, 150KB)

To regenerate talent data after a game update:

python scripts/pak_extract.py              # Extract zlib chunks from data.pak
python scripts/pak_talent_extract.py       # Parse talent entries from chunks
python scripts/generate_talent_data.py     # Generate talent_data.py module

Bestiary Extraction Pipeline

Creature display names and lore are extracted from the game's bestiary DataTable:

data.pak (Icarus game file)
    │
    ▼
┌─────────────────────┐
│  extract_bestiary.py │  Scans pak chunks for D_Bestiary JSON
│                      │  Extracts CreatureName, Lore1, Lore2 per creature
└─────────┬───────────┘  → bestiary_data.json (77 creatures)
          │
          ▼
┌──────────────────────────┐
│  add_bestiary_to_talent  │  Maps 26 mount types to bestiary keys
│  _data.py                │  Injects BESTIARY_DATA dict into talent_data.py
└──────────────────────────┘

The bestiary provides: - Display names — "Hyena" for DesertWolf, "Mammoth" for WoollyMammoth, etc. - Lore1 — Primary bestiary description shown in the swap preview - Lore2 — Extended lore shown as mouseover tooltip

5 types have no game bestiary entry and use manually-crafted placeholders: Bull, Chew, Pig, Rooster, TundraMonkey.

Variation Extraction Pipeline

Phenotype variation data is extracted from the same pak file alongside talents:

data.pak (Icarus game file)
    │
    ▼
┌─────────────────────┐
│  pak_extract.py      │  (shared step — same chunks used for talents)
└─────────┬───────────┘
          │
          ▼
┌──────────────────────────┐
│  pak_variation_extract.py │  Parses IcarusMount DataTable for Variations arrays
│                           │  Extracts MeshMaterial, GFurMaterial, Weighting
└──────────┬───────────────┘  → pak_variations_extracted.json (3 types, 24 variants)
           │
           ▼
┌──────────────────────────┐
│  generate_variation_     │  Computes rarity tiers from weightings
│  data.py                 │  Generates Python module with public API
└──────────┬───────────────┘
           │
           ▼
    variation_data.py          (Buffalo/Moa/Wolf, 8 variations each)

To regenerate all game data after a game update (one command):

python scripts/refresh_talent_data.py    # Runs all 5 extraction steps

Bug Report Data Flow

User clicks "🐛 Report Bug" in toolbar
    │
    ▼
┌─────────────────────────┐
│  BugReportDialog         │  Modal dialog with structured form
│  (bug_report.py)         │  Collects title, description, steps, expected
└─────────┬───────────────┘
          │  User clicks Submit
          ▼
┌─────────────────────────┐
│  Background thread       │  1. Collect system info (version, OS, mount context)
│                          │  2. Capture window screenshot (optional, via PIL)
│                          │  3. Upload files to GitLab /uploads endpoint
│                          │  4. POST issue to icarus-bug-reports project
└─────────┬───────────────┘
          │
          ▼
    GitLab Issue created     (icarus-bug-reports project, ID=2)
    with labels + attachments

Species Swap Data Flow

User selects target species in swap dropdown
    │
    ▼
┌─────────────────────┐
│  Bestiary preview    │  BESTIARY_DATA provides display name, lore, lore2
│  (overview_tab.py)   │  get_talent_tree() computes compatibility stats
└─────────┬───────────┘
          │  User confirms
          ▼
┌─────────────────────┐
│  MountModel.swap_    │  Updates 6 identity fields from SPECIES_DATA
│  species()           │  Rewrites ObjectFName/ActorPathName patterns
│  (mount_model.py)    │  Calls remap_talents() for display-name matching
└─────────┬───────────┘
          │
          ▼
    All tabs reload from updated model

Testing

test_roundtrip.py — Binary Roundtrip

Parses every mount across all locale fixtures in tests/sample_mounts/, serializes back to binary, and compares byte-for-byte. Auto-discovers locale subdirectories (EN-US, FR-FR, etc.) and handles both UTF-8 and UTF-16 LE encoded save files.

Currently tests 14 mounts across 2 locales (6 EN-US + 8 FR-FR), including non-ASCII mount names with accented characters.

To add a new locale sample, just drop a Mounts.json into a new subdirectory (e.g., tests/sample_mounts/DE-DE/).

python tests/test_roundtrip.py

test_species_swap.py — Species Swap Core (13 tests)

Core logic (#20): - SPECIES_DATA completeness — all 26 types have actor_class, ai_setup, mount_type - get_swap_targets() — same-category filtering, self-exclusion, unknown type handling - remap_talents() — display-name matching, rank transfer, lost point counting, rank clamping - MountModel.swap_species() — identity field updates, path rewriting, talent remapping - Cross-category rejection — ValueError on mount→combatpet - Wolf→Boar swap — combatpet-to-combatpet with known talent overlap

Additional cases (#22): - Buffalo→Horse — mount-to-mount with HorseStandard talent suffix, 3 shared / 5 lost - Full roundtrip after swap — serialize→parse→serialize produces identical binary (639 bytes) - Exhaustive cross-category check — all 26 types verified: no self-targets, no cross-category leaks

python tests/test_species_swap.py

test_species_swap_edge.py — Species Swap Edge Cases (22 tests)

Full matrix (#47): - Every type → every valid target (244 swaps across all 26 types) - Swap target count validation per category

Double swap: - A→B→A preserves shared talents (lossless roundtrip) - A→B→A with type-specific talents (lossy roundtrip, correct accounting)

Max-rank and overflow: - All talents at max rank for every type, swap to first target, verify point accounting - Rank exceeding new tree's max_rank → clamped, overflow refunded

Edge cases: - Same-type swap (no-op), unknown type (ValueError), empty talents, zero-rank, negative rank - Nonexistent talent names refunded, corrupted talent data resilience - Cross-category: mount→farm, combatpet→farm, farm→mount all rejected

Specific pairs: Cat→Dog, Chicken→Sheep, Mammoth→Zebra, field preservation, JSON consistency

python tests/test_species_swap_edge.py

tests/run_all.py — Unified Test Runner (#51)

Auto-discovers and runs all test_*.py files with timing and summary:

python tests/run_all.py           # run all tests
python tests/run_all.py --list    # list discovered tests

Key Design Decisions

Modular Tab Architecture

Each tab is a self-contained ttk.Frame subclass with load(model) and save() methods. This makes it easy to add new tabs without touching existing code.

MountModel as Abstraction Layer

The model provides a clean Pythonic API (model.name, model.get_genetics(), model.set_talents({...})) so tabs never need to understand the raw property dict structure. Changes flow through the model back to the same property dicts the serializer reads.

In-Place Property Editing

The Advanced tab edits property dicts directly (by reference). The Overview/Genetics/Talents tabs work through the MountModel which also modifies the same dicts. This means all edits converge on a single data source — no sync issues.

Binary-Perfect Roundtrip

The parser and serializer are designed so that serialize(parse(data)) == data for all tested save files. This ensures edits don't corrupt unrelated data.

Auto-Generated Talent Data

Rather than manually entering talent definitions, we extract them directly from the game's pak files. This ensures accuracy and makes it trivial to update after game patches — just re-run the extraction pipeline.

No External Dependencies

The entire project uses only Python 3 standard library (tkinter, json, struct, zlib). No pip install required.

See also: GUI Guide · Binary Format · Species & Types · Talent & Genetics Data

Back to Docs