architecture

Architecture

Overview

The portal sits between the public internet and GitLab. Every user-facing request is handled by the portal — GitLab is never accessed directly by clients.

User browser
    │
    ▼
Caddy (SSL termination, icarus.eurekaendeavors.com)
    │
    ▼
Flask portal (Gunicorn, localhost:8000)
    │  ├── / ─────────── Landing page (Jinja2)
    │  ├── /pets ──────── Pet editor web app (Flask Blueprint)
    │  ├── /prospects ─── Prospect save editor (Flask Blueprint)
    │  ├── /catalog ───── Data catalog explorer (Flask Blueprint)
    │  └── Records all download/clone events → SQLite
    │
    ▼
GitLab API / git HTTP (git.eurekaendeavors.com, PAT auth, server-side only)

GitLab repositories are private. The portal's personal access token (PAT) is stored server-side in .env — clients never see it.

Application Structure

The portal is organised as a Flask application with Blueprints. Each tool is a self-contained Blueprint under portal/blueprints/. The shared UE4 library lives in icarus/core and is installed as a local pip package.

icarus/portal (this repo)
├── portal/
│   ├── __init__.py           # App factory (create_app), landing route at /
│   ├── config.py             # Dev/Test/Prod configuration
│   ├── extensions.py         # Shared Flask extensions (SQLAlchemy)
│   ├── templates/
│   │   ├── base.html         # ✅ v0.05 — Shared portal base (nav, landmarks, a11y)
│   │   └── index.html        # ✅ v0.05 — Landing page (tool cards)
│   ├── static/
│   │   ├── css/app.css       # ✅ v0.05 — Shared styles + WCAG AA palette
│   │   └── js/app.js         # ✅ v0.05 — Shared JS (announce, disclaimer modal)
│   └── blueprints/
│       ├── prospects/        # ✅ v0.03 — Prospect save editor
│       │   ├── __init__.py   # Blueprint factory
│       │   ├── models.py     # SQLAlchemy models (sessions, visits, events)
│       │   ├── routes.py     # Upload, editor, API routes
│       │   ├── services/     # Mission parsing, cascade, reset logic
│       │   ├── data/         # Game data (gh_chains.json, metadata)
│       │   ├── templates/    # Jinja2 templates (extend portal base)
│       │   └── static/       # CSS + JS (AJAX editor, a11y)
│       ├── pets/             # ✅ v0.04 — Pet editor
│       │   ├── __init__.py   # Blueprint factory
│       │   ├── models.py     # SQLAlchemy models
│       │   ├── routes.py     # Upload, editor, API routes
│       │   ├── services/     # Mount parsing, talents, genetics, species swap
│       │   ├── data/         # Bestiary, talent, variation data
│       │   ├── templates/    # Jinja2 templates (extend portal base)
│       │   └── static/       # CSS + JS (AJAX editor, a11y)
│       └── # catalog/        # 🔲 v0.06 — Data catalog
├── tests/                    # Pytest test suite (77 tests)
├── deploy/                   # Caddy config, systemd unit
├── scripts/                  # GitLab admin scripts
└── wiki/                     # This wiki (separate git repo)

icarus/core                   # pip install -e ../icarus-core
├── icarus_core/
│   ├── ue4_parser.py         # BinaryReader/Writer + property parser
│   ├── ue4_serializer.py     # Property tree → binary serialization
│   └── save_io.py            # JSON I/O, blob compress/decompress, actor helpers
└── pyproject.toml

icarus/data-catalog           # extraction scripts, not a pip package

Requirements

R1 — GitLab API Proxy

  • All project data fetched server-side via GitLab API v4
  • Authenticated with a PAT stored in .env, never exposed to clients
  • No API response caching — every request fetches fresh data
  • Only explicitly configured projects are displayed (not an open proxy)

R2 — Pet Editor (/pets)

The pet editor is a web application (Flask Blueprint from icarus/pets). The portal provides the full in-browser editing experience:

  • Upload an Icarus save file
  • Edit pets, mounts, farm animals — 26 creature types, 457 talents, phenotype editing, species swap
  • Download the modified save file
  • No desktop application or installation required

R3 — Metrics & Download Tracking

  • Every file download recorded in SQLite: project slug, asset filename, timestamp, client IP, User-Agent
  • Every git clone/fetch logged via the git smart HTTP proxy

R4 — Wiki / Documentation

  • Wiki content pulled from GitLab via API and rendered through the portal
  • Users never see the GitLab web UI
  • API: GET /api/v4/projects/:id/wikis and GET /api/v4/projects/:id/wikis/:slug

R5 — Landing Page (/)

  • Overview of all hosted tools
  • David Beauchamp / Eureka Endeavors branding
  • Clean, minimal design

R6 — Git Clone Proxy

The portal proxies the git smart HTTP protocol so anonymous users can git clone from icarus.eurekaendeavors.com while GitLab repositories remain private.

See Git Clone Proxy for full details.

R7 — Prospect Save Editor (/prospects)

The prospect editor Blueprint is sourced from icarus/prospects:

  • Upload a prospect save file (.json)
  • Select missions to reset (Great Hunts chains + Open World, dependency cascading)
  • Download the modified save file
  • Session-based state management (no persistent user accounts)

R8 — Data Catalog Explorer (/catalog)

Frontend for icarus/data-catalog generated output:

  • Browse all 76+ game data categories
  • Search and filter entries
  • View patch diffs (what changed between game versions)
  • Powered by catalog_summary.json / full_catalog.json (static or dynamic)

Slug-to-Project Mapping

Slug GitLab Project ID Purpose
pets icarus/pets 1 Pet editor
prospects icarus/prospects 4 Prospect editor
data-catalog icarus/data-catalog 5 Data catalog

Data Flows

Pet editor

  1. User requests icarus.eurekaendeavors.com/pets
  2. Pet editor Blueprint (from icarus/pets) handles the request directly
  3. Parses/serializes save file using icarus/core library
  4. Modified file streamed back to user

Prospect editor

  1. User uploads save file at /prospects/upload
  2. Blueprint validates JSON, extracts ProspectBlob, decompresses + parses UE4 binary (via icarus_core)
  3. Creates ProspectEditSession in SQLite, stores compressed blob + parsed mission state
  4. Editor page (/prospects/editor) renders interactive mission checkboxes
  5. User toggles missions via AJAX (/prospects/api/toggle) — dependency cascading applied server-side
  6. User clicks Apply (/prospects/api/apply) — resets applied to original blob, verification run
  7. User downloads modified save (/prospects/api/download)
  8. Session expires after TTL, cleaned up by before_request hook

Git clone

  1. git clone https://icarus.eurekaendeavors.com/pets.git
  2. Portal proxies info/refs → GitLab, logs clone event to SQLite
  3. Portal proxies git-upload-pack POST → GitLab (streaming)
  4. Client receives packfile, clone completes

See Git Clone Proxy for details on URL migration and web UI lockdown.

Back to Docs