source2rosetta/README.md

# source2rosetta

**Current CS2 and Dota 2 gamedata — re-derived from every Valve build, proven on a live server, published automatically.**

When Valve ships an engine update, every Metamod / CounterStrikeSharp plugin breaks until someone hand-reverse-engineers fresh gamedata: function signatures, vtable offsets, netvar layouts. That has historically taken days, sometimes weeks.

Here it takes **about half an hour, with nobody involved.** A timer notices the new build, re-derives the whole surface from the stripped `.so` libraries the dedicated server maps, launches its own vanilla server and *calls the functions* to prove they resolve, then publishes to a fixed URL. No one is paged and nothing is hand-checked — and if any stage fails, the run stops and the previous release stays up. What ships is never a guess.

```sh
R=https://git.lo.sh/kamal/source2rosetta/releases/download/cs2-latest   # always the newest build
curl -fsSLO $R/gamedata-cs2.json   # WHERE functions are — signatures + vtable offsets
curl -fsSLO $R/netvars-cs2.json    # field offsets and types
curl -fsSLO $R/abi-cs2.json        # HOW to call them — parameter and return types, re-judged per build
```

Also published: `bindings-<game>.json` (the callable surface the binary declares about itself — Pulse
bindings, entity IO, console commands) and `manifest.json` (which build you got). The
[artifacts section](#artifacts-schemas--output-formats) covers all of them.

The output is framework-neutral; `source2rosetta-gen` renders it into whatever your stack speaks — the gamedata into your framework's locator format, and `abi-<game>.json` into **typed call sites** for the same functions. The deriver behind it is a standalone Rust tool — you only need that if you're self-hosting the pipeline or adding a game.

## Docs

- **[ATTRIBUTIONS.md](ATTRIBUTIONS.md) — start here.** This tool stands on a decade of community reverse-engineering, catalogues, dumpers, and research. The credits come first because the work does.
- **🎯 Render a release for your framework** → **[source2rosetta-gen](crates/source2rosetta-core/README.md)** — one command turns the JSON above into CounterStrikeSharp, Metamod/SourceMod, ModSharp, Swiftly, Plugify, or a typed C# SDK. No build, no corpus; what most people are here for.
- [CONTRIBUTING.md](CONTRIBUTING.md) — add or back-fill a gamedata entry.
- [LICENSE](LICENSE) — AGPL-3.0.

## Results

Ballpark from a recent build, on a 16-core desktop. These move build-to-build — treat them as orders of magnitude, not guarantees.

| | derived functions | typed schema | model | one-time distill |
|---|---|---|---|---|
| **CS2** | ~1,125 `core` + ~2,620 `high_confidence`, plus ~4,375 `experimental` name guesses | ~1,900 classes / ~12,300 fields | ~48 MB (a few MB gzipped) | ~15 min |
| **Dota 2** | ~1,930 `core` + ~2,450 `high_confidence`, plus ~5,950 `experimental` | ~2,960 classes / ~17,700 fields | ~570 MB | ~1 hr |

A full run live-validates what it ships and reports **0 dropped** on both games — for CS2 that is ~2,610 signatures and ~1,120 vtable offsets checked against a running server. Distilling the model is a one-time cost; after that each build's re-derive is minutes of compute, and the half hour in the headline is the whole loop: notice, update, derive, validate, publish.

---

## Staying current — the part with no human in it

Each game runs its own loop, independently:

1. A timer polls Steam every 15 minutes, comparing the installed build id against the live one.
2. On a change it updates the install and runs a single `produce`: derive → live-validate → typed netvars → roll the model forward.
3. It publishes an immutable `<game>-<buildid>-<patch>` snapshot, then moves `<game>-latest` onto it.

A CS2 update never rebuilds Dota, and vice versa. Two rules keep it honest: every stage **hard-fails rather than substituting** an older or on-disk input, and every `core` / `high_confidence` entry that *can* be checked against the live process is checked before it ships. (A handful legitimately cannot — see [`validated` is three-valued](#validated-is-three-valued).) A failed run publishes nothing and leaves the previous release standing.

| you want | use |
|---|---|
| the newest build, always | `…/releases/download/cs2-latest/gamedata-cs2.json` |
| a specific build, pinned | `…/releases/download/cs2-<buildid>-0/gamedata-cs2.json` |
| to know what you got | `manifest.json` — carries `version = <game>-<buildid>-<patch>` |

Follow `-latest` to adopt updates as they land, or pin a buildid tag to adopt them deliberately; old snapshots stay up either way. Whichever you choose, **check the manifest's build id against the server you're actually running** before loading — that is what stops stale offsets meeting a changed binary. (`patch` counts rebuilds on the same binary, e.g. a merged contribution.)

---

## How it works

Six stages: **read → derive → locate → measure → validate → emit.** The organising distinction, which everything else hangs off:

> **Some sources LOCATE a function. Others only DOCUMENT it.**
> A source that pairs a name with an address gives you a locator. A source that pairs a name with
> documentation gives you a catalogue entry and nothing to call. Conflating the two is the single most
> expensive mistake this project has made, and the artifacts keep them apart deliberately.

### 1. Read — the whole server, not just `libserver`

A dedicated server maps roughly 22 shared libraries, and `libserver.so` is only a fraction of the reachable engine surface. source2rosetta reads **all of them** (multilib) — `libserver`, `libengine2`, `libtier0`, `libnetworksystem`, `libschemasystem`, and the rest — and every locator it emits carries the library it belongs to.

ELF parsing is by hand. Three things make a stripped binary readable at all:

- **Relocations are resolved.** Every `SHT_RELA` section is processed (matched by type and flags, never by section name), so `.data.rel.ro` pointer slots — zero on disk — come back as their true as-loaded values. Three relocation types are handled; anything else is left alone.
- **`.eh_frame` enumeration** via `PT_GNU_EH_FRAME` recovers function extents where unwind info survives. Valve strips it from the *game* code, so this covers the statically-linked runtime tail and not much else — which is exactly why the next item exists.
- **Candidate entries** = relocation code-pointers (every vtable slot) ∪ decoded near-call targets, unioned with the `.eh_frame` starts. This is the function list everything downstream iterates.

On top of that sits a whole-binary **cross-reference index**: for every referenced address, the instructions that reference it. That gives the string-anchored locator ("which function uses this string?"), which several later stages depend on.

All Source-2 `.so` files link at vaddr 0, so a runtime address is simply `load_base + file_vaddr`.

### 2. Derive from the binary's own reflection

Valve compiles two reflection systems into every module, and both are read directly.

**RTTI** gives the type hierarchy and vtable layout: which classes exist, what they inherit, and the ordering of every vtable. Where a slot is read *directly* — the fold's offset locators and the experimental band — a slot index is a fact, not a guess.

**SchemaSystem** gives class → field metadata. Be precise about what is static here: **name and offset are in the file; the TYPE is not.** A field's type pointer is a null placeholder on disk and is populated only at runtime, which is why typed netvars require a live process and why an offline run ships no `netvars-<game>.json` at all.

### 3. Names Valve ships in the binary — three sources, and only two locate

Every Source-2 module names some of its own functions. This is ground truth from the shipped binary — not inference, not cross-game transfer — so it outranks every derived name, and it re-derives on every build with no input to maintain and nothing to bootstrap: the source travels with the binary.

| source | what it pairs | locates? |
|---|---|---|
| **entity-IO datadesc** | the C++ handler name (`InputKill`) with the handler's address | **yes** |
| **console-command registration** | the command name (`bot_add`) with its callback | **yes** |
| **Pulse binding registry** | a qualified `Class::Method` with display name, description, call policy and a full typed signature | **no** |

**The Pulse registry does not locate anything, and the mistake is instructive.** Its records carry two code pointers, which an early pass folded as locators for a headline `+782` names. They are *descriptor accessors* — every CS2 `libserver` binding measures the same empty `int=0 float=0` footprint, and they disassemble to a lazy-init singleton that returns a static vector. Folding them would have shipped `CBaseEntityAPI::GetAbsOrigin` pointing at a zero-argument accessor: a locator that resolves, passes live validation as executable code, and is still the wrong function. The honest yield from that table is zero locators — and a complete typed API surface, which ships separately as `bindings-<game>.json`.

**The datadesc handlers are class-qualified** by joining each record's array against the SchemaSystem: a datadesc array also carries field descriptors, and a `(member, offset)` pair is something the schema states from an entirely different table, so the class whose schema satisfies every pair in the array owns it. 653 of 715 CS2 handlers qualify this way. This is what makes `InputEnable` — a distinct handler on 48 classes — nameable at all; unqualified names that resolve to several addresses are still **dropped, not guessed**. Note for consumers: qualification **renamed 343 shipped keys** (`InputActivateSkybox` → `CAmbientGeneric::InputActivateSkybox`).

**Console commands are read from the registration call, not a registry walk.** CS2 registers through a handle-based `ConCommandRef` whose registry lives in tier0, so there is no static `ConCommand` object to scan for and nothing in the file points at a command name. The registration *call* still does — an ordinary call from a static initializer whose name, handler, description and flags are all constants in the instruction stream. source2rosetta tracks what each argument register provably holds and reads the vector at every call: **784 commands across 20 of the 22 libraries on CS2, 855 on Dota** (the two misses are libraries whose registrar the shape test does not find). The registrar is identified by what it *does* — it opens by writing the invalid-handle sentinel into its `ConCommandRef` — never by address and never by ranking call sites, so a library with three commands is as readable as one with three hundred.

Because name and handler come from one instruction sequence, Valve's own published command dump checks the result: **746 of 746 descriptions agree exactly, with no disagreements**, 742 of 780 distinct names appear in that dump, and 12 flag bits are matched to the flag names it prints. Three further flag bits occur and are deliberately left unnamed.

**Typed Pulse signatures, read offline.** A binding's record does not carry its signature; the accessors beside it return the parameter and return vectors, which are built at runtime and therefore zero in the file. The *code* that builds them is not, and every field is written to a fixed address from a `lea` or an immediate — so constant-propagating through the initializer recovers the full typed signature with no running process: **999/999 CS2 records and 859/859 Dota**, with parameter names, `PulseValueType_t` types, and the schema type a value refers to where the binding names one. Cross-checked against Valve's published metadata dump: **435 of 437** CS2 server bindings agree exactly, **293 of 294** on Dota (every difference a global-event binding, where Valve documents the same vector as an out-param).

### 4. Locate across builds — for everything Valve doesn't name

A stripped non-virtual function has no slot and no symbol, so it has to be *found*.

**The fingerprint** is a recompilation-invariant statistical description of a function: bounded-CFG structural counts, an 18-bucket mnemonic-class histogram, a 32-bucket sketch of the printable `.rodata` strings it references, and one hop of call-graph context (distinct callees plus an aggregate of each callee's own instructions, calls and branches) — **63 dimensions**, deliberately abstracted statistics and never raw bytes.

To be exact about what this is *not*: **no trained model, no machine learning, no learned or weighted metric, no embedding network.** Matching is nearest-history under a plain unweighted **L1** distance, accepted within a fixed threshold. The per-game "model" is a bundle of derived *facts* — vtable-alignment hops, reference-fingerprint windows, ABI-shape consensus, slot timelines — not a network.

**A signature resolves by one of three outcomes, and the artifact says which:**

1. **Strict** — a fingerprint match inside the threshold.
2. **Lenient** — no fingerprint check, but two or more distinct era-signatures vote for the same address.
3. **Unverified fallback** — a single candidate that every check rejected, shipped anyway and marked **`catalogue-unverified`**. This is **0.4% of CS2's core and 69.8% of Dota's**, so on Dota it is the common case, not an edge case. Read the marker.

Whatever the route, the shipped byte-signature is regenerated at the resolved address and confirmed **unique in its library**. That is a *usability* check, not corroboration of identity — the pattern is generated *from* that address, so a wrong address yields a signature that is unique and equally wrong. Identity comes from the match, the vote, or live validation; uniqueness only ensures a loader can find it.

**Catalogue vtable offsets are chained, not read.** For a named method, the slot in a *new* build is derived: dated slots are chained forward through per-build-pair vtable alignments that are themselves fingerprint-scored, then a recency-weighted vote emits a slot only above **80% confidence**. RTTI supplies the vtable and the ordering; which slot a named method occupies is an inference with a stated bar.

### 5. Measure the ABI — the guard a byte-signature can't provide

A signature sees a function's *body* drift and re-derives it. It cannot see the *argument list* change while the prologue stays recognisable — the signature still resolves and points at real code, yet a caller using the old prototype loads the wrong registers.

So each function's observable **SysV-AMD64 shape** (which argument registers are live-in, plus the return class) is recovered by a bounded backward-liveness pass and diffed across builds, flagging exactly those prototype changes and marking struct-by-value returns that are unsafe to blind-call.

The footprint is a deliberate **lower bound** — a callee that ignores an argument reads fewer registers than it is passed — and that is checked rather than asserted. Valve's entity-IO datadesc declares hundreds of independent handlers to one fixed prototype, and every one measures within it: **CS2 715/715, Dota 624/624**, on every derive.

Types cannot be recovered from a stripped binary, so `abi-<game>.json` joins *declared* prototypes to that measurement and judges each one. See [the manifest](#abi-gamejson--declared-prototypes-judged-against-this-build).

### 6. Validate — against a live server, not a spec

This is what separates source2rosetta from a static dumper. `produce` and `integration-test` **launch their own** vanilla dedicated server (bots on an empty deathmatch for pawn games; a pawn-less game like Dota waits on a `ready_class` proxy) — no Steam, no separate instance, no human.

Two access paths, and they differ:

- **Reading** is `/proc/<pid>/mem` — no attach, no stop, no injection.
- **Calling** is a real debugger attach: `PTRACE_ATTACH`, save registers, write a scratch frame, run, restore. No injected *code*, but the process is stopped and its registers are written.

What gets checked:

- every **offset** lands on a real vtable slot, and every **signature** on live executable code;
- a gamedata function is **actually called** to prove it is the semantically right function, not a plausible byte-match (pawn games);
- derived probes are **fuzzed across changing game state** for many iterations;
- field **types** are read live for the typed netvars — and fields that are non-null live but zero on disk confirm the reader is seeing real runtime state, not stale disk bytes;
- a **hooked** function (a mod detoured it) is detected by a byte diff at a uniquely-resolved prologue and reported rather than failed.

The contract is blunt: **"degrades or stops loudly, never lies."** An entry live validation confidently rejects is dropped, not shipped under a banner claiming it resolves; if the oracle cannot run, the run fails rather than emit an unverified result.

### 7. Confidence tiers — nothing vanishes silently

| tier | meaning |
|---|---|
| `core` | derived and, in a full run, live-validated — the load-bearing gamedata |
| `high_confidence` | names folded in as verified offsets/sigs — Valve's own in-binary sources (`valve-table` provenance, ground truth) first, then macOS ground-truth transfer, dictionary-exact, and gated extrapolation |
| `experimental` | the least-filtered band — every graded name guess, each with a **resolvable locator** but an **unverified name**. **Never live-validated.** |
| `unresolved` | catalogued but not confidently produced this build, with a closed-vocabulary reason (`sig-drifted`, `offset-low-conf`, `unresolved`, `abi-drift`) and no locator |

A function that couldn't be derived this build shows up as `unresolved` with a reason — it never just disappears.

**Naming is not derivation.** AI-assisted name extrapolation exists but is producer-side dev tooling, not part of the shipped deriver; every proposed name is gated on self-naming or ground-truth corroboration and then live-validated. A wrong *name* mislabels a real slot; it never touches the **offset**, which comes from RTTI.

---

## What it guarantees — and what it refuses to ship

The tool's strongest claim is not what it produces but what it declines to produce.

### Release floors

A `produce` run **aborts rather than publish** a collapsed artifact. Each surface is gated separately, because each is matched by its own record shape — a Valve reshape that breaks one leaves the others intact, and a summed floor would stay satisfied while a whole surface silently vanished:

| surface | floored on | why it needs its own |
|---|---|---|
| Pulse bindings | registry record count | the registry can move independently of the typed signatures |
| typed Pulse signatures | recovered-signature count | the registry can read perfectly while the descriptor layout moves — every binding would ship signature-less |
| entity-IO records | inputs + outputs | |
| entity classnames | factory-record count | |
| console commands | recovered-command count | the registrar is found by SHAPE, so a reworked constructor yields **zero** commands rather than wrong ones — correct, and invisible without this |
| typed netvars | fraction of fields typed | a runtime type-layout reshape resolves every field "untyped" and would otherwise ship a typeless schema at exit 0 |
| schema enums | enum count | read by shape like the class table, so a reshape yields zero rather than wrong |
| live validation | pass rate, above a minimum sample | |

### `validated` is three-valued

`true` / `false` / `null`, and `null` is not a synonym for failure. Three cases ship `null` legitimately: the library is not mapped in the vanilla server, the class is not a vtable class (an engine special or a carried member offset), or the entry has no locator to check. An **offline run ships the whole monolith `null`** — absence of validation, not failed validation. Treat `null` as "not checked here", never as "checked and passed".

### The standing oracles

A dozen free, mostly two-sided checks run on **every** derive and are reported. Two-sided means the two halves are read from different places by readers that don't know about each other, so agreement is evidence and disagreement is a defect:

- **entity-IO ABI** — hundreds of independent handlers against one declared prototype: CS2 715/715, Dota 624/624.
- **console-command ABI** — the callback *form* comes from the registration site, the *arity* from a liveness pass over a different function body: CS2 784/784, Dota 855/855.
- **console-command distinctness** — 784 commands → 783 distinct handlers (855 → 854 on Dota). Catches a layout change that would start returning a shared dispatch thunk: still resolving, still validating, still wrong.
- **Pulse receiver cross-check** — the registry's policy flag vs the independently recovered parameter list: 999/999 and 859/859 registrations.
- **entity-output ↔ schema join** — 226/226 CS2, 186/187 Dota.
- **EHANDLE class grouping** — Valve's naming vs the binary's destructor addresses: 0 of 44 CS2 / 41 Dota groups carry two classes.
- **Pulse element stride** — derived by consensus per image, unanimous across six libraries in both games.
- **live schema oracle** — offline layout vs the running process: 852/852 CS2, 1,912/1,912 Dota.
- **field-gap size calibration**, the semantic call sweep, and a 500-iteration live fuzz.

One of these found a real defect on its first run: 34 of 715 handlers measured float arguments a `void(ptr, ref)` cannot have, which traced to the ABI reader treating a `call` as fall-through so a callee's *return* propagated backwards as a phantom argument.

### What is NOT gated

Stated because "we check things" is worthless without a boundary. There is **no** floor on the validate-live drop rate, the live-fuzz fault rate, the schema class count, or the RTTI class / base-graph size. A regression in any of those is reported, not refused.

**One oracle is currently reporting.** The Pulse registry is keyed by qualified name, so a binding registered by several modules keeps one row, and every duplicate is compared against the row already present. On the current builds **CS2 flags 331 of 419 repeat registrations and Dota 271 of 359** as disagreeing — so for those names `bindings-<game>.json` carries one module's account of the signature, not a merged one. It is flagged on every run and is not yet resolved; if you consume `params`/`returns` for a multiply-registered binding, know that.

---

## Install & CLI usage

The two binaries have different audiences. **`source2rosetta`** (the deriver) is only needed to run the pipeline yourself, fork it, or add a game. **`source2rosetta-gen`** is needed by anyone using a release — a release is framework-neutral JSON, so something has to render it into your stack's format — but you can download the prebuilt binary from a `gen-v*` release instead of building it, provided you're on Linux x86-64. Anywhere else, build it from source.

```sh
# The deriver (`source2rosetta`) — the root binary.
cargo build --release                        # → ./target/release/source2rosetta

# The renderer (`source2rosetta-gen`) lives in the core crate and is NOT built by the root
# build — build it explicitly (or use --workspace). See crates/source2rosetta-core/README.md.
cargo build --release -p source2rosetta-core # → ./target/release/source2rosetta-gen
```

**Prerequisites.** Stable Rust for both binaries. The live half additionally needs a game install and **ptrace permission** (same user with `yama/ptrace_scope=0`, or `CAP_SYS_PTRACE`). The CI fuzz gate needs nightly Rust, `cargo-fuzz` and GNU `parallel`. The release runner needs `steamcmd`, `jq` and `python3`. Dependencies are deliberately few — ELF parsing, disassembly and the parallel primitive are in-tree rather than pulled in.

`--game <cs2|dota2>` is a global flag (default `cs2`), given before the subcommand: `source2rosetta --game dota2 produce …`.

| command | one line |
|---|---|
| `produce` | The whole per-game build in one command: derive → fold → (with `--game-dir`) validate-live + typed netvars → roll the model forward, into `--out-dir`. **`--game-dir` present = full live-validated build; absent = fast offline build (gamedata + model only). That flag is the entire offline/full switch.** |
| `corpus-model` | Distill a corpus of past builds into one shippable model (vtable-alignment hops, reference fingerprints, slot timelines), so future derivation needs only the model + the target binary, not the corpus. |
| `fold-model` | Roll an existing model forward by ONE build (`model N + build → N+1`), reading only the model and that one binary. The production update path (also a sidecar inside `produce`). |
| `integration-test` | Stand-alone CI live oracle: launch a vanilla server, populate it, and verify derived gamedata against it — schema oracle, a semantic ptrace CALL on a live pawn, and (with `--gamedata`) a full validate-live plus optional live fuzzing. |
| `backfill` | Give an extrapolated name a real cross-build timeline — resolve its string anchor in every corpus build, or chain a vtable slot through the model — and report history depth + consistency (how a guess graduates to first-class). |
| `classify-change` | `--prev`/`--new` → `skip` / `normal` / `shift` + the exact % of function bodies that changed, comparing with position-dependent bytes masked so a pure layout shift reads as unchanged. An **operator primitive** — nothing in the shipped pipeline invokes it; the poller dispatches a derive on any buildid change. |
| `filter-corpus` | Collapse runs of code-identical builds to one representative, label each transition `normal`/`shift`, and segment the timeline into toolchain eras. Writes an **advisory** selection manifest; the distill does not read it (see [corpus curation](#getting-the-corpus-only-to-bootstrap-a-model)). |

### Quickstart

Releases live on the **[releases page](https://git.lo.sh/kamal/source2rosetta/releases)**. For the offline path you need one file from there — the model (`model-<game>.json`) — plus the derive inputs, which ship in this repo under `mappings/`.

```sh
# OFFLINE — derive gamedata + roll the model forward. No server, fully deterministic.
./target/release/source2rosetta --game cs2 produce \
    --seed         mappings/seed-cs2.json \
    --corpus-model model-cs2.json \
    --target       <build-dir> \
    --out-dir      out

# FULL — the same, plus it launches its own vanilla+bots server to validate on the live
# process and read field types for the typed netvars. Adding --game-dir is the only change.
./target/release/source2rosetta --game cs2 produce \
    --seed         mappings/seed-cs2.json \
    --corpus-model model-cs2.json \
    --target       <build-dir> \
    --game-dir     <cs2-install> \
    --out-dir      out
```

- `--target <dir>` — the build **directory** to derive from; `produce` requires a directory and its libraries are searched by name. (Other subcommands accept a bare `.so` as well, which is how `classify-change --prev` is used.)
- `--game-dir <install>` — must be the **`game/` subtree** of the install, the same directory layout the dedicated server is launched from.
- `--seed <bundle>` — one file bundling every derive input. The loose equivalent is `--catalogue <file>` plus the optional `--promotable` / `--candidates` / `--full-names` / `--extra-offsets` / `--extra-sigs`, all defaulting to empty — **so a brand-new game needs only a catalogue to start deriving.**
- Corpus signal — exactly one of `--corpus-model <model.json>` (the normal path: forward-derive from the model + target binary, rolling the model N→N+1 as a sidecar) or `--corpus <dir>` (fingerprint raw build binaries on the fly).

Model-based derives are **forward-only**: the model describes history up to its newest build, so pointing one at an *older* target is not supported.

The launched server writes its log to `TMPDIR/<token>-produce.log`, one file per game, and binds a **fixed port** — so two games cannot be produced concurrently on one host without changing it.

---

## Fork it & distill your own model

Nothing is hosted — fork it, `cargo build --release`, and point it at a build on disk.

### What a fork inherits, and what it must supply

Everything engine-generic is inherited: ELF/RTTI/SchemaSystem/SysV reading, fingerprinting, the model machinery, live validation, the emitters. Two things a fork must produce for itself:

- **A catalogue** — the one required input. It is the list of functions you want gamedata *for*: each entry a name plus whatever historical evidence exists (dated vtable slots, per-era signatures, string anchors). Everything else in the seed is optional and defaults to empty. Without a catalogue the tool has nothing to look for.
- **A model**, distilled from a corpus of past builds — or downloaded from a release if you're forking this project's games.

The **seed bundle** collapses the loose inputs into one file with sections for catalogue, promotable names, candidates, full names, extra offsets, extra sigs, and contributions. `mappings/naming/` is a large frozen input with no in-repo producer — it is data, not something a build regenerates.

### Distilling

```sh
# Distill a corpus into a model (streaming, bounded RAM even over Dota's ~1k builds).
./target/release/source2rosetta --game cs2 corpus-model \
    --seed   mappings/seed-cs2.json \
    --corpus corpus/binaries \
    --out    model-cs2.json
```

`--class-scope` picks which classes get vtable-slot hops — the timelines that let a consumer derive an offset from the model alone:

- `clean` (default) — every real game class, dropping template instantiations, protobuf message shapes and NetworkVar chainers, whose hops nobody derives an offset from.
- `all` — those too.
- `catalogue` — only what the catalogue names. The catalogue's own classes are always included regardless of scope.

**Whatever scope you distill with, `fold-model` and `produce`'s sidecar fold must use the same one.** The model records its scope and the fold asserts on it, so a mismatch fails loudly — but CI does not pass the flag, so a non-default scope requires a workflow change too.

### Keeping a model fresh — the incremental fold

Once a model exists you never need the corpus again. `fold-model` rolls it forward one build, reading only the model plus the single new binary:

```sh
./target/release/source2rosetta --game cs2 fold-model \
    --model model-cs2.json \
    --seed  mappings/seed-cs2.json \
    --build <new-build-dir> \
    --out   model-cs2.next.json
```

`produce --corpus-model` runs exactly this fold as a sidecar, so a full build both derives *and* advances the model in one command.

The fold equals a full re-distill over the same builds under **three** conditions: the same `--class-scope` (asserted), the same catalogue the model was distilled from (**not** checked — production always folds with the distill's catalogue, but a newcomer gets only a warning), and no class re-appearing across the model's latest-build boundary. In that last case a class the model has never seen is back-filled with absent history — reduced coverage for that name until the next full re-distill, never a wrong offset.

### Getting the corpus (only to bootstrap a model)

A corpus is a directory of past builds, one subdirectory of `.so` files per build (`corpus/binaries/<label>/*.so`). Fetch it yourself, one time:

1. Use **DepotDownloader** — the self-contained release binary, **not** `dotnet tool install` (its NuGet package is pinned ancient).
2. Pull manifests from the **Linux binaries depot `2347773`** — *not* the content depot `2347770`. `2347773`'s manifest only advances when the binaries actually change, so its history already *is* the list of real recompiles; content micropatches only bump `2347770`. Read the manifest history off SteamDB, not the Steam client.
3. Download **oldest-first** (chronological = version order), then content-hash-dedup.

`filter-corpus` then collapses code-identical builds and segments toolchain eras, so you never fingerprint the same code twice. Its manifest is **advisory** — `corpus-model` reads a *directory*, not the manifest — so the usual pattern is to materialise the kept set as a directory of symlinks and point the distill at that. Nothing checks that the directory matches the manifest; that is on you.

A **partial corpus is fine** — fewer labels is a shallower history, not a broken model.

### Adding a game

Three edits: a `profile::GameProfile` const, a `Game` enum variant, and a match arm in `main`. The profile carries the library set, launch spec, pawn anchor, dead-weight vocabulary, per-surface floors and the game/content keys.

That is the mechanical part. The untested-per-game work is everything the profile *cannot* express: whether the engine era's SchemaSystem layout matches an existing one, whether the game boots to a state where the live oracle can run, and whether its dead-weight vocabulary actually filters that game's junk.

---

## Artifacts, schemas & output formats

A full `produce` run writes a self-contained release set per game into `--out-dir`:

| File | What it is | When |
|------|-----------|------|
| `gamedata-<game>.json` | The **monolith** — the tiered function catalogue (signatures + vtable offsets) with provenance and live-validation folded inline | always |
| `abi-<game>.json` | The **prototype manifest** — declared parameter/return types, each judged against the footprint measured in this build | always |
| `bindings-<game>.json` | The **declared callable surface** — what the binary says about itself: Pulse bindings, entity IO, entity classnames, console commands | always |
| `netvars-<game>.json` | The **typed schema** — every SchemaSystem class → field → offset/type, plus the base graph and type layouts | full (`--game-dir`) runs only |
| `model-<game>.json` | The **per-game model** — the distilled facts derivation reads instead of the corpus | when the run folds an existing model |
| `manifest.json` | Volatile release metadata: `{ version, artifacts: [...] }` | always |

Wall-clock and other volatile metadata live only in `manifest.json`; the other artifacts carry no timestamp, so they are **byte-reproducible** — the same build in yields the same JSON out.

### `gamedata-<game>.json` — the monolith

```jsonc
{
  "meta": { "game_key", "game", "source_build", "version",
            "counts": { "core", "high_confidence", "experimental", "unresolved" } },
  "core":            { "<fn name>": <MonoEntry>, ... },
  "high_confidence": { "<fn name>": <MonoEntry>, ... },
  "experimental":    { "<fn name>": <MonoEntry>, ... },
  "unresolved":      { "<fn name>": { "reason", "detail" }, ... }
}
```

A **`MonoEntry`** is a locator flattened to the top level, plus its grading and — where the address resolved offline — its measured `abi` footprint inline. A virtual method ships as a bare integer `offset` (its RTTI vtable slot index); a non-virtual function as a `signature` object with the `library` it scans and a space-hex `linux` pattern with `?` wildcards. By deriver convention an entry carries one or the other; the readers handle the rare both-present case deterministically.

`reason` on an unresolved entry comes from a closed vocabulary: `sig-drifted`, `offset-low-conf`, `unresolved`, `abi-drift`.

Console-command handlers ship under the key **`ConCommand::<name>`**. The prefix says what the entry *is* — the handler bound to that command — rather than claiming a C++ symbol; `ent_fire`'s real method name appears nowhere in the binary.

### `abi-<game>.json` — declared prototypes, judged against this build

Gamedata says *where* a function is; it never says what it takes. Types cannot be recovered from a stripped binary, so they come from declarations — and a declaration must be checked before anything calls through it, because a stale one produces a call that resolves, validates, and loads the wrong registers.

**The verdict is the product**, and there are six:

| verdict | meaning |
|---|---|
| `verified` | declared arity matches the footprint measured in *this* build |
| `lower-bound` | the declaration passes registers the callee never reads, and contradicts it in no register class — safe to call, but not the same claim as an exact match |
| `mismatch` | the callee reads a register the declaration does not mention — **do not call through it** |
| `return-only` | a return type is known and no parameter list, so there is no arity claim to check |
| `unverified` | nothing to check it against |
| `ambiguous` | several signatures on offer and no measurement to separate them |

Types come from three places, and `provenance` says which: a source declaration; the engine's own **dispatch contract**; or, for a return with neither, the measured register class (written `ret=…` so it can never be mistaken for a declared type — the measured class is wrong about known-void functions roughly seven times in eight).

**Two dispatch contracts exist**, and both are stronger than any header because nobody has to have written the function down for the way the engine invokes it to be known: an entity-IO handler is invoked through `void(CEntityInstance*, InputData_t&)`; a console-command handler through the command-context and command pair, plus a receiver where the registration dispatches through an object. Which of the three callback forms a command uses is recorded at the registration site, so the contract is keyed on it rather than assumed. A contract is judged as a **lower bound** — it describes how the function is *invoked*, so only an over-count refutes it.

In the artifact, parameters are spelled as **pointers** (`CCommandContext*`, `CCommand*`), not as the C++ reference types.

`source2rosetta-gen --abi … --format <framework>` turns this into **call sites**: typed C# fields for CounterStrikeSharp, a C++ typedef header for Metamod plugins, an `[AddressKey]` interface for ModSharp, and runtime type descriptors for Swiftly and Plugify. Both `verified` and `lower-bound` entries with a settled receiver are emitted, and every output marks the lower-bound ones.

### `bindings-<game>.json` — the declared callable surface

What the binary *says about itself*, as opposed to what the derivation *infers about it*. Kept out of the monolith deliberately: `gamedata` answers "where is this function", this answers "what may I do with it, and how". **Five sections:**

- `pulse` — bindings keyed by qualified `Class::Method`, each with Valve's display name and description, a decoded call policy (receiver kind, mutating, blocking) with the raw words beside it, and the recovered typed signature.
- `entity_inputs` — the map-facing input name, the C++ handler, its owning class where the schema join qualified it, and the handler's **address** (these also ship as gamedata).
- `entity_outputs` — the events an entity fires and where the subscriber list lives on the instance.
- `entity_classes` — map classname → the C++ class it constructs (`func_door` → `CBaseDoor`). Names to names, no addresses.
- `commands` — console commands with description, decoded flags, the raw flags word, the callback form, the measured ABI shape and the handler **address** (these also ship as gamedata).

**`descriptor` on a Pulse binding is NOT a locator** — see [above](#3-names-valve-ships-in-the-binary--three-sources-and-only-two-locate). Any Pulse count is a count of *registrations*, not distinct bindings.

### `netvars-<game>.json` — the typed schema

Every SchemaSystem class → field → offset and type, plus two sections that are easy to miss and load-bearing:

- `bases` — the class base graph. Without it an inherited field is unresolvable.
- `types` — per-type size and SysV register class, needed to compute a by-value argument's register cost.

Of a field's attributes, `type` and `kind` are read from the **live process**; `size` and the name hash are derived offline.

### `model-<game>.json` — the per-game model

The distilled facts derivation reads instead of the corpus. Not a consumer artifact.

### Rendering — the `gen` binary

`source2rosetta-gen` takes the monolith (`--from`), the schema (`--netvars`) or the manifest (`--abi`) and renders the matching format. The **input** chooses what is rendered; the `--format` id chooses for whom.

Two things to know before you diff outputs:

- The default `cssharp` gamedata output is **JSONC** — it carries comment banners, so a strict JSON parser will reject it.
- The `swiftly` gamedata format emits **signature entries only**; vtable-offset entries are omitted (that is over a thousand entries on CS2), because that framework takes offsets through a separate file.
- The `model` format emits a tier-selected `Gamedata` with no `meta`, so its output cannot be fed back in via `--from`.

---

## Provenance

This is a publishable tool, so where declarations and names come from is a hard boundary, not a footnote.

**Harvested:** Valve-published artifacts (the shipped binaries themselves, Valve's own metadata and command dumps, symbolicated older macOS builds) and legitimately licensed open-source projects, credited in [ATTRIBUTIONS.md](ATTRIBUTIONS.md).

**Excluded, and this is enforced rather than intended:** the 2020 CS:GO source leak and anything descending from it; `hl2sdk`-derived vendored SDK trees, which are quarantined on ambiguous provenance; and one widely-copied engine header that carries an annotation tying it to a fork whose own commit messages reference leaked code — every copy of it across the ecosystem shares that lineage, so all of them are denylisted. Where a harvested repository vendors a rejected tree as a submodule, only that project's own sources are read, and the extractor **asserts** the exclusion holds rather than assuming it — a denylist that matches nothing reads as a guarantee while enforcing nothing.

**Joining is by function, never by name.** A third-party plugin keys its gamedata by its own labels, so a declaration is attached only when the label *is* one of our names, or when its byte signature matches exactly one of ours in the same library. Matching on a bare method name is gated hard and withdrawn when the binary contradicts it.

**The corpus is never redistributed.** It is Valve's binaries; the model distilled from it contains derived facts, not code.

---

## Known limitations

- **Linux x86-64 only.** SysV register classification, `/proc`-based validation and ptrace are all platform-specific.
- **Dota's core leans on the unverified fallback** — 69.8% of it, against CS2's 0.4%. Those entries are marked; treat the marker as real.
- **`experimental` is never live-validated.** Resolvable locator, unverified name.
- **Offline runs ship no netvars and no `validated` state**, because field types and validation both require a running process.
- **Two games cannot be produced concurrently** on one host (fixed server port).
- **The Pulse duplicate-registration disagreement is open** — see [What is NOT gated](#what-is-not-gated).
- **Some declared returns are decided by source order.** Where sources disagree on a return type, the untrusted source is ranked last, but among the trusted ones the order is the order they were merged. A handful of names are settled that way, which is stated rather than papered over.

---

## Copyright

Valve, Counter-Strike, Dota and Source 2 are trademarks of Valve Corporation. This project is not affiliated with or endorsed by Valve. It ships no Valve code and no Valve binaries — only facts derived from publicly shipped files.

## License

AGPL-3.0. See [LICENSE](LICENSE).