source2rosetta

Re-derives Source-2 engine gamedata from stripped Valve binaries — and proves it on a live server.

When Valve ships a CS2 or Dota 2 engine update, every Metamod / CounterStrikeSharp plugin breaks until someone hand-reverse-engineers fresh gamedata — function signatures, vtable offsets, netvar layouts. Historically that can be up to weeks. source2rosetta reads the stripped .so libraries a dedicated server maps and re-derives the whole surface in minutes, then launches its own vanilla server and calls the functions to prove they're right.

It's a standalone Rust tool, not a plugin or a framework — the gamedata it emits renders into whatever your stack already speaks (CounterStrikeSharp, Metamod/SourceMod, ModSharp, Swiftly, Plugify, or a typed C# SDK). CS2 and Dota 2 are both live-validated; a new game is a --game arm away.

Docs

• ATTRIBUTIONS.md — start here. This tool stands on a decade of community reverse-engineering, catalogues, dumpers, and research. The credits come first because the work does.
• 🎯 Just want the gamedata? → Grab a release and render it for your framework. Download the published CS2 / Dota 2 gamedata, point source2rosetta-gen at it, and get CounterStrikeSharp / Metamod / ModSharp / Swiftly / Plugify / a typed C# SDK in one command. No build, no corpus — the 30-second path, and what most people are here for.
• CONTRIBUTING.md — add or back-fill a gamedata entry.
• LICENSE — AGPL-3.0.

Results

Ballpark from a recent build, on a 16-core desktop. These move build-to-build — treat them as orders of magnitude, not guarantees.

	derived functions	typed schema	model	one-time distill
CS2	~1,150 core + ~1,200 high_confidence, all live-validated, plus ~4,400 experimental name guesses	~1,900 classes / ~12,300 fields	~48 MB (a few MB gzipped)	~15 min
Dota 2	~1,900 core + ~1,000 high_confidence, plus ~6,100 experimental	~2,960 classes / ~17,700 fields	~570 MB	~1 hr

Both derive 0-dropped — every offset and signature that ships passed live validation. Distilling the model is a one-time cost; after that, a per-build update (fold the new build in, re-derive) is a couple of minutes — the "minutes, not weeks" the headline is about.

---

How it works — read → derive → validate → emit

1. Read — the whole server, not just libserver

A dedicated server maps roughly 22 shared libraries, and libserver.so is only a fraction of the reachable engine surface. source2rosetta reads all of them (multilib): libserver, libengine2, libtier0, libnetworksystem, libschemasystem, and the rest — each locator it emits carries the library it belongs to. ELF parsing is done by hand; .rela.dyn relocations are resolved so that .data.rel.ro pointer slots (zero on disk) come back as their true as-loaded values.

2. Derive

Offsets & netvars — near-deterministic. Vtable offsets come from Itanium C++ RTTI: the type hierarchy and vtable layout are read straight out of the binary, so a method's slot index is a fact, not a guess. Field layouts come from Valve's own SchemaSystem reflection tables — the engine emits class/field metadata (name, type, offset) as static data for its own use, and source2rosetta reads it directly. No fingerprints, no guessing.

Non-virtual signatures — located, then verified. A stripped, non-virtual function has no slot and no symbol, so it has to be found. source2rosetta computes a recompilation-invariant fingerprint of each catalogued function — CFG shape, call-graph degree, mnemonic histograms, imported-symbol references. These are deliberately abstracted statistics, never raw bytes. Locating a function in a new build is nearest-history under a plain, unweighted L1 distance over those features, accepted only within a small fixed recompile threshold. To be exact about what this is not: there is no trained model, no machine learning, no learned or weighted metric, no embedding network — it's a deterministic nearest-neighbour lookup against recent history, and the per-game "model" it reads is a bundle of derived facts, not a network. Every located address is then re-verified independently of the match: a fresh byte-signature is regenerated at the predicted address and confirmed to be unique in the target library and on a function prologue — the same check a loader does. A match that doesn't verify isn't shipped.

ABI-shape guard. A byte-signature sees a function's body drift and re-derives it, but it can't see the argument list change while the prologue stays recognisable — the sig still resolves and points at real code, yet a caller using the old prototype passes the wrong registers. source2rosetta recovers each function's observable SysV-AMD64 ABI shape (which argument registers are live-in, plus the return class) via a bounded backward-liveness pass and diffs it across builds, flagging exactly those prototype changes and marking struct-by-value (sret) returns that are unsafe to blind-call.

3. Validate — against a live server, not a spec

This is what separates source2rosetta from a static dumper. produce and integration-test launch their own vanilla dedicated server for the game (bots on an empty deathmatch for pawn games; a pawn-less game like Dota waits on a ready_class proxy) — no Steam, no separate instance, no human. Then, reading the running process through /proc/<pid>/mem (read-only ptrace — no injection, no debugger), it checks against ground truth:

• every offset lands on a real vtable slot, and every signature on live executable code;
• a gamedata function is actually called via ptrace to prove it's the semantically right function, not a plausible byte-match (pawn games);
• derived probes are fuzzed across changing game state for many iterations;
• field types are read from the live process for the typed netvars — and fields that are non-null live but zero on disk (e.g. m_pSchemaBinding) confirm the reader is seeing real live state, not stale disk bytes.

The contract is blunt: "degrades or stops loudly, never lies." An entry live validation confidently rejects is dropped, not shipped under a banner claiming it resolves; if the oracle can't run, it fails loudly rather than emit an unverified result.

4. Confidence tiers — nothing vanishes silently

The output is a per-game monolith in which every catalogue entry is accounted for, sorted into four tiers:

tier	meaning
core	derived and, in a full run, live-validated — the load-bearing gamedata
high_confidence	corroborated names folded in as verified offsets/sigs (dictionary-exact or macOS ground-truth transfer)
experimental	the least-filtered band — every graded name guess, each with a resolvable locator but an unverified name
unresolved	catalogued but not confidently produced this build, with a reason (sig-drifted, offset-low-conf, …) and no locator

A function that couldn't be derived this build shows up as unresolved with a reason — it never just disappears.

5. Emit

The canonical model and every format emitter live in crates/source2rosetta-core; the deriver writes format-neutral JSON and the tiny source2rosetta-gen binary renders it into any framework's shape (see Artifacts & formats).

Two things to keep straight. Multi-game: the ELF / RTTI / SysV / SchemaSystem machinery is engine-generic; game-specific knobs (library set, launch spec, schema-probe classes, pawn anchor) live on a profile::GameProfile, and --game <cs2|dota2> selects one — adding a game is a const plus a clap arm. Naming is not derivation: AI-assisted name extrapolation exists but is producer-side dev tooling, not part of the shipped deriver, and every proposed name is gated on self-naming or ground-truth corroboration and then live-validated. A wrong name only mislabels a real slot; it never touches the offset, which comes from RTTI.

---

Install & CLI usage

# The deriver (`source2rosetta`) — the root binary.
cargo build --release                        # → ./target/release/source2rosetta

# The renderer (`source2rosetta-gen`) lives in the core crate and is NOT built by the root
# build — build it explicitly (or use --workspace). See crates/source2rosetta-core/README.md.
cargo build --release -p source2rosetta-core # → ./target/release/source2rosetta-gen

--game <cs2|dota2> is a global flag (default cs2), given before the subcommand: source2rosetta --game dota2 produce ….

command	one line
produce	The whole per-game build in one command: derive → fold → (with --game-dir) validate-live + typed netvars → roll the model forward, into --out-dir. --game-dir present = full live-validated build; absent = fast offline build (gamedata + model only). That flag is the entire offline/full switch.
corpus-model	Distill a corpus of past builds into one shippable model (vtable-alignment hops, reference fingerprints, slot timelines), so future derivation needs only the model + the target binary, not the corpus.
fold-model	Roll an existing model forward by ONE build (model N + build → N+1), reading only the model and that one binary — equal to a full re-distill. The production update path (also a sidecar inside produce).
integration-test	Stand-alone CI live oracle: launch a vanilla server, populate it, and verify derived gamedata against it — schema oracle, a semantic ptrace CALL on a live pawn, and (with --gamedata) a full validate-live plus optional live fuzzing.
backfill	Give an extrapolated name a real cross-build timeline — resolve its string anchor in every corpus build, or chain a vtable slot through the model — and report history depth + consistency (how a guess graduates to first-class).
classify-change	--prev/--new → skip / normal / shift + the exact % of function bodies that changed, comparing with position-dependent bytes masked so a pure layout shift reads as unchanged. Decides whether a build even warrants a re-derive.
filter-corpus	Collapse runs of code-identical builds to one representative, label each transition normal/shift, and segment the timeline into toolchain eras. Writes the selection manifest the distill reads.

Quickstart

Releases live on the releases page. For the offline path you need one file from there — the model (model-<game>.json) — plus the derive inputs, which ship in this repo under mappings/. Put the downloaded model wherever you like; the examples assume it's in the working directory.

# OFFLINE — derive gamedata + roll the model forward. No server, fully deterministic.
./target/release/source2rosetta --game cs2 produce \
    --seed         mappings/seed-cs2.json \
    --corpus-model model-cs2.json \
    --target       <build-dir> \
    --out-dir      out

# FULL — the same, plus it launches its own vanilla+bots server to validate on the live
# process and read field types for the typed netvars. Adding --game-dir is the only change.
./target/release/source2rosetta --game cs2 produce \
    --seed         mappings/seed-cs2.json \
    --corpus-model model-cs2.json \
    --target       <build-dir> \
    --game-dir     <cs2-install> \
    --out-dir      out

• --target <dir> (required) — the build directory to derive from; its libraries are searched by name, so pass the directory, not a bare .so.
• --seed <bundle> — one file bundling every derive input (catalogue + optional naming/offset/sig sections). The loose equivalent is --catalogue <file> plus the optional --promotable / --candidates / --full-names / --extra-offsets / --extra-sigs, all defaulting to empty — so a brand-new game needs only a catalogue to start deriving.
• Corpus signal — exactly one of --corpus-model <model.json> (the normal path: forward-derive from the model + the target binary, and roll the model N→N+1 as a sidecar) or --corpus <dir> (fingerprint the raw build binaries on the fly).

---

Fork it & distill your own model

Nothing is hosted — fork it, cargo build --release, and point it at a build on disk. Two ways to run, depending on whether you already have a model:

• Have a model (downloaded from releases, or distilled): --corpus-model model-<game>.json forward-derives from the model plus only the target binary — no corpus on disk. This is the normal path.
• No model yet: distill one from a corpus of past builds. "Distilling" is what this project means by "training" — there's no ML (see Derive above); the model is the bundle of facts already described: vtable-alignment hops, reference-fingerprint windows, ABI-shape consensus, slot timelines.

# Distill a corpus into a model (streaming, bounded RAM even over Dota's ~1k builds).
./target/release/source2rosetta --game cs2 corpus-model \
    --seed   mappings/seed-cs2.json \
    --corpus corpus/binaries \
    --out    model-cs2.json

--class-scope (default clean — every real game class, enough for any modding offset to derive model-only) picks which classes get slot hops. Whatever scope you distill with, fold-model and produce's sidecar fold must use the same one.

Keeping a model fresh — the incremental fold

Once a model exists you never need the corpus again. fold-model rolls it forward one build, reading only the model plus the single new binary — identical to a full re-distill:

./target/release/source2rosetta --game cs2 fold-model \
    --model model-cs2.json \
    --seed  mappings/seed-cs2.json \
    --build <new-build-dir> \
    --out   model-cs2.next.json

produce --corpus-model runs exactly this fold as a sidecar, so a full build both derives and advances the model in one command. (--class-scope must match the model's.)

Getting the corpus (only to bootstrap a model)

A corpus is a directory of past builds, one subdirectory of .so files per build (corpus/binaries/<label>/*.so). Fetch it yourself, one time:

1. Use DepotDownloader — the self-contained release binary from https://github.com/SteamRE/DepotDownloader/releases, not dotnet tool install (its NuGet package is pinned ancient).
2. Pull manifests from the Linux binaries depot 2347773 — not the content depot 2347770. 2347773's manifest only advances when the binaries actually change, so its history already is the list of real recompiles; content micropatches only bump 2347770. Read the manifest history off SteamDB, not the Steam client.
3. Download oldest-first (chronological = version order), then content-hash-dedup. filter-corpus further collapses code-identical builds and segments toolchain eras before you distill, so you never fingerprint the same code twice.

A partial corpus is fine — fewer labels is a shallower history, not a broken model; skip very old manifests if they're un-downloadable.

Adding a game

Add a profile::GameProfile const (library set, schema-probe classes, launch spec, pawn anchor, dead-weight knobs) plus one --game clap-enum arm, then point corpus-model at that game's corpus. The rest is engine-generic.

---

Artifacts, schemas & output formats

A full produce run writes a small, self-contained release set per game into --out-dir:

File	What it is	When
gamedata-<game>.json	The monolith — the tiered function catalogue (signatures + vtable offsets) with provenance and live-validation folded inline	always
netvars-<game>.json	The typed schema — every SchemaSystem class → field → offset/type	full (--game-dir) runs only
model-<game>.json	The per-game model — the distilled facts derivation reads instead of the corpus (the shippable artifact)	when the run folds an existing model (--corpus-model)
manifest.json	Volatile release metadata: { version, artifacts: [...] }	always

Wall-clock and other volatile metadata live only in manifest.json; the monolith and schema carry no timestamp, so they're byte-reproducible — the same build in yields the same JSON out.

gamedata-<game>.json — the monolith

{
  "meta": { "game_key", "game", "source_build", "version",
            "counts": { "core", "high_confidence", "experimental", "unresolved" } },
  "core":            { "<fn name>": <MonoEntry>, ... },
  "high_confidence": { "<fn name>": <MonoEntry>, ... },
  "experimental":    { "<fn name>": <MonoEntry>, ... },
  "unresolved":      { "<fn name>": { "reason", "detail" }, ... }
}

A MonoEntry is a locator (flattened to the top level) plus its grading. The locator obeys a strict signature-XOR-offset invariant — a virtual method ships as a bare integer offset (its RTTI vtable slot index), a non-virtual function as a signature object with the library it scans and a space-hex linux byte pattern with ? wildcards:

{
  "signature": { "library": "server", "linux": "55 48 89 ? E5" },  // non-virtual: located scan pattern
  "offset": 158,                                                    // virtual: RTTI slot index (a bare int)
  "class": "CCSPlayerPawn",          // experimental offsets only: the vtable class, for an eyeball check
  "provenance": { "tier", "source", "confidence", "self_named", "by_value",
                  "rationale", "corroboration", "abi_drift", ... },  // grading; fields present by tier
  "validated": true                  // true = passed live validation · false = rejected · null = not validated (offline)
}

validated: false entries stay in the file for transparency but are dropped by every renderer.

The source2rosetta-gen renderers write a slightly different on-disk shape (plural {"signatures":{…}} / {"offsets":{…}} for CounterStrikeSharp-family output). The keys above are the canonical model JSON as gamedata-<game>.json stores it.

netvars-<game>.json — the typed schema

{
  "meta":    { "game_key", "source_build", "typed", "untyped" },
  "classes": { "<class>": { "<field>": { "offset", "type", "kind", "size", "name_hash" } } }
}

Offsets come from SchemaSystem tables (available offline); type / kind / size are read from the live process during a full run (kind ∈ ref | ptr | fixed_array). A full run refuses to ship a schema whose fields resolved wholesale-untyped rather than emit a typeless file — the same "stop loudly" contract.

model-<game>.json — the per-game model

The distilled per-game facts — vtable-alignment hops, reference-fingerprint windows, ABI-shape consensus, slot timelines — the artifact derivation reads instead of the corpus. What "distill a model" produces (above).

Rendering — the gen binary

The monolith and schema are format-neutral; source2rosetta-gen renders them, so the deriver never changes when a new consumer format is added.

Gamedata (--from gamedata-<game>.json, at a cumulative --tier core | high_confidence | experimental, default high_confidence): cssharp (CounterStrikeSharp), metamod (Metamod/SourceMod VDF), modsharp, swiftly, plugify, model (the canonical model re-serialized). Schema (--netvars netvars-<game>.json): cs-sdk (a typed C# SDK — one static class per schema class, const int field offsets tagged with their type), netvars (a flat { class: { field: offset } } map). Full render walkthrough: gen binary README.

---

Copyright

The build corpus — Valve's .so files (~86 GB) — is never shipped and never baked into a release. The published artifacts (model, gamedata, netvars) are designed to contain derived facts — vtable offsets, abstracted fingerprint statistics, and byte scan-patterns — rather than copies of the original code. That a statistic about code is a fact and not a copy is the project's position, not settled law — reverse-engineering Valve binaries under AGPL is exactly the territory a court hasn't ruled on, so use accordingly. Leaked or proprietary game source was deliberately kept out of the corpus and the naming pipeline; see ATTRIBUTIONS.md.

License

AGPL-3.0. Built on a decade of community work — see ATTRIBUTIONS.md first.