clean up bootstrapping stuff from CI, update README
All checks were successful
CI / lint (push) Successful in 16s
CI / fuzz (push) Successful in 1m39s
CI / test (push) Successful in 20s

This commit is contained in:
Kamal Tufekcic 2026-07-27 11:59:28 +03:00
commit 54ef572202
2 changed files with 41 additions and 28 deletions

View file

@ -23,7 +23,6 @@ jobs:
STEAM_APPS: /home/cs2/.steam/SteamApps STEAM_APPS: /home/cs2/.steam/SteamApps
STEAM_USER: source2rosetta STEAM_USER: source2rosetta
RELEASE_BASE: ${{ github.server_url }}/${{ github.repository }}/releases/download RELEASE_BASE: ${{ github.server_url }}/${{ github.repository }}/releases/download
BOOTSTRAP_DIR: /home/cs2/rosetta-bootstrap
steps: steps:
- uses: actions/checkout@v6.0.2 - uses: actions/checkout@v6.0.2
@ -56,25 +55,10 @@ jobs:
- name: Fetch the previous model + seed (the two non-user-facing release artifacts) - name: Fetch the previous model + seed (the two non-user-facing release artifacts)
run: | run: |
mkdir -p in dist work mkdir -p in dist work
code=$(curl -sSL -o in/model.gz -w '%{http_code}' "$RELEASE_BASE/$GAME-latest/model-$GAME.json.gz" || echo 000) curl -fsSL -o in/model.gz "$RELEASE_BASE/$GAME-latest/model-$GAME.json.gz"
if [ "$code" = "200" ]; then gunzip -c in/model.gz > "in/model-$GAME.json"
gunzip -c in/model.gz > "in/model-$GAME.json" curl -fsSL -o in/seed.gz "$RELEASE_BASE/$GAME-latest/seed-$GAME.json.gz"
echo "model: from the $GAME-latest release" gunzip -c in/seed.gz > "in/seed-$GAME.json"
elif [ "$code" = "404" ] && [ -f "${BOOTSTRAP_DIR:-}/model-$GAME.json" ]; then
# First run only. ONLY a genuine 404 falls back: on a transient failure we must NOT quietly
# re-derive from a stale on-disk model and then overwrite `latest` with the result.
cp "${BOOTSTRAP_DIR}/model-$GAME.json" "in/model-$GAME.json"
echo "model: BOOTSTRAP from $BOOTSTRAP_DIR — clear BOOTSTRAP_DIR once this run has published"
else
echo "model fetch failed (HTTP $code) and no bootstrap copy at '${BOOTSTRAP_DIR:-<unset>}/model-$GAME.json' — refusing to derive"
exit 1
fi
if curl -fsSL -o in/seed.gz "$RELEASE_BASE/$GAME-latest/seed-$GAME.json.gz"; then
gunzip -c in/seed.gz > "mappings/seed-$GAME.json"
echo "seed: from the $GAME-latest release"
else
echo "seed: none published yet — using the repo's mappings/seed-$GAME.json"
fi
- name: Produce — derive + validate-live + typed netvars + fold model N -> N+1 - name: Produce — derive + validate-live + typed netvars + fold model N -> N+1
run: | run: |
@ -84,14 +68,13 @@ jobs:
# would be rejected as not-newer. Buildids are monotonic and sort after the corpus's date labels. # would be rejected as not-newer. Buildids are monotonic and sort after the corpus's date labels.
ln -sfn "$GAME_DIR" "work/$BUILDID" ln -sfn "$GAME_DIR" "work/$BUILDID"
./target/release/source2rosetta --game "$GAME" produce \ ./target/release/source2rosetta --game "$GAME" produce \
--seed "mappings/seed-$GAME.json" \ --seed "in/seed-$GAME.json" \
--corpus-model "in/model-$GAME.json" \ --corpus-model "in/model-$GAME.json" \
--target "work/$BUILDID" \ --target "work/$BUILDID" \
--game-dir "$GAME_DIR" \ --game-dir "$GAME_DIR" \
--version "$GAME-$BUILDID-0" \ --version "$GAME-$BUILDID-0" \
--wait 300 \
--out-dir dist --out-dir dist
cp "mappings/seed-$GAME.json" "dist/seed-$GAME.json" cp "in/seed-$GAME.json" "dist/seed-$GAME.json"
- name: Compress the internal artifacts for release - name: Compress the internal artifacts for release
run: | run: |

View file

@ -1,15 +1,23 @@
# source2rosetta # source2rosetta
**Re-derives Source-2 engine gamedata from stripped Valve binaries — and proves it on a live server.** **Current CS2 and Dota 2 gamedata — re-derived from every Valve build, proven on a live server, published automatically.**
When Valve ships a CS2 or Dota 2 engine update, every Metamod / CounterStrikeSharp plugin breaks until someone hand-reverse-engineers fresh gamedata — function signatures, vtable offsets, netvar layouts. Historically that can be up to **weeks**. source2rosetta reads the stripped `.so` libraries a dedicated server maps and re-derives the whole surface in **minutes**, then launches its own vanilla server and *calls the functions* to prove they're right. When Valve ships an engine update, every Metamod / CounterStrikeSharp plugin breaks until someone hand-reverse-engineers fresh gamedata: function signatures, vtable offsets, netvar layouts. That has historically taken days, sometimes weeks.
It's a **standalone Rust tool, not a plugin or a framework** — the gamedata it emits renders into whatever your stack already speaks (CounterStrikeSharp, Metamod/SourceMod, ModSharp, Swiftly, Plugify, or a typed C# SDK). CS2 and Dota 2 are both live-validated; a new game is a `--game` arm away. Here it takes **about half an hour, with nobody involved.** A timer notices the new build, re-derives the whole surface from the stripped `.so` libraries the dedicated server maps, launches its own vanilla server and *calls the functions* to prove they resolve, then publishes to a fixed URL. No one is paged and nothing is hand-checked — and if any stage fails, the run stops and the previous release stays up. What ships is never a guess.
```sh
# always the newest build
curl -fsSLO https://git.lo.sh/kamal/source2rosetta/releases/download/cs2-latest/gamedata-cs2.json
curl -fsSLO https://git.lo.sh/kamal/source2rosetta/releases/download/cs2-latest/netvars-cs2.json
```
The output is framework-neutral; `source2rosetta-gen` renders it into whatever your stack speaks. The deriver behind it is a standalone Rust tool — you only need that if you're self-hosting the pipeline or adding a game.
## Docs ## Docs
- **[ATTRIBUTIONS.md](ATTRIBUTIONS.md) — start here.** This tool stands on a decade of community reverse-engineering, catalogues, dumpers, and research. The credits come first because the work does. - **[ATTRIBUTIONS.md](ATTRIBUTIONS.md) — start here.** This tool stands on a decade of community reverse-engineering, catalogues, dumpers, and research. The credits come first because the work does.
- **🎯 Just want the gamedata?** → **[Grab a release and render it for your framework.](crates/source2rosetta-core/README.md)** Download the published CS2 / Dota 2 gamedata, point `source2rosetta-gen` at it, and get CounterStrikeSharp / Metamod / ModSharp / Swiftly / Plugify / a typed C# SDK in one command. No build, no corpus — the 30-second path, and what most people are here for. - **🎯 Render a release for your framework** → **[source2rosetta-gen](crates/source2rosetta-core/README.md)** — one command turns the JSON above into CounterStrikeSharp, Metamod/SourceMod, ModSharp, Swiftly, Plugify, or a typed C# SDK. No build, no corpus; what most people are here for.
- [CONTRIBUTING.md](CONTRIBUTING.md) — add or back-fill a gamedata entry. - [CONTRIBUTING.md](CONTRIBUTING.md) — add or back-fill a gamedata entry.
- [LICENSE](LICENSE) — AGPL-3.0. - [LICENSE](LICENSE) — AGPL-3.0.
@ -22,7 +30,27 @@ Ballpark from a recent build, on a 16-core desktop. These move build-to-build
| **CS2** | ~1,150 `core` + ~1,200 `high_confidence`, all live-validated, plus ~4,400 `experimental` name guesses | ~1,900 classes / ~12,300 fields | ~48 MB (a few MB gzipped) | ~15 min | | **CS2** | ~1,150 `core` + ~1,200 `high_confidence`, all live-validated, plus ~4,400 `experimental` name guesses | ~1,900 classes / ~12,300 fields | ~48 MB (a few MB gzipped) | ~15 min |
| **Dota 2** | ~1,900 `core` + ~1,000 `high_confidence`, plus ~6,100 `experimental` | ~2,960 classes / ~17,700 fields | ~570 MB | ~1 hr | | **Dota 2** | ~1,900 `core` + ~1,000 `high_confidence`, plus ~6,100 `experimental` | ~2,960 classes / ~17,700 fields | ~570 MB | ~1 hr |
Both derive **0-dropped** — every offset and signature that ships passed live validation. Distilling the model is a one-time cost; after that, a per-build update (fold the new build in, re-derive) is **a couple of minutes** — the "minutes, not weeks" the headline is about. Both derive **0-dropped** — every offset and signature that ships passed live validation. Distilling the model is a one-time cost; after that each build's re-derive is minutes of compute, and the half hour in the headline is the whole loop: notice, update, derive, validate, publish.
---
## Staying current — the part with no human in it
Each game runs its own loop, independently:
1. A timer polls Steam every 15 minutes, comparing the installed build id against the live one.
2. On a change it updates the install and runs a single `produce`: derive → live-validate → typed netvars → roll the model forward.
3. It publishes an immutable `<game>-<buildid>-<patch>` snapshot, then moves `<game>-latest` onto it.
A CS2 update never rebuilds Dota, and vice versa. Two rules keep it honest: every stage **hard-fails rather than substituting** an older or on-disk input, and every entry in `core` / `high_confidence` is confirmed against the live process before it ships. A failed run publishes nothing and leaves the previous release standing.
| you want | use |
|---|---|
| the newest build, always | `…/releases/download/cs2-latest/gamedata-cs2.json` |
| a specific build, pinned | `…/releases/download/cs2-<buildid>-0/gamedata-cs2.json` |
| to know what you got | `manifest.json` — carries `version = <game>-<buildid>-<patch>` |
Follow `-latest` to adopt updates as they land, or pin a buildid tag to adopt them deliberately; old snapshots stay up either way. Whichever you choose, **check the manifest's build id against the server you're actually running** before loading — that is what stops stale offsets meeting a changed binary. (`patch` counts rebuilds on the same binary, e.g. a merged contribution.)
--- ---
@ -74,6 +102,8 @@ Two things to keep straight. **Multi-game:** the ELF / RTTI / SysV / SchemaSyste
## Install & CLI usage ## Install & CLI usage
The two binaries have different audiences. **`source2rosetta`** (the deriver) is only needed to run the pipeline yourself, fork it, or add a game. **`source2rosetta-gen`** is needed by anyone using a release — a release is framework-neutral JSON, so something has to render it into your stack's format — but you can download the prebuilt binary from a `gen-v*` release instead of building it, provided you're on Linux x86-64. Anywhere else, build it from source.
```sh ```sh
# The deriver (`source2rosetta`) — the root binary. # The deriver (`source2rosetta`) — the root binary.
cargo build --release # → ./target/release/source2rosetta cargo build --release # → ./target/release/source2rosetta