act on what the binary declares: callable Pulse shims, ConVars, string anchors; gen v2.1
This commit is contained in:
parent
3de955c4ff
commit
71ce34edd2
14 changed files with 1507 additions and 54 deletions
57
README.md
57
README.md
|
|
@ -14,7 +14,8 @@ curl -fsSLO $R/abi-cs2.json # HOW to call them — parameter and return t
|
|||
```
|
||||
|
||||
Also published: `bindings-<game>.json` (the callable surface the binary declares about itself — Pulse
|
||||
bindings, entity IO, console commands) and `manifest.json` (which build you got). The
|
||||
bindings with a callable shim, entity IO, console commands, ConVars) and `manifest.json` (which build you
|
||||
got). The
|
||||
[artifacts section](#artifacts-schemas--output-formats) covers all of them.
|
||||
|
||||
The output is framework-neutral; `source2rosetta-gen` renders it into whatever your stack speaks — the gamedata into your framework's locator format, and `abi-<game>.json` into **typed call sites** for the same functions. The deriver behind it is a standalone Rust tool — you only need that if you're self-hosting the pipeline or adding a game.
|
||||
|
|
@ -30,10 +31,12 @@ The output is framework-neutral; `source2rosetta-gen` renders it into whatever y
|
|||
|
||||
Ballpark from a recent build, on a 16-core desktop. These move build-to-build — treat them as orders of magnitude, not guarantees.
|
||||
|
||||
| | derived functions | typed schema | model | one-time distill |
|
||||
|---|---|---|---|---|
|
||||
| **CS2** | ~1,125 `core` + ~2,620 `high_confidence`, plus ~4,375 `experimental` name guesses | ~1,900 classes / ~12,300 fields | ~48 MB (a few MB gzipped) | ~15 min |
|
||||
| **Dota 2** | ~1,930 `core` + ~2,450 `high_confidence`, plus ~5,950 `experimental` | ~2,960 classes / ~17,700 fields | ~570 MB | ~1 hr |
|
||||
| | derived functions | declared surface | typed prototypes | typed schema | model | one-time distill |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **CS2** | ~1,125 `core` + ~2,620 `high_confidence`, plus ~4,375 `experimental` name guesses | 580 Pulse bindings (127 host-callable), 784 commands, **1,551 ConVars**, 715 entity inputs / 226 outputs, 474 classnames | ~2,055 `verified` + ~80 `lower-bound`, 55 `mismatch` | ~1,900 classes / ~12,300 fields | ~48 MB (a few MB gzipped) | ~15 min |
|
||||
| **Dota 2** | ~1,930 `core` + ~2,450 `high_confidence`, plus ~5,950 `experimental` | 500 Pulse bindings (99 host-callable), 855 commands, **1,170 ConVars**, 624 entity inputs / 187 outputs, 3,528 classnames | ~3,635 `verified` + ~100 `lower-bound`, 44 `mismatch` | ~2,960 classes / ~17,700 fields | ~570 MB | ~1 hr |
|
||||
|
||||
**Declared surface** is what the binary states about itself, and it is a different kind of fact from the rest: no inference, no cross-build chaining, no confidence tier. The *host-callable* count is the subset of Pulse bindings invocable with an argument array alone — verified by calling each one on a live server of both games.
|
||||
|
||||
A full run live-validates what it ships and reports **0 dropped** on both games — for CS2 that is ~2,610 signatures and ~1,120 vtable offsets checked against a running server. Distilling the model is a one-time cost; after that each build's re-derive is minutes of compute, and the half hour in the headline is the whole loop: notice, update, derive, validate, publish.
|
||||
|
||||
|
|
@ -66,9 +69,13 @@ The artifacts answer four different questions, and most useful work joins two or
|
|||
- **`gamedata-<game>.json` — where the code is.** Every entry is a hook point or a call target: a byte signature or an RTTI vtable slot, tiered and, for `core`/`high_confidence`, checked against a running server.
|
||||
- **`netvars-<game>.json` — what the state is.** Field offsets and types for every SchemaSystem class, plus the base graph, the enum tables and per-type sizes. This is the half that needs no hooking at all: a great deal of gameplay is readable and writable as plain memory.
|
||||
- **`abi-<game>.json` — whether it is safe to call.** A declared prototype joined to the register footprint measured in *this* build, with a verdict per function. `verified` and `lower-bound` are callable; `mismatch` says the prototype in circulation is wrong for this binary.
|
||||
- **`bindings-<game>.json` — what the binary declares about itself.** Console commands with flags and handler addresses, entity inputs and outputs, map classname → C++ class, and the typed Pulse registry.
|
||||
- **`bindings-<game>.json` — what the binary declares about itself.** Console commands and ConVars with decoded flags, entity inputs and outputs, map classname → C++ class, and the typed Pulse registry with a callable entry point per binding.
|
||||
|
||||
One thing shapes every capability below: **there are no ConVars in these artifacts, in either game.** The command surface is 784 ConCommands on CS2 and 855 on Dota, and not one `sv_airaccelerate`, `mp_freezetime`, `bot_quota` or `dota_gold_per_tick` appears anywhere. This is a *function-and-layout* release, not a configuration release. A plugin built on it changes behaviour by detouring code and writing fields — never by setting a cvar.
|
||||
Two of those are newer than the rest and worth calling out, because they change what a plugin can do:
|
||||
|
||||
**ConVars ship with their flags.** 1,551 on CS2 across four libraries, 781 in Dota's `libserver` — with `cheat`, `replicated`, `archive` and `notify` decoded, and the raw word beside them. The names are not the point: a consumer finds a convar by name at runtime with no gamedata at all. The *flags* are, because they are engine-declared authority. A host that wants to say "this module may change gameplay settings but not cheat-protected ones" can key that on what the engine itself declares instead of maintaining an allowlist by hand.
|
||||
|
||||
**Most of the Pulse surface is callable.** Each binding carries a `shim` address and a `call.needs` verdict; the `args-only` tier — roughly 110 on CS2, 82 on Dota within `libserver` — is invocable with an argument array and nothing else, through Valve's own marshalling, which enforces the binding's declared types. Those are *actions* (teleport, ignite, change team, start a mover, spawn a template), which is the half no field write can do; reading state remains the schema's job and is better served there.
|
||||
|
||||
Tiers stay visible throughout. `core` and `high_confidence` are buildable today; the `experimental` band is fenced off at the end and is a different kind of thing entirely.
|
||||
|
||||
|
|
@ -152,7 +159,7 @@ That is one leg of a defensive integrity monitor for the server process an opera
|
|||
|
||||
The console surface is the most mechanically reliable thing in the release. 755 of CS2's 784 registered commands carry a hookable `ConCommand::<name>` locator with a `verified` prototype — that is *all* commands, not a diagnostics subset, though the diagnostics family within it is broad (`stats_print`, `sv_packstats` with its `clear` argument, the `vprof_*` family, `mem_dump`, `net_stats_json`, `status_json`, `lrucache_stats`, `check_nofilefd`). Hooking is the reliable direction: the artifacts give you the callback address, not the `CCommand` layout needed to synthesize a call. `logaddress_add_http` ships log fan-out to an arbitrary URI with no sidecar, and tier0 carries a complete scripted-test harness (`Test_StartScript`, `Test_LoopForNumSeconds`, `Test_Checkpoint`, `Test_ExitProcess` with a chosen exit code) that is a CI rig Valve already wrote.
|
||||
|
||||
Because command flags are decoded, the client-reachable attack surface is exactly enumerable rather than folklore: **30 CS2 commands carry `client_can_execute`**, including `ent_setpos` and `ent_setang` — which move *arbitrary entities* — alongside `give`, `god`, `noclip`, `kill`, `explode`, `setpos_player`, `callvote` and `replay_start`. Exactly two carry `server_can_execute` (`echo`, `play`), which answers a question plugin authors argue about: the server cannot push arbitrary console commands to clients through the normal path. That static audit is solid. Detecting a *runtime* change to those flags is not — every cvar-registry accessor (`CCvar::GetConVarFlags`, `CCvar::FindCommand`, `CCvar::RegisterConCommand`) is experimental with a guessed name.
|
||||
Because command flags are decoded, the client-reachable attack surface is exactly enumerable rather than folklore: **30 CS2 commands carry `client_can_execute`**, including `ent_setpos` and `ent_setang` — which move *arbitrary entities* — alongside `give`, `god`, `noclip`, `kill`, `explode`, `setpos_player`, `callvote` and `replay_start`. Exactly two carry `server_can_execute` (`echo`, `play`), which answers a question plugin authors argue about: the server cannot push arbitrary console commands to clients through the normal path. That static audit is solid, and it now extends to ConVars: their flags are decoded the same way, so `cheat`-guarded and `replicated` tunables are enumerable rather than assumed. Detecting a *runtime* change to any of those flags is a different matter — every cvar-registry accessor (`CCvar::GetConVarFlags`, `CCvar::FindCommand`, `CCvar::RegisterConCommand`) is experimental with a guessed name.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -234,7 +241,7 @@ The only defensible workflow for anything in this band: pick a candidate, check
|
|||
|
||||
### What is not covered
|
||||
|
||||
- **No ConVars, in either game.** Commands only. No `sv_*`/`mp_*`/`dota_*` tunables, no defaults, no flags, no ranges.
|
||||
- **ConVars ship names, help and flags — but no defaults or ranges.** The default value is built in a stack structure at the registration site rather than passed as a literal, so it is not recoverable the way the rest is. `min`/`max` likewise. If you need the shipped default, read it off a running server.
|
||||
- **No protobuf field layouts.** You get message ids and class names; you must supply the `.proto` definitions.
|
||||
- **No game-event name tables.** The event *system* is there (post, register, the legacy bridge); the names (`player_death`, `dota_player_gained_level`) are not. On CS2 the practical substitutes are function-level equivalents and `logic_gameevent_listener`, which needs only a string.
|
||||
- **No content names.** No `.vpcf` particle systems, no sound events, no model paths, no Dota KeyValues gameplay data (no ability special values, no hero base stats, no item costs).
|
||||
|
|
@ -285,9 +292,13 @@ Every Source-2 module names some of its own functions. This is ground truth from
|
|||
|---|---|---|
|
||||
| **entity-IO datadesc** | the C++ handler name (`InputKill`) with the handler's address | **yes** |
|
||||
| **console-command registration** | the command name (`bot_add`) with its callback | **yes** |
|
||||
| **Pulse binding registry** | a qualified `Class::Method` with display name, description, call policy and a full typed signature | **no** |
|
||||
| **Pulse binding registry** | a qualified `Class::Method` with display name, description, call policy, a full typed signature and an invocation shim | **not as a C++ symbol** |
|
||||
|
||||
**The Pulse registry does not locate anything, and the mistake is instructive.** Its records carry two code pointers, which an early pass folded as locators for a headline `+782` names. They are *descriptor accessors* — every CS2 `libserver` binding measures the same empty `int=0 float=0` footprint, and they disassemble to a lazy-init singleton that returns a static vector. Folding them would have shipped `CBaseEntityAPI::GetAbsOrigin` pointing at a zero-argument accessor: a locator that resolves, passes live validation as executable code, and is still the wrong function. The honest yield from that table is zero locators — and a complete typed API surface, which ships separately as `bindings-<game>.json`.
|
||||
**Two of the Pulse record's three code pointers do not locate anything, and the mistake is instructive.** An early pass folded the pair at `+24`/`+32` as locators for a headline `+782` names. They are *descriptor accessors* — every CS2 `libserver` binding measures the same empty `int=0 float=0` footprint, and they disassemble to a lazy-init singleton returning a static vector. Folding them would have shipped `CBaseEntityAPI::GetAbsOrigin` pointing at a zero-argument accessor: a locator that resolves, passes live validation as executable code, and is still the wrong function.
|
||||
|
||||
**The third pointer, at `+72`, is a real entry point** — one per binding, never shared. It is not the bound C++ method and cannot be folded as one (that method's address is genuinely unrecoverable offline; the shim dispatches indirectly). It is a fixed-signature marshalling stub, and calling it *invokes the binding*. That is verified by doing it: `SetRenderAlpha` and `SetRenderColor` were called on a live CS2 server and moved the entity's `m_clrRender`, and a `SetRenderColor` on Dota set the RGB bytes while leaving the alpha byte untouched — proof that dispatch honours the binding's DECLARED `PulseValueType_t`, so a caller cannot smuggle a mistyped argument past it. Every eligible binding is re-checked on each derive by calling it with a sentinel handle (see [the standing oracles](#the-standing-oracles)).
|
||||
|
||||
So the honest yield from this table is **zero C++ symbol locators, a complete typed API surface, and a callable entry point per binding** — all of which ship in `bindings-<game>.json`, with `call.needs` stating what a host must supply for each.
|
||||
|
||||
**The datadesc handlers are class-qualified** by joining each record's array against the SchemaSystem: a datadesc array also carries field descriptors, and a `(member, offset)` pair is something the schema states from an entirely different table, so the class whose schema satisfies every pair in the array owns it. 653 of 715 CS2 handlers qualify this way. This is what makes `InputEnable` — a distinct handler on 48 classes — nameable at all; unqualified names that resolve to several addresses are still **dropped, not guessed**. Note for consumers: qualification **renamed 343 shipped keys** (`InputActivateSkybox` → `CAmbientGeneric::InputActivateSkybox`).
|
||||
|
||||
|
|
@ -374,6 +385,8 @@ A `produce` run **aborts rather than publish** a collapsed artifact. Each surfac
|
|||
| entity-IO records | inputs + outputs | |
|
||||
| entity classnames | factory-record count | |
|
||||
| console commands | recovered-command count | the registrar is found by SHAPE, so a reworked constructor yields **zero** commands rather than wrong ones — correct, and invisible without this |
|
||||
| host-callable Pulse shims | count of `call.needs == "args-only"` | the registry can read perfectly and every signature recover perfectly while a codegen change makes each shim appear to read another argument — retiring the one callable tier without failing anything |
|
||||
| ConVars | recovered-convar count | found by a DIFFERENT test than commands — convergence of registrar wrappers on a shared core, not a sentinel in the callee — so it can collapse while commands keep working |
|
||||
| typed netvars | fraction of fields typed | a runtime type-layout reshape resolves every field "untyped" and would otherwise ship a typeless schema at exit 0 |
|
||||
| schema enums | enum count | read by shape like the class table, so a reshape yields zero rather than wrong |
|
||||
| live validation | pass rate, above a minimum sample | |
|
||||
|
|
@ -394,6 +407,7 @@ A dozen free, mostly two-sided checks run on **every** derive and are reported.
|
|||
- **EHANDLE class grouping** — Valve's naming vs the binary's destructor addresses: 0 of 44 CS2 / 41 Dota groups carry two classes.
|
||||
- **Pulse element stride** — derived by consensus per image, unanimous across six libraries in both games.
|
||||
- **live schema oracle** — offline layout vs the running process: 852/852 CS2, 1,912/1,912 Dota.
|
||||
- **Pulse shim invocation** — the only *behavioural* oracle here: every binding the artifact calls `args-only` is invoked on the live server with a sentinel entity handle, which the engine's own resolve rejects before touching anything. CS2 **67/67 clean**. It verifies a claim the artifact makes rather than a value it reports, and it is safe to run in CI precisely because the sentinel path mutates nothing — every argument slot the measurement calls unused is passed as null, so a slot that is actually used faults, and a fault is caught and the thread restored.
|
||||
- **field-gap size calibration**, the semantic call sweep, and a 500-iteration live fuzz.
|
||||
|
||||
One of these found a real defect on its first run: 34 of 715 handlers measured float arguments a `void(ptr, ref)` cannot have, which traced to the ABI reader treating a `call` as fall-through so a callee's *return* propagated backwards as a phantom argument.
|
||||
|
|
@ -541,7 +555,7 @@ A full `produce` run writes a self-contained release set per game into `--out-di
|
|||
|------|-----------|------|
|
||||
| `gamedata-<game>.json` | The **monolith** — the tiered function catalogue (signatures + vtable offsets) with provenance and live-validation folded inline | always |
|
||||
| `abi-<game>.json` | The **prototype manifest** — declared parameter/return types, each judged against the footprint measured in this build | always |
|
||||
| `bindings-<game>.json` | The **declared callable surface** — what the binary says about itself: Pulse bindings, entity IO, entity classnames, console commands | always |
|
||||
| `bindings-<game>.json` | The **declared callable surface** — what the binary says about itself: Pulse bindings (with a callable shim), entity IO, entity classnames, console commands, ConVars | always |
|
||||
| `netvars-<game>.json` | The **typed schema** — every SchemaSystem class → field → offset/type, plus the base graph and type layouts | full (`--game-dir`) runs only |
|
||||
| `model-<game>.json` | The **per-game model** — the distilled facts derivation reads instead of the corpus | when the run folds an existing model |
|
||||
| `manifest.json` | Volatile release metadata: `{ version, artifacts: [...] }` | always |
|
||||
|
|
@ -563,6 +577,11 @@ Wall-clock and other volatile metadata live only in `manifest.json`; the other a
|
|||
|
||||
A **`MonoEntry`** is a locator flattened to the top level, plus its grading and — where the address resolved offline — its measured `abi` footprint inline. A virtual method ships as a bare integer `offset` (its RTTI vtable slot index); a non-virtual function as a `signature` object with the `library` it scans and a space-hex `linux` pattern with `?` wildcards. By deriver convention an entry carries one or the other; the readers handle the rare both-present case deterministically.
|
||||
|
||||
Two further keys appear where they were established, and both are part of locating rather than decoration:
|
||||
|
||||
- **`class`** — for an `offset` entry, the class whose vtable the slot was measured on. A slot index alone locates nothing, since it only means anything relative to a particular vtable. Taken from the class the derivation actually chained the offset through, never parsed out of the entry name: a base-declared method routinely sits in a derived class's vtable, so those are different facts and only the measured one locates.
|
||||
- **`anchors`** — distinctive string literals the function references, each unique to it within its library. Not a third locator competing with the sig-XOR-offset pair, but a supplement with a *different failure mode*: a byte signature is a snapshot of one build's codegen, while a string survives a recompile that moves instructions. Emitted alongside the signature, never instead of it.
|
||||
|
||||
`reason` on an unresolved entry comes from a closed vocabulary: `sig-drifted`, `offset-low-conf`, `unresolved`, `abi-drift`.
|
||||
|
||||
Console-command handlers ship under the key **`ConCommand::<name>`**. The prefix says what the entry *is* — the handler bound to that command — rather than claiming a C++ symbol; `ent_fire`'s real method name appears nowhere in the binary.
|
||||
|
|
@ -592,13 +611,20 @@ In the artifact, parameters are spelled as **pointers** (`CCommandContext*`, `CC
|
|||
|
||||
### `bindings-<game>.json` — the declared callable surface
|
||||
|
||||
What the binary *says about itself*, as opposed to what the derivation *infers about it*. Kept out of the monolith deliberately: `gamedata` answers "where is this function", this answers "what may I do with it, and how". **Five sections:**
|
||||
What the binary *says about itself*, as opposed to what the derivation *infers about it*. Kept out of the monolith deliberately: `gamedata` answers "where is this function", this answers "what may I do with it, and how". **Six sections:**
|
||||
|
||||
- `pulse` — bindings keyed by qualified `Class::Method`, each with Valve's display name and description, a decoded call policy (receiver kind, mutating, blocking) with the raw words beside it, and the recovered typed signature.
|
||||
- `pulse` — bindings keyed by qualified `Class::Method`, each with Valve's display name and description, a decoded call policy (receiver kind, mutating, blocking) with the raw words beside it, the recovered typed signature, and — where the record carries one — a `shim` address plus a `call` block saying how to invoke it. `call.needs` is the field to read:
|
||||
- `args-only` — callable with an argument array and nothing else; the other argument slots may be null. **Validated by calling every such binding on a live server of both games.**
|
||||
- `output-sink` — it returns a value, and writes through a register-file object a host does not have. Read the state through `netvars-<game>.json` instead; the Pulse getters are redundant with schema fields.
|
||||
- `pulse-context` — needs a live `CPulseExecCursor` or graph instance. Not host-callable.
|
||||
- `other-slots` — reads an argument whose role is not established (the `CPulseCell_*` family, which are graph *node* implementations rather than API bindings). Not host-callable.
|
||||
|
||||
`call.reads` carries the measured slot list beside the decoding, so a build that changes the contract is re-readable rather than silently mis-labelled.
|
||||
- `entity_inputs` — the map-facing input name, the C++ handler, its owning class where the schema join qualified it, and the handler's **address** (these also ship as gamedata).
|
||||
- `entity_outputs` — the events an entity fires and where the subscriber list lives on the instance.
|
||||
- `entity_classes` — map classname → the C++ class it constructs (`func_door` → `CBaseDoor`). Names to names, no addresses.
|
||||
- `commands` — console commands with description, decoded flags, the raw flags word, the callback form, the measured ABI shape and the handler **address** (these also ship as gamedata).
|
||||
- `convars` — the configuration half of the console surface: name, help text, decoded flags, the raw flags word, and the ConVar object's address. Emitted for the **metadata**, not as a locator — a consumer finds a convar by name at runtime with no gamedata at all, so the name alone would add nothing. The flags are the payload: `cheat`, `replicated`, `archive`, `notify` are engine-*declared* authority, which is what lets a host decide what a plugin may change without a hand-maintained allowlist. Decoded by a **convar-specific** bit table, not the command one — see [Known limitations](#known-limitations).
|
||||
|
||||
**`descriptor` on a Pulse binding is NOT a locator** — see [above](#3-names-valve-ships-in-the-binary--three-sources-and-only-two-locate). Any Pulse count is a count of *registrations*, not distinct bindings.
|
||||
|
||||
|
|
@ -624,6 +650,7 @@ Two things to know before you diff outputs:
|
|||
- The default `cssharp` gamedata output is **JSONC** — it carries comment banners, so a strict JSON parser will reject it.
|
||||
- The `swiftly` gamedata format emits **signature entries only**; vtable-offset entries are omitted (that is over a thousand entries on CS2), because that framework takes offsets through a separate file.
|
||||
- The `model` format emits a tier-selected `Gamedata` with no `meta`, so its output cannot be fed back in via `--from`.
|
||||
- The `modsharp` format now emits a **`refs`** block where one was derived — `refs.strings` for string anchors and `refs.vtable` for an offset entry's class, both ModSharp's own keys. An entry with anchors and no byte pattern is emitted as `refs`-only, which is how several of ModSharp's own hand-written entries are written; if you diff against an older render, those rows are additions rather than changes.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -649,6 +676,8 @@ This is a publishable tool, so where declarations and names come from is a hard
|
|||
- **Offline runs ship no netvars and no `validated` state**, because field types and validation both require a running process.
|
||||
- **Two games cannot be produced concurrently** on one host (fixed server port).
|
||||
- **The Pulse duplicate-registration disagreement is open** — see [What is NOT gated](#what-is-not-gated).
|
||||
- **ConVar flag bits are decoded by a convar-specific table, and three bits are unnamed.** FCVAR is not one flag space across object types — decoding convars with the *command* table mislabels bit 0 on 185 Dota and 56 CS2 convars with a name Valve's own dumps give to none of them. The convar table was derived against both games' published dumps (1,939 convars pooled) keeping only bits that hold at 100% precision; bits 0, 1 and 2 are set often and match nothing cleanly, so they stay unnamed and survive in `flags_raw`.
|
||||
- **String anchors cover a minority, by design.** 480 catalogued + 77 derived on CS2, 103 + 22 on Dota. Two conditions do the filtering: an anchor names a *function*, so an entry whose locator points mid-function (a hook site rather than a prologue) gets none; and the string must be referenced only from inside that function. Most shipped functions reference no string unique to them, which is the real ceiling — 859 of 1,505 CS2 candidates fail on that alone.
|
||||
- **Some declared returns are decided by source order.** Where sources disagree on a return type, the untrusted source is ranked last, but among the trusted ones the order is the order they were merged. A handful of names are settled that way, which is stated rather than papered over.
|
||||
|
||||
---
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue