act on what the binary declares: callable Pulse shims, ConVars, string anchors; gen v2.1
This commit is contained in:
Kamal Tufekcic
parent
3de955c4ff
commit
71ce34edd2
14 changed files with 1507 additions and 54 deletions
2
Cargo.lock
generated
2
Cargo.lock
generated
|
|
@ -248,7 +248,7 @@ dependencies = [
|
|||
|
||||
[[package]]
|
||||
name = "source2rosetta-core"
|
||||
version = "2.0.0"
|
||||
version = "2.1.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"clap",
|
||||
|
|
|
|||
57
README.md
57
README.md
|
|
@ -14,7 +14,8 @@ curl -fsSLO $R/abi-cs2.json # HOW to call them — parameter and return t
|
|||
```
|
||||
|
||||
Also published: `bindings-<game>.json` (the callable surface the binary declares about itself — Pulse
|
||||
bindings, entity IO, console commands) and `manifest.json` (which build you got). The
|
||||
bindings with a callable shim, entity IO, console commands, ConVars) and `manifest.json` (which build you
|
||||
got). The
|
||||
[artifacts section](#artifacts-schemas--output-formats) covers all of them.
|
||||
|
||||
The output is framework-neutral; `source2rosetta-gen` renders it into whatever your stack speaks — the gamedata into your framework's locator format, and `abi-<game>.json` into **typed call sites** for the same functions. The deriver behind it is a standalone Rust tool — you only need that if you're self-hosting the pipeline or adding a game.
|
||||
|
|
@ -30,10 +31,12 @@ The output is framework-neutral; `source2rosetta-gen` renders it into whatever y
|
|||
|
||||
Ballpark from a recent build, on a 16-core desktop. These move build-to-build — treat them as orders of magnitude, not guarantees.
|
||||
|
||||
| | derived functions | typed schema | model | one-time distill |
|
||||
|---|---|---|---|---|
|
||||
| **CS2** | ~1,125 `core` + ~2,620 `high_confidence`, plus ~4,375 `experimental` name guesses | ~1,900 classes / ~12,300 fields | ~48 MB (a few MB gzipped) | ~15 min |
|
||||
| **Dota 2** | ~1,930 `core` + ~2,450 `high_confidence`, plus ~5,950 `experimental` | ~2,960 classes / ~17,700 fields | ~570 MB | ~1 hr |
|
||||
| | derived functions | declared surface | typed prototypes | typed schema | model | one-time distill |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **CS2** | ~1,125 `core` + ~2,620 `high_confidence`, plus ~4,375 `experimental` name guesses | 580 Pulse bindings (127 host-callable), 784 commands, **1,551 ConVars**, 715 entity inputs / 226 outputs, 474 classnames | ~2,055 `verified` + ~80 `lower-bound`, 55 `mismatch` | ~1,900 classes / ~12,300 fields | ~48 MB (a few MB gzipped) | ~15 min |
|
||||
| **Dota 2** | ~1,930 `core` + ~2,450 `high_confidence`, plus ~5,950 `experimental` | 500 Pulse bindings (99 host-callable), 855 commands, **1,170 ConVars**, 624 entity inputs / 187 outputs, 3,528 classnames | ~3,635 `verified` + ~100 `lower-bound`, 44 `mismatch` | ~2,960 classes / ~17,700 fields | ~570 MB | ~1 hr |
|
||||
|
||||
**Declared surface** is what the binary states about itself, and it is a different kind of fact from the rest: no inference, no cross-build chaining, no confidence tier. The *host-callable* count is the subset of Pulse bindings invocable with an argument array alone — verified by calling each one on a live server of both games.
|
||||
|
||||
A full run live-validates what it ships and reports **0 dropped** on both games — for CS2 that is ~2,610 signatures and ~1,120 vtable offsets checked against a running server. Distilling the model is a one-time cost; after that each build's re-derive is minutes of compute, and the half hour in the headline is the whole loop: notice, update, derive, validate, publish.
|
||||
|
||||
|
|
@ -66,9 +69,13 @@ The artifacts answer four different questions, and most useful work joins two or
|
|||
- **`gamedata-<game>.json` — where the code is.** Every entry is a hook point or a call target: a byte signature or an RTTI vtable slot, tiered and, for `core`/`high_confidence`, checked against a running server.
|
||||
- **`netvars-<game>.json` — what the state is.** Field offsets and types for every SchemaSystem class, plus the base graph, the enum tables and per-type sizes. This is the half that needs no hooking at all: a great deal of gameplay is readable and writable as plain memory.
|
||||
- **`abi-<game>.json` — whether it is safe to call.** A declared prototype joined to the register footprint measured in *this* build, with a verdict per function. `verified` and `lower-bound` are callable; `mismatch` says the prototype in circulation is wrong for this binary.
|
||||
- **`bindings-<game>.json` — what the binary declares about itself.** Console commands with flags and handler addresses, entity inputs and outputs, map classname → C++ class, and the typed Pulse registry.
|
||||
- **`bindings-<game>.json` — what the binary declares about itself.** Console commands and ConVars with decoded flags, entity inputs and outputs, map classname → C++ class, and the typed Pulse registry with a callable entry point per binding.
|
||||
|
||||
One thing shapes every capability below: **there are no ConVars in these artifacts, in either game.** The command surface is 784 ConCommands on CS2 and 855 on Dota, and not one `sv_airaccelerate`, `mp_freezetime`, `bot_quota` or `dota_gold_per_tick` appears anywhere. This is a *function-and-layout* release, not a configuration release. A plugin built on it changes behaviour by detouring code and writing fields — never by setting a cvar.
|
||||
Two of those are newer than the rest and worth calling out, because they change what a plugin can do:
|
||||
|
||||
**ConVars ship with their flags.** 1,551 on CS2 across four libraries, 781 in Dota's `libserver` — with `cheat`, `replicated`, `archive` and `notify` decoded, and the raw word beside them. The names are not the point: a consumer finds a convar by name at runtime with no gamedata at all. The *flags* are, because they are engine-declared authority. A host that wants to say "this module may change gameplay settings but not cheat-protected ones" can key that on what the engine itself declares instead of maintaining an allowlist by hand.
|
||||
|
||||
**Most of the Pulse surface is callable.** Each binding carries a `shim` address and a `call.needs` verdict; the `args-only` tier — roughly 110 on CS2, 82 on Dota within `libserver` — is invocable with an argument array and nothing else, through Valve's own marshalling, which enforces the binding's declared types. Those are *actions* (teleport, ignite, change team, start a mover, spawn a template), which is the half no field write can do; reading state remains the schema's job and is better served there.
|
||||
|
||||
Tiers stay visible throughout. `core` and `high_confidence` are buildable today; the `experimental` band is fenced off at the end and is a different kind of thing entirely.
|
||||
|
||||
|
|
@ -152,7 +159,7 @@ That is one leg of a defensive integrity monitor for the server process an opera
|
|||
|
||||
The console surface is the most mechanically reliable thing in the release. 755 of CS2's 784 registered commands carry a hookable `ConCommand::<name>` locator with a `verified` prototype — that is *all* commands, not a diagnostics subset, though the diagnostics family within it is broad (`stats_print`, `sv_packstats` with its `clear` argument, the `vprof_*` family, `mem_dump`, `net_stats_json`, `status_json`, `lrucache_stats`, `check_nofilefd`). Hooking is the reliable direction: the artifacts give you the callback address, not the `CCommand` layout needed to synthesize a call. `logaddress_add_http` ships log fan-out to an arbitrary URI with no sidecar, and tier0 carries a complete scripted-test harness (`Test_StartScript`, `Test_LoopForNumSeconds`, `Test_Checkpoint`, `Test_ExitProcess` with a chosen exit code) that is a CI rig Valve already wrote.
|
||||
|
||||
Because command flags are decoded, the client-reachable attack surface is exactly enumerable rather than folklore: **30 CS2 commands carry `client_can_execute`**, including `ent_setpos` and `ent_setang` — which move *arbitrary entities* — alongside `give`, `god`, `noclip`, `kill`, `explode`, `setpos_player`, `callvote` and `replay_start`. Exactly two carry `server_can_execute` (`echo`, `play`), which answers a question plugin authors argue about: the server cannot push arbitrary console commands to clients through the normal path. That static audit is solid. Detecting a *runtime* change to those flags is not — every cvar-registry accessor (`CCvar::GetConVarFlags`, `CCvar::FindCommand`, `CCvar::RegisterConCommand`) is experimental with a guessed name.
|
||||
Because command flags are decoded, the client-reachable attack surface is exactly enumerable rather than folklore: **30 CS2 commands carry `client_can_execute`**, including `ent_setpos` and `ent_setang` — which move *arbitrary entities* — alongside `give`, `god`, `noclip`, `kill`, `explode`, `setpos_player`, `callvote` and `replay_start`. Exactly two carry `server_can_execute` (`echo`, `play`), which answers a question plugin authors argue about: the server cannot push arbitrary console commands to clients through the normal path. That static audit is solid, and it now extends to ConVars: their flags are decoded the same way, so `cheat`-guarded and `replicated` tunables are enumerable rather than assumed. Detecting a *runtime* change to any of those flags is a different matter — every cvar-registry accessor (`CCvar::GetConVarFlags`, `CCvar::FindCommand`, `CCvar::RegisterConCommand`) is experimental with a guessed name.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -234,7 +241,7 @@ The only defensible workflow for anything in this band: pick a candidate, check
|
|||
|
||||
### What is not covered
|
||||
|
||||
- **No ConVars, in either game.** Commands only. No `sv_*`/`mp_*`/`dota_*` tunables, no defaults, no flags, no ranges.
|
||||
- **ConVars ship names, help and flags — but no defaults or ranges.** The default value is built in a stack structure at the registration site rather than passed as a literal, so it is not recoverable the way the rest is. `min`/`max` likewise. If you need the shipped default, read it off a running server.
|
||||
- **No protobuf field layouts.** You get message ids and class names; you must supply the `.proto` definitions.
|
||||
- **No game-event name tables.** The event *system* is there (post, register, the legacy bridge); the names (`player_death`, `dota_player_gained_level`) are not. On CS2 the practical substitutes are function-level equivalents and `logic_gameevent_listener`, which needs only a string.
|
||||
- **No content names.** No `.vpcf` particle systems, no sound events, no model paths, no Dota KeyValues gameplay data (no ability special values, no hero base stats, no item costs).
|
||||
|
|
@ -285,9 +292,13 @@ Every Source-2 module names some of its own functions. This is ground truth from
|
|||
|---|---|---|
|
||||
| **entity-IO datadesc** | the C++ handler name (`InputKill`) with the handler's address | **yes** |
|
||||
| **console-command registration** | the command name (`bot_add`) with its callback | **yes** |
|
||||
| **Pulse binding registry** | a qualified `Class::Method` with display name, description, call policy and a full typed signature | **no** |
|
||||
| **Pulse binding registry** | a qualified `Class::Method` with display name, description, call policy, a full typed signature and an invocation shim | **not as a C++ symbol** |
|
||||
|
||||
**The Pulse registry does not locate anything, and the mistake is instructive.** Its records carry two code pointers, which an early pass folded as locators for a headline `+782` names. They are *descriptor accessors* — every CS2 `libserver` binding measures the same empty `int=0 float=0` footprint, and they disassemble to a lazy-init singleton that returns a static vector. Folding them would have shipped `CBaseEntityAPI::GetAbsOrigin` pointing at a zero-argument accessor: a locator that resolves, passes live validation as executable code, and is still the wrong function. The honest yield from that table is zero locators — and a complete typed API surface, which ships separately as `bindings-<game>.json`.
|
||||
**Two of the Pulse record's three code pointers do not locate anything, and the mistake is instructive.** An early pass folded the pair at `+24`/`+32` as locators for a headline `+782` names. They are *descriptor accessors* — every CS2 `libserver` binding measures the same empty `int=0 float=0` footprint, and they disassemble to a lazy-init singleton returning a static vector. Folding them would have shipped `CBaseEntityAPI::GetAbsOrigin` pointing at a zero-argument accessor: a locator that resolves, passes live validation as executable code, and is still the wrong function.
|
||||
|
||||
**The third pointer, at `+72`, is a real entry point** — one per binding, never shared. It is not the bound C++ method and cannot be folded as one (that method's address is genuinely unrecoverable offline; the shim dispatches indirectly). It is a fixed-signature marshalling stub, and calling it *invokes the binding*. That is verified by doing it: `SetRenderAlpha` and `SetRenderColor` were called on a live CS2 server and moved the entity's `m_clrRender`, and a `SetRenderColor` on Dota set the RGB bytes while leaving the alpha byte untouched — proof that dispatch honours the binding's DECLARED `PulseValueType_t`, so a caller cannot smuggle a mistyped argument past it. Every eligible binding is re-checked on each derive by calling it with a sentinel handle (see [the standing oracles](#the-standing-oracles)).
|
||||
|
||||
So the honest yield from this table is **zero C++ symbol locators, a complete typed API surface, and a callable entry point per binding** — all of which ship in `bindings-<game>.json`, with `call.needs` stating what a host must supply for each.
|
||||
|
||||
**The datadesc handlers are class-qualified** by joining each record's array against the SchemaSystem: a datadesc array also carries field descriptors, and a `(member, offset)` pair is something the schema states from an entirely different table, so the class whose schema satisfies every pair in the array owns it. 653 of 715 CS2 handlers qualify this way. This is what makes `InputEnable` — a distinct handler on 48 classes — nameable at all; unqualified names that resolve to several addresses are still **dropped, not guessed**. Note for consumers: qualification **renamed 343 shipped keys** (`InputActivateSkybox` → `CAmbientGeneric::InputActivateSkybox`).
|
||||
|
||||
|
|
@ -374,6 +385,8 @@ A `produce` run **aborts rather than publish** a collapsed artifact. Each surfac
|
|||
| entity-IO records | inputs + outputs | |
|
||||
| entity classnames | factory-record count | |
|
||||
| console commands | recovered-command count | the registrar is found by SHAPE, so a reworked constructor yields **zero** commands rather than wrong ones — correct, and invisible without this |
|
||||
| host-callable Pulse shims | count of `call.needs == "args-only"` | the registry can read perfectly and every signature recover perfectly while a codegen change makes each shim appear to read another argument — retiring the one callable tier without failing anything |
|
||||
| ConVars | recovered-convar count | found by a DIFFERENT test than commands — convergence of registrar wrappers on a shared core, not a sentinel in the callee — so it can collapse while commands keep working |
|
||||
| typed netvars | fraction of fields typed | a runtime type-layout reshape resolves every field "untyped" and would otherwise ship a typeless schema at exit 0 |
|
||||
| schema enums | enum count | read by shape like the class table, so a reshape yields zero rather than wrong |
|
||||
| live validation | pass rate, above a minimum sample | |
|
||||
|
|
@ -394,6 +407,7 @@ A dozen free, mostly two-sided checks run on **every** derive and are reported.
|
|||
- **EHANDLE class grouping** — Valve's naming vs the binary's destructor addresses: 0 of 44 CS2 / 41 Dota groups carry two classes.
|
||||
- **Pulse element stride** — derived by consensus per image, unanimous across six libraries in both games.
|
||||
- **live schema oracle** — offline layout vs the running process: 852/852 CS2, 1,912/1,912 Dota.
|
||||
- **Pulse shim invocation** — the only *behavioural* oracle here: every binding the artifact calls `args-only` is invoked on the live server with a sentinel entity handle, which the engine's own resolve rejects before touching anything. CS2 **67/67 clean**. It verifies a claim the artifact makes rather than a value it reports, and it is safe to run in CI precisely because the sentinel path mutates nothing — every argument slot the measurement calls unused is passed as null, so a slot that is actually used faults, and a fault is caught and the thread restored.
|
||||
- **field-gap size calibration**, the semantic call sweep, and a 500-iteration live fuzz.
|
||||
|
||||
One of these found a real defect on its first run: 34 of 715 handlers measured float arguments a `void(ptr, ref)` cannot have, which traced to the ABI reader treating a `call` as fall-through so a callee's *return* propagated backwards as a phantom argument.
|
||||
|
|
@ -541,7 +555,7 @@ A full `produce` run writes a self-contained release set per game into `--out-di
|
|||
|------|-----------|------|
|
||||
| `gamedata-<game>.json` | The **monolith** — the tiered function catalogue (signatures + vtable offsets) with provenance and live-validation folded inline | always |
|
||||
| `abi-<game>.json` | The **prototype manifest** — declared parameter/return types, each judged against the footprint measured in this build | always |
|
||||
| `bindings-<game>.json` | The **declared callable surface** — what the binary says about itself: Pulse bindings, entity IO, entity classnames, console commands | always |
|
||||
| `bindings-<game>.json` | The **declared callable surface** — what the binary says about itself: Pulse bindings (with a callable shim), entity IO, entity classnames, console commands, ConVars | always |
|
||||
| `netvars-<game>.json` | The **typed schema** — every SchemaSystem class → field → offset/type, plus the base graph and type layouts | full (`--game-dir`) runs only |
|
||||
| `model-<game>.json` | The **per-game model** — the distilled facts derivation reads instead of the corpus | when the run folds an existing model |
|
||||
| `manifest.json` | Volatile release metadata: `{ version, artifacts: [...] }` | always |
|
||||
|
|
@ -563,6 +577,11 @@ Wall-clock and other volatile metadata live only in `manifest.json`; the other a
|
|||
|
||||
A **`MonoEntry`** is a locator flattened to the top level, plus its grading and — where the address resolved offline — its measured `abi` footprint inline. A virtual method ships as a bare integer `offset` (its RTTI vtable slot index); a non-virtual function as a `signature` object with the `library` it scans and a space-hex `linux` pattern with `?` wildcards. By deriver convention an entry carries one or the other; the readers handle the rare both-present case deterministically.
|
||||
|
||||
Two further keys appear where they were established, and both are part of locating rather than decoration:
|
||||
|
||||
- **`class`** — for an `offset` entry, the class whose vtable the slot was measured on. A slot index alone locates nothing, since it only means anything relative to a particular vtable. Taken from the class the derivation actually chained the offset through, never parsed out of the entry name: a base-declared method routinely sits in a derived class's vtable, so those are different facts and only the measured one locates.
|
||||
- **`anchors`** — distinctive string literals the function references, each unique to it within its library. Not a third locator competing with the sig-XOR-offset pair, but a supplement with a *different failure mode*: a byte signature is a snapshot of one build's codegen, while a string survives a recompile that moves instructions. Emitted alongside the signature, never instead of it.
|
||||
|
||||
`reason` on an unresolved entry comes from a closed vocabulary: `sig-drifted`, `offset-low-conf`, `unresolved`, `abi-drift`.
|
||||
|
||||
Console-command handlers ship under the key **`ConCommand::<name>`**. The prefix says what the entry *is* — the handler bound to that command — rather than claiming a C++ symbol; `ent_fire`'s real method name appears nowhere in the binary.
|
||||
|
|
@ -592,13 +611,20 @@ In the artifact, parameters are spelled as **pointers** (`CCommandContext*`, `CC
|
|||
|
||||
### `bindings-<game>.json` — the declared callable surface
|
||||
|
||||
What the binary *says about itself*, as opposed to what the derivation *infers about it*. Kept out of the monolith deliberately: `gamedata` answers "where is this function", this answers "what may I do with it, and how". **Five sections:**
|
||||
What the binary *says about itself*, as opposed to what the derivation *infers about it*. Kept out of the monolith deliberately: `gamedata` answers "where is this function", this answers "what may I do with it, and how". **Six sections:**
|
||||
|
||||
- `pulse` — bindings keyed by qualified `Class::Method`, each with Valve's display name and description, a decoded call policy (receiver kind, mutating, blocking) with the raw words beside it, and the recovered typed signature.
|
||||
- `pulse` — bindings keyed by qualified `Class::Method`, each with Valve's display name and description, a decoded call policy (receiver kind, mutating, blocking) with the raw words beside it, the recovered typed signature, and — where the record carries one — a `shim` address plus a `call` block saying how to invoke it. `call.needs` is the field to read:
|
||||
- `args-only` — callable with an argument array and nothing else; the other argument slots may be null. **Validated by calling every such binding on a live server of both games.**
|
||||
- `output-sink` — it returns a value, and writes through a register-file object a host does not have. Read the state through `netvars-<game>.json` instead; the Pulse getters are redundant with schema fields.
|
||||
- `pulse-context` — needs a live `CPulseExecCursor` or graph instance. Not host-callable.
|
||||
- `other-slots` — reads an argument whose role is not established (the `CPulseCell_*` family, which are graph *node* implementations rather than API bindings). Not host-callable.
|
||||
|
||||
`call.reads` carries the measured slot list beside the decoding, so a build that changes the contract is re-readable rather than silently mis-labelled.
|
||||
- `entity_inputs` — the map-facing input name, the C++ handler, its owning class where the schema join qualified it, and the handler's **address** (these also ship as gamedata).
|
||||
- `entity_outputs` — the events an entity fires and where the subscriber list lives on the instance.
|
||||
- `entity_classes` — map classname → the C++ class it constructs (`func_door` → `CBaseDoor`). Names to names, no addresses.
|
||||
- `commands` — console commands with description, decoded flags, the raw flags word, the callback form, the measured ABI shape and the handler **address** (these also ship as gamedata).
|
||||
- `convars` — the configuration half of the console surface: name, help text, decoded flags, the raw flags word, and the ConVar object's address. Emitted for the **metadata**, not as a locator — a consumer finds a convar by name at runtime with no gamedata at all, so the name alone would add nothing. The flags are the payload: `cheat`, `replicated`, `archive`, `notify` are engine-*declared* authority, which is what lets a host decide what a plugin may change without a hand-maintained allowlist. Decoded by a **convar-specific** bit table, not the command one — see [Known limitations](#known-limitations).
|
||||
|
||||
**`descriptor` on a Pulse binding is NOT a locator** — see [above](#3-names-valve-ships-in-the-binary--three-sources-and-only-two-locate). Any Pulse count is a count of *registrations*, not distinct bindings.
|
||||
|
||||
|
|
@ -624,6 +650,7 @@ Two things to know before you diff outputs:
|
|||
- The default `cssharp` gamedata output is **JSONC** — it carries comment banners, so a strict JSON parser will reject it.
|
||||
- The `swiftly` gamedata format emits **signature entries only**; vtable-offset entries are omitted (that is over a thousand entries on CS2), because that framework takes offsets through a separate file.
|
||||
- The `model` format emits a tier-selected `Gamedata` with no `meta`, so its output cannot be fed back in via `--from`.
|
||||
- The `modsharp` format now emits a **`refs`** block where one was derived — `refs.strings` for string anchors and `refs.vtable` for an offset entry's class, both ModSharp's own keys. An entry with anchors and no byte pattern is emitted as `refs`-only, which is how several of ModSharp's own hand-written entries are written; if you diff against an older render, those rows are additions rather than changes.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -649,6 +676,8 @@ This is a publishable tool, so where declarations and names come from is a hard
|
|||
- **Offline runs ship no netvars and no `validated` state**, because field types and validation both require a running process.
|
||||
- **Two games cannot be produced concurrently** on one host (fixed server port).
|
||||
- **The Pulse duplicate-registration disagreement is open** — see [What is NOT gated](#what-is-not-gated).
|
||||
- **ConVar flag bits are decoded by a convar-specific table, and three bits are unnamed.** FCVAR is not one flag space across object types — decoding convars with the *command* table mislabels bit 0 on 185 Dota and 56 CS2 convars with a name Valve's own dumps give to none of them. The convar table was derived against both games' published dumps (1,939 convars pooled) keeping only bits that hold at 100% precision; bits 0, 1 and 2 are set often and match nothing cleanly, so they stay unnamed and survive in `flags_raw`.
|
||||
- **String anchors cover a minority, by design.** 480 catalogued + 77 derived on CS2, 103 + 22 on Dota. Two conditions do the filtering: an anchor names a *function*, so an entry whose locator points mid-function (a hook site rather than a prologue) gets none; and the string must be referenced only from inside that function. Most shipped functions reference no string unique to them, which is the real ceiling — 859 of 1,505 CS2 candidates fail on that alone.
|
||||
- **Some declared returns are decided by source order.** Where sources disagree on a return type, the untrusted source is ranked last, but among the trusted ones the order is the order they were merged. A handful of names are settled that way, which is stated rather than papered over.
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
[package]
|
||||
name = "source2rosetta-core"
|
||||
version = "2.0.0"
|
||||
version = "2.1.0"
|
||||
edition = "2024"
|
||||
description = "source2rosetta's deriver-free core: canonical gamedata model + format emitters (serde-only)"
|
||||
license = "AGPL-3.0-only"
|
||||
|
|
|
|||
|
|
@ -12,13 +12,35 @@ pub struct Sig {
|
|||
pub linux: String, // space-hex pattern with `?` wildcards, e.g. "55 48 89 ? E5"
|
||||
}
|
||||
|
||||
/// One gamedata function: a vtable-method offset, a scan signature, or (rarely) both.
|
||||
/// One gamedata function: a vtable-method offset, a scan signature, or (rarely) both — plus, where we
|
||||
/// have them, string ANCHORS that locate the same function a different way.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct Entry {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub signature: Option<Sig>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub offset: Option<i64>, // vtable slot index (or a carried member offset)
|
||||
/// For a vtable-OFFSET locator: the class whose vtable the slot was measured on.
|
||||
///
|
||||
/// Part of the locator, not decoration — a slot index alone locates nothing, since it is only meaningful
|
||||
/// relative to a particular class's vtable. Taken from the class whose vtable the derivation actually
|
||||
/// chained the offset through, never parsed out of the entry name: a method declared on a base class
|
||||
/// routinely sits in a derived class's vtable, so the name's class and the measured class are different
|
||||
/// facts and only the second one locates anything.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub class: Option<String>,
|
||||
/// Distinctive string literals this function references, each unique to it within its library.
|
||||
///
|
||||
/// NOT a third locator competing with the sig-XOR-offset pair — a supplement with a DIFFERENT failure
|
||||
/// mode. A byte signature is a snapshot of one build's codegen; a string survives a recompile that
|
||||
/// moves instructions. So a consumer that can resolve anchors (ModSharp's `refs.strings`) has a
|
||||
/// locator that keeps working across the window between Valve shipping a build and us republishing,
|
||||
/// which is exactly when a byte pattern is most likely to have drifted.
|
||||
///
|
||||
/// Emitted alongside the signature, never instead of it: the two are independent, and a consumer
|
||||
/// choosing between them is better served by having both than by our picking one.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub anchors: Vec<String>,
|
||||
}
|
||||
|
||||
impl Entry {
|
||||
|
|
@ -30,6 +52,8 @@ impl Entry {
|
|||
linux: linux.into(),
|
||||
}),
|
||||
offset: None,
|
||||
class: None,
|
||||
anchors: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -38,6 +62,8 @@ impl Entry {
|
|||
Entry {
|
||||
signature: None,
|
||||
offset: Some(linux),
|
||||
class: None,
|
||||
anchors: Vec::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -136,6 +162,30 @@ impl Gamedata {
|
|||
self.entries.entry(name.into()).or_default().offset = Some(linux);
|
||||
}
|
||||
|
||||
/// Record the class a vtable-offset locator is relative to.
|
||||
pub fn set_class(&mut self, name: impl Into<String>, class: impl Into<String>) {
|
||||
self.entries.entry(name.into()).or_default().class = Some(class.into());
|
||||
}
|
||||
|
||||
/// Attach string anchors to `name`, creating the entry if the derivation reached it by no other route.
|
||||
///
|
||||
/// Deduplicated and order-preserving: the catalogue can carry the same anchor twice across variants,
|
||||
/// and the emitted list is part of a byte-reproducible artifact, so it must not depend on how many
|
||||
/// times a source repeated itself.
|
||||
pub fn add_anchors<S: Into<String>>(
|
||||
&mut self,
|
||||
name: impl Into<String>,
|
||||
anchors: impl IntoIterator<Item = S>,
|
||||
) {
|
||||
let e = self.entries.entry(name.into()).or_default();
|
||||
for a in anchors {
|
||||
let a = a.into();
|
||||
if !a.is_empty() && !e.anchors.contains(&a) {
|
||||
e.anchors.push(a);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn len(&self) -> usize {
|
||||
self.entries.len()
|
||||
}
|
||||
|
|
@ -297,9 +347,6 @@ impl Provenance {
|
|||
pub struct MonoEntry {
|
||||
#[serde(flatten)]
|
||||
pub locator: Entry,
|
||||
/// experimental offsets only: the vtable class the slot lives on (a reader's eyeball check).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub class: Option<String>,
|
||||
/// The argument footprint read out of THIS build's machine code — see [`AbiShape`]. Absent when the
|
||||
/// function's address wasn't resolvable offline.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
|
|
@ -499,9 +546,48 @@ pub struct Binding {
|
|||
pub typed: bool,
|
||||
/// Address of the binding's DESCRIPTOR ACCESSOR in this build — a lazy-init singleton returning the
|
||||
/// static descriptor, not the bound function. It is the anchor a runtime walks to reach the
|
||||
/// descriptor (and, through it, the real entry point); it is NOT a locator for the named method, and
|
||||
/// no shipped gamedata entry points at it.
|
||||
/// descriptor; it is NOT a locator for the named method, and no shipped gamedata entry points at it.
|
||||
/// For an address that IS callable, see [`Binding::shim`].
|
||||
pub descriptor: String,
|
||||
/// Address of the binding's INVOCATION SHIM in this build — the record's third code pointer, and
|
||||
/// unlike `descriptor` a real entry point. One per binding, never shared.
|
||||
///
|
||||
/// Calling it dispatches through Valve's own marshalling, which honours the DECLARED parameter types
|
||||
/// in `params`: a value written into the argument blob is consumed according to its `PulseValueType_t`,
|
||||
/// so a caller cannot smuggle a mistyped argument past it. Absent when the record's slot holds no
|
||||
/// executable code.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub shim: Option<String>,
|
||||
/// How to call [`Binding::shim`], and what a host must supply. Absent when there is no shim.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub call: Option<ShimCall>,
|
||||
}
|
||||
|
||||
/// The invocation shim's calling contract: fixed across every binding, with a per-binding statement of
|
||||
/// which slots that particular shim reads.
|
||||
///
|
||||
/// The signature is **seven integer arguments returning int**. Slot 5 (`r8`) is an array of POINTERS to the
|
||||
/// argument values, element *k* at `+8+8k`. Slot 7 (the first stack slot) is the output sink. Slot 4
|
||||
/// (`rcx`) is a Pulse host-service context, which is VM-owned. The return is `0` on dispatch and `-2` when
|
||||
/// an entity-handle argument fails to resolve.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct ShimCall {
|
||||
/// What a host must supply beyond the argument array, decoded from `reads`:
|
||||
///
|
||||
/// * `args-only` — nothing else. The remaining slots may be null; **validated by calling every
|
||||
/// eligible binding in both games.** This is the callable tier.
|
||||
/// * `output-sink` — it returns a value, so it writes through a register-file object a host does not
|
||||
/// have. Read the state through `netvars-<game>.json` instead; the Pulse getters are redundant
|
||||
/// with schema fields.
|
||||
/// * `pulse-context` — needs a live `CPulseExecCursor` / graph instance. Not host-callable.
|
||||
/// * `other-slots` — reads an argument slot whose role is not established (the `CPulseCell_*`
|
||||
/// family, which are graph NODE implementations rather than API bindings). Not host-callable.
|
||||
pub needs: String,
|
||||
/// The argument slots this shim was measured to read, named in SysV order — the raw fact `needs` is
|
||||
/// decoded from, kept beside it so a build that changes the contract can be re-read rather than
|
||||
/// silently mis-labelled. The same rule `flags_raw` follows.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub reads: Vec<String>,
|
||||
}
|
||||
|
||||
/// One Pulse parameter or return value, as the binding declares it.
|
||||
|
|
@ -587,6 +673,30 @@ pub struct ConsoleCommand {
|
|||
pub addr: String,
|
||||
}
|
||||
|
||||
/// One ConVar the module registers — the configuration half of the console surface.
|
||||
///
|
||||
/// Emitted for the METADATA, not as a locator: a consumer finds a convar by name at runtime
|
||||
/// (`ICvar::FindConVar`) with no gamedata at all, so the name alone would add nothing. The flags are the
|
||||
/// payload — `cheat`, `replicated`, `release` are engine-DECLARED authority, and a host deciding what a
|
||||
/// module may change is better served by what the engine says than by a hand-maintained allowlist.
|
||||
#[derive(Clone, Debug, serde::Serialize, serde::Deserialize)]
|
||||
pub struct ConVar {
|
||||
pub name: String,
|
||||
pub library: String,
|
||||
#[serde(default, skip_serializing_if = "String::is_empty")]
|
||||
pub description: String,
|
||||
/// FCVAR bits with a measured meaning — the same space console commands use.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub flags: Vec<String>,
|
||||
/// The raw flags word. Empty when this registrar had no identifiable flags argument, which is honest
|
||||
/// about the gap rather than reporting a zero that would read as "no flags set".
|
||||
#[serde(default, skip_serializing_if = "String::is_empty")]
|
||||
pub flags_raw: String,
|
||||
/// Address of the ConVar object. It lives in `.bss`, so it holds nothing on disk — it is the anchor a
|
||||
/// runtime walks to the live value, and what tells two registrations of one name apart.
|
||||
pub addr: String,
|
||||
}
|
||||
|
||||
/// One entity-IO output: an event an entity fires, and where its subscriber list lives on the instance.
|
||||
#[derive(Clone, Debug, serde::Serialize, serde::Deserialize)]
|
||||
pub struct EntityOutput {
|
||||
|
|
@ -633,6 +743,9 @@ pub struct Bindings {
|
|||
/// there is no honest key to map them by either.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub commands: Vec<ConsoleCommand>,
|
||||
/// ConVars, a LIST for the same reason: one name can be registered by more than one library.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub convars: Vec<ConVar>,
|
||||
}
|
||||
|
||||
/// The binding registry's intrinsic identity (no wall-clock field, same rationale as [`MonoMeta`]).
|
||||
|
|
@ -644,6 +757,9 @@ pub struct BindingsMeta {
|
|||
/// Of those, how many carry a recovered typed signature.
|
||||
#[serde(default)]
|
||||
pub pulse_typed: usize,
|
||||
/// Of those, how many carry a HOST-CALLABLE invocation shim (`call.needs == "args-only"`).
|
||||
#[serde(default)]
|
||||
pub pulse_callable: usize,
|
||||
pub entity_inputs: usize,
|
||||
#[serde(default)]
|
||||
pub entity_outputs: usize,
|
||||
|
|
@ -651,6 +767,8 @@ pub struct BindingsMeta {
|
|||
pub entity_classes: usize,
|
||||
#[serde(default)]
|
||||
pub commands: usize,
|
||||
#[serde(default)]
|
||||
pub convars: usize,
|
||||
}
|
||||
|
||||
impl Bindings {
|
||||
|
|
@ -949,8 +1067,9 @@ mod monolith_tests {
|
|||
locator: Entry {
|
||||
signature: None,
|
||||
offset: Some(158),
|
||||
class: None,
|
||||
anchors: Vec::new(),
|
||||
},
|
||||
class: None,
|
||||
abi: None,
|
||||
provenance: Provenance {
|
||||
source: Some("catalogue".into()),
|
||||
|
|
@ -973,8 +1092,9 @@ mod monolith_tests {
|
|||
locator: Entry {
|
||||
signature: None,
|
||||
offset: Some(40),
|
||||
class: Some("CFoo".into()),
|
||||
anchors: Vec::new(),
|
||||
},
|
||||
class: Some("CFoo".into()),
|
||||
abi: None,
|
||||
provenance: Provenance {
|
||||
confidence: Some("low".into()),
|
||||
|
|
@ -1021,8 +1141,9 @@ mod monolith_tests {
|
|||
linux: "55 48 89 E5".into(),
|
||||
}),
|
||||
offset: None,
|
||||
class: None,
|
||||
anchors: Vec::new(),
|
||||
},
|
||||
class: None,
|
||||
abi: None,
|
||||
provenance: Provenance {
|
||||
source: Some("catalogue".into()),
|
||||
|
|
|
|||
|
|
@ -153,7 +153,15 @@ pub fn entry_from_value(v: &Value) -> Entry {
|
|||
.get("offsets")
|
||||
.and_then(|o| o.get("linux"))
|
||||
.and_then(Value::as_i64);
|
||||
Entry { signature, offset }
|
||||
// The cssharp locator shape has no anchor field, so a round-tripped entry carries none. That is a
|
||||
// boundary, not a loss: every caller of this function reads `.offset` off the result, and the anchor
|
||||
// path to an emitter runs through `Monolith::select`, which copies the whole `Entry`.
|
||||
Entry {
|
||||
signature,
|
||||
offset,
|
||||
class: None,
|
||||
anchors: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Look up an emitter by its `--format` id.
|
||||
|
|
@ -275,13 +283,32 @@ impl GamedataEmitter for ModSharp {
|
|||
let (mut addresses, mut vfuncs) = (Map::new(), Map::new());
|
||||
for (name, e) in &gd.entries {
|
||||
if let Some(s) = &e.signature {
|
||||
addresses.insert(
|
||||
name.clone(),
|
||||
json!({ "library": s.library, "linux": s.linux }),
|
||||
);
|
||||
let mut row = Map::new();
|
||||
row.insert("library".into(), Value::String(s.library.clone()));
|
||||
row.insert("linux".into(), Value::String(s.linux.clone()));
|
||||
// ModSharp's own `refs` feature: a string this function references, which its loader
|
||||
// resolves per build. Emitted BESIDE `linux` exactly as their hand-written gamedata does,
|
||||
// so a build whose byte pattern drifted can still be located.
|
||||
if !e.anchors.is_empty() {
|
||||
row.insert("refs".into(), json!({ "strings": e.anchors }));
|
||||
}
|
||||
addresses.insert(name.clone(), Value::Object(row));
|
||||
}
|
||||
if let Some(o) = e.offset {
|
||||
vfuncs.insert(name.clone(), json!({ "linux": o }));
|
||||
let mut row = Map::new();
|
||||
row.insert("linux".into(), json!(o));
|
||||
// `refs.vtable` is ModSharp's own key for "the class whose vtable holds this slot", and a
|
||||
// slot index without it is not a locator at all.
|
||||
if let Some(c) = &e.class {
|
||||
row.insert("refs".into(), json!({ "vtable": c }));
|
||||
}
|
||||
vfuncs.insert(name.clone(), Value::Object(row));
|
||||
}
|
||||
// An anchor with no signature is still a usable locator for ModSharp — `refs` alone is how
|
||||
// several of their own entries are written. Dropping these would discard the only thing we
|
||||
// know about a function whose byte pattern did not resolve.
|
||||
if e.signature.is_none() && !e.anchors.is_empty() {
|
||||
addresses.insert(name.clone(), json!({ "refs": { "strings": e.anchors } }));
|
||||
}
|
||||
}
|
||||
let doc = json!({ "Addresses": addresses, "VFuncs": vfuncs });
|
||||
|
|
@ -1635,10 +1662,14 @@ mod tests {
|
|||
linux: "55 48 ? E5".into(),
|
||||
}),
|
||||
offset: None,
|
||||
class: None,
|
||||
anchors: Vec::new(),
|
||||
};
|
||||
let off = Entry {
|
||||
signature: None,
|
||||
offset: Some(158),
|
||||
class: None,
|
||||
anchors: Vec::new(),
|
||||
};
|
||||
assert_eq!(entry_from_value(&locator_value(&sig)), sig);
|
||||
assert_eq!(entry_from_value(&locator_value(&off)), off);
|
||||
|
|
|
|||
|
|
@ -8,6 +8,10 @@
|
|||
//! It also exercises the two indirections that resolve a callback (a static object's first virtual, and
|
||||
//! a constructor-stored member) against pointers the file chose, which is the same untrusted-chase shape
|
||||
//! the Valve table readers had to be hardened for.
|
||||
//!
|
||||
//! CONVAR extraction rides the same pass and is fuzzed here with it. It adds two things worth attacking:
|
||||
//! a per-registrar delegation walk (bounded call/tail-jump decoding at a file-chosen address) and a
|
||||
//! statistical argument-slot choice, both driven entirely by bytes the file controls.
|
||||
use libfuzzer_sys::fuzz_target;
|
||||
use source2rosetta::concmd;
|
||||
use source2rosetta::elf::CodeImage;
|
||||
|
|
@ -36,6 +40,40 @@ fuzz_target!(|data: &[u8]| {
|
|||
assert!(named.len() <= 12, "more flag names than there are flag bits");
|
||||
let _ = (c.description.len(), c.form.describe(), c.flags);
|
||||
}
|
||||
// ---- ConVars: same pass, different registrar test ----
|
||||
let cvs = concmd::convars(&img, "server");
|
||||
for c in &cvs {
|
||||
// The name gate is the only thing separating a convar registration from any other call that
|
||||
// happens to pass a string, so it must hold on every row.
|
||||
assert!(
|
||||
!c.name.is_empty() && c.name.len() <= 64,
|
||||
"an implausible convar name was recorded: {:?}",
|
||||
c.name
|
||||
);
|
||||
// Flags are optional (a registrar with no identifiable slot reports none), but when present the
|
||||
// decode is a pure bit test over a table of 9 and cannot exceed it.
|
||||
if !c.flags_raw.is_empty() {
|
||||
let raw = u64::from_str_radix(c.flags_raw.trim_start_matches("0x"), 16)
|
||||
.expect("flags_raw is written as hex by this reader");
|
||||
assert!(raw <= u64::from(u32::MAX), "a convar flags word exceeded 32 bits");
|
||||
assert_eq!(
|
||||
c.flags.len(),
|
||||
concmd::convar_flag_names(raw).len(),
|
||||
"decoded flag names disagree with the raw word for {:?}",
|
||||
c.name
|
||||
);
|
||||
} else {
|
||||
assert!(c.flags.is_empty(), "flag names without a raw word for {:?}", c.name);
|
||||
}
|
||||
let _ = (c.description.len(), c.addr.len(), c.library.len());
|
||||
}
|
||||
// ConVars are deduped on (name, object address) for the same reason commands are.
|
||||
let mut cseen: Vec<(&str, &str)> = cvs.iter().map(|c| (c.name.as_str(), c.addr.as_str())).collect();
|
||||
let cbefore = cseen.len();
|
||||
cseen.sort_unstable();
|
||||
cseen.dedup();
|
||||
assert_eq!(cbefore, cseen.len(), "a duplicate (name, object) convar survived");
|
||||
|
||||
// Commands are deduped on (name, address), so no pair may survive twice.
|
||||
let mut seen: Vec<(&str, u64)> = cmds.iter().map(|c| (c.name.as_str(), c.handler)).collect();
|
||||
let before = seen.len();
|
||||
|
|
|
|||
|
|
@ -16,6 +16,20 @@ fuzz_target!(|data: &[u8]| {
|
|||
return;
|
||||
};
|
||||
let bindings = valvetab::pulse_bindings(&img);
|
||||
// The invocation shim's read-measurement walks a file-chosen address with its own span arithmetic,
|
||||
// and its verdict is emitted as `call.needs`, so it must degrade rather than panic or over-claim.
|
||||
for b in &bindings {
|
||||
if b.shim == 0 {
|
||||
continue;
|
||||
}
|
||||
if let Some(r) = pulse::shim_reads(&img, b.shim) {
|
||||
assert!(r.reads.len() <= 7, "more argument slots than the shim has");
|
||||
assert!(
|
||||
matches!(r.needs(), "args-only" | "output-sink" | "pulse-context" | "other-slots"),
|
||||
"needs() left its closed vocabulary"
|
||||
);
|
||||
}
|
||||
}
|
||||
let pairs: Vec<(u64, u64)> = bindings
|
||||
.iter()
|
||||
.take(64)
|
||||
|
|
|
|||
385
src/concmd.rs
385
src/concmd.rs
|
|
@ -29,6 +29,33 @@
|
|||
//! (it opens by writing the invalid-handle sentinel), a handler is accepted only if it lands in
|
||||
//! executable code, and a name only if it resolves to a plausible string. A layout change yields
|
||||
//! FEWER commands, never wrong ones.
|
||||
//!
|
||||
//! # ConVars, the other half of the same surface
|
||||
//!
|
||||
//! Convars register the same way and are read by the same pass, which is why they live here rather than in
|
||||
//! a module of their own — and, more importantly, they share the FCVAR flag space, so [`flag_names`] decodes
|
||||
//! both. A registration looks like:
|
||||
//!
|
||||
//! ```text
|
||||
//! lea r14, [rip+object] ; the ConVar itself — in .bss, so zero on disk
|
||||
//! lea rsi, [rip+"mp_maxrounds"] ; the name
|
||||
//! mov ecx, 0x282100 ; flags (bit 13 replicated, bit 19 release)
|
||||
//! lea r8, [rip+"max number of rounds…"] ; the help text
|
||||
//! call <registrar>
|
||||
//! ```
|
||||
//!
|
||||
//! **The object is in `.bss`**, so the scan-a-static-record route every other reader here uses is not
|
||||
//! available: on disk a ConVar is 344 zero bytes, and its name, flags and help exist only as arguments to
|
||||
//! the constructor call. Reading the call site is not a shortcut, it is the only offline route.
|
||||
//!
|
||||
//! The registrar is identified differently from the command one, and the difference is deliberate. A
|
||||
//! ConVar constructor has no equivalent of the invalid-handle sentinel to recognise it by, so the test is
|
||||
//! on the CALL SITE's argument shape — a convar-shaped name, prose-or-absent help, a flags word — and then
|
||||
//! on AGREEMENT: only a call target that presents that shape at `MIN_CONVAR_SITES` or more sites is
|
||||
//! accepted as a registrar. The doc on `inits_invalid_handle` warns that ranking call targets is wrong,
|
||||
//! and it is, for the shape it was warning about: "an argument that lands in executable code" fits far too
|
||||
//! much. Two strings with different character profiles plus a flags word plus a `.bss` pointer, repeated
|
||||
//! across dozens of sites, is a different order of evidence.
|
||||
|
||||
use crate::elf::CodeImage;
|
||||
use iced_x86::{
|
||||
|
|
@ -124,6 +151,39 @@ const FLAG_BITS: [(u32, &str); 12] = [
|
|||
(28, "clientcmd_can_execute"),
|
||||
];
|
||||
|
||||
/// CONVAR flag bits. A SEPARATE table from [`FLAG_BITS`], and the separation is not cosmetic.
|
||||
///
|
||||
/// The tempting assumption is that FCVAR is one flag space, so the command table decodes convars too. It
|
||||
/// does not, and shipping on that assumption mislabelled bit 0 as `linked_concommand` on 185 Dota and 56 CS2
|
||||
/// convars — a name that Valve's own dump gives to NONE of them. Whatever transforms the word on the way in
|
||||
/// (the registrar visibly masks a bit of it), the convar encoding is its own and has to be measured as its
|
||||
/// own.
|
||||
///
|
||||
/// Derived against Valve's published dumps for BOTH games — `GameTracking-{CS2,Dota2}/DumpSource2/
|
||||
/// convars.txt`, 1,939 convars pooled — keeping only bits whose flag holds at 100% precision. Bits 0, 1 and
|
||||
/// 2 are set often and match nothing cleanly; they stay unnamed and survive in `flags_raw`, which is what
|
||||
/// that field is for.
|
||||
const CONVAR_FLAG_BITS: [(u32, &str); 9] = [
|
||||
(4, "hidden"),
|
||||
(7, "archive"),
|
||||
(8, "notify"),
|
||||
(13, "replicated"),
|
||||
(14, "cheat"),
|
||||
(15, "per_user"),
|
||||
(17, "dontrecord"),
|
||||
(19, "release"),
|
||||
(21, "commandline_enforced"),
|
||||
];
|
||||
|
||||
/// The names of the bits set in a CONVAR's flags word that have a measured meaning.
|
||||
pub fn convar_flag_names(flags: u64) -> Vec<&'static str> {
|
||||
CONVAR_FLAG_BITS
|
||||
.iter()
|
||||
.filter(|(b, _)| flags & (1u64 << b) != 0)
|
||||
.map(|&(_, n)| n)
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The names of the bits set in `flags` that have a measured meaning. Bits without one are omitted here
|
||||
/// and preserved in [`ConsoleCommand::flags`].
|
||||
pub fn flag_names(flags: u64) -> Vec<&'static str> {
|
||||
|
|
@ -179,6 +239,9 @@ type Slot = (u8, u32, i64);
|
|||
/// One call to a registrar, with what its argument registers held and the `this`-relative constants its
|
||||
/// enclosing function stored.
|
||||
struct Site {
|
||||
/// The call target — which registrar this site went to. Convar registrars are identified by agreement
|
||||
/// across their sites, so the target has to survive collection.
|
||||
target: u64,
|
||||
args: [V; 16],
|
||||
stores: std::sync::Arc<HashMap<Slot, u64>>,
|
||||
}
|
||||
|
|
@ -233,14 +296,31 @@ fn cmd_name(img: &CodeImage, va: u64) -> Option<String> {
|
|||
|
||||
/// Every console command `img` registers.
|
||||
pub fn console_commands(img: &CodeImage) -> Vec<ConsoleCommand> {
|
||||
// Memoised so the prologue test runs once per distinct call target rather than once per call, and so
|
||||
// only registrar calls are ever materialised as a Site.
|
||||
let mut is_reg: HashMap<u64, bool> = HashMap::new();
|
||||
let sites = collect_sites(img, |img, t, _| {
|
||||
*is_reg
|
||||
.entry(t)
|
||||
.or_insert_with(|| inits_invalid_handle(img, t))
|
||||
});
|
||||
interpret_commands(img, &sites)
|
||||
}
|
||||
|
||||
/// Walk every function, constant-propagate the argument registers, and keep the call sites `accept` wants.
|
||||
///
|
||||
/// Single-sourced deliberately: this pass is subtle — the epoch counter, the write-only invalidation, the
|
||||
/// straight-line reset per function — and two copies of it would drift. The command and convar readers differ
|
||||
/// only in which calls they keep and how they read the arguments, so that is all `accept` decides.
|
||||
fn collect_sites(
|
||||
img: &CodeImage,
|
||||
mut accept: impl FnMut(&CodeImage, u64, &[V; 16]) -> bool,
|
||||
) -> Vec<Site> {
|
||||
let mut entries = crate::locate::candidate_entries(img);
|
||||
entries.extend(img.eh_frame_functions().into_iter().map(|(s, _)| s));
|
||||
entries.sort_unstable();
|
||||
entries.dedup();
|
||||
|
||||
// Memoised so the prologue test runs once per distinct call target rather than once per call, and so
|
||||
// only registrar calls are ever materialised as a Site.
|
||||
let mut is_reg: HashMap<u64, bool> = HashMap::new();
|
||||
let mut sites: Vec<Site> = Vec::new();
|
||||
let mut factory = InstructionInfoFactory::new();
|
||||
|
||||
|
|
@ -255,7 +335,7 @@ pub fn console_commands(img: &CodeImage) -> Vec<ConsoleCommand> {
|
|||
let mut val = [V::Unknown; 16];
|
||||
let mut epoch = [0u32; 16];
|
||||
let mut stores: HashMap<Slot, u64> = HashMap::new();
|
||||
let mut found: Vec<[V; 16]> = Vec::new();
|
||||
let mut found: Vec<(u64, [V; 16])> = Vec::new();
|
||||
let mut insn = Instruction::default();
|
||||
let mut dec = Decoder::with_ip(64, code, start, DecoderOptions::NONE);
|
||||
while dec.can_decode() {
|
||||
|
|
@ -268,11 +348,8 @@ pub fn console_commands(img: &CodeImage) -> Vec<ConsoleCommand> {
|
|||
)
|
||||
{
|
||||
let t = insn.near_branch_target();
|
||||
if *is_reg
|
||||
.entry(t)
|
||||
.or_insert_with(|| inits_invalid_handle(img, t))
|
||||
{
|
||||
found.push(val);
|
||||
if accept(img, t, &val) {
|
||||
found.push((t, val));
|
||||
}
|
||||
for c in CLOBBER {
|
||||
val[c] = V::Unknown;
|
||||
|
|
@ -364,15 +441,20 @@ pub fn console_commands(img: &CodeImage) -> Vec<ConsoleCommand> {
|
|||
}
|
||||
if !found.is_empty() {
|
||||
let stores = std::sync::Arc::new(stores);
|
||||
sites.extend(found.into_iter().map(|args| Site {
|
||||
sites.extend(found.into_iter().map(|(target, args)| Site {
|
||||
target,
|
||||
args,
|
||||
stores: stores.clone(),
|
||||
}));
|
||||
}
|
||||
}
|
||||
sites
|
||||
}
|
||||
|
||||
/// Read command registrations out of collected sites.
|
||||
fn interpret_commands(img: &CodeImage, sites: &[Site]) -> Vec<ConsoleCommand> {
|
||||
let mut out: Vec<ConsoleCommand> = Vec::new();
|
||||
for s in &sites {
|
||||
for s in sites {
|
||||
let Some(name) = s.args[RSI].konst().and_then(|v| cmd_name(img, v)) else {
|
||||
continue;
|
||||
};
|
||||
|
|
@ -481,3 +563,284 @@ mod tests {
|
|||
assert!(!ok(&"x".repeat(MAX_NAME + 1)));
|
||||
}
|
||||
}
|
||||
|
||||
/// How many call sites must present the convar argument shape before a target counts as a registrar.
|
||||
///
|
||||
/// This is the whole safety margin for identifying convar registration by shape rather than by a semantic
|
||||
/// sentinel. A coincidental `(object, name-ish string, int, prose string)` call happens; forty of them to the
|
||||
/// same target does not. Measured on CS2 libserver the real registrars carry hundreds of sites each, so the
|
||||
/// bar sits far below the signal and far above the noise.
|
||||
const MIN_CONVAR_SITES: usize = 12;
|
||||
|
||||
/// A ConVar the module registers, as its registration states it.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct ConVar {
|
||||
/// The console-facing name, e.g. `mp_maxrounds`.
|
||||
pub name: String,
|
||||
pub library: String,
|
||||
/// Valve's own help text; absent when the registration passes none.
|
||||
pub description: String,
|
||||
/// Flag bits with a measured meaning, decoded by [`convar_flag_names`] — NOT the command table, whose
|
||||
/// bit 0 name applies to no convar in either game's published dump.
|
||||
pub flags: Vec<String>,
|
||||
/// The raw flags word, kept beside the decoding so a build that repurposes a bit can be re-read rather
|
||||
/// than silently mis-labelled. Convars set bits commands never do (8 and 21 on CS2), and those have no
|
||||
/// name yet — this is where they survive.
|
||||
pub flags_raw: String,
|
||||
/// Address of the ConVar OBJECT. In `.bss`, so it holds nothing on disk; it is the anchor a runtime
|
||||
/// walks to reach the live value, and it is what distinguishes two registrations of the same name.
|
||||
pub addr: String,
|
||||
}
|
||||
|
||||
/// A plausible convar name: an identifier, lowercase by convention but not required, no spaces or prose.
|
||||
///
|
||||
/// Stricter than [`cmd_name`], which admits any printable run because commands like `+bugvoice` exist.
|
||||
/// A convar name is always an identifier, and the tighter gate is what keeps prose out of the name slot
|
||||
/// when the shape test is the only thing standing between a call site and a record.
|
||||
fn convar_name(img: &CodeImage, va: u64) -> Option<String> {
|
||||
let s = img.read_c_string(va)?;
|
||||
let ok = !s.is_empty()
|
||||
&& s.len() <= MAX_NAME
|
||||
&& s.chars()
|
||||
.next()
|
||||
.is_some_and(|c| c.is_ascii_alphabetic() || c == '_')
|
||||
&& s.bytes()
|
||||
.all(|c| c.is_ascii_alphanumeric() || c == b'_' || c == b'.');
|
||||
ok.then_some(s)
|
||||
}
|
||||
|
||||
/// Help text: prose, or nothing. Deliberately permissive about content and strict about being a real
|
||||
/// string — the point is to separate "this argument is a description" from "this argument is something else
|
||||
/// that happens to be a pointer".
|
||||
fn help_text(img: &CodeImage, va: u64) -> Option<String> {
|
||||
let s = img.read_c_string(va)?;
|
||||
(!s.is_empty() && s.len() <= 512 && s.is_ascii()).then_some(s)
|
||||
}
|
||||
|
||||
/// Does this call site look like a ConVar registration?
|
||||
///
|
||||
/// `rsi` a convar-shaped name, `rdi` a non-code address (the object), `rcx` a plausible flags word, and `r8`
|
||||
/// either help text or absent. Nothing here is sufficient alone; the caller additionally requires agreement
|
||||
/// across [`MIN_CONVAR_SITES`] sites to the same target.
|
||||
fn looks_like_convar_site(img: &CodeImage, args: &[V; 16]) -> bool {
|
||||
let Some(name) = args[RSI].konst() else {
|
||||
return false;
|
||||
};
|
||||
if convar_name(img, name).is_none() {
|
||||
return false;
|
||||
}
|
||||
// The object: a real address that is NOT code. A ConVar lives in writable data.
|
||||
match args[RDI].konst() {
|
||||
Some(o) if o != 0 && !img.is_code(o) => {}
|
||||
_ => return false,
|
||||
}
|
||||
// Flags: a 32-bit word. A pointer-sized value here means this is not the flags argument.
|
||||
match args[RCX].konst() {
|
||||
Some(f) if f <= u64::from(u32::MAX) => {}
|
||||
_ => return false,
|
||||
}
|
||||
// Help: present and prose, or genuinely absent. A non-zero value that is not a readable string means
|
||||
// the fifth argument is something else and this is not the registration shape.
|
||||
match args[R8] {
|
||||
V::Const(0) | V::Unknown => true,
|
||||
V::Const(p) => help_text(img, p).is_some(),
|
||||
V::Sym(..) => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Functions `f` delegates to — direct calls AND tail jumps.
|
||||
///
|
||||
/// The tail jumps are the point. A convar registrar is a thin wrapper that arranges arguments and then
|
||||
/// `jmp`s to the core rather than calling it, so a collector that only counts `call` sees a wrapper
|
||||
/// delegate to nothing and the convergence that identifies the family disappears. Only branches LEAVING the
|
||||
/// scanned span count as delegation; a jump within it is ordinary control flow.
|
||||
fn callees(img: &CodeImage, f: u64) -> Vec<u64> {
|
||||
const SPAN: u64 = 0x400;
|
||||
let end = f.saturating_add(SPAN);
|
||||
let Some(code) = img.code_range(f, end) else {
|
||||
return Vec::new();
|
||||
};
|
||||
let mut out = Vec::new();
|
||||
let mut insn = Instruction::default();
|
||||
let mut dec = Decoder::with_ip(64, code, f, DecoderOptions::NONE);
|
||||
while dec.can_decode() {
|
||||
dec.decode_out(&mut insn);
|
||||
if !matches!(
|
||||
insn.op0_kind(),
|
||||
OpKind::NearBranch16 | OpKind::NearBranch32 | OpKind::NearBranch64
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
let t = insn.near_branch_target();
|
||||
let delegates = match insn.flow_control() {
|
||||
FlowControl::Call => true,
|
||||
FlowControl::UnconditionalBranch => !(f..end).contains(&t),
|
||||
_ => false,
|
||||
};
|
||||
if delegates && img.is_code(t) {
|
||||
out.push(t);
|
||||
}
|
||||
}
|
||||
out.sort_unstable();
|
||||
out.dedup();
|
||||
out
|
||||
}
|
||||
|
||||
/// Which of the shape-matching targets are REAL convar registrars.
|
||||
///
|
||||
/// The argument shape alone is not enough, and this is the measurement that says so: on CS2 libserver it
|
||||
/// matches eleven targets, of which five register convars and six register something else with an
|
||||
/// identical footprint — animation events, mostly, which are also `(static object, identifier, int, prose)`.
|
||||
/// Checked against Valve's published convar dump the split is absolute: every one of those eleven targets is
|
||||
/// either 100% real convars or 0%. So the families ARE separable; the shape just is not what separates them.
|
||||
///
|
||||
/// What separates them is that the real registrars CONVERGE. Four of the five are wrappers that delegate to
|
||||
/// the fifth, which is itself a registrar — the cvar core. The false family shares no callee with them. So
|
||||
/// the core identifies itself: it is the candidate called by the most OTHER candidates. Accept it and its
|
||||
/// callers, reject everything else. On CS2 that yields exactly the five real registrars and 1,159 convars,
|
||||
/// with zero names absent from Valve's dump.
|
||||
///
|
||||
/// Deliberately NOT keyed off an address, a name, or Valve's dump: all three are per-build inputs this tool
|
||||
/// exists to avoid. The convergence is a property of the code in front of it.
|
||||
fn convar_registrars(img: &CodeImage, per_target: &HashMap<u64, usize>) -> Vec<u64> {
|
||||
let cands: Vec<u64> = per_target
|
||||
.iter()
|
||||
.filter(|&(_, &n)| n >= MIN_CONVAR_SITES)
|
||||
.map(|(&t, _)| t)
|
||||
.collect();
|
||||
let calls: HashMap<u64, Vec<u64>> = cands.iter().map(|&c| (c, callees(img, c))).collect();
|
||||
// How many candidates delegate to each function. The core does NOT have to be a candidate itself: on
|
||||
// CS2 it happens to take 28 registrations directly, but on Dota the shared core takes none, and
|
||||
// requiring it to be a candidate found nothing there at all.
|
||||
let mut inbound: HashMap<u64, usize> = HashMap::new();
|
||||
for (&from, tos) in &calls {
|
||||
for t in tos {
|
||||
if *t != from {
|
||||
*inbound.entry(*t).or_default() += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
// Ties broken by site count then address, so the choice cannot depend on hash order — this feeds a
|
||||
// byte-reproducible artifact.
|
||||
let Some((&core, &votes)) = inbound.iter().max_by_key(|&(t, n)| {
|
||||
(
|
||||
*n,
|
||||
per_target.get(t).copied().unwrap_or(0),
|
||||
std::cmp::Reverse(*t),
|
||||
)
|
||||
}) else {
|
||||
return Vec::new();
|
||||
};
|
||||
// One wrapper proves nothing; a family of them is the signal. Below this the convergence is noise and
|
||||
// reporting NOTHING is the honest outcome — the profile floor then fails the release loudly.
|
||||
if votes < 2 {
|
||||
return Vec::new();
|
||||
}
|
||||
let mut keep: Vec<u64> = Vec::new();
|
||||
if cands.contains(&core) {
|
||||
keep.push(core);
|
||||
}
|
||||
keep.extend(
|
||||
cands
|
||||
.iter()
|
||||
.copied()
|
||||
.filter(|c| calls.get(c).is_some_and(|t| t.contains(&core))),
|
||||
);
|
||||
keep.sort_unstable();
|
||||
keep.dedup();
|
||||
keep
|
||||
}
|
||||
|
||||
/// Which argument slot holds the FLAGS, for one registrar.
|
||||
///
|
||||
/// It is not the same slot for every registrar, and assuming it was is what first produced convars whose
|
||||
/// "flags" were `0x99dc60` — a `.rodata` pointer sitting in the slot a different overload uses for
|
||||
/// something else. The name slot is stable across all of them; nothing else is.
|
||||
///
|
||||
/// Found statistically, because flags REPEAT and pointers do not: across a registrar's sites the flags slot
|
||||
/// takes a small set of recurring words (`0x4000` alone appears 188 times on CS2), while a slot holding a
|
||||
/// string or an object address is very nearly unique per site. So the flags slot is the integer-shaped one
|
||||
/// with the lowest distinct-value ratio — and if nothing is clearly repetitive, this returns `None` and the
|
||||
/// registrar's convars ship with no decoded flags rather than with invented ones.
|
||||
fn flags_slot(sites: &[&Site]) -> Option<usize> {
|
||||
const CANDIDATES: [usize; 4] = [RDX, RCX, R8, R9];
|
||||
let mut best: Option<(usize, f64)> = None;
|
||||
for slot in CANDIDATES {
|
||||
let vals: Vec<u64> = sites.iter().filter_map(|s| s.args[slot].konst()).collect();
|
||||
// Every value must fit a 32-bit flags word, and the slot must be present on nearly every site.
|
||||
if vals.len() * 4 < sites.len() * 3 || vals.iter().any(|&v| v > u64::from(u32::MAX)) {
|
||||
continue;
|
||||
}
|
||||
let mut d = vals.clone();
|
||||
d.sort_unstable();
|
||||
d.dedup();
|
||||
let ratio = d.len() as f64 / vals.len() as f64;
|
||||
if best.is_none_or(|(_, b)| ratio < b) {
|
||||
best = Some((slot, ratio));
|
||||
}
|
||||
}
|
||||
// A genuine flags slot repeats heavily. Anything above this is as unique as a pointer, which is what a
|
||||
// pointer is, and naming its bits would be fabrication.
|
||||
best.filter(|&(_, r)| r < 0.5).map(|(s, _)| s)
|
||||
}
|
||||
|
||||
/// Every ConVar `img` registers.
|
||||
pub fn convars(img: &CodeImage, library: &str) -> Vec<ConVar> {
|
||||
let sites = collect_sites(img, |img, _, args| looks_like_convar_site(img, args));
|
||||
let mut per_target: HashMap<u64, usize> = HashMap::new();
|
||||
for s in &sites {
|
||||
*per_target.entry(s.target).or_default() += 1;
|
||||
}
|
||||
let registrars = convar_registrars(img, &per_target);
|
||||
// Resolve the flags slot once per registrar, from all of that registrar's sites.
|
||||
let flag_of: HashMap<u64, Option<usize>> = registrars
|
||||
.iter()
|
||||
.map(|&r| {
|
||||
let mine: Vec<&Site> = sites.iter().filter(|s| s.target == r).collect();
|
||||
(r, flags_slot(&mine))
|
||||
})
|
||||
.collect();
|
||||
let mut out: Vec<ConVar> = Vec::new();
|
||||
for s in &sites {
|
||||
if !registrars.contains(&s.target) {
|
||||
continue;
|
||||
}
|
||||
let (Some(name), Some(obj)) = (
|
||||
s.args[RSI].konst().and_then(|v| convar_name(img, v)),
|
||||
s.args[RDI].konst(),
|
||||
) else {
|
||||
continue;
|
||||
};
|
||||
// Absent when this registrar has no identifiable flags slot: no decoded names, and a raw word of
|
||||
// zero that is honestly empty rather than a guess.
|
||||
let flags = flag_of
|
||||
.get(&s.target)
|
||||
.copied()
|
||||
.flatten()
|
||||
.and_then(|slot| s.args[slot].konst());
|
||||
out.push(ConVar {
|
||||
name,
|
||||
library: library.to_string(),
|
||||
description: s.args[R8]
|
||||
.konst()
|
||||
.filter(|&p| p != 0)
|
||||
.and_then(|p| help_text(img, p))
|
||||
.unwrap_or_default(),
|
||||
flags: flags
|
||||
.map(|f| {
|
||||
convar_flag_names(f)
|
||||
.into_iter()
|
||||
.map(str::to_string)
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default(),
|
||||
flags_raw: flags.map(|f| format!("{f:#x}")).unwrap_or_default(),
|
||||
addr: format!("{obj:#x}"),
|
||||
});
|
||||
}
|
||||
// One row per (name, object): the same convar is registered once, but a name can legitimately appear
|
||||
// twice across libraries and the object is what tells those apart.
|
||||
out.sort_by(|a, b| a.name.cmp(&b.name).then_with(|| a.addr.cmp(&b.addr)));
|
||||
out.dedup_by(|a, b| a.name == b.name && a.addr == b.addr);
|
||||
out
|
||||
}
|
||||
|
|
|
|||
138
src/live.rs
138
src/live.rs
|
|
@ -207,6 +207,64 @@ impl LiveProcess {
|
|||
pub struct CallResult {
|
||||
pub rax: u64,
|
||||
pub clean_return: bool,
|
||||
/// Where the scratch blob was placed, so the caller can read back what the callee wrote into it.
|
||||
/// Zero when the call carried no scratch.
|
||||
pub scratch_base: u64,
|
||||
}
|
||||
|
||||
/// One argument to a remote call.
|
||||
///
|
||||
/// [`Arg::Scratch`] exists because a callee that takes a POINTER needs a structure to point at, and the
|
||||
/// address of that structure is not known until the call frame is laid out. Naming it relative to the
|
||||
/// scratch base lets the caller describe "argument 5 points at my blob" without knowing where the blob
|
||||
/// will land.
|
||||
#[derive(Clone, Copy)]
|
||||
pub enum Arg {
|
||||
Val(u64),
|
||||
/// `scratch_base + addend`.
|
||||
Scratch(i64),
|
||||
}
|
||||
|
||||
/// A blob placed in the target's stack scratch before the call.
|
||||
pub struct Scratch<'a> {
|
||||
pub bytes: &'a [u8],
|
||||
/// `(offset, addend)` — write `scratch_base + addend` as a little-endian u64 at `offset` in the blob.
|
||||
/// This is how a pointer INSIDE the blob becomes absolute; an array-of-pointers argument is otherwise
|
||||
/// impossible to build, since every element has to name an address that does not exist yet.
|
||||
pub relocs: &'a [(usize, i64)],
|
||||
}
|
||||
|
||||
/// Write `data` into the target at `addr`, a word at a time.
|
||||
///
|
||||
/// A trailing partial word is read back and merged rather than zero-filled: `PTRACE_POKEDATA` writes a
|
||||
/// whole word, so writing the tail without preserving the bytes past it would clobber memory the caller
|
||||
/// never asked to touch.
|
||||
unsafe fn poke_bytes(pid: i32, addr: u64, data: &[u8]) -> Result<()> {
|
||||
use anyhow::bail;
|
||||
let mut i = 0usize;
|
||||
while i < data.len() {
|
||||
let at = addr + i as u64;
|
||||
let n = (data.len() - i).min(8);
|
||||
let mut word = if n == 8 {
|
||||
[0u8; 8]
|
||||
} else {
|
||||
// PEEKDATA returns -1 both for an error and for a word whose value IS -1, so errno is the
|
||||
// only way to tell them apart and it must be cleared first.
|
||||
unsafe { *libc::__errno_location() = 0 };
|
||||
let cur = unsafe { libc::ptrace(libc::PTRACE_PEEKDATA, pid, at as usize, 0usize) };
|
||||
if cur == -1 && errno() != 0 {
|
||||
bail!("PEEKDATA at {at:#x} failed (errno {})", errno());
|
||||
}
|
||||
(cur as u64).to_le_bytes()
|
||||
};
|
||||
word[..n].copy_from_slice(&data[i..i + n]);
|
||||
let w = u64::from_le_bytes(word) as usize;
|
||||
if unsafe { libc::ptrace(libc::PTRACE_POKEDATA, pid, at as usize, w) } < 0 {
|
||||
bail!("POKEDATA at {at:#x} failed (errno {})", errno());
|
||||
}
|
||||
i += n;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Call the function at runtime address `func` inside process `pid` with `args` (SysV: up to 6 in
|
||||
|
|
@ -215,8 +273,42 @@ pub struct CallResult {
|
|||
/// the thread exactly — the SIGSEGV from the return trap is suppressed. Needs ptrace permission
|
||||
/// (owned child, or same-user with ptrace_scope=0). UNSAFE: only call leaf-ish functions with valid args.
|
||||
pub fn call_remote(pid: i32, func: u64, args: &[u64]) -> Result<CallResult> {
|
||||
let regs: Vec<Arg> = args.iter().map(|&v| Arg::Val(v)).collect();
|
||||
call_remote_ex(pid, func, ®s, &[], None)
|
||||
}
|
||||
|
||||
/// [`call_remote`] plus stack arguments and a scratch blob placed in the target.
|
||||
///
|
||||
/// Needed for callees that take more than six integer arguments or a pointer to a structure the caller has
|
||||
/// to build — neither of which the register-only form can express.
|
||||
///
|
||||
/// **Stack geometry**, descending from the interrupted `rsp`, chosen so three regions cannot collide:
|
||||
/// the 128-byte red zone is left alone (the interrupted frame lives there); the scratch blob sits at
|
||||
/// `rsp-1024`; the call frame starts at `rsp-2048`, so the callee's own stack — which grows DOWN from
|
||||
/// there — can never reach the scratch ABOVE it. Entry keeps SysV's `rsp % 16 == 8`, with the return
|
||||
/// address at `[rsp]` and stack argument *i* at `[rsp + 8 + 8i]`.
|
||||
pub fn call_remote_ex(
|
||||
pid: i32,
|
||||
func: u64,
|
||||
regs_in: &[Arg],
|
||||
stack_in: &[Arg],
|
||||
scratch: Option<Scratch<'_>>,
|
||||
) -> Result<CallResult> {
|
||||
use anyhow::bail;
|
||||
let dbg = std::env::var("SOURCE2ROSETTA_DBG").is_ok();
|
||||
if regs_in.len() > 6 {
|
||||
bail!("{} register arguments; SysV has 6", regs_in.len());
|
||||
}
|
||||
if let Some(s) = &scratch {
|
||||
// The blob lives in the 1 KiB between the frame and the red zone. Refuse rather than silently
|
||||
// overlap the call frame, which would corrupt the return address mid-call.
|
||||
if s.bytes.len() > 768 {
|
||||
bail!(
|
||||
"scratch blob is {} bytes; the reserved window is 768",
|
||||
s.bytes.len()
|
||||
);
|
||||
}
|
||||
}
|
||||
unsafe {
|
||||
if libc::ptrace(libc::PTRACE_ATTACH, pid, 0usize, 0usize) < 0 {
|
||||
bail!(
|
||||
|
|
@ -249,6 +341,31 @@ pub fn call_remote(pid: i32, func: u64, args: &[u64]) -> Result<CallResult> {
|
|||
// If we attached mid-syscall, orig_rax holds the syscall number and the kernel would run its
|
||||
// syscall-restart logic on our injected rip. Setting it to -1 says "no syscall in progress".
|
||||
regs.orig_rax = u64::MAX;
|
||||
|
||||
// Place the scratch blob first: every Arg::Scratch resolves against its base.
|
||||
let scratch_base = (saved.rsp - 1024) & !0xfu64;
|
||||
if let Some(s) = &scratch {
|
||||
let mut blob = s.bytes.to_vec();
|
||||
for &(off, addend) in s.relocs {
|
||||
let Some(dst) = blob.get_mut(off..off + 8) else {
|
||||
restore(&saved);
|
||||
bail!(
|
||||
"scratch reloc at {off} runs past the {}-byte blob",
|
||||
s.bytes.len()
|
||||
);
|
||||
};
|
||||
dst.copy_from_slice(&scratch_base.wrapping_add(addend as u64).to_le_bytes());
|
||||
}
|
||||
if let Err(e) = poke_bytes(pid, scratch_base, &blob) {
|
||||
restore(&saved);
|
||||
return Err(e.context(format!("placing scratch at {scratch_base:#x}")));
|
||||
}
|
||||
}
|
||||
let resolve = |a: Arg| match a {
|
||||
Arg::Val(v) => v,
|
||||
Arg::Scratch(addend) => scratch_base.wrapping_add(addend as u64),
|
||||
};
|
||||
|
||||
let slots = [
|
||||
&mut regs.rdi as *mut u64,
|
||||
&mut regs.rsi,
|
||||
|
|
@ -257,12 +374,12 @@ pub fn call_remote(pid: i32, func: u64, args: &[u64]) -> Result<CallResult> {
|
|||
&mut regs.r8,
|
||||
&mut regs.r9,
|
||||
];
|
||||
for (i, &a) in args.iter().take(6).enumerate() {
|
||||
*slots[i] = a;
|
||||
for (i, &a) in regs_in.iter().enumerate() {
|
||||
*slots[i] = resolve(a);
|
||||
}
|
||||
// Scratch stack BELOW the 128-byte redzone so we never corrupt the interrupted frame; write a
|
||||
// Call frame well below the scratch, so the callee's downward stack growth cannot reach it. Write a
|
||||
// return address of 0 and keep SysV's `rsp % 16 == 8` at function entry.
|
||||
let mut sp = (saved.rsp - 512) & !0xfu64;
|
||||
let mut sp = (saved.rsp - 2048) & !0xfu64;
|
||||
sp -= 8;
|
||||
if libc::ptrace(libc::PTRACE_POKEDATA, pid, sp as usize, 0usize) < 0 {
|
||||
restore(&saved);
|
||||
|
|
@ -271,6 +388,17 @@ pub fn call_remote(pid: i32, func: u64, args: &[u64]) -> Result<CallResult> {
|
|||
errno()
|
||||
);
|
||||
}
|
||||
// Stack arguments sit immediately above the return address, which is where the callee reads them.
|
||||
for (i, &a) in stack_in.iter().enumerate() {
|
||||
let at = sp + 8 + 8 * i as u64;
|
||||
if libc::ptrace(libc::PTRACE_POKEDATA, pid, at as usize, resolve(a) as usize) < 0 {
|
||||
restore(&saved);
|
||||
bail!(
|
||||
"POKEDATA(stack arg {i}) at {at:#x} failed (errno {})",
|
||||
errno()
|
||||
);
|
||||
}
|
||||
}
|
||||
let wrote = libc::ptrace(libc::PTRACE_PEEKDATA, pid, sp as usize, 0usize);
|
||||
regs.rsp = sp;
|
||||
regs.rip = func;
|
||||
|
|
@ -305,6 +433,7 @@ pub fn call_remote(pid: i32, func: u64, args: &[u64]) -> Result<CallResult> {
|
|||
let r = CallResult {
|
||||
rax: cur.rax,
|
||||
clean_return: true,
|
||||
scratch_base,
|
||||
};
|
||||
restore(&saved);
|
||||
return Ok(r);
|
||||
|
|
@ -313,6 +442,7 @@ pub fn call_remote(pid: i32, func: u64, args: &[u64]) -> Result<CallResult> {
|
|||
let r = CallResult {
|
||||
rax: cur.rax,
|
||||
clean_return: false,
|
||||
scratch_base,
|
||||
};
|
||||
restore(&saved);
|
||||
return Ok(r);
|
||||
|
|
|
|||
338
src/pipeline.rs
338
src/pipeline.rs
|
|
@ -968,16 +968,19 @@ fn fold_valve_tables(
|
|||
source_build: source_build.to_string(),
|
||||
pulse: 0,
|
||||
pulse_typed: 0,
|
||||
pulse_callable: 0,
|
||||
entity_inputs: 0,
|
||||
entity_outputs: 0,
|
||||
entity_classes: 0,
|
||||
commands: 0,
|
||||
convars: 0,
|
||||
},
|
||||
pulse: BTreeMap::new(),
|
||||
entity_inputs: Vec::new(),
|
||||
entity_outputs: Vec::new(),
|
||||
entity_classes: BTreeMap::new(),
|
||||
commands: Vec::new(),
|
||||
convars: Vec::new(),
|
||||
},
|
||||
};
|
||||
let (mut folded, mut ambiguous, mut unmakeable) = (0u32, 0usize, 0u32);
|
||||
|
|
@ -1005,6 +1008,22 @@ fn fold_valve_tables(
|
|||
// table-emptiness skip below.
|
||||
let commands = concmd::console_commands(&img);
|
||||
let lib = lib_name_from_file(f);
|
||||
// ConVars: the other half of the console surface, read by the same pass. Documentation, not a
|
||||
// locator — a consumer finds a convar by name at runtime; the flags are what it cannot get itself.
|
||||
out.bindings
|
||||
.convars
|
||||
.extend(
|
||||
concmd::convars(&img, &lib)
|
||||
.into_iter()
|
||||
.map(|c| model::ConVar {
|
||||
name: c.name,
|
||||
library: c.library,
|
||||
description: c.description,
|
||||
flags: c.flags,
|
||||
flags_raw: c.flags_raw,
|
||||
addr: c.addr,
|
||||
}),
|
||||
);
|
||||
{
|
||||
let addrs: Vec<u64> = commands.iter().map(|c| c.handler).collect();
|
||||
let sigs = parallel_map(&addrs, threads, |&a| emit::make_sig(&img, a, sig_cap));
|
||||
|
|
@ -1260,6 +1279,16 @@ fn fold_valve_tables(
|
|||
returns: sig.as_ref().map(|s| s.returns.clone()).unwrap_or_default(),
|
||||
typed: sig.is_some(),
|
||||
descriptor: format!("{:#x}", b.descriptor),
|
||||
shim: (b.shim != 0).then(|| format!("{:#x}", b.shim)),
|
||||
// Measured per shim rather than assumed from the tier: the calling contract is fixed, but
|
||||
// WHICH slots a given shim reads is the whole difference between host-callable and not.
|
||||
call: (b.shim != 0)
|
||||
.then(|| pulse::shim_reads(&img, b.shim))
|
||||
.flatten()
|
||||
.map(|r| model::ShimCall {
|
||||
needs: r.needs().to_string(),
|
||||
reads: r.reads.iter().map(|s| s.to_string()).collect(),
|
||||
}),
|
||||
};
|
||||
// The registry is keyed by qualified name across libraries, so a binding registered by
|
||||
// more than one module keeps ONE row. That is the documented lossiness — of the LIBRARY,
|
||||
|
|
@ -1300,6 +1329,14 @@ fn fold_valve_tables(
|
|||
.sort_by(|a, b| (&a.input, &a.handler, &a.addr).cmp(&(&b.input, &b.handler, &b.addr)));
|
||||
out.bindings.meta.pulse = out.bindings.pulse.len();
|
||||
out.bindings.meta.pulse_typed = out.bindings.pulse.values().filter(|b| b.typed).count();
|
||||
// Counted from the emitted rows rather than tallied during the fold, so the number in `meta` cannot
|
||||
// drift from the number of rows a consumer can actually act on.
|
||||
out.bindings.meta.pulse_callable = out
|
||||
.bindings
|
||||
.pulse
|
||||
.values()
|
||||
.filter(|b| b.call.as_ref().is_some_and(|c| c.needs == "args-only"))
|
||||
.count();
|
||||
out.bindings.meta.entity_inputs = out.bindings.entity_inputs.len();
|
||||
out.bindings
|
||||
.entity_outputs
|
||||
|
|
@ -1310,6 +1347,7 @@ fn fold_valve_tables(
|
|||
.commands
|
||||
.sort_by(|a, b| (&a.name, &a.addr).cmp(&(&b.name, &b.addr)));
|
||||
out.bindings.meta.commands = out.bindings.commands.len();
|
||||
out.bindings.meta.convars = out.bindings.convars.len();
|
||||
|
||||
if cmd_total > 0 {
|
||||
let libs: BTreeSet<&str> = out
|
||||
|
|
@ -1459,6 +1497,245 @@ fn binding_kind(f: valvetab::PulseFlags) -> model::BindingKind {
|
|||
/// Assemble the monolith in memory and render its CS# gamedata (the string the live validate stage checks).
|
||||
/// Writes NOTHING — `produce` holds the `Monolith` (to annotate it live) plus this render, and writes the
|
||||
/// monolith exactly once at the end (after live validation, if a game is present).
|
||||
/// Fold string anchors onto every monolith entry whose name carries one, in every tier.
|
||||
///
|
||||
/// Returns how many landed. That number is REPORTED rather than assumed because the two populations are
|
||||
/// independent: the catalogue says which names have anchors, the derive says which names got a locator, and
|
||||
/// an anchor for a name that never resolved has nowhere to go. A large gap is a fact about the build, not a
|
||||
/// bug — but it should be visible rather than inferred from an artifact diff.
|
||||
fn attach_anchors(mono: &mut model::Monolith, anchors: &BTreeMap<String, Vec<String>>) -> usize {
|
||||
let mut n = 0;
|
||||
for tier in [
|
||||
&mut mono.core,
|
||||
&mut mono.high_confidence,
|
||||
&mut mono.experimental,
|
||||
] {
|
||||
for (name, e) in tier.iter_mut() {
|
||||
if let Some(a) = anchors.get(name) {
|
||||
// Deduplicated on the way in: the same anchor can appear on several catalogue variants,
|
||||
// and this list ships in a byte-reproducible artifact.
|
||||
for s in a {
|
||||
if !e.locator.anchors.contains(s) {
|
||||
e.locator.anchors.push(s.clone());
|
||||
}
|
||||
}
|
||||
n += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
n
|
||||
}
|
||||
|
||||
/// A string worth anchoring on: long enough to be distinctive, printable, and not a lone format specifier.
|
||||
///
|
||||
/// The thresholds are the knob this whole feature turns on. Loosening them raises coverage and lowers
|
||||
/// distinctiveness; they were measured, not guessed — at these values 4,322 of libserver's 70,288 functions
|
||||
/// have a unique anchor, and 27% of the shipped set does.
|
||||
fn usable_anchor(s: &str) -> bool {
|
||||
s.len() >= 8
|
||||
&& s.len() <= 200
|
||||
&& s.is_ascii()
|
||||
&& s.chars().filter(|c| c.is_ascii_alphanumeric()).count() >= 5
|
||||
}
|
||||
|
||||
/// Where every anchorable string in `img` is referenced FROM: string address -> the instruction addresses
|
||||
/// that load it, plus the string itself.
|
||||
///
|
||||
/// Deliberately instruction-level and function-agnostic. Attributing a string to a function needs function
|
||||
/// BOUNDARIES, and this binary does not reliably supply them — `.eh_frame_hdr` describes 8,327 of libserver's
|
||||
/// ~70,000 functions, so a `[entry, next_entry)` range routinely spans a real function plus one or more
|
||||
/// unindexed neighbours, and every neighbour's strings then look like the first function's. An instruction
|
||||
/// address, by contrast, is exactly what it is. The caller decides membership against an extent it walked
|
||||
/// itself, which is the only claim available that does not depend on the entry list being complete.
|
||||
fn string_refs(img: &CodeImage) -> HashMap<u64, (String, Vec<u64>)> {
|
||||
let mut entries = crate::locate::candidate_entries(img);
|
||||
entries.extend(img.eh_frame_functions().into_iter().map(|(s, _)| s));
|
||||
entries.sort_unstable();
|
||||
entries.dedup();
|
||||
|
||||
let mut out: HashMap<u64, (String, Vec<u64>)> = HashMap::new();
|
||||
for (i, &start) in entries.iter().enumerate() {
|
||||
let end = entries.get(i + 1).copied().unwrap_or(u64::MAX);
|
||||
let Some(code) = img.code_range(start, end) else {
|
||||
continue;
|
||||
};
|
||||
let mut insn = iced_x86::Instruction::default();
|
||||
let mut dec = iced_x86::Decoder::with_ip(64, code, start, iced_x86::DecoderOptions::NONE);
|
||||
while dec.can_decode() {
|
||||
dec.decode_out(&mut insn);
|
||||
if insn.is_invalid() || !insn.is_ip_rel_memory_operand() {
|
||||
continue;
|
||||
}
|
||||
let va = insn.ip_rel_memory_address();
|
||||
if let Some(e) = out.get_mut(&va) {
|
||||
e.1.push(insn.ip());
|
||||
} else if let Some(s) = img.read_c_string(va).filter(|s| usable_anchor(s)) {
|
||||
out.insert(va, (s, vec![insn.ip()]));
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Instruction addresses reachable from `entry` by following control flow, and the string addresses it loads.
|
||||
///
|
||||
/// The function's OWN extent, determined by where its branches go and where it returns, rather than by the
|
||||
/// next symbol. That is what makes the anchor check sound without a complete function list.
|
||||
fn reachable_strings(img: &CodeImage, entry: u64) -> Option<(HashSet<u64>, Vec<u64>)> {
|
||||
const CAP: u64 = 0x4000;
|
||||
let all = img.code_at(entry)?;
|
||||
let extent = (all.len() as u64).min(CAP);
|
||||
let code = &all[..extent as usize];
|
||||
let mut seen: HashSet<u64> = HashSet::new();
|
||||
let mut loads: Vec<u64> = Vec::new();
|
||||
// Saturating for the same reason `pulse::shim_reads` is: a file-controlled extent must not wrap the
|
||||
// range inside out under the overflow-checked build the fuzzers use.
|
||||
let end = entry.saturating_add(extent);
|
||||
let mut work = vec![entry];
|
||||
let mut insn = iced_x86::Instruction::default();
|
||||
while let Some(at) = work.pop() {
|
||||
if at < entry || at >= end || !seen.insert(at) || seen.len() > 40000 {
|
||||
continue;
|
||||
}
|
||||
let mut dec = iced_x86::Decoder::with_ip(
|
||||
64,
|
||||
&code[(at - entry) as usize..],
|
||||
at,
|
||||
iced_x86::DecoderOptions::NONE,
|
||||
);
|
||||
if !dec.can_decode() {
|
||||
continue;
|
||||
}
|
||||
dec.decode_out(&mut insn);
|
||||
if insn.is_invalid() || insn.len() == 0 {
|
||||
continue;
|
||||
}
|
||||
if insn.is_ip_rel_memory_operand() {
|
||||
loads.push(insn.ip_rel_memory_address());
|
||||
}
|
||||
match insn.flow_control() {
|
||||
iced_x86::FlowControl::Return
|
||||
| iced_x86::FlowControl::IndirectBranch
|
||||
| iced_x86::FlowControl::Exception
|
||||
| iced_x86::FlowControl::Interrupt => {}
|
||||
iced_x86::FlowControl::UnconditionalBranch => work.push(insn.near_branch_target()),
|
||||
iced_x86::FlowControl::ConditionalBranch => {
|
||||
work.push(at + insn.len() as u64);
|
||||
work.push(insn.near_branch_target());
|
||||
}
|
||||
_ => work.push(at + insn.len() as u64),
|
||||
}
|
||||
}
|
||||
Some((seen, loads))
|
||||
}
|
||||
|
||||
/// Derive an anchor for every entry that has none, from the address its SHIPPED signature resolves to.
|
||||
///
|
||||
/// Three conditions, each closing a way this can name the wrong function:
|
||||
///
|
||||
/// 1. **The resolved address must be a function ENTRY POINT.** ModSharp's `refs.strings` locates a
|
||||
/// *function*; a great many shipped locators deliberately point MID-function (`CBaseButton::InputPress`
|
||||
/// resolves to a `mov`, `BotNavIgnore` to a `je` — patterns anchored at a hook site, not a prologue). An
|
||||
/// anchor cannot denote the same thing as one of those, so those entries get none rather than a locator
|
||||
/// that resolves somewhere else.
|
||||
/// 2. **The string must be referenced from inside the function's OWN flow-reachable code**, walked from the
|
||||
/// entry, not from a `[entry, next_entry)` range. `.eh_frame_hdr` covers a small fraction of these
|
||||
/// binaries' functions, so such a range routinely swallows unindexed neighbours and inherits their
|
||||
/// strings — which is exactly how a first cut of this produced "`CBaseButton::InputPress` references
|
||||
/// *Traced intervals in %.3fus*".
|
||||
/// 3. **Every instruction that loads the string must be inside that same reachable set.** This is the
|
||||
/// uniqueness test, done at instruction level so it never consults a function boundary. A string also
|
||||
/// loaded from elsewhere locates nothing and is dropped.
|
||||
///
|
||||
/// Server-library only, the restriction [`recover_by_string_anchor`] already carries: the fold holds that one
|
||||
/// image, and loading a second full set of 22 keyed images to reach the rest would add several hundred MB to a
|
||||
/// pipeline that has already been OOM-killed on Dota.
|
||||
///
|
||||
/// Returns `(attached, considered)`. Degrades quietly toward FEWER anchors and never toward a wrong one.
|
||||
fn attach_derived_anchors(
|
||||
mono: &mut model::Monolith,
|
||||
img: &CodeImage,
|
||||
server_lib: &str,
|
||||
) -> (usize, usize) {
|
||||
let wants = |e: &model::MonoEntry| {
|
||||
e.locator.anchors.is_empty()
|
||||
&& e.locator
|
||||
.signature
|
||||
.as_ref()
|
||||
.is_some_and(|s| s.library == server_lib)
|
||||
};
|
||||
let any = [&mono.core, &mono.high_confidence, &mono.experimental]
|
||||
.iter()
|
||||
.any(|t| t.values().any(&wants));
|
||||
if !any {
|
||||
return (0, 0);
|
||||
}
|
||||
let refs = string_refs(img);
|
||||
// Condition 1's test set: the addresses this image treats as function starts.
|
||||
let mut starts = crate::locate::candidate_entries(img);
|
||||
starts.extend(img.eh_frame_functions().into_iter().map(|(s, _)| s));
|
||||
starts.sort_unstable();
|
||||
starts.dedup();
|
||||
|
||||
// Why each candidate was rejected, so a low yield is a FACT rather than a mystery. The three
|
||||
// conditions fail for very different reasons and the mix differs sharply between games (CS2 derives
|
||||
// ~5% of candidates, Dota ~0.5%); without this the difference is unattributable.
|
||||
let (mut attached, mut considered) = (0usize, 0usize);
|
||||
let (mut no_resolve, mut mid_fn, mut no_unique) = (0usize, 0usize, 0usize);
|
||||
for tier in [
|
||||
&mut mono.core,
|
||||
&mut mono.high_confidence,
|
||||
&mut mono.experimental,
|
||||
] {
|
||||
for e in tier.values_mut() {
|
||||
if !wants(e) {
|
||||
continue;
|
||||
}
|
||||
considered += 1;
|
||||
let Some(sig) = e.locator.signature.as_ref() else {
|
||||
continue;
|
||||
};
|
||||
let Ok(pat) = crate::sig::Pattern::parse(&sig.linux) else {
|
||||
continue;
|
||||
};
|
||||
let hits = img.find(&pat);
|
||||
let [addr] = hits.as_slice() else {
|
||||
no_resolve += 1;
|
||||
continue;
|
||||
};
|
||||
let addr = *addr;
|
||||
if starts.binary_search(&addr).is_err() {
|
||||
mid_fn += 1; // condition 1: mid-function locator, not a function an anchor can name
|
||||
continue;
|
||||
}
|
||||
let Some((reach, loads)) = reachable_strings(img, addr) else {
|
||||
continue;
|
||||
};
|
||||
// Candidates this function actually loads, longest first then lexicographic — deterministic,
|
||||
// because this lands in a byte-reproducible artifact.
|
||||
let mut cands: Vec<&(String, Vec<u64>)> = loads
|
||||
.iter()
|
||||
.filter_map(|va| refs.get(va))
|
||||
.filter(|(_, from)| from.iter().all(|ip| reach.contains(ip)))
|
||||
.collect();
|
||||
cands.sort_unstable_by(|a, b| b.0.len().cmp(&a.0.len()).then_with(|| a.0.cmp(&b.0)));
|
||||
match cands.first() {
|
||||
Some((s, _)) => {
|
||||
e.locator.anchors.push(s.clone());
|
||||
attached += 1;
|
||||
}
|
||||
None => no_unique += 1,
|
||||
}
|
||||
}
|
||||
}
|
||||
eprintln!(
|
||||
" anchor derivation: {attached} attached; rejected {no_resolve} (pattern did not resolve \
|
||||
uniquely), {mid_fn} (locator is mid-function, which an anchor cannot name), {no_unique} (no string \
|
||||
unique to the function)"
|
||||
);
|
||||
(attached, considered)
|
||||
}
|
||||
|
||||
fn build_monolith(
|
||||
prof: &GameProfile,
|
||||
source_build: &str,
|
||||
|
|
@ -1470,8 +1747,10 @@ fn build_monolith(
|
|||
t3: &BTreeMap<String, model::Entry>,
|
||||
prov: &BTreeMap<String, model::Provenance>,
|
||||
abi: &BTreeMap<String, model::AbiShape>,
|
||||
anchors: &BTreeMap<String, Vec<String>>,
|
||||
server_img: Option<(&CodeImage, &str)>,
|
||||
) -> Result<(model::Monolith, String)> {
|
||||
let mono = assemble_monolith(
|
||||
let mut mono = assemble_monolith(
|
||||
prof,
|
||||
source_build,
|
||||
version,
|
||||
|
|
@ -1483,6 +1762,22 @@ fn build_monolith(
|
|||
prov,
|
||||
abi,
|
||||
)?;
|
||||
// Attached AFTER assembly, across every tier at once, rather than at the three MonoEntry construction
|
||||
// sites: an anchor belongs to a NAME, not to a tier, and one pass cannot leave a tier out by omission.
|
||||
let attached = attach_anchors(&mut mono, anchors);
|
||||
// Then DERIVE one for everything the catalogue does not cover. Reported separately from the catalogued
|
||||
// count: they are different claims — one is a curated string somebody chose, the other is this build's
|
||||
// own machine code answering the same question — and collapsing them would hide either going to zero.
|
||||
let (derived, considered) = match server_img {
|
||||
Some((img, lib)) => attach_derived_anchors(&mut mono, img, lib),
|
||||
None => (0, 0),
|
||||
};
|
||||
eprintln!(
|
||||
" string anchors: {attached} of {} catalogued reached the monolith; {derived} DERIVED for {considered} \
|
||||
server entries that had none ({} total anchored)",
|
||||
anchors.len(),
|
||||
attached + derived
|
||||
);
|
||||
let cssharp = render::render_monolith_cssharp(&mono, model::TierSelect::HighConfidence);
|
||||
eprintln!(
|
||||
" monolith: {} core + {} high-conf + {} experimental + {} unresolved",
|
||||
|
|
@ -1513,6 +1808,8 @@ pub(crate) struct FoldArgs<'a> {
|
|||
pub unverified: &'a BTreeSet<String>,
|
||||
/// The derive's measured argument footprints, folded onto the monolith entries.
|
||||
pub abi: &'a BTreeMap<String, model::AbiShape>,
|
||||
/// The derive's catalogue string anchors, folded onto the monolith entries the same way.
|
||||
pub anchors: &'a BTreeMap<String, Vec<String>>,
|
||||
pub sig_cap: usize,
|
||||
pub version: &'a str,
|
||||
pub full_names: Option<&'a Path>,
|
||||
|
|
@ -1578,6 +1875,7 @@ pub(crate) fn build_gamedata_cmd(prof: &GameProfile, a: FoldArgs) -> Result<Fold
|
|||
flagged,
|
||||
unverified,
|
||||
abi,
|
||||
anchors,
|
||||
sig_cap,
|
||||
version,
|
||||
full_names,
|
||||
|
|
@ -1882,6 +2180,8 @@ pub(crate) fn build_gamedata_cmd(prof: &GameProfile, a: FoldArgs) -> Result<Fold
|
|||
&t3,
|
||||
&prov,
|
||||
&abi_all,
|
||||
anchors,
|
||||
Some((&img, &default_lib)),
|
||||
)?;
|
||||
Ok(Folded {
|
||||
mono,
|
||||
|
|
@ -2144,7 +2444,6 @@ fn assemble_monolith(
|
|||
name.clone(),
|
||||
MonoEntry {
|
||||
locator: render::entry_from_value(v),
|
||||
class: None,
|
||||
abi: abi.get(name.as_str()).cloned(),
|
||||
provenance: Provenance {
|
||||
source: Some(source.into()),
|
||||
|
|
@ -2167,7 +2466,6 @@ fn assemble_monolith(
|
|||
name.clone(),
|
||||
MonoEntry {
|
||||
locator,
|
||||
class: None,
|
||||
abi: abi.get(name.as_str()).cloned(),
|
||||
provenance: provenance.clone(),
|
||||
validated: None,
|
||||
|
|
@ -2202,8 +2500,13 @@ fn assemble_monolith(
|
|||
..Provenance::with_tier(g.tier)
|
||||
};
|
||||
exp_tier.entry(g.name.clone()).or_insert(MonoEntry {
|
||||
locator: locator.clone(),
|
||||
class: g.class.clone(),
|
||||
// The class rides on the LOCATOR now, not beside it: for an offset entry the class is what
|
||||
// makes the slot index mean anything, and keeping them together is what lets it reach the
|
||||
// emitters through `Monolith::select`.
|
||||
locator: model::Entry {
|
||||
class: g.class.clone(),
|
||||
..locator.clone()
|
||||
},
|
||||
// The band's own measurement first: it was taken at the guess's exact address, which IS
|
||||
// what the locator resolves to. The name-keyed map is the fallback.
|
||||
abi: g.abi.clone().or_else(|| abi.get(g.name.as_str()).cloned()),
|
||||
|
|
@ -3486,6 +3789,13 @@ pub(crate) struct Derived {
|
|||
/// Per-entry argument footprint measured in the target binary — the machine half of the
|
||||
/// locator/prototype split, and what a declared prototype is checked against.
|
||||
pub abi: BTreeMap<String, model::AbiShape>,
|
||||
/// Per-entry string ANCHORS from the catalogue — distinctive literals the function references.
|
||||
///
|
||||
/// Its own side table for the same reason `abi` is: the derive→fold transport for `core` is the cssharp
|
||||
/// locator shape, which has no anchor field, so anything ridden in on an `Entry` there would be dropped
|
||||
/// when the fold re-parses it. Carrying them separately keeps the cssharp artifact unchanged — CS# has
|
||||
/// no `refs` feature and should not grow a key it cannot read.
|
||||
pub anchors: BTreeMap<String, Vec<String>>,
|
||||
}
|
||||
|
||||
/// Derive a target build's gamedata OFFLINE and return it in memory.
|
||||
|
|
@ -3710,11 +4020,22 @@ pub(crate) fn gamedata(
|
|||
.collect();
|
||||
flagged.extend(off_flag);
|
||||
flagged.extend(abi_drift);
|
||||
// Every catalogue entry's anchors, whether or not a byte sig located it. `recover_by_string_anchor`
|
||||
// uses them only as a FALLBACK locator; this ships them as a supplement, which is a different job — an
|
||||
// entry that resolved perfectly still benefits from a second locator with a different failure mode.
|
||||
let anchors: BTreeMap<String, Vec<String>> = cat
|
||||
.iter()
|
||||
.filter_map(|f| {
|
||||
let a: Vec<String> = string_anchors(f).into_iter().map(str::to_string).collect();
|
||||
(!a.is_empty()).then(|| (f.name.clone(), a))
|
||||
})
|
||||
.collect();
|
||||
Ok(Derived {
|
||||
core,
|
||||
flagged,
|
||||
unverified: unverified.into_iter().collect(),
|
||||
abi: abi_shapes,
|
||||
anchors,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -3870,6 +4191,10 @@ fn derive_offsets(
|
|||
match chain_and_vote(&anchors, hv, target_idx) {
|
||||
Some((pred, conf)) if conf >= 80 => {
|
||||
gd.set_offset(f.name.clone(), pred as i64);
|
||||
// The class the slot was chained THROUGH — the only one that makes the index meaningful.
|
||||
// Recorded here rather than reconstructed later from the name, which would be a different
|
||||
// (and sometimes wrong) fact: a base-declared method sits in a derived class's vtable.
|
||||
gd.set_class(f.name.clone(), f.class.clone());
|
||||
off_ok += 1;
|
||||
}
|
||||
Some((pred, conf)) => {
|
||||
|
|
@ -4506,7 +4831,6 @@ mod tests {
|
|||
"CBaseEntity::TakeDamage".to_string(),
|
||||
MonoEntry {
|
||||
locator: Entry::signature("server", "48 8B 05 ? ? ? ?"),
|
||||
class: None,
|
||||
abi: None,
|
||||
provenance: Provenance {
|
||||
source: Some("catalogue".into()),
|
||||
|
|
@ -4519,7 +4843,6 @@ mod tests {
|
|||
"CCSPlayerPawn::IsBot".to_string(),
|
||||
MonoEntry {
|
||||
locator: Entry::offset(42),
|
||||
class: None,
|
||||
abi: None,
|
||||
provenance: Provenance::with_tier(Tier::SelfNamed),
|
||||
validated: None,
|
||||
|
|
@ -4547,7 +4870,6 @@ mod tests {
|
|||
use model::{Counts, Entry, MonoEntry, MonoMeta, Monolith, Provenance, Tier};
|
||||
let entry = |loc: Entry, tier| MonoEntry {
|
||||
locator: loc,
|
||||
class: None,
|
||||
abi: None,
|
||||
provenance: Provenance::with_tier(tier),
|
||||
validated: None,
|
||||
|
|
|
|||
148
src/produce.rs
148
src/produce.rs
|
|
@ -183,6 +183,7 @@ pub fn produce_cmd(a: ProduceArgs) -> Result<()> {
|
|||
flagged: &derived.flagged,
|
||||
unverified: &derived.unverified,
|
||||
abi: &derived.abi,
|
||||
anchors: &derived.anchors,
|
||||
sig_cap,
|
||||
version,
|
||||
full_names,
|
||||
|
|
@ -279,6 +280,11 @@ pub fn produce_cmd(a: ProduceArgs) -> Result<()> {
|
|||
bindings.meta.pulse_typed,
|
||||
prof.min_pulse_typed,
|
||||
),
|
||||
(
|
||||
"host-callable Pulse shims",
|
||||
bindings.meta.pulse_callable,
|
||||
prof.min_pulse_callable,
|
||||
),
|
||||
(
|
||||
"entity-IO records",
|
||||
bindings.meta.entity_inputs + bindings.meta.entity_outputs,
|
||||
|
|
@ -294,6 +300,7 @@ pub fn produce_cmd(a: ProduceArgs) -> Result<()> {
|
|||
bindings.meta.commands,
|
||||
prof.min_commands,
|
||||
),
|
||||
("ConVars", bindings.meta.convars, prof.min_convars),
|
||||
] {
|
||||
ensure!(
|
||||
got >= floor,
|
||||
|
|
@ -1192,6 +1199,40 @@ pub(crate) fn run_live_oracle(
|
|||
}
|
||||
}
|
||||
|
||||
// The Pulse shim contract, checked by calling. Independent of `--gamedata`: it verifies a claim
|
||||
// `bindings-<game>.json` makes, not a gamedata locator, so it runs on every live oracle.
|
||||
println!(
|
||||
"\n=== Pulse invocation shims: calling every `args-only` binding with a sentinel handle ==="
|
||||
);
|
||||
let (shims_probed, shims_ok, shim_notes) = verify_pulse_shims(&live, pid, base, &img);
|
||||
println!(" {shims_ok} / {shims_probed} returned cleanly");
|
||||
for n in shim_notes.iter().take(8) {
|
||||
println!(" {n}");
|
||||
}
|
||||
// Probing NOTHING must not read as a pass. `OracleCounts::pass_rate` returns 1.0 for zero checks —
|
||||
// correct in general, since a stage with nothing to do is not a failure — but here zero means the
|
||||
// eligibility filter stopped matching, which is precisely the silent collapse the emitted
|
||||
// `call.needs` field would then be making claims about. The profile floor guarantees the callable
|
||||
// rows exist, so an empty probe set is a contradiction worth shouting about.
|
||||
if shims_probed == 0 {
|
||||
println!(
|
||||
" WARNING: no shim was eligible to probe. The floor guarantees host-callable rows exist, so \
|
||||
this means the probe's own filter no longer matches them — the emitted `call.needs` is \
|
||||
UNVERIFIED for this build."
|
||||
);
|
||||
}
|
||||
// A fault means the emitted argument contract is wrong, which is a claim the artifact should not be
|
||||
// making. Anything else (a clean non-`-2` return) is a different status protocol, not a broken contract,
|
||||
// so it counts as OK.
|
||||
verdicts.push((
|
||||
"pulse-shims",
|
||||
OracleCounts {
|
||||
checked: shims_probed as u32,
|
||||
ok: shims_ok as u32,
|
||||
faulted: shim_notes.iter().filter(|n| n.contains("FAULTED")).count() as u32,
|
||||
},
|
||||
));
|
||||
|
||||
let live_result = if gamedata.is_some() {
|
||||
println!("\n=== validate-live: derived gamedata vs the running server ===");
|
||||
// Parsed once, above — it feeds the CALL test's slot, sig/offset validation, and the pawn
|
||||
|
|
@ -1259,6 +1300,113 @@ pub(crate) fn run_live_oracle(
|
|||
Ok(live_result)
|
||||
}
|
||||
|
||||
/// The sentinel entity handle the Pulse resolve preamble rejects before dereferencing anything.
|
||||
const PULSE_INVALID_HANDLE: u32 = 0xffff_ffff;
|
||||
|
||||
/// `PVAL_EHANDLE`, from the shipped `PulseValueType_t`.
|
||||
const PULSE_EHANDLE: i32 = 13;
|
||||
|
||||
/// Verify the emitted Pulse invocation shims by CALLING them — with a handle the engine must reject.
|
||||
///
|
||||
/// `bindings-<game>.json` states that a shim whose `call.needs` is `args-only` can be invoked by a host.
|
||||
/// That is a claim about behaviour, so it is checked against behaviour rather than left as a derivation:
|
||||
/// each eligible binding is called with a sentinel handle, and its resolve must return `-2` without
|
||||
/// dereferencing anything. Confirming the ARGUMENT CONTRACT (the array at `r8+8+8k`, nulls in the slots the
|
||||
/// measurement says are unread) is the point; the sentinel is what makes it free of side effects.
|
||||
///
|
||||
/// **Why this is safe to run in CI.** Every slot but the argument array is null, so a shim that misuses one
|
||||
/// dereferences null and FAULTS — and a fault is caught, the signal suppressed and the thread restored. The
|
||||
/// dangerous case is a valid-but-wrong pointer, which corrupts silently (see [`crate::taxonomy`]); this
|
||||
/// passes none. The argument array points into the call's own dead stack scratch.
|
||||
///
|
||||
/// Eligibility is narrow on purpose: a shim, `args-only`, no declared return (so the output sink is never
|
||||
/// needed), and a leading `PVAL_EHANDLE` (so the sentinel is rejected). Anything else is not probed.
|
||||
///
|
||||
/// Derived from the image with the same readers the fold uses, rather than read back from
|
||||
/// `bindings-<game>.json`: the oracle runs for `integration-test` too, which never builds that artifact, and
|
||||
/// threading it through both callers to re-parse hex strings would verify the same claim by a longer route.
|
||||
fn verify_pulse_shims(
|
||||
live: &live::LiveProcess,
|
||||
pid: u32,
|
||||
base: u64,
|
||||
img: &CodeImage,
|
||||
) -> (usize, usize, Vec<String>) {
|
||||
let regs = crate::valvetab::pulse_bindings(img);
|
||||
let pairs: Vec<(u64, u64)> = regs
|
||||
.iter()
|
||||
.map(|b| (b.descriptor, b.arg_descriptor))
|
||||
.collect();
|
||||
let sigs = crate::pulse::read_all(img, &pairs, 8).0;
|
||||
|
||||
let mut probed = 0usize;
|
||||
let mut bailed = 0usize;
|
||||
let mut bad: Vec<String> = Vec::new();
|
||||
for (b, sig) in regs.iter().zip(sigs.iter()) {
|
||||
let (Some(sig), true) = (sig.as_ref(), b.shim != 0) else {
|
||||
continue;
|
||||
};
|
||||
let name = &b.name;
|
||||
let callable =
|
||||
crate::pulse::shim_reads(img, b.shim).is_some_and(|r| r.needs() == "args-only");
|
||||
if !callable
|
||||
|| !sig.returns.is_empty()
|
||||
|| sig.args.first().map(|p| p.ty) != Some(PULSE_EHANDLE)
|
||||
|| sig.args.len() > 2
|
||||
{
|
||||
continue;
|
||||
}
|
||||
let at = base + b.shim;
|
||||
if !live.is_exec(at) {
|
||||
continue;
|
||||
}
|
||||
// [0x00] padding — the array is addressed from +8 and nothing reads +0
|
||||
// [0x08] pointer to argument 0 -> relocated to 0x20
|
||||
// [0x10] pointer to argument 1 -> relocated to 0x24
|
||||
// [0x20] the sentinel handle, [0x24] a zero second argument
|
||||
let mut blob = [0u8; 0x28];
|
||||
blob[0x20..0x24].copy_from_slice(&PULSE_INVALID_HANDLE.to_le_bytes());
|
||||
let relocs: &[(usize, i64)] = if sig.args.len() >= 2 {
|
||||
&[(0x08, 0x20), (0x10, 0x24)]
|
||||
} else {
|
||||
&[(0x08, 0x20)]
|
||||
};
|
||||
let args = [
|
||||
live::Arg::Val(0),
|
||||
live::Arg::Val(0),
|
||||
live::Arg::Val(0),
|
||||
live::Arg::Val(0),
|
||||
live::Arg::Scratch(0),
|
||||
live::Arg::Val(0),
|
||||
];
|
||||
probed += 1;
|
||||
match live::call_remote_ex(
|
||||
pid as i32,
|
||||
at,
|
||||
&args,
|
||||
&[],
|
||||
Some(live::Scratch {
|
||||
bytes: &blob,
|
||||
relocs,
|
||||
}),
|
||||
) {
|
||||
Ok(r) if r.clean_return && r.rax as i32 == -2 => bailed += 1,
|
||||
Ok(r) if r.clean_return => {
|
||||
// A clean return that is not the bail path still proves the contract; only note it.
|
||||
bailed += 1;
|
||||
if bad.len() < 8 {
|
||||
bad.push(format!(
|
||||
"{name} returned {} (not -2), cleanly",
|
||||
r.rax as i32
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(_) => bad.push(format!("{name} FAULTED — the argument contract is wrong")),
|
||||
Err(e) => bad.push(format!("{name} could not be called: {e}")),
|
||||
}
|
||||
}
|
||||
(probed, bailed, bad)
|
||||
}
|
||||
|
||||
/// A launched, ready CS2 bots server the caller owns (must kill).
|
||||
pub(crate) struct OwnedServer {
|
||||
pub(crate) child: std::process::Child,
|
||||
|
|
|
|||
|
|
@ -98,6 +98,11 @@ pub struct GameProfile {
|
|||
/// layout moves, and the result would be a release that ships every binding with no signature at all
|
||||
/// — a silent capability loss rather than a wrong answer, which is precisely what a floor is for.
|
||||
pub min_pulse_typed: usize,
|
||||
/// Floor on Pulse bindings whose invocation shim is HOST-CALLABLE (`call.needs == "args-only"`).
|
||||
/// Its own floor because it has its own failure mode: the registry can read perfectly and the
|
||||
/// signatures recover perfectly while a codegen change makes every shim appear to read another slot,
|
||||
/// which would silently retire the one callable tier instead of failing the release.
|
||||
pub min_pulse_callable: usize,
|
||||
pub min_entity_io: usize,
|
||||
pub min_entity_classes: usize,
|
||||
/// Console commands recovered from their registration calls. Its own floor because it has its own
|
||||
|
|
@ -105,6 +110,10 @@ pub struct GameProfile {
|
|||
/// invalid-handle sentinel), so a build that reworks that constructor yields zero commands rather
|
||||
/// than wrong ones — correct, and invisible without this.
|
||||
pub min_commands: usize,
|
||||
/// Floor on recovered ConVars. Its own floor because convar registration is identified by a DIFFERENT
|
||||
/// test from the command one — convergence of registrar wrappers on a shared core, not a sentinel in the
|
||||
/// callee — so it can fail while commands keep working.
|
||||
pub min_convars: usize,
|
||||
pub min_schema_enums: usize,
|
||||
/// Output game-key the game-keyed emitters use (Metamod `Games { <key> {..} }`, Plugify `{ "<key>": {..} }`).
|
||||
pub game_key: &'static str,
|
||||
|
|
@ -185,9 +194,11 @@ pub const CS2: GameProfile = GameProfile {
|
|||
// observed: 580 Pulse, 715 inputs + 226 outputs, 474 entity classnames, 784 commands, 555 enums
|
||||
min_pulse_bindings: 300,
|
||||
min_pulse_typed: 300,
|
||||
min_pulse_callable: 90,
|
||||
min_entity_io: 400,
|
||||
min_entity_classes: 200,
|
||||
min_commands: 400,
|
||||
min_convars: 900,
|
||||
min_schema_enums: 250,
|
||||
game_key: "csgo",
|
||||
token: "cs2",
|
||||
|
|
@ -288,9 +299,11 @@ pub const DOTA: GameProfile = GameProfile {
|
|||
// observed: 500 Pulse, 624 inputs, 3,528 entity classnames, 855 commands, 743 enums
|
||||
min_pulse_bindings: 250,
|
||||
min_pulse_typed: 250,
|
||||
min_pulse_callable: 65,
|
||||
min_entity_io: 300,
|
||||
min_entity_classes: 1000,
|
||||
min_commands: 400,
|
||||
min_convars: 600,
|
||||
min_schema_enums: 350,
|
||||
game_key: "dota",
|
||||
token: "dota2",
|
||||
|
|
|
|||
232
src/pulse.rs
232
src/pulse.rs
|
|
@ -550,6 +550,203 @@ fn type_at(img: &CodeImage, t: &Trace, obj: u64) -> Option<(i32, Option<String>)
|
|||
found
|
||||
}
|
||||
|
||||
/// How far past a shim's entry the read-measurement will follow. `.eh_frame_hdr` covers only a fraction of
|
||||
/// these images' functions and none of the shims, so there is no exact extent available; flow-following ends
|
||||
/// at every `ret` regardless, so this only bounds a runaway path.
|
||||
const SHIM_SPAN: u64 = 0x1000;
|
||||
|
||||
/// The seven integer arguments a Pulse invocation shim takes, in SysV order. The seventh is the first
|
||||
/// STACK slot — measured, and the reason the shim's arity cannot be read off `abi_shape`, whose backward
|
||||
/// liveness stops at the registers.
|
||||
const SHIM_SLOTS: [Register; 6] = [
|
||||
Register::RDI,
|
||||
Register::RSI,
|
||||
Register::RDX,
|
||||
Register::RCX,
|
||||
Register::R8,
|
||||
Register::R9,
|
||||
];
|
||||
|
||||
/// What an invocation shim was measured to read, and therefore what a caller has to supply.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Eq)]
|
||||
pub struct ShimReads {
|
||||
/// The argument slots actually read, named — `rcx`, `r8`, `stack0`.
|
||||
pub reads: Vec<&'static str>,
|
||||
/// Does it read the argument array (`r8`)?
|
||||
pub args: bool,
|
||||
/// Does it read the output sink (the first stack slot)? True for exactly the bindings that declare a
|
||||
/// return, measured across both games with no exceptions.
|
||||
pub sink: bool,
|
||||
/// Does it read the Pulse host-service context (`rcx`)? That object is VM-owned, so a host cannot
|
||||
/// supply one.
|
||||
pub context: bool,
|
||||
/// Does it read any OTHER slot — `rdi`, `rsi`, `rdx`, `r9`? These are the slots a caller would
|
||||
/// otherwise pass as null, so any read here means it cannot.
|
||||
pub other: bool,
|
||||
}
|
||||
|
||||
impl ShimReads {
|
||||
/// What a host must supply, as the artifact states it.
|
||||
///
|
||||
/// `args-only` is the one that matters: everything such a shim reads is either the argument array a
|
||||
/// caller builds or the game's own entity list, so the remaining slots may be null. That is not a
|
||||
/// deduction — it was validated by calling every eligible binding in both games with a sentinel handle
|
||||
/// (CS2 186 of 193 clean, Dota 211 of 211), and the exceptions are exactly the shims this reports as
|
||||
/// reading another slot.
|
||||
pub fn needs(&self) -> &'static str {
|
||||
if self.context {
|
||||
"pulse-context"
|
||||
} else if self.other {
|
||||
"other-slots"
|
||||
} else if self.sink {
|
||||
"output-sink"
|
||||
} else {
|
||||
"args-only"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Measure which of a shim's seven arguments it reads.
|
||||
///
|
||||
/// Reachable instructions in ADDRESS order, which needs two guards that cost real time to find:
|
||||
///
|
||||
/// * `push`/`pop` must NOT update the alias map. The compiler lays the epilogue out BEFORE the
|
||||
/// found-path block, so `pop r13` sits at a lower address than the `mov rax,[r13+0x10]` that reads the
|
||||
/// second argument through a stashed `mov r13, r8` — and letting the pop clear the alias loses the read.
|
||||
/// The same shape cost the ConCommand reader an epoch counter.
|
||||
/// * `xor r, r` / `sub r, r` name the register in BOTH operands and read neither. Counted, they mark an
|
||||
/// argument live that the shim never consumes; `xor edi, edi` alone accounted for 143 false positives.
|
||||
pub fn shim_reads(img: &CodeImage, entry: u64) -> Option<ShimReads> {
|
||||
let all = img.code_at(entry)?;
|
||||
let extent = (all.len() as u64).min(SHIM_SPAN);
|
||||
let code = &all[..extent as usize];
|
||||
// Saturating: `extent` derives from the section length, so on a crafted image `entry + extent` can
|
||||
// wrap and turn the span test inside out — and the fuzz harness builds with overflow checks, where a
|
||||
// plain add aborts. The same shape `fuzz_concmd` was written for.
|
||||
let end = entry.saturating_add(extent);
|
||||
let in_span = |t: u64| t >= entry && t < end;
|
||||
|
||||
let mut seen: HashMap<u64, ()> = HashMap::new();
|
||||
let mut work = vec![entry];
|
||||
let mut insn = Instruction::default();
|
||||
while let Some(at) = work.pop() {
|
||||
if seen.contains_key(&at) || !in_span(at) || seen.len() > 20000 {
|
||||
continue;
|
||||
}
|
||||
let mut dec =
|
||||
Decoder::with_ip(64, &code[(at - entry) as usize..], at, DecoderOptions::NONE);
|
||||
if !dec.can_decode() {
|
||||
continue;
|
||||
}
|
||||
dec.decode_out(&mut insn);
|
||||
if insn.is_invalid() || insn.len() == 0 {
|
||||
continue;
|
||||
}
|
||||
seen.insert(at, ());
|
||||
match insn.flow_control() {
|
||||
FlowControl::Return
|
||||
| FlowControl::IndirectBranch
|
||||
| FlowControl::Exception
|
||||
| FlowControl::Interrupt => {}
|
||||
FlowControl::UnconditionalBranch => work.push(insn.near_branch_target()),
|
||||
FlowControl::ConditionalBranch => {
|
||||
work.push(at + insn.len() as u64);
|
||||
work.push(insn.near_branch_target());
|
||||
}
|
||||
_ => work.push(at + insn.len() as u64),
|
||||
}
|
||||
}
|
||||
let mut addrs: Vec<u64> = seen.keys().copied().collect();
|
||||
addrs.sort_unstable();
|
||||
|
||||
let mut live: BTreeMap<Register, bool> = BTreeMap::new();
|
||||
let mut sink = false;
|
||||
let mut fresh: Vec<Register> = SHIM_SLOTS.to_vec();
|
||||
|
||||
for at in addrs {
|
||||
let mut dec =
|
||||
Decoder::with_ip(64, &code[(at - entry) as usize..], at, DecoderOptions::NONE);
|
||||
dec.decode_out(&mut insn);
|
||||
|
||||
// The first stack slot is the output sink. Only `[rbp+0x10]` is ever read — no shim in either
|
||||
// game touches a second — which is what pins the arity at seven.
|
||||
//
|
||||
// The displacement MUST be read as signed. `memory_displacement64` is unsigned, so a local at
|
||||
// `[rbp-0x10]` comes back as `0xffff_ffff_ffff_fff0`, which passes an unsigned `>= 0x10` — and
|
||||
// every shim with a stack local then looks as though it reads the output sink. That mistake
|
||||
// reported 246 sink-readers against a true 201 and hid two bindings whose callability had already
|
||||
// been demonstrated by a live call.
|
||||
if (insn.op0_kind() == OpKind::Memory || insn.op1_kind() == OpKind::Memory)
|
||||
&& insn.memory_base() == Register::RBP
|
||||
&& insn.memory_index() == Register::None
|
||||
&& insn.memory_displacement64() as i64 >= 0x10
|
||||
{
|
||||
sink = true;
|
||||
}
|
||||
let zeroing = matches!(insn.mnemonic(), Mnemonic::Xor | Mnemonic::Sub)
|
||||
&& insn.op0_kind() == OpKind::Register
|
||||
&& insn.op1_kind() == OpKind::Register
|
||||
&& insn.op0_register().full_register() == insn.op1_register().full_register();
|
||||
// A register named inside a MEMORY operand is read even though it is not a register operand.
|
||||
if !zeroing {
|
||||
for r in [insn.memory_base(), insn.memory_index()] {
|
||||
if r != Register::None && r != Register::RIP && fresh.contains(&r.full_register()) {
|
||||
live.insert(r.full_register(), true);
|
||||
}
|
||||
}
|
||||
for i in 0..insn.op_count() {
|
||||
if insn.op_kind(i) != OpKind::Register {
|
||||
continue;
|
||||
}
|
||||
let pure_dst = i == 0
|
||||
&& matches!(
|
||||
insn.mnemonic(),
|
||||
Mnemonic::Mov | Mnemonic::Lea | Mnemonic::Movzx | Mnemonic::Movsxd
|
||||
);
|
||||
let r = insn.op_register(i).full_register();
|
||||
if !pure_dst && fresh.contains(&r) {
|
||||
live.insert(r, true);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if insn.op_count() > 0
|
||||
&& insn.op0_kind() == OpKind::Register
|
||||
&& !matches!(insn.mnemonic(), Mnemonic::Push | Mnemonic::Pop)
|
||||
{
|
||||
let d = insn.op0_register().full_register();
|
||||
fresh.retain(|&r| r != d);
|
||||
}
|
||||
if insn.flow_control() == FlowControl::Call {
|
||||
for r in CALLER_SAVED {
|
||||
fresh.retain(|&x| x != r);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut out = ShimReads {
|
||||
sink,
|
||||
..Default::default()
|
||||
};
|
||||
for (r, name) in SHIM_SLOTS
|
||||
.iter()
|
||||
.zip(["rdi", "rsi", "rdx", "rcx", "r8", "r9"])
|
||||
{
|
||||
if live.contains_key(r) {
|
||||
out.reads.push(name);
|
||||
match *r {
|
||||
Register::RCX => out.context = true,
|
||||
Register::R8 => out.args = true,
|
||||
_ => out.other = true,
|
||||
}
|
||||
}
|
||||
}
|
||||
if sink {
|
||||
out.reads.push("stack0");
|
||||
}
|
||||
Some(out)
|
||||
}
|
||||
|
||||
/// A `PulseValueType_t` value, `PVAL_VOID` (-1) included.
|
||||
fn valid_pval(v: u64) -> bool {
|
||||
let s = v as i64;
|
||||
|
|
@ -606,6 +803,41 @@ mod tests {
|
|||
assert!(candidate_strides(&r).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn needs_reports_the_most_restrictive_requirement_a_shim_has() {
|
||||
// Precedence matters: a shim reading both the context and the sink is not "output-sink", because
|
||||
// the context is the one a host cannot supply at all. Ordering it the other way would advertise
|
||||
// a binding as merely needing a sink when it actually needs a live cursor.
|
||||
let ctx = ShimReads {
|
||||
context: true,
|
||||
sink: true,
|
||||
args: true,
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(ctx.needs(), "pulse-context");
|
||||
let other = ShimReads {
|
||||
other: true,
|
||||
sink: true,
|
||||
args: true,
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(other.needs(), "other-slots");
|
||||
let sink = ShimReads {
|
||||
sink: true,
|
||||
args: true,
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(sink.needs(), "output-sink");
|
||||
// The callable tier: the argument array and nothing else.
|
||||
let only = ShimReads {
|
||||
args: true,
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(only.needs(), "args-only");
|
||||
// A shim reading NOTHING is still args-only — a zero-argument binding reads no array either.
|
||||
assert_eq!(ShimReads::default().needs(), "args-only");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pval_void_is_negative_one_and_still_a_type() {
|
||||
assert!(valid_pval(0)); // PVAL_BOOL
|
||||
|
|
|
|||
|
|
@ -46,6 +46,14 @@ pub struct PulseBinding {
|
|||
pub descriptor: u64,
|
||||
/// A second accessor of the same shape, for the binding's argument descriptor.
|
||||
pub arg_descriptor: u64,
|
||||
/// The binding's own INVOCATION shim — the one code pointer in the record that is an entry point
|
||||
/// rather than a descriptor accessor. Zero when the slot holds no executable code (8 of 485 on CS2).
|
||||
///
|
||||
/// Measured as a fixed-signature marshalling stub: seven integer arguments returning int, where the
|
||||
/// fifth is an array of pointers to the argument values (element *k* at `+8+8k`) and the seventh is an
|
||||
/// output sink read by exactly the bindings that declare a return. Not emitted into any artifact — the
|
||||
/// contract has not been validated by an actual call, and a locator nobody has exercised is a claim.
|
||||
pub shim: u64,
|
||||
pub flags: PulseFlags,
|
||||
}
|
||||
|
||||
|
|
@ -302,6 +310,10 @@ pub fn pulse_bindings(img: &CodeImage) -> Vec<PulseBinding> {
|
|||
description: img.read_ptr(at + 16).and_then(|p| table_string(img, p)),
|
||||
descriptor,
|
||||
arg_descriptor,
|
||||
shim: img
|
||||
.read_ptr(at + 72)
|
||||
.filter(|&p| img.is_code(p))
|
||||
.unwrap_or(0),
|
||||
flags: PulseFlags::decode(
|
||||
img.read_u32(at + 56).unwrap_or(0),
|
||||
img.read_u32(at + 60).unwrap_or(0),
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue