From 5266151cc5665f7e82048635e566efbdb828001e Mon Sep 17 00:00:00 2001 From: kamal Date: Thu, 2 Apr 2026 20:50:00 +0000 Subject: [PATCH] Add Audit and Testing --- Audit-and-Testing.md | 139 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 139 insertions(+) create mode 100644 Audit-and-Testing.md diff --git a/Audit-and-Testing.md b/Audit-and-Testing.md new file mode 100644 index 0000000..6c97ab9 --- /dev/null +++ b/Audit-and-Testing.md @@ -0,0 +1,139 @@ +# Audit & Testing + +Trust signals for evaluating libsoliton's reliability and security posture. + +## Test Suite + +| Location | Count | Description | +|----------|-------|-------------| +| Core unit tests | 488 | `soliton/src/**/` `#[cfg(test)]` — unit tests + proptests | +| Core integration tests | 61 | KEX → ratchet flows, storage round-trips, Argon2id, zeroization verification, Appendix F test vectors | +| CAPI tests | 287 | Null-pointer guards, invalid lengths, full round-trips, error code verification | +| Python tests | 49 | Full KEX → ratchet round-trip, serialize, reset, call keys, storage, streaming, auth, identity, primitives | +| Zig tests | 35 | Full KEX → ratchet round-trip, serialize, reset, call keys, storage, streaming, auth, AEAD, X-Wing, Argon2id | +| WASM tests | 36 | Full KEX → ratchet lifecycle, serialize/deserialize, call keys, storage, streaming, auth, identity, primitives, Argon2id (Node + Chromium) | +| Fuzz targets | 36 | 29 core + 7 CAPI (`cargo-fuzz` / libFuzzer) | + +### Cross-Cutting Properties + +| Property | Count | Description | +|----------|-------|-------------| +| MIRI-eligible | 455 | PQ-free subset safe for instruction-level interpretation (`--profile miri`) | +| KAT vectors | 27 | Known-answer tests against [Specification](Specification) Appendix F | + +## Fuzzing + +36 fuzz targets (29 core + 7 CAPI) with two corpora (core: `soliton/fuzz/corpus/`, CAPI: `soliton_capi/fuzz/corpus/`) totaling 165,540 entries (883 MB). All corpora are checked in via Git LFS; `ci_regression.sh` replays every seed with zero mutations as a CI regression gate. + +### Excluded Targets + +5 of 36 targets are excluded from overnight campaigns after saturation analysis — all reachable edges discovered; additional CPU time yields no new coverage. These retain their corpora and run in `ci_regression.sh`. + +| Target | Corpus | Edges | Reason | +|--------|--------|-------|--------| +| `fuzz_identity_from_bytes` | 84 | 107 | 3 length checks only | +| `fuzz_auth_verify` | ~15 | 88 | Single `ct_eq` call | +| `fuzz_xwing_roundtrip` | 7 | 763 | No adversarial input (keygen→encap→decap) | +| `fuzz_verification_phrase` | 29 | 88 | SHA3→wordlist index, trivial surface | +| `fuzz_auth_respond` | 4 | 1573 | Single KEM decap→HMAC, no branching on input | + +### Latest Campaign + +31 targets (24 core + 7 CAPI), 1 libFuzzer worker per target, Hetzner CCX53 (32 vCPUs, AMD EPYC, 128 GB RAM), 24 hours. + +| Metric | Value | +|--------|-------| +| Total targets | 36 (29 core + 7 CAPI) | +| Active in campaign | 31 | +| Total executions | **574.6 billion** | +| Corpus | 165,540 entries (883 MB) | +| Crashes / Timeouts / OOM | **0 / 0 / 0** | + +The state-machine fuzzer (`fuzz_ratchet_state_machine`) performs full cryptographic sessions with up to 200 protocol actions per execution (KEM, AEAD, HKDF, HMAC per message) — 12,714 coverage features, the highest of any target. + +```bash +./ci_regression.sh # Replay all seeds — fast, runs in CI +./fuzz_overnight.sh 24 1 # 31 active targets, 24 hours, 1 worker each +./fuzz_stats.sh # Per-target breakdown (edges, features, throughput, RSS) +``` + +## Benchmarks + +Measured with `cargo +nightly bench` across three platforms. ML-DSA-65, SHA3-256, and Keccak lack SIMD implementations in current RustCrypto crates — expect improvement as upstream adds platform-specific optimizations. All operations are pure ARX or field arithmetic; no hardware acceleration is required. + +| Platform | CPU | Arch | Clock | +|----------|-----|------|-------| +| **Desktop** | AMD Ryzen 7 7840HS | x86-64 (Zen 4) | ~5.1 GHz boost | +| **RPi 5** | Cortex-A76 | aarch64 | 2.4 GHz | +| **VisionFive 2** | SiFive U74 | riscv64gc | 1.5 GHz | + +### Per-Message Hot Paths + +| Operation | Desktop | RPi 5 | VisionFive 2 | Notes | +|-----------|---------|-------|--------------|-------| +| Ratchet encrypt (same epoch) | 4.3 µs | 7.2 µs | 47.6 µs | HMAC-SHA3-256 + XChaCha20-Poly1305 | +| Ratchet decrypt (same epoch) | 6.2 µs | 10.1 µs | 67.5 µs | Includes `from_bytes` deserialization | +| KDF root (isolated) | 5.7 µs | 9.2 µs | 59.5 µs | HKDF-SHA3-256, 64-byte output | +| Ratchet serialize | 1.6 µs | 6.1 µs | 38.1 µs | `to_bytes` | +| Ratchet deserialize | 0.7 µs | 2.7 µs | 17.8 µs | `from_bytes_with_min_epoch` | + +### Direction-Change Paths + +| Operation | Desktop | RPi 5 | VisionFive 2 | Notes | +|-----------|---------|-------|--------------|-------| +| Ratchet encrypt (direction change) | 182 µs | 651 µs | 2.46 ms | X-Wing keygen + encapsulate + KDF | +| Ratchet decrypt (direction change) | 127 µs | 473 µs | 1.76 ms | X-Wing decapsulate + KDF | + +### Session Establishment + +| Operation | Desktop | RPi 5 | VisionFive 2 | Notes | +|-----------|---------|-------|--------------|-------| +| Identity keygen | 417 µs | 1.05 ms | 4.90 ms | X-Wing + Ed25519 + ML-DSA-65 | +| Hybrid sign | 988 µs | 2.85 ms | 10.6 ms | Ed25519 + ML-DSA-65 (no SIMD) | +| Hybrid verify | 254 µs | 794 µs | 3.31 ms | Ed25519 + ML-DSA-65 (no SIMD) | +| X-Wing encapsulate | 104 µs | 417 µs | 1.44 ms | X25519 + ML-KEM-768 | +| X-Wing decapsulate | 112 µs | 451 µs | 1.61 ms | X25519 + ML-KEM-768 | +| Initiate session (Alice) | 1.41 ms | 3.95 ms | 17.1 ms | Ephemeral keygen + 3 encaps + HKDF + sign | +| Receive session (Bob) | 585 µs | 1.88 ms | 7.70 ms | Verify + 3 decaps + HKDF | + +### Streaming AEAD + +| Operation | Desktop | RPi 5 | VisionFive 2 | Notes | +|-----------|---------|-------|--------------|-------| +| Encrypt 1 MiB | 537 µs (1.95 GB/s) | 3.96 ms (265 MB/s) | 31.0 ms (33.8 MB/s) | XChaCha20-Poly1305, single chunk | +| Decrypt 1 MiB | 749 µs (1.40 GB/s) | 4.74 ms (221 MB/s) | 34.9 ms (30.0 MB/s) | Includes init overhead | +| Encrypt 4 MiB sequential | 2.58 ms (1.55 GB/s) | 15.8 ms (253 MB/s) | 124 ms (32.3 MB/s) | 4 chunks, sequential API | +| Encrypt 4 MiB parallel | 867 µs (4.61 GB/s) | 5.35 ms (748 MB/s) | 33.0 ms (121 MB/s) | 4 chunks via `encrypt_chunk_at`, rayon (4 cores) | +| Encrypt 8 MiB parallel | 1.29 ms (6.21 GB/s) | 10.7 ms (748 MB/s) | 67.8 ms (118 MB/s) | 8 chunks, 2 rounds on 4 cores | + +### Password KDF + +| Operation | Desktop | RPi 5 | VisionFive 2 | Notes | +|-----------|---------|-------|--------------|-------| +| Argon2id OWASP minimum | 10.9 ms | 42.0 ms | 256 ms | 19 MiB, 2 passes, 1 lane | +| Argon2id recommended | 65.0 ms | 227 ms | 1.35 s | 64 MiB, 3 passes, 4 lanes | + +## Dependencies + +All dependencies are pure Rust — no C libraries, no cmake, no system linker dependencies. + +| Crate | Role | +|-------|------| +| [ml-kem](https://crates.io/crates/ml-kem) | ML-KEM-768 (inside X-Wing) | +| [ml-dsa](https://crates.io/crates/ml-dsa) | ML-DSA-65 hybrid signatures | +| [ed25519-dalek](https://crates.io/crates/ed25519-dalek) | Ed25519 signing/verification | +| [x25519-dalek](https://crates.io/crates/x25519-dalek) | X25519 (inside X-Wing) | +| [chacha20poly1305](https://crates.io/crates/chacha20poly1305) | XChaCha20-Poly1305 AEAD | +| [sha3](https://crates.io/crates/sha3) | SHA3-256 (HMAC, HKDF, fingerprints, combiner) | +| [hmac](https://crates.io/crates/hmac) | HMAC-SHA3-256 | +| [hkdf](https://crates.io/crates/hkdf) | HKDF-SHA3-256 key derivation | +| [argon2](https://crates.io/crates/argon2) | Argon2id password hashing | +| [ruzstd](https://crates.io/crates/ruzstd) | Zstd decompression | +| [zeroize](https://crates.io/crates/zeroize) | Secret wiping | + +### Binding-Specific Dependencies + +| Crate | Role | +|-------|------| +| [pyo3](https://crates.io/crates/pyo3) | Python FFI bridge (`soliton_py`) | +| [wasm-bindgen](https://crates.io/crates/wasm-bindgen) | WASM/JS interop (`soliton_wasm`) |