feat: Rust yrs core with ops, view and wazero host
The Slides core, following Docs' pattern: a yrs CRDT compiled to wasm32-unknown-unknown with zero host imports, run under wazero. core/ (slides-core): the deck schema over yrs (schema.rs), the §7.2 operations applied as one batch in one transaction with a dry-run model so a failure applies nothing (ops.rs), the JSON view (view.rs), the Markdown subset ops accept text in (markdown.rs), and the C ABI with Docs' bounds-checking and no-borrow-across-yrs rules (abi.rs). The core never draws entropy: every id arrives from the host. internal/core: the wazero host copied from Docs (trap containment, rebuild by log replay, Recovered/Rejected) with State, JSON and Ops; an Ops refusal is a *RejectedError carrying the core's message, and the update Ops produces is retained for rebuilds. Vendored fastrand patch, .cargo/config.toml, clippy.toml, the reproducible build script and toolchain pin are Docs' with the rename. The committed testdata/slides_core.wasm is byte-identical to a fresh scripts/build-wasm.sh output.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01563XGCgxcSZToEXwuTBsKE
31 files changed,
+6873
−0
.cargo/config.toml+19 −0clippy.toml+5 −0core/.gitignore+1 −0core/Cargo.lock+584 −0core/Cargo.toml+36 −0core/src/abi.rs+427 −0core/src/lib.rs+146 −0core/src/markdown.rs+344 −0core/src/ops.rs+1089 −0core/src/schema.rs+523 −0core/src/view.rs+168 −0core/tests/no_host_imports.rs+56 −0core/tests/ops.rs+387 −0core/vendor/README.md+120 −0core/vendor/fastrand-2.5.0/CHANGELOG.md+140 −0core/vendor/fastrand-2.5.0/Cargo.toml+96 −0core/vendor/fastrand-2.5.0/LICENSE-APACHE+201 −0core/vendor/fastrand-2.5.0/LICENSE-MIT+23 −0core/vendor/fastrand-2.5.0/README.md+110 −0core/vendor/fastrand-2.5.0/benches/bench.rs+141 −0core/vendor/fastrand-2.5.0/src/global_rng.rs+234 −0core/vendor/fastrand-2.5.0/src/lib.rs+736 −0core/vendor/fastrand-2.5.0/tests/char.rs+44 −0core/vendor/fastrand-2.5.0/tests/smoke.rs+357 −0go.mod+1 −0go.sum+2 −0internal/core/core.go+637 −0internal/core/core_test.go+199 −0internal/core/testdata/slides_core.wasmbinaryrust-toolchain.toml+4 −0scripts/build-wasm.sh+43 −0
Intent
Attributed to this commit: 209 g CO₂e (16579485 tokens). This is an attribution, not a measurement — usage counted from this commit's own marker forward to the next one on worktree-slice-1, never backward from it. A turn that produced more than one commit books all of its tokens to the last one, and a concurrent session on this branch can add to the same figure. Some of it was priced using the shared default rate, because no model-specific factor row exists yet for at least one model involved. The Carbon view has the repo total, which is never derived from this attribution.
diff --git a/.cargo/config.toml b/.cargo/config.toml| new file mode 100644 |
| index 0000000..59839f9 |
| --- /dev/null |
| +++ b/.cargo/config.toml |
| @@ -0,0 +1,19 @@ |
| +# yrs (pinned in core/Cargo.toml) unconditionally generates a random |
| +# document GUID internally (`Doc::with_client_id` still calls `uuid_v4()`, |
| +# via `fastrand`, on every Doc construction — the "with_client_id" name only |
| +# refers to the collaboration client id, not the guid). On wasm32-unknown- |
| +# unknown, fastrand's transitively-required "js" feature makes `getrandom` |
| +# default to a `wasm-bindgen`-based backend that imports host JS functions |
| +# (`crypto.getRandomValues` etc.) — even when the randomness-using code path |
| +# is never actually reachable at runtime, referencing `yrs::Doc` anywhere in |
| +# this crate is enough for those wasm-bindgen exports to end up in the final |
| +# module. The Go/wazero host provides no JS glue, so that import breaks the |
| +# zero-imports contract enforced by core/tests/no_host_imports.rs. |
| +# |
| +# Selecting getrandom's "custom" backend (see core/src/lib.rs for the |
| +# `__getrandom_v03_custom` symbol this requires) routes entropy requests to |
| +# our own no-JS-dependency implementation instead, for the wasm32 target |
| +# only — native builds (tests, tooling) are unaffected and keep getrandom's |
| +# normal OS-backed source. |
| +[target.wasm32-unknown-unknown] |
| +rustflags = ["--cfg", 'getrandom_backend="custom"'] |
diff --git a/clippy.toml b/clippy.toml| new file mode 100644 |
| index 0000000..a4922e4 |
| --- /dev/null |
| +++ b/clippy.toml |
| @@ -0,0 +1,5 @@ |
| +disallowed-methods = [ |
| + { path = "yrs::Doc::new", reason = "draws a random client id; use Options::with_guid_and_client_id" }, |
| + { path = "yrs::Doc::with_client_id", reason = "draws a GUID via uuid_v4; use Options::with_guid_and_client_id" }, |
| + { path = "yrs::uuid_v4", reason = "constant seed on wasm means identical values across processes" }, |
| +] |
diff --git a/core/.gitignore b/core/.gitignore| new file mode 100644 |
| index 0000000..ea8c4bf |
| --- /dev/null |
| +++ b/core/.gitignore |
| @@ -0,0 +1 @@ |
| +/target |
diff --git a/core/Cargo.lock b/core/Cargo.lock| new file mode 100644 |
| index 0000000..4538ac6 |
| --- /dev/null |
| +++ b/core/Cargo.lock |
| @@ -0,0 +1,584 @@ |
| +# This file is automatically @generated by Cargo. |
| +# It is not intended for manual editing. |
| +version = 4 |
| + |
| +[[package]] |
| +name = "ahash" |
| +version = "0.8.12" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" |
| +dependencies = [ |
| + "cfg-if", |
| + "once_cell", |
| + "version_check", |
| + "zerocopy", |
| +] |
| + |
| +[[package]] |
| +name = "arc-swap" |
| +version = "1.9.2" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "c049c0be4daef0b145cb3555416b3b8ef5b7888a38aea1a3a155801fe7b0810b" |
| +dependencies = [ |
| + "rustversion", |
| +] |
| + |
| +[[package]] |
| +name = "arrayvec" |
| +version = "0.7.8" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "d3fb67a6e08acf24fdeccbac2cb6ac4305825bd1f117462e0e6f2f193345ad56" |
| + |
| +[[package]] |
| +name = "async-lock" |
| +version = "3.4.2" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "290f7f2596bd5b78a9fec8088ccd89180d7f9f55b94b0576823bbbdc72ee8311" |
| +dependencies = [ |
| + "event-listener", |
| + "event-listener-strategy", |
| + "pin-project-lite", |
| +] |
| + |
| +[[package]] |
| +name = "async-trait" |
| +version = "0.1.92" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667" |
| +dependencies = [ |
| + "proc-macro2", |
| + "quote", |
| + "syn 3.0.4", |
| +] |
| + |
| +[[package]] |
| +name = "autocfg" |
| +version = "1.5.1" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" |
| + |
| +[[package]] |
| +name = "bitflags" |
| +version = "2.13.1" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" |
| + |
| +[[package]] |
| +name = "cfg-if" |
| +version = "1.0.4" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" |
| + |
| +[[package]] |
| +name = "crossbeam-utils" |
| +version = "0.8.22" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17" |
| + |
| +[[package]] |
| +name = "dashmap" |
| +version = "6.2.1" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "e6361d5c062261c78a176addb82d4c821ae42bed6089de0e12603cd25de2059c" |
| +dependencies = [ |
| + "cfg-if", |
| + "crossbeam-utils", |
| + "hashbrown 0.14.5", |
| + "lock_api", |
| + "once_cell", |
| + "parking_lot_core", |
| +] |
| + |
| +[[package]] |
| +name = "downcast-rs" |
| +version = "1.2.1" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "75b325c5dbd37f80359721ad39aca5a29fb04c89279657cffdda8736d0c0b9d2" |
| + |
| +[[package]] |
| +name = "equivalent" |
| +version = "1.0.2" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" |
| + |
| +[[package]] |
| +name = "event-listener" |
| +version = "5.4.2" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "5a23add41df1562121a9393cb065eab5146a1242410f23a644851e90cfd669d2" |
| +dependencies = [ |
| + "parking", |
| + "pin-project-lite", |
| +] |
| + |
| +[[package]] |
| +name = "event-listener-strategy" |
| +version = "0.5.4" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "8be9f3dfaaffdae2972880079a491a1a8bb7cbed0b8dd7a347f668b4150a3b93" |
| +dependencies = [ |
| + "event-listener", |
| + "pin-project-lite", |
| +] |
| + |
| +[[package]] |
| +name = "fastrand" |
| +version = "2.5.0" |
| +dependencies = [ |
| + "getrandom", |
| +] |
| + |
| +[[package]] |
| +name = "foldhash" |
| +version = "0.2.0" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb" |
| + |
| +[[package]] |
| +name = "getrandom" |
| +version = "0.4.3" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099" |
| +dependencies = [ |
| + "cfg-if", |
| + "libc", |
| + "r-efi", |
| +] |
| + |
| +[[package]] |
| +name = "hashbrown" |
| +version = "0.14.5" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" |
| +dependencies = [ |
| + "ahash", |
| +] |
| + |
| +[[package]] |
| +name = "hashbrown" |
| +version = "0.17.1" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" |
| +dependencies = [ |
| + "foldhash", |
| + "serde", |
| + "serde_core", |
| +] |
| + |
| +[[package]] |
| +name = "indexmap" |
| +version = "2.14.1" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "07aa2048142242915a31d35844fb311e0e53fcca590c3a0a40dcf1b841fa09eb" |
| +dependencies = [ |
| + "equivalent", |
| + "hashbrown 0.17.1", |
| + "serde", |
| + "serde_core", |
| +] |
| + |
| +[[package]] |
| +name = "indexmap-nostd" |
| +version = "0.4.0" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "8e04e2fd2b8188ea827b32ef11de88377086d690286ab35747ef7f9bf3ccb590" |
| + |
| +[[package]] |
| +name = "itoa" |
| +version = "1.0.18" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" |
| + |
| +[[package]] |
| +name = "libc" |
| +version = "0.2.189" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2" |
| + |
| +[[package]] |
| +name = "libm" |
| +version = "0.2.16" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "b6d2cec3eae94f9f509c767b45932f1ada8350c4bdb85af2fcab4a3c14807981" |
| + |
| +[[package]] |
| +name = "lock_api" |
| +version = "0.4.14" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" |
| +dependencies = [ |
| + "scopeguard", |
| +] |
| + |
| +[[package]] |
| +name = "memchr" |
| +version = "2.8.3" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" |
| + |
| +[[package]] |
| +name = "multi-stash" |
| +version = "0.2.0" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "685a9ac4b61f4e728e1d2c6a7844609c16527aeb5e6c865915c08e619c16410f" |
| + |
| +[[package]] |
| +name = "num-derive" |
| +version = "0.4.2" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "ed3955f1a9c7c0c15e092f9c887db08b1fc683305fdf6eb6684f22555355e202" |
| +dependencies = [ |
| + "proc-macro2", |
| + "quote", |
| + "syn 2.0.119", |
| +] |
| + |
| +[[package]] |
| +name = "num-traits" |
| +version = "0.2.19" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" |
| +dependencies = [ |
| + "autocfg", |
| +] |
| + |
| +[[package]] |
| +name = "once_cell" |
| +version = "1.21.4" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" |
| + |
| +[[package]] |
| +name = "parking" |
| +version = "2.2.1" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba" |
| + |
| +[[package]] |
| +name = "parking_lot_core" |
| +version = "0.9.12" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" |
| +dependencies = [ |
| + "cfg-if", |
| + "libc", |
| + "redox_syscall", |
| + "smallvec", |
| + "windows-link", |
| +] |
| + |
| +[[package]] |
| +name = "paste" |
| +version = "1.0.15" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a" |
| + |
| +[[package]] |
| +name = "pin-project-lite" |
| +version = "0.2.17" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" |
| + |
| +[[package]] |
| +name = "proc-macro2" |
| +version = "1.0.107" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" |
| +dependencies = [ |
| + "unicode-ident", |
| +] |
| + |
| +[[package]] |
| +name = "quote" |
| +version = "1.0.47" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" |
| +dependencies = [ |
| + "proc-macro2", |
| +] |
| + |
| +[[package]] |
| +name = "r-efi" |
| +version = "6.0.0" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" |
| + |
| +[[package]] |
| +name = "redox_syscall" |
| +version = "0.5.18" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" |
| +dependencies = [ |
| + "bitflags", |
| +] |
| + |
| +[[package]] |
| +name = "rustversion" |
| +version = "1.0.23" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f" |
| + |
| +[[package]] |
| +name = "scopeguard" |
| +version = "1.2.0" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" |
| + |
| +[[package]] |
| +name = "semver" |
| +version = "1.0.28" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd" |
| + |
| +[[package]] |
| +name = "serde" |
| +version = "1.0.229" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba" |
| +dependencies = [ |
| + "serde_core", |
| + "serde_derive", |
| +] |
| + |
| +[[package]] |
| +name = "serde_core" |
| +version = "1.0.229" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48" |
| +dependencies = [ |
| + "serde_derive", |
| +] |
| + |
| +[[package]] |
| +name = "serde_derive" |
| +version = "1.0.229" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348" |
| +dependencies = [ |
| + "proc-macro2", |
| + "quote", |
| + "syn 3.0.4", |
| +] |
| + |
| +[[package]] |
| +name = "serde_json" |
| +version = "1.0.151" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14" |
| +dependencies = [ |
| + "itoa", |
| + "memchr", |
| + "serde", |
| + "serde_core", |
| + "zmij", |
| +] |
| + |
| +[[package]] |
| +name = "slides-core" |
| +version = "0.1.0" |
| +dependencies = [ |
| + "getrandom", |
| + "serde", |
| + "serde_json", |
| + "wasmi", |
| + "wasmparser", |
| + "yrs", |
| +] |
| + |
| +[[package]] |
| +name = "smallstr" |
| +version = "0.3.1" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "862077b1e764f04c251fe82a2ef562fd78d7cadaeb072ca7c2bcaf7217b1ff3b" |
| +dependencies = [ |
| + "smallvec", |
| +] |
| + |
| +[[package]] |
| +name = "smallvec" |
| +version = "1.15.2" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90" |
| + |
| +[[package]] |
| +name = "spin" |
| +version = "0.9.9" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "3763264f6b73151db08c50ff20d7d8a0b8796e021cdea7ceedad07b80155fa0e" |
| + |
| +[[package]] |
| +name = "string-interner" |
| +version = "0.17.0" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "1c6a0d765f5807e98a091107bae0a56ea3799f66a5de47b2c84c94a39c09974e" |
| +dependencies = [ |
| + "cfg-if", |
| + "hashbrown 0.14.5", |
| + "serde", |
| +] |
| + |
| +[[package]] |
| +name = "syn" |
| +version = "2.0.119" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" |
| +dependencies = [ |
| + "proc-macro2", |
| + "quote", |
| + "unicode-ident", |
| +] |
| + |
| +[[package]] |
| +name = "syn" |
| +version = "3.0.4" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "e6275cddf4610d1775e6d1fe9469b2e77d0f39fd98fb7450901b821e0c53649f" |
| +dependencies = [ |
| + "proc-macro2", |
| + "quote", |
| + "unicode-ident", |
| +] |
| + |
| +[[package]] |
| +name = "thiserror" |
| +version = "2.0.20" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "ec86235f5fcc2a73650310756d2ac5b138a5780bbbdfae3eeccec992c435ba4f" |
| +dependencies = [ |
| + "thiserror-impl", |
| +] |
| + |
| +[[package]] |
| +name = "thiserror-impl" |
| +version = "2.0.20" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af" |
| +dependencies = [ |
| + "proc-macro2", |
| + "quote", |
| + "syn 3.0.4", |
| +] |
| + |
| +[[package]] |
| +name = "unicode-ident" |
| +version = "1.0.24" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" |
| + |
| +[[package]] |
| +name = "version_check" |
| +version = "0.9.5" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" |
| + |
| +[[package]] |
| +name = "wasmi" |
| +version = "0.32.3" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "50386c99b9c32bd2ed71a55b6dd4040af2580530fae8bdb9a6576571a80d0cca" |
| +dependencies = [ |
| + "arrayvec", |
| + "multi-stash", |
| + "num-derive", |
| + "num-traits", |
| + "smallvec", |
| + "spin", |
| + "wasmi_collections", |
| + "wasmi_core", |
| + "wasmparser-nostd", |
| +] |
| + |
| +[[package]] |
| +name = "wasmi_collections" |
| +version = "0.32.3" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "9c128c039340ffd50d4195c3f8ce31aac357f06804cfc494c8b9508d4b30dca4" |
| +dependencies = [ |
| + "ahash", |
| + "hashbrown 0.14.5", |
| + "string-interner", |
| +] |
| + |
| +[[package]] |
| +name = "wasmi_core" |
| +version = "0.32.3" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "a23b3a7f6c8c3ceeec6b83531ee61f0013c56e51cbf2b14b0f213548b23a4b41" |
| +dependencies = [ |
| + "downcast-rs", |
| + "libm", |
| + "num-traits", |
| + "paste", |
| +] |
| + |
| +[[package]] |
| +name = "wasmparser" |
| +version = "0.258.0" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "d9a61719f93a87b16d325921e251800c4833f8fab50fa21c7de73aed50086313" |
| +dependencies = [ |
| + "bitflags", |
| + "hashbrown 0.17.1", |
| + "indexmap", |
| + "semver", |
| + "serde", |
| +] |
| + |
| +[[package]] |
| +name = "wasmparser-nostd" |
| +version = "0.100.2" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "d5a015fe95f3504a94bb1462c717aae75253e39b9dd6c3fb1062c934535c64aa" |
| +dependencies = [ |
| + "indexmap-nostd", |
| +] |
| + |
| +[[package]] |
| +name = "windows-link" |
| +version = "0.2.1" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" |
| + |
| +[[package]] |
| +name = "yrs" |
| +version = "0.23.5" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "197c2b4b298f35c3ba4d549884ac872a961112095b29f173ab45e3d5cf609018" |
| +dependencies = [ |
| + "arc-swap", |
| + "async-lock", |
| + "async-trait", |
| + "dashmap", |
| + "fastrand", |
| + "serde", |
| + "serde_json", |
| + "smallstr", |
| + "smallvec", |
| + "thiserror", |
| +] |
| + |
| +[[package]] |
| +name = "zerocopy" |
| +version = "0.8.56" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "556764e583adb45a9f8d413c2a147fa7e8d821e48e12b14fd560b607998b75eb" |
| +dependencies = [ |
| + "zerocopy-derive", |
| +] |
| + |
| +[[package]] |
| +name = "zerocopy-derive" |
| +version = "0.8.56" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "f2ab42fc20575779bd240faa45f94a74256f755c0fa9e89f0ede20d91d0cdfc1" |
| +dependencies = [ |
| + "proc-macro2", |
| + "quote", |
| + "syn 2.0.119", |
| +] |
| + |
| +[[package]] |
| +name = "zmij" |
| +version = "1.0.23" |
| +source = "registry+https://github.com/rust-lang/crates.io-index" |
| +checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" |
diff --git a/core/Cargo.toml b/core/Cargo.toml| new file mode 100644 |
| index 0000000..fbb5452 |
| --- /dev/null |
| +++ b/core/Cargo.toml |
| @@ -0,0 +1,36 @@ |
| +[package] |
| +name = "slides-core" |
| +version = "0.1.0" |
| +edition = "2021" |
| + |
| +[lib] |
| +crate-type = ["cdylib", "rlib"] |
| + |
| +[dependencies] |
| +yrs = "=0.23.5" |
| +# The op batch and the view both cross the ABI as JSON. Pinned exact, |
| +# like yrs, so a build is the same build everywhere the lockfile goes. |
| +serde = { version = "=1.0.229", features = ["derive"] } |
| +serde_json = "=1.0.151" |
| + |
| +# Only used by the __getrandom_v03_custom hook in src/lib.rs, which is |
| +# itself cfg(target_arch = "wasm32") — the custom getrandom backend has no |
| +# role on native builds (tests, tooling), so keep the dependency scoped to |
| +# match instead of pulling it into every target unconditionally. |
| +[target.'cfg(target_arch = "wasm32")'.dependencies] |
| +getrandom = "=0.4.3" |
| + |
| +[profile.release] |
| +opt-level = "s" |
| +lto = true |
| +panic = "abort" |
| + |
| +[dev-dependencies] |
| +wasmi = "=0.32.3" |
| +wasmparser = "=0.258.0" |
| + |
| +# See vendor/README.md — this fixes a `getrandom`/`fastrand` feature |
| +# conflict that only manifests when overriding getrandom's backend to |
| +# avoid a wasm-bindgen (JS) import on wasm32-unknown-unknown. |
| +[patch.crates-io] |
| +fastrand = { path = "vendor/fastrand-2.5.0" } |
diff --git a/core/src/abi.rs b/core/src/abi.rs| new file mode 100644 |
| index 0000000..45d3633 |
| --- /dev/null |
| +++ b/core/src/abi.rs |
| @@ -0,0 +1,427 @@ |
| +//! The C ABI the Go host calls through wazero. No wasm-bindgen: wazero |
| +//! provides no JS glue, so everything crosses as integers and linear-memory |
| +//! offsets. |
| +//! |
| +//! Handles are indices into a slab rather than raw pointers, so a stale |
| +//! handle from the host is a null lookup instead of a wild dereference. |
| +//! |
| +//! # Containment |
| +//! |
| +//! Each document gets its own wasm module instance, and a trapped |
| +//! instance is discarded and rebuilt by the host. That defence is |
| +//! architectural, not something this module can validate its way out of |
| +//! — `yrs` decodes wire bytes with `std::str::from_utf8_unchecked` in |
| +//! places, so a sufficiently corrupted update can reach real undefined |
| +//! behaviour deep inside the CRDT decode, and finding that requires |
| +//! actually parsing the update — which is `yrs`'s job, not ours to redo. |
| +//! |
| +//! What *is* this module's job is to not make an upstream trap worse than |
| +//! it has to be, and the rules are Docs' (`docs/core/src/abi.rs`): |
| +//! |
| +//! - No `RefCell` borrow is held across a call into `yrs`. A wasm trap |
| +//! does not unwind — with `panic = "abort"` no destructors run — so a |
| +//! borrow guard on the stack at the moment of a trap stays "borrowed" |
| +//! in the instance's memory forever, poisoning every later call that |
| +//! needs `borrow_mut()` even for documents the bad input never touched. |
| +//! `get_doc` clones the `Doc` out (an `Arc` bump, not a copy of CRDT |
| +//! state) and releases the borrow before doing anything that might trap. |
| +//! The `OUT`/`ERR` slabs are only ever borrowed for a plain insert or |
| +//! clone, after the `yrs` call has returned. |
| +//! - Every raw pointer/length pair this module receives from the host is |
| +//! bounds-checked against the module's actual linear memory |
| +//! (`in_bounds`) before a slice is ever constructed over it, turning a |
| +//! host bug into a returned error code instead of an out-of-bounds trap. |
| +//! - A handle that doesn't exist is distinguishable from a real, empty |
| +//! document: every `_len` and `_read` returns `ERR_NO_DOC` for the |
| +//! former, never 0 bytes. |
| +//! |
| +//! # The read pairs |
| +//! |
| +//! `state`, `json`, `out` and `err` all follow one convention: `_len(h)` |
| +//! returns the byte length (or a negative code), the host allocates that |
| +//! many bytes with `slides_core_alloc`, and `_read(h, ptr, cap)` copies |
| +//! the bytes in and returns how many it wrote. `out` and `err` are the |
| +//! two results of the last `slides_core_ops` call on that handle: a |
| +//! success stores the update in `out` and clears `err`; a refusal stores |
| +//! the message in `err` and clears `out`. Before any ops call both read |
| +//! as empty. |
| + |
| +use crate::{apply_ops, apply_update, state_bytes, view_json, CoreError}; |
| +use std::cell::RefCell; |
| +use std::collections::HashMap; |
| +use yrs::Doc; |
| + |
| +thread_local! { |
| + static DOCS: RefCell<HashMap<u32, Doc>> = RefCell::new(HashMap::new()); |
| + static OUT: RefCell<HashMap<u32, Vec<u8>>> = RefCell::new(HashMap::new()); |
| + static ERR: RefCell<HashMap<u32, String>> = RefCell::new(HashMap::new()); |
| + static NEXT: RefCell<u32> = const { RefCell::new(1) }; |
| +} |
| + |
| +pub const ERR_NO_DOC: i32 = -1; |
| +pub const ERR_DECODE: i32 = -2; |
| +pub const ERR_APPLY: i32 = -3; |
| +pub const ERR_BUFFER_TOO_SMALL: i32 = -4; |
| +/// The host handed us a `ptr`/`len` pair this module's linear memory |
| +/// cannot possibly satisfy: `len` alone exceeds `isize::MAX` (the limit |
| +/// `std::slice::from_raw_parts` requires), or `ptr..ptr+len` reaches past |
| +/// the module's currently instantiated memory, or the addition overflows |
| +/// `u64`. See `in_bounds`. |
| +pub const ERR_INVALID_LENGTH: i32 = -5; |
| +/// The op batch was refused and nothing applied; the message is in the |
| +/// `err` read pair. |
| +pub const ERR_OPS: i32 = -6; |
| + |
| +/// Clones the `Doc` for `handle` out of `DOCS` and releases the borrow |
| +/// before returning it, so callers never hold a `RefCell` guard across a |
| +/// call into `yrs` that might trap. |
| +fn get_doc(handle: u32) -> Option<Doc> { |
| + DOCS.with(|d| d.borrow().get(&handle).cloned()) |
| +} |
| + |
| +pub fn doc_new(guid: &str, client_id: u64) -> u32 { |
| + let id = NEXT.with(|n| { |
| + let mut n = n.borrow_mut(); |
| + let id = *n; |
| + // 0 is the failure sentinel every caller checks handles against. |
| + // Wrapping `u32` addition would land exactly on 0 after 2^32 |
| + // calls and hand out a "null" handle as if it were real; skip 0 |
| + // on wraparound instead of landing on it. |
| + let next = id.wrapping_add(1); |
| + *n = if next == 0 { 1 } else { next }; |
| + id |
| + }); |
| + let doc = crate::new_doc(guid, client_id); |
| + DOCS.with(|d| d.borrow_mut().insert(id, doc)); |
| + id |
| +} |
| + |
| +pub fn doc_free(handle: u32) { |
| + DOCS.with(|d| d.borrow_mut().remove(&handle)); |
| + OUT.with(|o| o.borrow_mut().remove(&handle)); |
| + ERR.with(|e| e.borrow_mut().remove(&handle)); |
| +} |
| + |
| +pub fn apply_slice(handle: u32, update: &[u8]) -> i32 { |
| + match get_doc(handle) { |
| + None => ERR_NO_DOC, |
| + Some(doc) => match apply_update(&doc, update) { |
| + Ok(()) => 0, |
| + Err(CoreError::Decode(_)) => ERR_DECODE, |
| + Err(CoreError::Apply(_)) | Err(CoreError::Ops(_)) => ERR_APPLY, |
| + }, |
| + } |
| +} |
| + |
| +/// Runs an op batch and files its result under the handle. Returns 0 on |
| +/// success (update in `out`), `ERR_OPS` on refusal (message in `err`), |
| +/// `ERR_NO_DOC` for an unknown handle. |
| +pub fn ops_str(handle: u32, ops: &str) -> i32 { |
| + let Some(doc) = get_doc(handle) else { |
| + return ERR_NO_DOC; |
| + }; |
| + match apply_ops(&doc, ops) { |
| + Ok(update) => { |
| + OUT.with(|o| o.borrow_mut().insert(handle, update)); |
| + ERR.with(|e| e.borrow_mut().remove(&handle)); |
| + 0 |
| + } |
| + Err(e) => { |
| + let msg = match e { |
| + CoreError::Ops(m) => m, |
| + other => other.to_string(), |
| + }; |
| + ERR.with(|e| e.borrow_mut().insert(handle, msg)); |
| + OUT.with(|o| o.borrow_mut().remove(&handle)); |
| + ERR_OPS |
| + } |
| + } |
| +} |
| + |
| +/// `None` for an unknown handle — distinct from `Some(vec![])`, which |
| +/// is never what `state_bytes` returns for a live document anyway (an |
| +/// empty document still encodes to a few bytes). |
| +pub fn state_vec(handle: u32) -> Option<Vec<u8>> { |
| + get_doc(handle).map(|doc| state_bytes(&doc)) |
| +} |
| + |
| +/// `None` for an unknown handle. Never `Some("")`: `view_json` always |
| +/// emits a complete object. |
| +pub fn json_string(handle: u32) -> Option<String> { |
| + get_doc(handle).map(|doc| view_json(&doc)) |
| +} |
| + |
| +/// The last ops call's update; empty if there was none or it failed. |
| +/// `None` for an unknown handle. |
| +pub fn out_vec(handle: u32) -> Option<Vec<u8>> { |
| + get_doc(handle)?; |
| + Some(OUT.with(|o| o.borrow().get(&handle).cloned().unwrap_or_default())) |
| +} |
| + |
| +/// The last ops call's refusal message; empty if there was none or it |
| +/// succeeded. `None` for an unknown handle. |
| +pub fn err_string(handle: u32) -> Option<String> { |
| + get_doc(handle)?; |
| + Some(ERR.with(|e| e.borrow().get(&handle).cloned().unwrap_or_default())) |
| +} |
| + |
| +// --- wasm exports --- |
| + |
| +/// The `Layout` for a `len`-byte buffer, or `None` if `len` isn't |
| +/// representable — `Layout::from_size_align` rejects any size that would |
| +/// round up past `isize::MAX` once aligned. That's a real possibility here: |
| +/// `len` is a host-supplied `u32`, and on wasm32 (where `usize` is 32-bit) |
| +/// `isize::MAX` is only `0x7FFF_FFFF`, well inside `u32`'s range. |
| +fn layout_for(len: u32) -> Option<std::alloc::Layout> { |
| + // align 1: these are byte buffers the host writes into. |
| + std::alloc::Layout::from_size_align(len.max(1) as usize, 1).ok() |
| +} |
| + |
| +/// Number of bytes in this module's currently instantiated linear memory. |
| +/// On wasm32 this is the real bound `in_bounds` checks host pointers |
| +/// against. Native builds have no such thing, so `in_bounds` degrades |
| +/// there to just the `isize::MAX` length check. |
| +#[cfg(target_arch = "wasm32")] |
| +fn memory_len_bytes() -> u64 { |
| + const WASM_PAGE_BYTES: u64 = 65536; |
| + core::arch::wasm32::memory_size(0) as u64 * WASM_PAGE_BYTES |
| +} |
| + |
| +#[cfg(not(target_arch = "wasm32"))] |
| +fn memory_len_bytes() -> u64 { |
| + u64::MAX |
| +} |
| + |
| +/// True if `ptr..ptr+len` is representable as a Rust slice and (on wasm32) |
| +/// fully inside this module's currently instantiated linear memory. Every |
| +/// export taking a pointer calls this before constructing a slice over |
| +/// host-supplied bytes, turning a bad pair into `ERR_INVALID_LENGTH` |
| +/// instead of an out-of-bounds-memory trap. It cannot validate the |
| +/// *contents* of the bytes — that's the CRDT decode's job. |
| +fn in_bounds(ptr: u32, len: u32) -> bool { |
| + if len as u64 > isize::MAX as u64 { |
| + return false; |
| + } |
| + match (ptr as u64).checked_add(len as u64) { |
| + Some(end) => end <= memory_len_bytes(), |
| + None => false, |
| + } |
| +} |
| + |
| +/// Allocates `len` bytes in this module's linear memory for the host to |
| +/// write into. Returns 0 on failure — `len` too large to form a valid |
| +/// `Layout` (see `layout_for`), or genuine allocator exhaustion — and |
| +/// **the host must check for a 0 return before using the pointer**; |
| +/// nothing here traps on a bad `len`. |
| +#[no_mangle] |
| +pub extern "C" fn slides_core_alloc(len: u32) -> u32 { |
| + match layout_for(len) { |
| + // SAFETY: `layout_for` never returns a zero-size Layout (it floors |
| + // `len` to 1), which is the one precondition `std::alloc::alloc` |
| + // places beyond having a valid Layout at all. |
| + Some(layout) => unsafe { std::alloc::alloc(layout) as u32 }, |
| + None => 0, |
| + } |
| +} |
| + |
| +/// # Safety |
| +/// |
| +/// `ptr` must be either 0 (a no-op) or a value previously returned by |
| +/// `slides_core_alloc`, not yet freed, and `len` must be exactly the `len` |
| +/// that allocation was made with — `layout_for(len)` must reproduce the |
| +/// same `Layout` used to allocate it, per `std::alloc::dealloc`'s contract. |
| +#[no_mangle] |
| +pub unsafe extern "C" fn slides_core_free(ptr: u32, len: u32) { |
| + if ptr == 0 { |
| + return; |
| + } |
| + // A `len` with no valid Layout cannot be the `len` slides_core_alloc |
| + // succeeded with — alloc would have already returned 0 for it. Rather |
| + // than call `dealloc` with a fabricated Layout (undefined behaviour if |
| + // it doesn't match the allocation), treat this as a no-op. |
| + if let Some(layout) = layout_for(len) { |
| + std::alloc::dealloc(ptr as *mut u8, layout); |
| + } |
| +} |
| + |
| +/// # Safety |
| +/// |
| +/// `guid_ptr` and `guid_len` must describe a byte range that lies entirely |
| +/// within this module's linear memory and is not mutated for the duration |
| +/// of this call. (Checked by `in_bounds` before any slice is constructed — |
| +/// this safety requirement covers what that check cannot see, such as |
| +/// concurrent mutation.) |
| +#[no_mangle] |
| +pub unsafe extern "C" fn slides_core_doc_new(guid_ptr: u32, guid_len: u32, client_id: u64) -> u32 { |
| + if !in_bounds(guid_ptr, guid_len) { |
| + return 0; // out-of-bounds ptr/len is a null handle, not a trap |
| + } |
| + let bytes = std::slice::from_raw_parts(guid_ptr as *const u8, guid_len as usize); |
| + match std::str::from_utf8(bytes) { |
| + Ok(guid) => doc_new(guid, client_id), |
| + Err(_) => 0, // invalid UTF-8 GUID is a null handle, not a trap |
| + } |
| +} |
| + |
| +#[no_mangle] |
| +pub extern "C" fn slides_core_doc_free(handle: u32) { |
| + doc_free(handle) |
| +} |
| + |
| +/// # Safety |
| +/// |
| +/// `ptr` and `len` must describe a byte range that lies entirely within |
| +/// this module's linear memory and is not mutated for the duration of this |
| +/// call. (Checked by `in_bounds` before any slice is constructed.) |
| +#[no_mangle] |
| +pub unsafe extern "C" fn slides_core_apply(handle: u32, ptr: u32, len: u32) -> i32 { |
| + if !in_bounds(ptr, len) { |
| + return ERR_INVALID_LENGTH; |
| + } |
| + let bytes = std::slice::from_raw_parts(ptr as *const u8, len as usize); |
| + apply_slice(handle, bytes) |
| +} |
| + |
| +/// # Safety |
| +/// |
| +/// `ptr` and `len` must describe a byte range that lies entirely within |
| +/// this module's linear memory and is not mutated for the duration of this |
| +/// call. (Checked by `in_bounds` before any slice is constructed.) |
| +#[no_mangle] |
| +pub unsafe extern "C" fn slides_core_ops(handle: u32, ptr: u32, len: u32) -> i32 { |
| + if !in_bounds(ptr, len) { |
| + return ERR_INVALID_LENGTH; |
| + } |
| + let bytes = std::slice::from_raw_parts(ptr as *const u8, len as usize); |
| + match std::str::from_utf8(bytes) { |
| + Ok(ops) => ops_str(handle, ops), |
| + Err(e) => { |
| + if get_doc(handle).is_none() { |
| + return ERR_NO_DOC; |
| + } |
| + ERR.with(|m| { |
| + m.borrow_mut() |
| + .insert(handle, format!("ops are not UTF-8: {e}")) |
| + }); |
| + OUT.with(|o| o.borrow_mut().remove(&handle)); |
| + ERR_OPS |
| + } |
| + } |
| +} |
| + |
| +/// The byte length of `bytes`, clamped to `i32` — on native 64-bit test |
| +/// builds a length can exceed it; on wasm32 it cannot but the clamp |
| +/// costs nothing. |
| +fn len_code(bytes: &[u8]) -> i32 { |
| + bytes.len().try_into().unwrap_or(i32::MAX) |
| +} |
| + |
| +/// Copies `bytes` to the host's buffer and returns how many were |
| +/// written, or `ERR_BUFFER_TOO_SMALL` / `ERR_INVALID_LENGTH`. |
| +/// |
| +/// # Safety |
| +/// |
| +/// `out_ptr` and `out_len` must describe a byte range that lies entirely |
| +/// within this module's linear memory and is valid for writes for the |
| +/// duration of this call. (Checked by `in_bounds` before any write.) |
| +unsafe fn read_into(bytes: &[u8], out_ptr: u32, out_len: u32) -> i32 { |
| + if bytes.len() > out_len as usize { |
| + return ERR_BUFFER_TOO_SMALL; |
| + } |
| + if !in_bounds(out_ptr, out_len) { |
| + return ERR_INVALID_LENGTH; |
| + } |
| + std::ptr::copy_nonoverlapping(bytes.as_ptr(), out_ptr as *mut u8, bytes.len()); |
| + len_code(bytes) |
| +} |
| + |
| +/// Byte length of the document's whole state as one v1 update, or |
| +/// `ERR_NO_DOC`. |
| +#[no_mangle] |
| +pub extern "C" fn slides_core_state_len(handle: u32) -> i32 { |
| + match state_vec(handle) { |
| + Some(v) => len_code(&v), |
| + None => ERR_NO_DOC, |
| + } |
| +} |
| + |
| +/// # Safety |
| +/// |
| +/// `out_ptr` and `out_len` must describe a byte range that lies entirely |
| +/// within this module's linear memory and is valid for writes for the |
| +/// duration of this call. (Checked by `in_bounds` before any write.) |
| +#[no_mangle] |
| +pub unsafe extern "C" fn slides_core_state_read(handle: u32, out_ptr: u32, out_len: u32) -> i32 { |
| + match state_vec(handle) { |
| + Some(v) => read_into(&v, out_ptr, out_len), |
| + None => ERR_NO_DOC, |
| + } |
| +} |
| + |
| +/// Byte length of the deck's JSON view, or `ERR_NO_DOC`. |
| +#[no_mangle] |
| +pub extern "C" fn slides_core_json_len(handle: u32) -> i32 { |
| + match json_string(handle) { |
| + Some(s) => len_code(s.as_bytes()), |
| + None => ERR_NO_DOC, |
| + } |
| +} |
| + |
| +/// # Safety |
| +/// |
| +/// `out_ptr` and `out_len` must describe a byte range that lies entirely |
| +/// within this module's linear memory and is valid for writes for the |
| +/// duration of this call. (Checked by `in_bounds` before any write.) |
| +#[no_mangle] |
| +pub unsafe extern "C" fn slides_core_json_read(handle: u32, out_ptr: u32, out_len: u32) -> i32 { |
| + match json_string(handle) { |
| + Some(s) => read_into(s.as_bytes(), out_ptr, out_len), |
| + None => ERR_NO_DOC, |
| + } |
| +} |
| + |
| +/// Byte length of the last ops call's update (0 if none), or |
| +/// `ERR_NO_DOC`. |
| +#[no_mangle] |
| +pub extern "C" fn slides_core_out_len(handle: u32) -> i32 { |
| + match out_vec(handle) { |
| + Some(v) => len_code(&v), |
| + None => ERR_NO_DOC, |
| + } |
| +} |
| + |
| +/// # Safety |
| +/// |
| +/// `out_ptr` and `out_len` must describe a byte range that lies entirely |
| +/// within this module's linear memory and is valid for writes for the |
| +/// duration of this call. (Checked by `in_bounds` before any write.) |
| +#[no_mangle] |
| +pub unsafe extern "C" fn slides_core_out_read(handle: u32, out_ptr: u32, out_len: u32) -> i32 { |
| + match out_vec(handle) { |
| + Some(v) => read_into(&v, out_ptr, out_len), |
| + None => ERR_NO_DOC, |
| + } |
| +} |
| + |
| +/// Byte length of the last ops call's refusal message (0 if none), or |
| +/// `ERR_NO_DOC`. |
| +#[no_mangle] |
| +pub extern "C" fn slides_core_err_len(handle: u32) -> i32 { |
| + match err_string(handle) { |
| + Some(s) => len_code(s.as_bytes()), |
| + None => ERR_NO_DOC, |
| + } |
| +} |
| + |
| +/// # Safety |
| +/// |
| +/// `out_ptr` and `out_len` must describe a byte range that lies entirely |
| +/// within this module's linear memory and is valid for writes for the |
| +/// duration of this call. (Checked by `in_bounds` before any write.) |
| +#[no_mangle] |
| +pub unsafe extern "C" fn slides_core_err_read(handle: u32, out_ptr: u32, out_len: u32) -> i32 { |
| + match err_string(handle) { |
| + Some(s) => read_into(s.as_bytes(), out_ptr, out_len), |
| + None => ERR_NO_DOC, |
| + } |
| +} |
diff --git a/core/src/lib.rs b/core/src/lib.rs| new file mode 100644 |
| index 0000000..204cd63 |
| --- /dev/null |
| +++ b/core/src/lib.rs |
| @@ -0,0 +1,146 @@ |
| +//! The Slides core: the deck's Yjs CRDT, compiled to wasm and run under |
| +//! wazero on the server. It owns three things — the deck schema over yrs |
| +//! (`schema`), the operations the drive surface applies (`ops`), and the |
| +//! JSON view the server reads a deck through (`view`) — plus the Markdown |
| +//! subset the operations accept text in (`markdown`). |
| +//! |
| +//! # Important: Doc construction |
| +//! |
| +//! Never call `yrs::Doc::new()` *or* `yrs::Doc::with_client_id()` in this |
| +//! crate. Both draw entropy from `fastrand`, not just `Doc::new()`: |
| +//! `Doc::new()` picks a random client id *and* GUID; `Doc::with_client_id()` |
| +//! takes the client id explicitly but still draws a random GUID internally |
| +//! (`Options::with_client_id`, `yrs` `src/doc.rs:916`, calls `uuid_v4()` |
| +//! regardless). The only construction that supplies everything itself, and |
| +//! so is safe to use, is `Doc::with_options(Options::with_guid_and_client_id |
| +//! (guid, client_id))` (`doc.rs:928`) with both values we choose. This is |
| +//! enforced by `clippy.toml` (`disallowed-methods`), wired into |
| +//! `scripts/build-wasm.sh` — not just documented here, because this rule |
| +//! has been gotten wrong more than once already in Docs (see below). |
| +//! |
| +//! The same rule reaches further here than in Docs: every slide and block |
| +//! id arrives in the op JSON from the Go host (`id`, `new_id`, |
| +//! `new_block_ids`). The core never draws entropy for anything, and an op |
| +//! that needs an id and lacks one is an error, not an id. |
| +//! |
| +//! # Zero host imports and the custom getrandom backend |
| +//! |
| +//! On `wasm32-unknown-unknown`, `yrs`'s pinned `fastrand` dependency |
| +//! requires the `js` feature, which makes the transitively-pulled |
| +//! `getrandom` crate default to a `wasm-bindgen`-based backend that imports |
| +//! host JS functions. Merely referencing `yrs::Doc` anywhere in this crate |
| +//! is enough for those `wasm-bindgen` exports (`__wbindgen_describe`, |
| +//! `__wbindgen_malloc`, ...) to end up linked into the final module, even |
| +//! along code paths that are never reached at runtime — which breaks the |
| +//! zero-imports contract with the Go/wazero host (see |
| +//! `core/tests/no_host_imports.rs`). |
| +//! |
| +//! `.cargo/config.toml` (repository root) selects getrandom's "custom" |
| +//! backend for the `wasm32-unknown-unknown` target only (native builds, |
| +//! including `cargo test`, are unaffected and keep the real OS entropy |
| +//! source). That backend calls out to the `__getrandom_v03_custom` symbol |
| +//! defined below instead of any JS API. It refuses the request (`Err`) |
| +//! rather than fabricating bytes — but that refusal is weaker than it |
| +//! sounds, and the next paragraph is the actual reason construction must |
| +//! never draw a GUID or client id from the library, not "so a panic tells |
| +//! us": |
| +//! |
| +//! `fastrand::Rng::new()` — what both `Observer::subscribe()` |
| +//! (`yrs` `src/observer.rs:128`/`:145`, subscription ids) and `uuid_v4()` |
| +//! (`yrs` `src/lib.rs:713`, document GUIDs) call — does not propagate our |
| +//! `Err`. It masks it: `vendor/fastrand-2.5.0/src/global_rng.rs:223` turns |
| +//! any `getrandom` failure into `None` and falls back to a constant seed. |
| +//! `Rng::new()` itself then *forks* the thread-local generator from that |
| +//! seed and advances it, so successive calls within one process still |
| +//! produce different values — subscription ids don't collide with each |
| +//! other, and this crate's own tests can't see anything wrong. The actual |
| +//! hazard is cross-process: the thread-local seed is the same fixed |
| +//! constant on every process, every time, so the *sequence* of "random" |
| +//! ids/GUIDs a process draws is identical across processes. Two |
| +//! independently-created documents that never draw a client id or GUID |
| +//! explicitly would collide. This crate avoids the whole problem by |
| +//! supplying both explicitly (see above) rather than relying on `fastrand` |
| +//! to fail loudly when it can't — it won't. |
| +pub mod abi; |
| +pub mod markdown; |
| +mod ops; |
| +mod schema; |
| +mod view; |
| + |
| +pub use ops::apply_ops; |
| +pub use schema::{Paragraph, Run}; |
| +pub use view::view_json; |
| + |
| +use yrs::updates::decoder::Decode; |
| +use yrs::{Doc, Options, ReadTxn, StateVector, Transact, Update}; |
| + |
| +#[cfg(target_arch = "wasm32")] |
| +#[no_mangle] |
| +extern "Rust" fn __getrandom_v03_custom( |
| + _dest: *mut u8, |
| + _len: usize, |
| +) -> Result<(), getrandom::Error> { |
| + Err(getrandom::Error::UNSUPPORTED) |
| +} |
| + |
| +/// Returns the ABI version this module exports. The Go host checks this on |
| +/// load and refuses a module it does not understand, so a stale .wasm on a |
| +/// box fails loudly instead of behaving strangely. |
| +/// |
| +/// Version 1 is the whole surface `abi.rs` exports today: doc lifecycle, |
| +/// apply, the state/json read pairs, and ops with its out/err pairs. An |
| +/// export added later forces a bump even though it is, strictly, backwards |
| +/// compatible on the wasm side: the module and the Go binary ship embedded |
| +/// in each other, and `internal/core`'s check is exact-match, so the bump |
| +/// is what turns "unknown export" deep inside a request into a refusal at |
| +/// load time. |
| +#[no_mangle] |
| +pub extern "C" fn slides_core_abi_version() -> u32 { |
| + 1 |
| +} |
| + |
| +#[derive(Debug)] |
| +pub enum CoreError { |
| + /// The bytes were not a valid Yjs v1 update. |
| + Decode(String), |
| + /// The update decoded but could not be applied. |
| + Apply(String), |
| + /// The operation batch was refused; nothing was applied. The message |
| + /// is what the host surfaces to the caller. |
| + Ops(String), |
| +} |
| + |
| +impl std::fmt::Display for CoreError { |
| + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { |
| + match self { |
| + CoreError::Decode(m) => write!(f, "decode: {m}"), |
| + CoreError::Apply(m) => write!(f, "apply: {m}"), |
| + CoreError::Ops(m) => write!(f, "ops: {m}"), |
| + } |
| + } |
| +} |
| + |
| +impl std::error::Error for CoreError {} |
| + |
| +/// A document with the caller's GUID and client id. The one sanctioned |
| +/// constructor (see the module comment): nothing here draws entropy. |
| +pub fn new_doc(guid: &str, client_id: u64) -> Doc { |
| + Doc::with_options(Options::with_guid_and_client_id(guid.into(), client_id)) |
| +} |
| + |
| +/// Applies a Yjs v1 update to `doc`. |
| +pub fn apply_update(doc: &Doc, update: &[u8]) -> Result<(), CoreError> { |
| + let update = Update::decode_v1(update).map_err(|e| CoreError::Decode(e.to_string()))?; |
| + let mut txn = doc.transact_mut(); |
| + txn.apply_update(update) |
| + .map_err(|e| CoreError::Apply(e.to_string())) |
| +} |
| + |
| +/// The whole document as one v1 update: `encode_state_as_update_v1` |
| +/// against the EMPTY state vector, which is the call that carries every |
| +/// block, the whole delete set and the pending queue (Docs' |
| +/// `snapshot.rs` explains why the `encode_diff_v1` neighbour is a trap). |
| +pub fn state_bytes(doc: &Doc) -> Vec<u8> { |
| + doc.transact() |
| + .encode_state_as_update_v1(&StateVector::default()) |
| +} |
diff --git a/core/src/markdown.rs b/core/src/markdown.rs| new file mode 100644 |
| index 0000000..77e7657 |
| --- /dev/null |
| +++ b/core/src/markdown.rs |
| @@ -0,0 +1,344 @@ |
| +//! The Markdown subset the operations accept text in. Deliberately |
| +//! small: this is how an agent or a script says "bold this word", not a |
| +//! document format. |
| +//! |
| +//! - Paragraphs split on blank lines; the lines of one paragraph join |
| +//! with a space. |
| +//! - A line starting `- ` or `1. ` (any digits) is its own paragraph with |
| +//! the marker stripped — a `list` block is one paragraph per item. |
| +//! - Inline: `**bold**`, `*italic*`, `` `code` ``, `==highlight==`, |
| +//! `[text](url)`. A delimiter with no closing partner is literal text. |
| +//! Inside a code span nothing else is parsed. A link's text may carry |
| +//! marks; a link inside a link is literal. |
| +//! - The only escape is `\*`, which yields a literal `*`. |
| +//! |
| +//! `parse_lines` is the plain variant for `code` blocks: one paragraph |
| +//! per line, nothing interpreted, blank lines kept as empty paragraphs. |
| + |
| +use crate::schema::{Paragraph, Run}; |
| + |
| +/// The Markdown subset as paragraphs of runs. |
| +pub fn parse(input: &str) -> Vec<Paragraph> { |
| + let mut sources: Vec<String> = Vec::new(); |
| + let mut current: Option<String> = None; |
| + for line in input.lines() { |
| + let line = line.trim(); |
| + if line.is_empty() { |
| + if let Some(c) = current.take() { |
| + sources.push(c); |
| + } |
| + continue; |
| + } |
| + if let Some(item) = list_item(line) { |
| + if let Some(c) = current.take() { |
| + sources.push(c); |
| + } |
| + current = Some(item.to_string()); |
| + continue; |
| + } |
| + match current.as_mut() { |
| + Some(c) => { |
| + c.push(' '); |
| + c.push_str(line); |
| + } |
| + None => current = Some(line.to_string()), |
| + } |
| + } |
| + if let Some(c) = current { |
| + sources.push(c); |
| + } |
| + sources.iter().map(|s| parse_inline(s)).collect() |
| +} |
| + |
| +/// One plain paragraph per line, for `code` blocks. |
| +pub fn parse_lines(input: &str) -> Vec<Paragraph> { |
| + input |
| + .lines() |
| + .map(|line| { |
| + if line.is_empty() { |
| + Vec::new() |
| + } else { |
| + vec![Run { |
| + t: line.to_string(), |
| + ..Run::default() |
| + }] |
| + } |
| + }) |
| + .collect() |
| +} |
| + |
| +/// The text after a list marker, if `line` starts with one. |
| +fn list_item(line: &str) -> Option<&str> { |
| + if let Some(rest) = line.strip_prefix("- ") { |
| + return Some(rest.trim_start()); |
| + } |
| + let digits = line.bytes().take_while(u8::is_ascii_digit).count(); |
| + if digits > 0 { |
| + if let Some(rest) = line[digits..].strip_prefix(". ") { |
| + return Some(rest.trim_start()); |
| + } |
| + } |
| + None |
| +} |
| + |
| +#[derive(Default, Clone)] |
| +struct Marks { |
| + b: bool, |
| + i: bool, |
| + c: bool, |
| + h: bool, |
| + href: Option<String>, |
| +} |
| + |
| +struct Parser { |
| + runs: Paragraph, |
| + buf: String, |
| + marks: Marks, |
| +} |
| + |
| +impl Parser { |
| + fn flush(&mut self) { |
| + if self.buf.is_empty() { |
| + return; |
| + } |
| + let run = Run { |
| + t: std::mem::take(&mut self.buf), |
| + b: self.marks.b, |
| + i: self.marks.i, |
| + c: self.marks.c, |
| + h: self.marks.h, |
| + href: self.marks.href.clone(), |
| + }; |
| + match self.runs.last_mut() { |
| + Some(last) if last.same_marks(&run) => last.t.push_str(&run.t), |
| + _ => self.runs.push(run), |
| + } |
| + } |
| + |
| + /// Toggles a mark on `\*\*`-style delimiters: closes it when open, |
| + /// opens it when a closing partner exists later in `rest`, and |
| + /// otherwise leaves the delimiter as text. Returns whether the |
| + /// delimiter was consumed. |
| + fn toggle( |
| + &mut self, |
| + delim: &str, |
| + rest: &str, |
| + get: fn(&Marks) -> bool, |
| + set: fn(&mut Marks, bool), |
| + ) -> bool { |
| + if get(&self.marks) { |
| + self.flush(); |
| + set(&mut self.marks, false); |
| + return true; |
| + } |
| + if has_partner(rest, delim) { |
| + self.flush(); |
| + set(&mut self.marks, true); |
| + return true; |
| + } |
| + false |
| + } |
| + |
| + fn parse(&mut self, src: &str) { |
| + let mut i = 0; |
| + while i < src.len() { |
| + let rest = &src[i..]; |
| + if rest.starts_with("\\*") { |
| + self.buf.push('*'); |
| + i += 2; |
| + continue; |
| + } |
| + if let Some(after) = rest.strip_prefix("**") { |
| + if self.toggle("**", after, |m| m.b, |m, v| m.b = v) { |
| + i += 2; |
| + continue; |
| + } |
| + } else if let Some(after) = rest.strip_prefix("==") { |
| + if self.toggle("==", after, |m| m.h, |m, v| m.h = v) { |
| + i += 2; |
| + continue; |
| + } |
| + } else if let Some(after) = rest.strip_prefix('*') { |
| + if self.toggle("*", after, |m| m.i, |m, v| m.i = v) { |
| + i += 1; |
| + continue; |
| + } |
| + } else if let Some(after) = rest.strip_prefix('`') { |
| + if let Some(end) = after.find('`') { |
| + self.flush(); |
| + let was = self.marks.c; |
| + self.marks.c = true; |
| + self.buf.push_str(&after[..end]); |
| + self.flush(); |
| + self.marks.c = was; |
| + i += 1 + end + 1; |
| + continue; |
| + } |
| + } else if rest.starts_with('[') && self.marks.href.is_none() { |
| + if let Some((text, url, len)) = link(rest) { |
| + self.flush(); |
| + let outer = self.marks.clone(); |
| + self.marks.href = Some(url.to_string()); |
| + self.parse(text); |
| + self.flush(); |
| + self.marks = outer; |
| + i += len; |
| + continue; |
| + } |
| + } |
| + let ch = rest.chars().next().unwrap_or('\0'); |
| + self.buf.push(ch); |
| + i += ch.len_utf8(); |
| + } |
| + } |
| +} |
| + |
| +/// Whether `delim` occurs in `rest` as itself — for a single `*`, one |
| +/// that is not half of a `**`, so `lone* **bold**` does not open italic |
| +/// on the bold marker. |
| +fn has_partner(rest: &str, delim: &str) -> bool { |
| + if delim != "*" { |
| + return rest.contains(delim); |
| + } |
| + let bytes = rest.as_bytes(); |
| + let mut i = 0; |
| + while i < bytes.len() { |
| + if bytes[i] == b'*' { |
| + if i + 1 < bytes.len() && bytes[i + 1] == b'*' { |
| + i += 2; |
| + continue; |
| + } |
| + return true; |
| + } |
| + i += 1; |
| + } |
| + false |
| +} |
| + |
| +/// `[text](url)` at the start of `s`: the text, the url, and the bytes |
| +/// the whole link spans. |
| +fn link(s: &str) -> Option<(&str, &str, usize)> { |
| + let close = s.find("](")?; |
| + let text = &s[1..close]; |
| + let url_start = close + 2; |
| + let url_end = url_start + s[url_start..].find(')')?; |
| + let url = &s[url_start..url_end]; |
| + if text.is_empty() || url.is_empty() { |
| + return None; |
| + } |
| + Some((text, url, url_end + 1)) |
| +} |
| + |
| +fn parse_inline(src: &str) -> Paragraph { |
| + let mut p = Parser { |
| + runs: Vec::new(), |
| + buf: String::new(), |
| + marks: Marks::default(), |
| + }; |
| + p.parse(src); |
| + p.flush(); |
| + p.runs |
| +} |
| + |
| +#[cfg(test)] |
| +mod tests { |
| + use super::*; |
| + |
| + fn plain(t: &str) -> Run { |
| + Run { |
| + t: t.to_string(), |
| + ..Run::default() |
| + } |
| + } |
| + |
| + #[test] |
| + fn paragraphs_split_on_blank_lines_and_list_markers() { |
| + let got = parse("a\nb\n\nc\n- d\n1. e\n\n\n"); |
| + assert_eq!(got.len(), 4); |
| + assert_eq!(got[0], vec![plain("a b")]); |
| + assert_eq!(got[1], vec![plain("c")]); |
| + assert_eq!(got[2], vec![plain("d")]); |
| + assert_eq!(got[3], vec![plain("e")]); |
| + } |
| + |
| + #[test] |
| + fn inline_marks() { |
| + let got = parse("**b** *i* `c *x*` ==h== [t](u) \\* lone* **open"); |
| + assert_eq!( |
| + got[0], |
| + vec![ |
| + Run { |
| + t: "b".into(), |
| + b: true, |
| + ..Run::default() |
| + }, |
| + plain(" "), |
| + Run { |
| + t: "i".into(), |
| + i: true, |
| + ..Run::default() |
| + }, |
| + plain(" "), |
| + Run { |
| + t: "c *x*".into(), |
| + c: true, |
| + ..Run::default() |
| + }, |
| + plain(" "), |
| + Run { |
| + t: "h".into(), |
| + h: true, |
| + ..Run::default() |
| + }, |
| + plain(" "), |
| + Run { |
| + t: "t".into(), |
| + href: Some("u".into()), |
| + ..Run::default() |
| + }, |
| + plain(" * lone* **open"), |
| + ] |
| + ); |
| + } |
| + |
| + #[test] |
| + fn marks_nest_and_links_carry_marks() { |
| + let got = parse("**a *b* c** [**x**](y)"); |
| + assert_eq!( |
| + got[0], |
| + vec![ |
| + Run { |
| + t: "a ".into(), |
| + b: true, |
| + ..Run::default() |
| + }, |
| + Run { |
| + t: "b".into(), |
| + b: true, |
| + i: true, |
| + ..Run::default() |
| + }, |
| + Run { |
| + t: " c".into(), |
| + b: true, |
| + ..Run::default() |
| + }, |
| + plain(" "), |
| + Run { |
| + t: "x".into(), |
| + b: true, |
| + href: Some("y".into()), |
| + ..Run::default() |
| + }, |
| + ] |
| + ); |
| + } |
| + |
| + #[test] |
| + fn empty_input_is_no_paragraphs_and_lines_keep_blanks() { |
| + assert!(parse("").is_empty()); |
| + assert!(parse(" \n\n ").is_empty()); |
| + let got = parse_lines("x\n\ny"); |
| + assert_eq!(got, vec![vec![plain("x")], vec![], vec![plain("y")]]); |
| + } |
| +} |
diff --git a/core/src/ops.rs b/core/src/ops.rs| new file mode 100644 |
| index 0000000..52de59d |
| --- /dev/null |
| +++ b/core/src/ops.rs |
| @@ -0,0 +1,1089 @@ |
| +//! The operations the drive surface applies (spec §7.2), as one batch in |
| +//! one transaction. |
| +//! |
| +//! # Atomicity |
| +//! |
| +//! A yrs `TransactionMut` cannot be rolled back — dropping it commits |
| +//! whatever was written. So "a failure applies nothing" has to be true |
| +//! before the first write, and it is arranged in two passes: |
| +//! |
| +//! 1. Every op is parsed (shape, vocabularies, required fields) and then |
| +//! checked against a *model* of the deck — just the slide ids, their |
| +//! block ids and types, and the asset ids — that each op updates as |
| +//! it is checked, so a reference that only becomes wrong halfway |
| +//! through the batch (`remove_block` on a block an earlier op removed) |
| +//! is caught here too. |
| +//! 2. Only then does the transaction open and the ops apply. By the |
| +//! model's construction every lookup in this pass succeeds; the |
| +//! `Err`s it can still return exist so a bug here surfaces as a |
| +//! message rather than a trap, not because they are expected. |
| +//! |
| +//! # Ids |
| +//! |
| +//! The core never draws entropy (`lib.rs`). Every new slide or block id |
| +//! arrives in the op (`id`, `new_id`, `new_block_ids`) from the Go host, |
| +//! and an op that needs one and lacks it is a parse error. |
| +//! |
| +//! # The update |
| +//! |
| +//! The batch's update is the transaction's own `encode_update_v1()`, |
| +//! taken after an explicit `commit()` (which squashes the delete set and |
| +//! fires observers) and before the guard drops. yrs 0.23.5's |
| +//! `TransactionMut::encode_update` reads `before_state`, the store and |
| +//! the transaction's delete set, all of which are intact after commit — |
| +//! so no observer, no captured buffer, and nothing that could see a |
| +//! different transaction's changes. |
| + |
| +use crate::markdown; |
| +use crate::schema::{self, Paragraph, Roots}; |
| +use crate::CoreError; |
| +use serde::{Deserialize, Deserializer}; |
| +use serde_json::{Map as JsonMap, Value}; |
| +use std::collections::{HashMap, HashSet}; |
| +use std::sync::Arc; |
| +use yrs::types::xml::XmlFragmentPrelim; |
| +use yrs::types::AsPrelim; |
| +use yrs::{Any, Array, ArrayRef, Doc, In, Map, MapRef, ReadTxn, Transact, TransactionMut}; |
| + |
| +type Props = JsonMap<String, Value>; |
| + |
| +/// `Option<Option<T>>` so an absent key (`None`) and an explicit `null` |
| +/// (`Some(None)`) can mean different things — `after` absent appends, |
| +/// `after: null` puts first; `accent: null` clears. |
| +fn nullable<'de, D, T>(d: D) -> Result<Option<Option<T>>, D::Error> |
| +where |
| + D: Deserializer<'de>, |
| + T: Deserialize<'de>, |
| +{ |
| + Option::<T>::deserialize(d).map(Some) |
| +} |
| + |
| +#[derive(Deserialize)] |
| +struct Fonts { |
| + heading: String, |
| + body: String, |
| +} |
| + |
| +#[derive(Deserialize, Clone, Copy)] |
| +struct BoxSpec { |
| + x: f64, |
| + y: f64, |
| + w: f64, |
| + h: f64, |
| +} |
| + |
| +impl BoxSpec { |
| + fn json(&self) -> Value { |
| + schema::box_json(self.x, self.y, self.w, self.h) |
| + } |
| +} |
| + |
| +#[derive(Deserialize)] |
| +struct Background { |
| + kind: String, |
| + #[serde(default)] |
| + value: String, |
| +} |
| + |
| +#[derive(Deserialize)] |
| +#[serde(deny_unknown_fields)] |
| +struct PhoneOverride { |
| + hidden: Option<bool>, |
| + order: Option<i64>, |
| + size: Option<String>, |
| + #[serde(rename = "box")] |
| + bx: Option<BoxSpec>, |
| +} |
| + |
| +#[derive(Deserialize)] |
| +#[serde(tag = "op", rename_all = "snake_case")] |
| +enum Op { |
| + SetMeta { |
| + title: Option<String>, |
| + theme: Option<String>, |
| + #[serde(default, deserialize_with = "nullable")] |
| + accent: Option<Option<String>>, |
| + #[serde(default, deserialize_with = "nullable")] |
| + fonts: Option<Option<Fonts>>, |
| + }, |
| + AddSlide { |
| + id: String, |
| + #[serde(default, deserialize_with = "nullable")] |
| + after: Option<Option<String>>, |
| + layout: String, |
| + template: Option<String>, |
| + }, |
| + RemoveSlide { |
| + id: String, |
| + }, |
| + MoveSlide { |
| + id: String, |
| + after: Option<String>, |
| + }, |
| + DuplicateSlide { |
| + id: String, |
| + new_id: String, |
| + new_block_ids: Vec<String>, |
| + }, |
| + SetSlide { |
| + id: String, |
| + layout: Option<String>, |
| + template: Option<String>, |
| + background: Option<Background>, |
| + phone: Option<String>, |
| + transition: Option<String>, |
| + }, |
| + SetNotes { |
| + id: String, |
| + markdown: String, |
| + }, |
| + AddBlock { |
| + slide: String, |
| + id: String, |
| + #[serde(rename = "type")] |
| + kind: String, |
| + #[serde(default)] |
| + props: Props, |
| + region: Option<String>, |
| + #[serde(rename = "box")] |
| + bx: Option<BoxSpec>, |
| + z: Option<f64>, |
| + #[serde(default, deserialize_with = "nullable")] |
| + after: Option<Option<String>>, |
| + markdown: Option<String>, |
| + }, |
| + RemoveBlock { |
| + slide: String, |
| + id: String, |
| + }, |
| + MoveBlock { |
| + slide: String, |
| + id: String, |
| + after: Option<String>, |
| + }, |
| + SetBlock { |
| + slide: String, |
| + id: String, |
| + props: Option<Props>, |
| + #[serde(default, deserialize_with = "nullable")] |
| + region: Option<Option<String>>, |
| + #[serde(default, deserialize_with = "nullable", rename = "box")] |
| + bx: Option<Option<BoxSpec>>, |
| + z: Option<f64>, |
| + }, |
| + SetText { |
| + slide: String, |
| + id: String, |
| + markdown: String, |
| + }, |
| + SetPhone { |
| + slide: String, |
| + id: String, |
| + phone: PhoneOverride, |
| + }, |
| + RegisterAsset { |
| + id: String, |
| + #[serde(rename = "ref")] |
| + reference: String, |
| + #[serde(default)] |
| + name: String, |
| + #[serde(default)] |
| + width: i64, |
| + #[serde(default)] |
| + height: i64, |
| + #[serde(default)] |
| + alt: String, |
| + }, |
| + RemoveAsset { |
| + id: String, |
| + }, |
| +} |
| + |
| +fn fail<T>(msg: impl Into<String>) -> Result<T, CoreError> { |
| + Err(CoreError::Ops(msg.into())) |
| +} |
| + |
| +fn in_vocab(what: &str, value: &str, vocab: &[&str]) -> Result<(), CoreError> { |
| + if vocab.contains(&value) { |
| + Ok(()) |
| + } else { |
| + fail(format!( |
| + "{what} {value:?} is not one of {}", |
| + vocab.join(", ") |
| + )) |
| + } |
| +} |
| + |
| +fn non_empty(what: &str, value: &str) -> Result<(), CoreError> { |
| + if value.is_empty() { |
| + fail(format!("{what} is empty")) |
| + } else { |
| + Ok(()) |
| + } |
| +} |
| + |
| +/// `rows`/`cols` from props when present: whole numbers within |
| +/// `1..=MAX_TABLE_SIDE`. |
| +fn table_side(props: &Props, key: &str) -> Result<Option<i64>, CoreError> { |
| + let Some(v) = props.get(key) else { |
| + return Ok(None); |
| + }; |
| + match v.as_i64() { |
| + Some(n) if (1..=schema::MAX_TABLE_SIDE).contains(&n) => Ok(Some(n)), |
| + _ => fail(format!( |
| + "table {key} must be a whole number from 1 to {}", |
| + schema::MAX_TABLE_SIDE |
| + )), |
| + } |
| +} |
| + |
| +// --- the model, and the checking pass --- |
| + |
| +struct BlockModel { |
| + id: String, |
| + kind: String, |
| +} |
| + |
| +struct SlideModel { |
| + id: String, |
| + blocks: Vec<BlockModel>, |
| +} |
| + |
| +struct Model { |
| + slides: Vec<SlideModel>, |
| + assets: HashSet<String>, |
| +} |
| + |
| +impl Model { |
| + fn read<T: ReadTxn>(txn: &T, roots: &Roots) -> Model { |
| + let slides = schema::maps_of(txn, &roots.slides) |
| + .into_iter() |
| + .map(|(_, m)| { |
| + let blocks = schema::get_array(txn, &m, "blocks") |
| + .map(|a| { |
| + schema::maps_of(txn, &a) |
| + .into_iter() |
| + .map(|(_, b)| BlockModel { |
| + id: schema::get_string(txn, &b, "id").unwrap_or_default(), |
| + kind: schema::get_string(txn, &b, "type").unwrap_or_default(), |
| + }) |
| + .collect() |
| + }) |
| + .unwrap_or_default(); |
| + SlideModel { |
| + id: schema::get_string(txn, &m, "id").unwrap_or_default(), |
| + blocks, |
| + } |
| + }) |
| + .collect(); |
| + let assets = roots.assets.keys(txn).map(|k| k.to_string()).collect(); |
| + Model { slides, assets } |
| + } |
| + |
| + fn slide_index(&self, id: &str) -> Result<usize, CoreError> { |
| + self.slides |
| + .iter() |
| + .position(|s| s.id == id) |
| + .ok_or_else(|| CoreError::Ops(format!("no slide {id:?}"))) |
| + } |
| + |
| + fn slide_mut(&mut self, id: &str) -> Result<&mut SlideModel, CoreError> { |
| + let i = self.slide_index(id)?; |
| + Ok(&mut self.slides[i]) |
| + } |
| + |
| + fn check(&mut self, op: &Op) -> Result<(), CoreError> { |
| + match op { |
| + Op::SetMeta { .. } => Ok(()), |
| + Op::AddSlide { |
| + id, |
| + after, |
| + layout, |
| + template, |
| + } => { |
| + non_empty("id", id)?; |
| + if self.slide_index(id).is_ok() { |
| + return fail(format!("slide {id:?} already exists")); |
| + } |
| + in_vocab("layout", layout, schema::LAYOUTS)?; |
| + if let Some(t) = template { |
| + in_vocab("template", t, schema::TEMPLATES)?; |
| + } |
| + let at = match after { |
| + None => self.slides.len(), |
| + Some(None) => 0, |
| + Some(Some(a)) => self.slide_index(a)? + 1, |
| + }; |
| + self.slides.insert( |
| + at, |
| + SlideModel { |
| + id: id.clone(), |
| + blocks: Vec::new(), |
| + }, |
| + ); |
| + Ok(()) |
| + } |
| + Op::RemoveSlide { id } => { |
| + let i = self.slide_index(id)?; |
| + self.slides.remove(i); |
| + Ok(()) |
| + } |
| + Op::MoveSlide { id, after } => { |
| + self.slide_index(id)?; |
| + if let Some(a) = after { |
| + if a == id { |
| + return fail("move_slide: after is the slide itself"); |
| + } |
| + self.slide_index(a)?; |
| + } |
| + let i = self.slide_index(id)?; |
| + let s = self.slides.remove(i); |
| + let at = match after { |
| + None => 0, |
| + Some(a) => self.slide_index(a)? + 1, |
| + }; |
| + self.slides.insert(at, s); |
| + Ok(()) |
| + } |
| + Op::DuplicateSlide { |
| + id, |
| + new_id, |
| + new_block_ids, |
| + } => { |
| + non_empty("new_id", new_id)?; |
| + let i = self.slide_index(id)?; |
| + if self.slide_index(new_id).is_ok() { |
| + return fail(format!("slide {new_id:?} already exists")); |
| + } |
| + let n = self.slides[i].blocks.len(); |
| + if new_block_ids.len() != n { |
| + return fail(format!( |
| + "duplicate_slide: {n} blocks but {} new_block_ids", |
| + new_block_ids.len() |
| + )); |
| + } |
| + let distinct: HashSet<&String> = new_block_ids.iter().collect(); |
| + if distinct.len() != n || new_block_ids.iter().any(String::is_empty) { |
| + return fail("duplicate_slide: new_block_ids must be distinct and non-empty"); |
| + } |
| + let blocks = self.slides[i] |
| + .blocks |
| + .iter() |
| + .zip(new_block_ids) |
| + .map(|(b, nid)| BlockModel { |
| + id: nid.clone(), |
| + kind: b.kind.clone(), |
| + }) |
| + .collect(); |
| + self.slides.insert( |
| + i + 1, |
| + SlideModel { |
| + id: new_id.clone(), |
| + blocks, |
| + }, |
| + ); |
| + Ok(()) |
| + } |
| + Op::SetSlide { |
| + id, |
| + layout, |
| + template, |
| + background, |
| + phone, |
| + transition, |
| + } => { |
| + self.slide_index(id)?; |
| + if let Some(l) = layout { |
| + in_vocab("layout", l, schema::LAYOUTS)?; |
| + } |
| + if let Some(t) = template { |
| + in_vocab("template", t, schema::TEMPLATES)?; |
| + } |
| + if let Some(b) = background { |
| + in_vocab("background.kind", &b.kind, schema::BACKGROUND_KINDS)?; |
| + } |
| + if let Some(p) = phone { |
| + in_vocab("phone", p, schema::SLIDE_PHONES)?; |
| + } |
| + if let Some(t) = transition { |
| + in_vocab("transition", t, schema::TRANSITIONS)?; |
| + } |
| + Ok(()) |
| + } |
| + Op::SetNotes { id, .. } => self.slide_index(id).map(|_| ()), |
| + Op::AddBlock { |
| + slide, |
| + id, |
| + kind, |
| + props, |
| + region, |
| + after, |
| + .. |
| + } => { |
| + non_empty("id", id)?; |
| + in_vocab("type", kind, schema::BLOCK_TYPES)?; |
| + if let Some(r) = region { |
| + in_vocab("region", r, schema::REGIONS)?; |
| + } |
| + if kind == "table" { |
| + table_side(props, "rows")?; |
| + table_side(props, "cols")?; |
| + } |
| + let s = self.slide_mut(slide)?; |
| + if s.blocks.iter().any(|b| &b.id == id) { |
| + return fail(format!("block {id:?} already exists in slide {slide:?}")); |
| + } |
| + let at = match after { |
| + None => s.blocks.len(), |
| + Some(None) => 0, |
| + Some(Some(a)) => block_index(s, a)? + 1, |
| + }; |
| + s.blocks.insert( |
| + at, |
| + BlockModel { |
| + id: id.clone(), |
| + kind: kind.clone(), |
| + }, |
| + ); |
| + Ok(()) |
| + } |
| + Op::RemoveBlock { slide, id } => { |
| + let s = self.slide_mut(slide)?; |
| + let i = block_index(s, id)?; |
| + s.blocks.remove(i); |
| + Ok(()) |
| + } |
| + Op::MoveBlock { slide, id, after } => { |
| + let s = self.slide_mut(slide)?; |
| + let i = block_index(s, id)?; |
| + if let Some(a) = after { |
| + if a == id { |
| + return fail("move_block: after is the block itself"); |
| + } |
| + block_index(s, a)?; |
| + } |
| + let b = s.blocks.remove(i); |
| + let at = match after { |
| + None => 0, |
| + Some(a) => block_index(s, a)? + 1, |
| + }; |
| + s.blocks.insert(at, b); |
| + Ok(()) |
| + } |
| + Op::SetBlock { |
| + slide, |
| + id, |
| + props, |
| + region, |
| + .. |
| + } => { |
| + let s = self.slide_mut(slide)?; |
| + let i = block_index(s, id)?; |
| + if let Some(Some(r)) = region { |
| + in_vocab("region", r, schema::REGIONS)?; |
| + } |
| + if s.blocks[i].kind == "table" { |
| + if let Some(p) = props { |
| + table_side(p, "rows")?; |
| + table_side(p, "cols")?; |
| + } |
| + } |
| + Ok(()) |
| + } |
| + Op::SetText { slide, id, .. } => { |
| + let s = self.slide_mut(slide)?; |
| + let i = block_index(s, id)?; |
| + if !schema::is_text_type(&s.blocks[i].kind) { |
| + return fail(format!( |
| + "block {id:?} is a {} and carries no text", |
| + s.blocks[i].kind |
| + )); |
| + } |
| + Ok(()) |
| + } |
| + Op::SetPhone { slide, id, phone } => { |
| + let s = self.slide_mut(slide)?; |
| + block_index(s, id)?; |
| + if let Some(size) = &phone.size { |
| + in_vocab("phone.size", size, schema::SIZES)?; |
| + } |
| + Ok(()) |
| + } |
| + Op::RegisterAsset { id, reference, .. } => { |
| + non_empty("id", id)?; |
| + non_empty("ref", reference)?; |
| + if !self.assets.insert(id.clone()) { |
| + return fail(format!("asset {id:?} is already registered")); |
| + } |
| + Ok(()) |
| + } |
| + Op::RemoveAsset { id } => { |
| + if !self.assets.remove(id) { |
| + return fail(format!("no asset {id:?}")); |
| + } |
| + Ok(()) |
| + } |
| + } |
| + } |
| +} |
| + |
| +fn block_index(s: &SlideModel, id: &str) -> Result<usize, CoreError> { |
| + s.blocks |
| + .iter() |
| + .position(|b| b.id == id) |
| + .ok_or_else(|| CoreError::Ops(format!("no block {id:?} in slide {:?}", s.id))) |
| +} |
| + |
| +// --- the applying pass --- |
| + |
| +fn slide_map(txn: &TransactionMut, roots: &Roots, id: &str) -> Result<(u32, MapRef), CoreError> { |
| + schema::find_by_id(txn, &roots.slides, id) |
| + .ok_or_else(|| CoreError::Ops(format!("no slide {id:?}"))) |
| +} |
| + |
| +/// The slide's block array, created if a peer wrote a slide without one. |
| +fn blocks_of(txn: &mut TransactionMut, slide: &MapRef) -> ArrayRef { |
| + match schema::get_array(txn, slide, "blocks") { |
| + Some(a) => a, |
| + None => slide.insert(txn, "blocks", yrs::ArrayPrelim::default()), |
| + } |
| +} |
| + |
| +fn block_map( |
| + txn: &TransactionMut, |
| + blocks: &ArrayRef, |
| + id: &str, |
| +) -> Result<(u32, MapRef), CoreError> { |
| + schema::find_by_id(txn, blocks, id).ok_or_else(|| CoreError::Ops(format!("no block {id:?}"))) |
| +} |
| + |
| +fn fragment_of(txn: &mut TransactionMut, map: &MapRef, key: &str) -> yrs::XmlFragmentRef { |
| + match schema::get_fragment(txn, map, key) { |
| + Some(f) => f, |
| + None => map.insert(txn, key, XmlFragmentPrelim::default()), |
| + } |
| +} |
| + |
| +fn map_of(txn: &mut TransactionMut, map: &MapRef, key: &str) -> MapRef { |
| + match schema::get_map(txn, map, key) { |
| + Some(m) => m, |
| + None => map.insert(txn, key, yrs::MapPrelim::default()), |
| + } |
| +} |
| + |
| +/// Where an `after` puts a new entry: absent appends, null goes first, |
| +/// an id goes right after it. |
| +fn insert_index( |
| + txn: &TransactionMut, |
| + array: &ArrayRef, |
| + after: &Option<Option<String>>, |
| +) -> Result<u32, CoreError> { |
| + Ok(match after { |
| + None => array.len(txn), |
| + Some(None) => 0, |
| + Some(Some(a)) => { |
| + schema::find_by_id(txn, array, a) |
| + .ok_or_else(|| CoreError::Ops(format!("no entry {a:?}")))? |
| + .0 |
| + + 1 |
| + } |
| + }) |
| +} |
| + |
| +/// Text for a block of `kind` from its Markdown: `code` is plain lines, |
| +/// everything else the subset. |
| +fn text_for(kind: &str, markdown: &str) -> Vec<Paragraph> { |
| + if kind == "code" { |
| + markdown::parse_lines(markdown) |
| + } else { |
| + markdown::parse(markdown) |
| + } |
| +} |
| + |
| +// --- layout conversion (§2.3) --- |
| + |
| +/// The region a block lands in when nobody said: headings in `heading`, |
| +/// images in `media`, the rest in `body`. |
| +fn default_region(kind: &str) -> &'static str { |
| + match kind { |
| + "heading" => "heading", |
| + "image" => "media", |
| + _ => "body", |
| + } |
| +} |
| + |
| +/// The frame fraction each region occupies: `(x, y, w, h)`. `body` |
| +/// narrows to make room when the slide has an `aside`. |
| +fn region_rect(region: &str, has_aside: bool) -> (f64, f64, f64, f64) { |
| + match region { |
| + "heading" => (0.06, 0.08, 0.88, 0.18), |
| + "aside" => (0.54, 0.30, 0.40, 0.62), |
| + "media" => (0.50, 0.08, 0.44, 0.84), |
| + _ if has_aside => (0.06, 0.30, 0.44, 0.62), |
| + _ => (0.06, 0.30, 0.88, 0.62), |
| + } |
| +} |
| + |
| +fn region_of<T: ReadTxn>(txn: &T, block: &MapRef) -> String { |
| + match schema::get_string(txn, block, "region") { |
| + Some(r) if schema::REGIONS.contains(&r.as_str()) => r, |
| + _ => { |
| + default_region(&schema::get_string(txn, block, "type").unwrap_or_default()).to_string() |
| + } |
| + } |
| +} |
| + |
| +/// Flow to canvas: every block gets the box its region gave it, blocks |
| +/// sharing a region split its height equally in order, `z` is the |
| +/// array index, and `region` goes. |
| +fn to_canvas(txn: &mut TransactionMut, slide: &MapRef) { |
| + let blocks = blocks_of(txn, slide); |
| + let maps = schema::maps_of(txn, &blocks); |
| + let regions: Vec<String> = maps.iter().map(|(_, m)| region_of(txn, m)).collect(); |
| + let has_aside = regions.iter().any(|r| r == "aside"); |
| + let mut counts: HashMap<&str, usize> = HashMap::new(); |
| + for r in ®ions { |
| + *counts.entry(r.as_str()).or_default() += 1; |
| + } |
| + let mut seen: HashMap<&str, usize> = HashMap::new(); |
| + for ((i, m), region) in maps.iter().zip(®ions) { |
| + let n = counts[region.as_str()] as f64; |
| + let k = seen.entry(region.as_str()).or_default(); |
| + let (x, y, w, h) = region_rect(region, has_aside); |
| + let slot = h / n; |
| + let bx = schema::box_json(x, y + slot * (*k as f64), w, slot); |
| + *k += 1; |
| + m.insert(txn, "box", schema::json_to_any(&bx)); |
| + m.insert(txn, "z", Any::Number(*i as f64)); |
| + m.remove(txn, "region"); |
| + } |
| +} |
| + |
| +/// Canvas to flow: boxes go, regions come from position (a heading in |
| +/// the top quarter is `heading`, an image is `media`, the rest `body`), |
| +/// and the array is reordered top to bottom, left to right. |
| +fn to_flow(txn: &mut TransactionMut, slide: &MapRef) { |
| + let blocks = blocks_of(txn, slide); |
| + let maps = schema::maps_of(txn, &blocks); |
| + let mut order: Vec<(f64, f64, usize)> = Vec::with_capacity(maps.len()); |
| + for (pos, (_, m)) in maps.iter().enumerate() { |
| + let kind = schema::get_string(txn, m, "type").unwrap_or_default(); |
| + let bx = schema::get_object(txn, m, "box"); |
| + let coord = |k: &str, default: f64| { |
| + bx.as_ref() |
| + .and_then(|b| b.get(k)) |
| + .and_then(Value::as_f64) |
| + .unwrap_or(default) |
| + }; |
| + let (x, y) = (coord("x", 0.06), coord("y", 0.30)); |
| + let region = match kind.as_str() { |
| + "heading" if y < 0.25 => "heading", |
| + "image" => "media", |
| + _ => "body", |
| + }; |
| + m.insert(txn, "region", Any::from(region)); |
| + m.remove(txn, "box"); |
| + m.remove(txn, "z"); |
| + order.push((y, x, pos)); |
| + } |
| + order.sort_by(|a, b| { |
| + a.0.partial_cmp(&b.0) |
| + .unwrap_or(std::cmp::Ordering::Equal) |
| + .then(a.1.partial_cmp(&b.1).unwrap_or(std::cmp::Ordering::Equal)) |
| + }); |
| + if order.iter().enumerate().all(|(i, (_, _, pos))| i == *pos) { |
| + return; |
| + } |
| + let prelims: Vec<yrs::MapPrelim> = order |
| + .iter() |
| + .map(|(_, _, pos)| maps[*pos].1.as_prelim(txn)) |
| + .collect(); |
| + let len = blocks.len(txn); |
| + blocks.remove_range(txn, 0, len); |
| + for p in prelims { |
| + blocks.push_back(txn, p); |
| + } |
| +} |
| + |
| +// --- duplicate --- |
| + |
| +/// Gives a copied slide its new ids: the slide's own, each block's, and |
| +/// any `columns.children` list that named the old block ids. |
| +fn rewrite_ids(copy: &mut yrs::MapPrelim, new_id: &str, new_block_ids: &[String]) { |
| + copy.insert(Arc::from("id"), schema::string_in(new_id)); |
| + let Some(In::Array(blocks)) = copy.get_mut("blocks") else { |
| + return; |
| + }; |
| + let mut mapping: HashMap<String, String> = HashMap::new(); |
| + let mut next = new_block_ids.iter(); |
| + for b in blocks.iter_mut() { |
| + let In::Map(bm) = b else { continue }; |
| + let Some(nid) = next.next() else { break }; |
| + if let Some(In::Any(Any::String(old))) = bm.get("id") { |
| + mapping.insert(old.to_string(), nid.clone()); |
| + } |
| + bm.insert(Arc::from("id"), schema::string_in(nid)); |
| + } |
| + for b in blocks.iter_mut() { |
| + let In::Map(bm) = b else { continue }; |
| + let Some(In::Map(props)) = bm.get_mut("props") else { |
| + continue; |
| + }; |
| + let Some(In::Any(Any::Array(children))) = props.get("children") else { |
| + continue; |
| + }; |
| + let remapped: Vec<Any> = children |
| + .iter() |
| + .map(|c| match c { |
| + Any::String(s) => match mapping.get(s.as_ref()) { |
| + Some(n) => Any::from(n.as_str()), |
| + None => c.clone(), |
| + }, |
| + other => other.clone(), |
| + }) |
| + .collect(); |
| + props.insert(Arc::from("children"), In::Any(Any::Array(remapped.into()))); |
| + } |
| +} |
| + |
| +// --- apply --- |
| + |
| +fn apply_one(txn: &mut TransactionMut, roots: &Roots, op: &Op) -> Result<(), CoreError> { |
| + match op { |
| + Op::SetMeta { |
| + title, |
| + theme, |
| + accent, |
| + fonts, |
| + } => { |
| + if let Some(t) = title { |
| + roots.meta.insert(txn, "title", Any::from(t.as_str())); |
| + } |
| + if let Some(t) = theme { |
| + roots.meta.insert(txn, "theme", Any::from(t.as_str())); |
| + } |
| + match accent { |
| + None => {} |
| + Some(None) => { |
| + roots.meta.remove(txn, "accent"); |
| + } |
| + Some(Some(a)) => { |
| + roots.meta.insert(txn, "accent", Any::from(a.as_str())); |
| + } |
| + } |
| + match fonts { |
| + None => {} |
| + Some(None) => { |
| + roots.meta.remove(txn, "fonts"); |
| + } |
| + Some(Some(f)) => { |
| + let v = serde_json::json!({"heading": f.heading, "body": f.body}); |
| + roots.meta.insert(txn, "fonts", schema::json_to_any(&v)); |
| + } |
| + } |
| + } |
| + Op::AddSlide { |
| + id, |
| + after, |
| + layout, |
| + template, |
| + } => { |
| + let at = insert_index(txn, &roots.slides, after)?; |
| + let template = template.as_deref().unwrap_or(schema::DEFAULT_TEMPLATE); |
| + roots |
| + .slides |
| + .insert(txn, at, schema::slide_prelim(id, layout, template)); |
| + } |
| + Op::RemoveSlide { id } => { |
| + let (i, _) = slide_map(txn, roots, id)?; |
| + roots.slides.remove(txn, i); |
| + } |
| + Op::MoveSlide { id, after } => { |
| + let (i, m) = slide_map(txn, roots, id)?; |
| + let copy = m.as_prelim(txn); |
| + roots.slides.remove(txn, i); |
| + let at = insert_index(txn, &roots.slides, &Some(after.clone()))?; |
| + roots.slides.insert(txn, at, copy); |
| + } |
| + Op::DuplicateSlide { |
| + id, |
| + new_id, |
| + new_block_ids, |
| + } => { |
| + let (i, m) = slide_map(txn, roots, id)?; |
| + let mut copy = m.as_prelim(txn); |
| + rewrite_ids(&mut copy, new_id, new_block_ids); |
| + roots.slides.insert(txn, i + 1, copy); |
| + } |
| + Op::SetSlide { |
| + id, |
| + layout, |
| + template, |
| + background, |
| + phone, |
| + transition, |
| + } => { |
| + let (_, m) = slide_map(txn, roots, id)?; |
| + if let Some(l) = layout { |
| + let current = |
| + schema::get_string(txn, &m, "layout").unwrap_or_else(|| "flow".into()); |
| + if *l != current { |
| + match l.as_str() { |
| + "canvas" => to_canvas(txn, &m), |
| + _ => to_flow(txn, &m), |
| + } |
| + m.insert(txn, "layout", Any::from(l.as_str())); |
| + } |
| + } |
| + if let Some(t) = template { |
| + m.insert(txn, "template", Any::from(t.as_str())); |
| + } |
| + if let Some(b) = background { |
| + m.insert( |
| + txn, |
| + "background", |
| + schema::background_prelim(&b.kind, &b.value), |
| + ); |
| + } |
| + if let Some(p) = phone { |
| + m.insert(txn, "phone", Any::from(p.as_str())); |
| + } |
| + if let Some(t) = transition { |
| + m.insert(txn, "transition", Any::from(t.as_str())); |
| + } |
| + } |
| + Op::SetNotes { id, markdown } => { |
| + let (_, m) = slide_map(txn, roots, id)?; |
| + let notes = fragment_of(txn, &m, "notes"); |
| + schema::replace_text(txn, ¬es, &markdown::parse(markdown)); |
| + } |
| + Op::AddBlock { |
| + slide, |
| + id, |
| + kind, |
| + props, |
| + region, |
| + bx, |
| + z, |
| + after, |
| + markdown, |
| + } => { |
| + let (_, s) = slide_map(txn, roots, slide)?; |
| + let blocks = blocks_of(txn, &s); |
| + let at = insert_index(txn, &blocks, after)?; |
| + let canvas = schema::get_string(txn, &s, "layout").as_deref() == Some("canvas"); |
| + let mut props = props.clone(); |
| + let cells = if kind == "table" { |
| + let rows = table_side(&props, "rows")?.unwrap_or(schema::DEFAULT_TABLE_ROWS); |
| + let cols = table_side(&props, "cols")?.unwrap_or(schema::DEFAULT_TABLE_COLS); |
| + props.insert("rows".into(), Value::from(rows)); |
| + props.insert("cols".into(), Value::from(cols)); |
| + Some((rows * cols) as usize) |
| + } else { |
| + None |
| + }; |
| + let text: Option<Vec<Paragraph>> = if schema::is_text_type(kind) { |
| + Some( |
| + markdown |
| + .as_deref() |
| + .map(|md| text_for(kind, md)) |
| + .unwrap_or_default(), |
| + ) |
| + } else { |
| + None |
| + }; |
| + let region = region.as_deref().unwrap_or_else(|| default_region(kind)); |
| + let (region_key, box_value, z_value) = if canvas { |
| + let bx = bx.map(|b| b.json()).unwrap_or_else(|| { |
| + let (x, y, w, h) = region_rect(region, false); |
| + schema::box_json(x, y, w, h) |
| + }); |
| + (None, Some(bx), Some(z.unwrap_or(at as f64))) |
| + } else { |
| + (Some(region), None, None) |
| + }; |
| + let prelim = schema::block_prelim( |
| + id, |
| + kind, |
| + &props, |
| + region_key, |
| + box_value.as_ref(), |
| + z_value, |
| + text.as_deref(), |
| + cells, |
| + ); |
| + blocks.insert(txn, at, prelim); |
| + } |
| + Op::RemoveBlock { slide, id } => { |
| + let (_, s) = slide_map(txn, roots, slide)?; |
| + let blocks = blocks_of(txn, &s); |
| + let (i, _) = block_map(txn, &blocks, id)?; |
| + blocks.remove(txn, i); |
| + } |
| + Op::MoveBlock { slide, id, after } => { |
| + let (_, s) = slide_map(txn, roots, slide)?; |
| + let blocks = blocks_of(txn, &s); |
| + let (i, m) = block_map(txn, &blocks, id)?; |
| + let copy = m.as_prelim(txn); |
| + blocks.remove(txn, i); |
| + let at = insert_index(txn, &blocks, &Some(after.clone()))?; |
| + blocks.insert(txn, at, copy); |
| + } |
| + Op::SetBlock { |
| + slide, |
| + id, |
| + props, |
| + region, |
| + bx, |
| + z, |
| + } => { |
| + let (_, s) = slide_map(txn, roots, slide)?; |
| + let blocks = blocks_of(txn, &s); |
| + let (_, m) = block_map(txn, &blocks, id)?; |
| + if let Some(p) = props { |
| + let pm = map_of(txn, &m, "props"); |
| + for (k, v) in p { |
| + if v.is_null() { |
| + pm.remove(txn, k); |
| + } else { |
| + pm.insert(txn, k.as_str(), schema::json_to_any(v)); |
| + } |
| + } |
| + let is_table = schema::get_string(txn, &m, "type").as_deref() == Some("table"); |
| + if is_table && (p.contains_key("rows") || p.contains_key("cols")) { |
| + let rows = schema::get_number(txn, &pm, "rows") |
| + .unwrap_or(schema::DEFAULT_TABLE_ROWS as f64) |
| + as i64; |
| + let cols = schema::get_number(txn, &pm, "cols") |
| + .unwrap_or(schema::DEFAULT_TABLE_COLS as f64) |
| + as i64; |
| + let want = (rows.max(1) * cols.max(1)) as u32; |
| + let cells = match schema::get_array(txn, &m, "cells") { |
| + Some(c) => c, |
| + None => m.insert(txn, "cells", yrs::ArrayPrelim::default()), |
| + }; |
| + let have = cells.len(txn); |
| + if have > want { |
| + cells.remove_range(txn, want, have - want); |
| + } |
| + for _ in have..want { |
| + cells.push_back(txn, XmlFragmentPrelim::default()); |
| + } |
| + } |
| + } |
| + match region { |
| + None => {} |
| + Some(None) => { |
| + m.remove(txn, "region"); |
| + } |
| + Some(Some(r)) => { |
| + m.insert(txn, "region", Any::from(r.as_str())); |
| + } |
| + } |
| + match bx { |
| + None => {} |
| + Some(None) => { |
| + m.remove(txn, "box"); |
| + } |
| + Some(Some(b)) => { |
| + m.insert(txn, "box", schema::json_to_any(&b.json())); |
| + } |
| + } |
| + if let Some(z) = z { |
| + m.insert(txn, "z", Any::Number(*z)); |
| + } |
| + } |
| + Op::SetText { |
| + slide, |
| + id, |
| + markdown, |
| + } => { |
| + let (_, s) = slide_map(txn, roots, slide)?; |
| + let blocks = blocks_of(txn, &s); |
| + let (_, m) = block_map(txn, &blocks, id)?; |
| + let kind = schema::get_string(txn, &m, "type").unwrap_or_default(); |
| + let text = fragment_of(txn, &m, "text"); |
| + schema::replace_text(txn, &text, &text_for(&kind, markdown)); |
| + } |
| + Op::SetPhone { slide, id, phone } => { |
| + let (_, s) = slide_map(txn, roots, slide)?; |
| + let blocks = blocks_of(txn, &s); |
| + let (_, m) = block_map(txn, &blocks, id)?; |
| + let pm = map_of(txn, &m, "phone"); |
| + pm.clear(txn); |
| + if let Some(h) = phone.hidden { |
| + pm.insert(txn, "hidden", Any::Bool(h)); |
| + } |
| + if let Some(o) = phone.order { |
| + pm.insert(txn, "order", Any::Number(o as f64)); |
| + } |
| + if let Some(sz) = &phone.size { |
| + pm.insert(txn, "size", Any::from(sz.as_str())); |
| + } |
| + if let Some(b) = &phone.bx { |
| + pm.insert(txn, "box", schema::json_to_any(&b.json())); |
| + } |
| + } |
| + Op::RegisterAsset { |
| + id, |
| + reference, |
| + name, |
| + width, |
| + height, |
| + alt, |
| + } => { |
| + let v = serde_json::json!({ |
| + "ref": reference, |
| + "name": name, |
| + "width": width, |
| + "height": height, |
| + "alt": alt, |
| + }); |
| + roots |
| + .assets |
| + .insert(txn, id.as_str(), schema::json_to_any(&v)); |
| + } |
| + Op::RemoveAsset { id } => { |
| + roots.assets.remove(txn, id); |
| + } |
| + } |
| + Ok(()) |
| +} |
| + |
| +/// Applies `ops` (a JSON array of operations) to `doc` in one |
| +/// transaction with origin `"ops"`, returning the v1 update that |
| +/// transaction produced. On `Err` nothing was written — see the module |
| +/// comment for how that is arranged. |
| +pub fn apply_ops(doc: &Doc, ops: &str) -> Result<Vec<u8>, CoreError> { |
| + let raw: Vec<Value> = serde_json::from_str(ops) |
| + .map_err(|e| CoreError::Ops(format!("ops must be a JSON array: {e}")))?; |
| + let mut parsed = Vec::with_capacity(raw.len()); |
| + for (i, v) in raw.into_iter().enumerate() { |
| + let op: Op = |
| + serde_json::from_value(v).map_err(|e| CoreError::Ops(format!("op {i}: {e}")))?; |
| + parsed.push(op); |
| + } |
| + |
| + // Roots first: get_or_insert_* takes its own write transaction. |
| + let roots = schema::roots(doc); |
| + { |
| + let txn = doc.transact(); |
| + let mut model = Model::read(&txn, &roots); |
| + for (i, op) in parsed.iter().enumerate() { |
| + model.check(op).map_err(|e| match e { |
| + CoreError::Ops(m) => CoreError::Ops(format!("op {i}: {m}")), |
| + other => other, |
| + })?; |
| + } |
| + } |
| + |
| + let mut txn = doc.transact_mut_with("ops"); |
| + for (i, op) in parsed.iter().enumerate() { |
| + apply_one(&mut txn, &roots, op).map_err(|e| match e { |
| + CoreError::Ops(m) => CoreError::Ops(format!("op {i} (applying): {m}")), |
| + other => other, |
| + })?; |
| + } |
| + txn.commit(); |
| + Ok(txn.encode_update_v1()) |
| +} |
diff --git a/core/src/schema.rs b/core/src/schema.rs| new file mode 100644 |
| index 0000000..ddc7bdb |
| --- /dev/null |
| +++ b/core/src/schema.rs |
| @@ -0,0 +1,523 @@ |
| +//! The deck's shape over yrs: root names, key names, the vocabularies the |
| +//! spec closes, the text schema, and the small set of constructors and |
| +//! accessors `ops` and `view` share. Spec §2 in code. |
| +//! |
| +//! What lives where (§2.1, §2.2, §2.4): |
| +//! |
| +//! - `meta` (Y.Map): `title`, `theme`, `accent`, `fonts`, `aspect` as |
| +//! plain values (`fonts` is one `{heading, body}` object). |
| +//! - `slides` (Y.Array of Y.Map): each slide map holds `id`, `layout`, |
| +//! `template`, `phone`, `transition` as strings, `background` as a |
| +//! Y.Map `{kind, value}`, `notes` as a Y.XmlFragment, `blocks` as a |
| +//! Y.Array of block Y.Maps. |
| +//! - a block map holds `id`, `type`, `props` (Y.Map of plain values — |
| +//! `columns.children` is a plain array inside it), `phone` (Y.Map of |
| +//! the §2.5 overrides), and then either `region` (flow) or `box` + |
| +//! `z` (canvas). `box` is one plain `{x, y, w, h}` value — the four |
| +//! numbers move together, and merging two peers' drags coordinate by |
| +//! coordinate would produce a box neither of them made. Text-bearing |
| +//! types carry `text` (Y.XmlFragment); a `table` carries `cells` |
| +//! (Y.Array of Y.XmlFragment, row-major, `rows * cols` long) as a key |
| +//! of its own on the block map, which the view folds into |
| +//! `props.cells`. |
| +//! - `assets` (Y.Map): id to one plain `{ref, name, width, height, alt}` |
| +//! value — an asset is registered once and never edited in place. |
| +//! |
| +//! The text schema (§2.6): an XmlFragment of `<paragraph>` elements, each |
| +//! holding one XmlText. Marks are formatting attributes on the text's |
| +//! runs: `b`, `i`, `c`, `h` (value `true`) and `href` (a string). The |
| +//! reader is more tolerant than the writer — several texts in one |
| +//! paragraph concatenate, a bare text at fragment level is its own |
| +//! paragraph — because updates reach the server from browsers as well. |
| + |
| +use serde::Serialize; |
| +use std::collections::HashMap; |
| +use std::sync::Arc; |
| +use yrs::types::text::YChange; |
| +use yrs::types::xml::{XmlDeltaPrelim, XmlElementPrelim, XmlFragmentPrelim, XmlIn, XmlOut}; |
| +use yrs::types::{Attrs, Delta}; |
| +use yrs::{ |
| + Any, Array, ArrayRef, Doc, GetString, In, Map, MapRef, Out, ReadTxn, Text, TransactionMut, |
| + XmlFragment, XmlFragmentRef, |
| +}; |
| + |
| +pub const ROOT_META: &str = "meta"; |
| +pub const ROOT_SLIDES: &str = "slides"; |
| +pub const ROOT_ASSETS: &str = "assets"; |
| + |
| +pub const LAYOUTS: &[&str] = &["flow", "canvas"]; |
| +pub const TEMPLATES: &[&str] = &[ |
| + "title", |
| + "title-body", |
| + "two-column", |
| + "section", |
| + "image-full", |
| + "quote", |
| + "blank", |
| +]; |
| +pub const REGIONS: &[&str] = &["heading", "body", "aside", "media"]; |
| +pub const BACKGROUND_KINDS: &[&str] = &["theme", "color", "image"]; |
| +pub const SLIDE_PHONES: &[&str] = &["inherit", "reflow"]; |
| +pub const TRANSITIONS: &[&str] = &["none", "fade"]; |
| +pub const BLOCK_TYPES: &[&str] = &[ |
| + "heading", "text", "list", "quote", "image", "columns", "code", "shape", "table", "divider", |
| + "spacer", |
| +]; |
| +/// Types that carry a `text` fragment (§2.4's "Text: yes" column, plus |
| +/// `shape`'s optional label). |
| +pub const TEXT_TYPES: &[&str] = &["heading", "text", "list", "quote", "code", "shape"]; |
| +pub const SIZES: &[&str] = &["s", "m", "l"]; |
| + |
| +pub const DEFAULT_THEME: &str = "paper"; |
| +pub const DEFAULT_ASPECT: &str = "16:9"; |
| +pub const DEFAULT_TEMPLATE: &str = "blank"; |
| +pub const DEFAULT_TABLE_ROWS: i64 = 2; |
| +pub const DEFAULT_TABLE_COLS: i64 = 2; |
| +/// The most cells one table can hold. Not a product limit — a guard on |
| +/// `set_block {props: {rows: 1000000}}` allocating a million fragments |
| +/// inside one request. |
| +pub const MAX_TABLE_SIDE: i64 = 100; |
| + |
| +pub fn is_text_type(kind: &str) -> bool { |
| + TEXT_TYPES.contains(&kind) |
| +} |
| + |
| +// --- text --- |
| + |
| +/// One formatted run of a paragraph. Serialises as `{t, b?, i?, c?, h?, |
| +/// href?}` with false and absent marks omitted, which is the view's run |
| +/// shape verbatim. |
| +#[derive(Debug, Clone, PartialEq, Eq, Default, Serialize)] |
| +pub struct Run { |
| + pub t: String, |
| + #[serde(skip_serializing_if = "std::ops::Not::not")] |
| + pub b: bool, |
| + #[serde(skip_serializing_if = "std::ops::Not::not")] |
| + pub i: bool, |
| + #[serde(skip_serializing_if = "std::ops::Not::not")] |
| + pub c: bool, |
| + #[serde(skip_serializing_if = "std::ops::Not::not")] |
| + pub h: bool, |
| + #[serde(skip_serializing_if = "Option::is_none")] |
| + pub href: Option<String>, |
| +} |
| + |
| +impl Run { |
| + pub fn same_marks(&self, other: &Run) -> bool { |
| + self.b == other.b |
| + && self.i == other.i |
| + && self.c == other.c |
| + && self.h == other.h |
| + && self.href == other.href |
| + } |
| +} |
| + |
| +pub type Paragraph = Vec<Run>; |
| + |
| +const MARK_KEYS: [&str; 4] = ["b", "i", "c", "h"]; |
| + |
| +fn run_attrs(run: &Run) -> Attrs { |
| + let mut attrs = Attrs::new(); |
| + for (key, on) in MARK_KEYS.iter().zip([run.b, run.i, run.c, run.h]) { |
| + if on { |
| + attrs.insert(Arc::from(*key), Any::Bool(true)); |
| + } |
| + } |
| + if let Some(href) = &run.href { |
| + attrs.insert(Arc::from("href"), Any::from(href.as_str())); |
| + } |
| + attrs |
| +} |
| + |
| +fn paragraph_prelim(paragraph: &[Run]) -> XmlElementPrelim { |
| + let delta: Vec<Delta<In>> = paragraph |
| + .iter() |
| + .filter(|run| !run.t.is_empty()) |
| + .map(|run| { |
| + let attrs = run_attrs(run); |
| + let attrs = if attrs.is_empty() { |
| + None |
| + } else { |
| + Some(Box::new(attrs)) |
| + }; |
| + Delta::Inserted(In::Any(Any::from(run.t.as_str())), attrs) |
| + }) |
| + .collect(); |
| + let text = XmlDeltaPrelim { |
| + attributes: HashMap::default(), |
| + delta, |
| + }; |
| + XmlElementPrelim::new("paragraph", [XmlIn::Text(text)]) |
| +} |
| + |
| +/// A fragment holding `text` in the §2.6 schema, ready to insert. |
| +pub fn text_prelim(text: &[Paragraph]) -> XmlFragmentPrelim { |
| + XmlFragmentPrelim::new::<_, ()>(text.iter().map(|p| XmlIn::Element(paragraph_prelim(p)))) |
| +} |
| + |
| +/// Replaces the whole content of `frag` with `text`. |
| +pub fn replace_text(txn: &mut TransactionMut, frag: &XmlFragmentRef, text: &[Paragraph]) { |
| + let len = frag.len(txn); |
| + if len > 0 { |
| + frag.remove_range(txn, 0, len); |
| + } |
| + for p in text { |
| + frag.push_back(txn, paragraph_prelim(p)); |
| + } |
| +} |
| + |
| +fn truthy(v: &Any) -> bool { |
| + !matches!(v, Any::Null | Any::Undefined | Any::Bool(false)) |
| +} |
| + |
| +fn push_runs<T: ReadTxn>(txn: &T, text: &yrs::XmlTextRef, out: &mut Paragraph) { |
| + for chunk in text.diff(txn, YChange::identity) { |
| + let Out::Any(Any::String(s)) = &chunk.insert else { |
| + continue; // an embed: nothing in this schema, nothing to show |
| + }; |
| + if s.is_empty() { |
| + continue; |
| + } |
| + let mut run = Run { |
| + t: s.to_string(), |
| + ..Run::default() |
| + }; |
| + if let Some(attrs) = &chunk.attributes { |
| + run.b = attrs.get("b").is_some_and(truthy); |
| + run.i = attrs.get("i").is_some_and(truthy); |
| + run.c = attrs.get("c").is_some_and(truthy); |
| + run.h = attrs.get("h").is_some_and(truthy); |
| + if let Some(Any::String(href)) = attrs.get("href") { |
| + run.href = Some(href.to_string()); |
| + } |
| + } |
| + match out.last_mut() { |
| + Some(last) if last.same_marks(&run) => last.t.push_str(&run.t), |
| + _ => out.push(run), |
| + } |
| + } |
| +} |
| + |
| +/// The paragraphs of a text fragment. |
| +pub fn read_text<T: ReadTxn>(txn: &T, frag: &XmlFragmentRef) -> Vec<Paragraph> { |
| + let mut paragraphs = Vec::new(); |
| + for child in frag.children(txn) { |
| + match child { |
| + XmlOut::Element(e) => { |
| + let mut p = Paragraph::new(); |
| + for inner in e.children(txn) { |
| + match inner { |
| + XmlOut::Text(t) => push_runs(txn, &t, &mut p), |
| + // An element inside a paragraph is outside this |
| + // schema; keep its plain text rather than lose it. |
| + other => { |
| + let s = other_string(txn, &other); |
| + if !s.is_empty() { |
| + p.push(Run { |
| + t: s, |
| + ..Run::default() |
| + }); |
| + } |
| + } |
| + } |
| + } |
| + paragraphs.push(p); |
| + } |
| + XmlOut::Text(t) => { |
| + let mut p = Paragraph::new(); |
| + push_runs(txn, &t, &mut p); |
| + paragraphs.push(p); |
| + } |
| + XmlOut::Fragment(f) => paragraphs.extend(read_text(txn, &f)), |
| + } |
| + } |
| + paragraphs |
| +} |
| + |
| +fn other_string<T: ReadTxn>(txn: &T, node: &XmlOut) -> String { |
| + match node { |
| + XmlOut::Element(e) => e.get_string(txn), |
| + XmlOut::Fragment(f) => f.get_string(txn), |
| + XmlOut::Text(t) => t.get_string(txn), |
| + } |
| +} |
| + |
| +// --- roots and lookups --- |
| + |
| +pub struct Roots { |
| + pub meta: MapRef, |
| + pub slides: ArrayRef, |
| + pub assets: MapRef, |
| +} |
| + |
| +/// The three root types. `get_or_insert_*` takes a write transaction of |
| +/// its own internally, so this must run BEFORE any transaction the caller |
| +/// opens (Docs' `snapshot.rs` records the self-deadlock otherwise). |
| +pub fn roots(doc: &Doc) -> Roots { |
| + Roots { |
| + meta: doc.get_or_insert_map(ROOT_META), |
| + slides: doc.get_or_insert_array(ROOT_SLIDES), |
| + assets: doc.get_or_insert_map(ROOT_ASSETS), |
| + } |
| +} |
| + |
| +pub fn get_string<T: ReadTxn>(txn: &T, map: &MapRef, key: &str) -> Option<String> { |
| + match map.get(txn, key)? { |
| + Out::Any(Any::String(s)) => Some(s.to_string()), |
| + _ => None, |
| + } |
| +} |
| + |
| +pub fn get_number<T: ReadTxn>(txn: &T, map: &MapRef, key: &str) -> Option<f64> { |
| + match map.get(txn, key)? { |
| + Out::Any(Any::Number(n)) => Some(n), |
| + Out::Any(Any::BigInt(n)) => Some(n as f64), |
| + _ => None, |
| + } |
| +} |
| + |
| +pub fn get_map<T: ReadTxn>(txn: &T, map: &MapRef, key: &str) -> Option<MapRef> { |
| + match map.get(txn, key)? { |
| + Out::YMap(m) => Some(m), |
| + _ => None, |
| + } |
| +} |
| + |
| +pub fn get_array<T: ReadTxn>(txn: &T, map: &MapRef, key: &str) -> Option<ArrayRef> { |
| + match map.get(txn, key)? { |
| + Out::YArray(a) => Some(a), |
| + _ => None, |
| + } |
| +} |
| + |
| +pub fn get_fragment<T: ReadTxn>(txn: &T, map: &MapRef, key: &str) -> Option<XmlFragmentRef> { |
| + match map.get(txn, key)? { |
| + Out::YXmlFragment(f) => Some(f), |
| + _ => None, |
| + } |
| +} |
| + |
| +/// The maps of an array of maps, in order, with their indices. Entries |
| +/// that are not maps (nothing this crate writes, but an update from a |
| +/// browser could) are skipped rather than trapped on. |
| +pub fn maps_of<T: ReadTxn>(txn: &T, array: &ArrayRef) -> Vec<(u32, MapRef)> { |
| + array |
| + .iter(txn) |
| + .enumerate() |
| + .filter_map(|(i, out)| match out { |
| + Out::YMap(m) => Some((i as u32, m)), |
| + _ => None, |
| + }) |
| + .collect() |
| +} |
| + |
| +/// The map in `array` whose `id` is `id`, with its index. |
| +pub fn find_by_id<T: ReadTxn>(txn: &T, array: &ArrayRef, id: &str) -> Option<(u32, MapRef)> { |
| + maps_of(txn, array) |
| + .into_iter() |
| + .find(|(_, m)| get_string(txn, m, "id").as_deref() == Some(id)) |
| +} |
| + |
| +// --- JSON <-> yrs values --- |
| + |
| +/// A plain JSON value as a yrs `Any`. Objects and arrays become plain |
| +/// nested values, never shared types: this is for the leaves of the |
| +/// schema (`props` entries, `box`, `fonts`, an asset), which are edited |
| +/// whole. |
| +pub fn json_to_any(v: &serde_json::Value) -> Any { |
| + match v { |
| + serde_json::Value::Null => Any::Null, |
| + serde_json::Value::Bool(b) => Any::Bool(*b), |
| + serde_json::Value::Number(n) => { |
| + if let Some(i) = n.as_i64() { |
| + if i.unsigned_abs() < (1u64 << 53) { |
| + Any::Number(i as f64) |
| + } else { |
| + Any::BigInt(i) |
| + } |
| + } else { |
| + Any::Number(n.as_f64().unwrap_or(0.0)) |
| + } |
| + } |
| + serde_json::Value::String(s) => Any::from(s.as_str()), |
| + serde_json::Value::Array(items) => { |
| + Any::Array(items.iter().map(json_to_any).collect::<Vec<_>>().into()) |
| + } |
| + serde_json::Value::Object(entries) => Any::Map(Arc::new( |
| + entries |
| + .iter() |
| + .map(|(k, v)| (k.clone(), json_to_any(v))) |
| + .collect(), |
| + )), |
| + } |
| +} |
| + |
| +/// A JSON number for `f`: an integer when it is one (so `level: 1` reads |
| +/// back as `1`, not `1.0`), a float otherwise, null for NaN/inf. |
| +pub fn number_json(f: f64) -> serde_json::Value { |
| + if f.fract() == 0.0 && f.abs() < (1u64 << 53) as f64 { |
| + serde_json::Value::from(f as i64) |
| + } else { |
| + serde_json::Number::from_f64(f) |
| + .map(serde_json::Value::Number) |
| + .unwrap_or(serde_json::Value::Null) |
| + } |
| +} |
| + |
| +pub fn any_to_json(v: &Any) -> serde_json::Value { |
| + match v { |
| + Any::Null | Any::Undefined => serde_json::Value::Null, |
| + Any::Bool(b) => serde_json::Value::Bool(*b), |
| + Any::Number(n) => number_json(*n), |
| + Any::BigInt(n) => serde_json::Value::from(*n), |
| + Any::String(s) => serde_json::Value::String(s.to_string()), |
| + Any::Buffer(b) => { |
| + serde_json::Value::Array(b.iter().map(|x| serde_json::Value::from(*x)).collect()) |
| + } |
| + Any::Array(items) => serde_json::Value::Array(items.iter().map(any_to_json).collect()), |
| + Any::Map(entries) => serde_json::Value::Object( |
| + entries |
| + .iter() |
| + .map(|(k, v)| (k.clone(), any_to_json(v))) |
| + .collect(), |
| + ), |
| + } |
| +} |
| + |
| +/// Any value the document holds, as JSON. Shared types are rendered by |
| +/// content — a map as an object, an array as an array, a text fragment |
| +/// as its paragraphs — so the view need not know which of a key's |
| +/// possible representations a peer chose. |
| +pub fn out_to_json<T: ReadTxn>(txn: &T, out: &Out) -> serde_json::Value { |
| + match out { |
| + Out::Any(a) => any_to_json(a), |
| + Out::YMap(m) => map_to_json(txn, m), |
| + Out::YArray(a) => { |
| + serde_json::Value::Array(a.iter(txn).map(|item| out_to_json(txn, &item)).collect()) |
| + } |
| + Out::YXmlFragment(f) => serde_json::to_value(read_text(txn, f)).unwrap_or_default(), |
| + Out::YText(t) => serde_json::Value::String(t.get_string(txn)), |
| + Out::YXmlText(t) => serde_json::Value::String(t.get_string(txn)), |
| + Out::YXmlElement(e) => serde_json::Value::String(e.get_string(txn)), |
| + _ => serde_json::Value::Null, |
| + } |
| +} |
| + |
| +pub fn map_to_json<T: ReadTxn>(txn: &T, map: &MapRef) -> serde_json::Value { |
| + serde_json::Value::Object( |
| + map.iter(txn) |
| + .map(|(k, v)| (k.to_string(), out_to_json(txn, &v))) |
| + .collect(), |
| + ) |
| +} |
| + |
| +/// `Some(json)` when the value is a map (shared or plain), else `None`. |
| +pub fn get_object<T: ReadTxn>(txn: &T, map: &MapRef, key: &str) -> Option<serde_json::Value> { |
| + match map.get(txn, key)? { |
| + Out::YMap(m) => Some(map_to_json(txn, &m)), |
| + Out::Any(a @ Any::Map(_)) => Some(any_to_json(&a)), |
| + _ => None, |
| + } |
| +} |
| + |
| +// --- constructors --- |
| + |
| +pub fn string_in(s: &str) -> In { |
| + In::Any(Any::from(s)) |
| +} |
| + |
| +pub fn number_in(n: f64) -> In { |
| + In::Any(Any::Number(n)) |
| +} |
| + |
| +/// A Y.Map prelim whose entries are plain values from a JSON object. |
| +pub fn plain_map_prelim(entries: &serde_json::Map<String, serde_json::Value>) -> yrs::MapPrelim { |
| + entries |
| + .iter() |
| + .map(|(k, v)| (k.as_str(), In::Any(json_to_any(v)))) |
| + .collect() |
| +} |
| + |
| +/// A `{kind, value}` background as a Y.Map prelim. |
| +pub fn background_prelim(kind: &str, value: &str) -> yrs::MapPrelim { |
| + [("kind", string_in(kind)), ("value", string_in(value))] |
| + .into_iter() |
| + .collect() |
| +} |
| + |
| +/// An array of `n` empty text fragments — a table's cells. |
| +pub fn cells_prelim(n: usize) -> yrs::ArrayPrelim { |
| + (0..n) |
| + .map(|_| In::XmlFragment(XmlFragmentPrelim::default())) |
| + .collect() |
| +} |
| + |
| +/// A block map prelim. `text` is written for text-bearing types only, |
| +/// `cells` for tables only; flow blocks get `region`, canvas blocks |
| +/// `box` and `z` — the caller has already decided which. |
| +#[allow(clippy::too_many_arguments)] |
| +pub fn block_prelim( |
| + id: &str, |
| + kind: &str, |
| + props: &serde_json::Map<String, serde_json::Value>, |
| + region: Option<&str>, |
| + r#box: Option<&serde_json::Value>, |
| + z: Option<f64>, |
| + text: Option<&[Paragraph]>, |
| + cells: Option<usize>, |
| +) -> yrs::MapPrelim { |
| + let mut m: yrs::MapPrelim = [ |
| + ("id", string_in(id)), |
| + ("type", string_in(kind)), |
| + ("props", In::Map(plain_map_prelim(props))), |
| + ("phone", In::Map(yrs::MapPrelim::default())), |
| + ] |
| + .into_iter() |
| + .collect(); |
| + if let Some(r) = region { |
| + m.insert(Arc::from("region"), string_in(r)); |
| + } |
| + if let Some(b) = r#box { |
| + m.insert(Arc::from("box"), In::Any(json_to_any(b))); |
| + } |
| + if let Some(z) = z { |
| + m.insert(Arc::from("z"), number_in(z)); |
| + } |
| + if let Some(t) = text { |
| + m.insert(Arc::from("text"), In::XmlFragment(text_prelim(t))); |
| + } |
| + if let Some(n) = cells { |
| + m.insert(Arc::from("cells"), In::Array(cells_prelim(n))); |
| + } |
| + m |
| +} |
| + |
| +/// A slide map prelim with no blocks and empty notes. |
| +pub fn slide_prelim(id: &str, layout: &str, template: &str) -> yrs::MapPrelim { |
| + [ |
| + ("id", string_in(id)), |
| + ("layout", string_in(layout)), |
| + ("template", string_in(template)), |
| + ("phone", string_in("inherit")), |
| + ("transition", string_in("none")), |
| + ("background", In::Map(background_prelim("theme", ""))), |
| + ("notes", In::XmlFragment(XmlFragmentPrelim::default())), |
| + ("blocks", In::Array(yrs::ArrayPrelim::default())), |
| + ] |
| + .into_iter() |
| + .collect() |
| +} |
| + |
| +/// Rounds a box coordinate to the four decimals §2.3 specifies. |
| +pub fn round4(f: f64) -> f64 { |
| + (f * 10_000.0).round() / 10_000.0 |
| +} |
| + |
| +pub fn box_json(x: f64, y: f64, w: f64, h: f64) -> serde_json::Value { |
| + serde_json::json!({ |
| + "x": round4(x), |
| + "y": round4(y), |
| + "w": round4(w), |
| + "h": round4(h), |
| + }) |
| +} |
diff --git a/core/src/view.rs b/core/src/view.rs| new file mode 100644 |
| index 0000000..7b6ea53 |
| --- /dev/null |
| +++ b/core/src/view.rs |
| @@ -0,0 +1,168 @@ |
| +//! The JSON view of a deck: what `GET /api/v1/decks/{sqid}` returns and |
| +//! what the renderer, the thumbnailer and the PDF exporter read. Its |
| +//! shape is the contract in the task brief, verbatim: |
| +//! |
| +//! ```json |
| +//! {"meta":{"title":"","theme":"paper","accent":null,"fonts":null,"aspect":"16:9"}, |
| +//! "slides":[{"id":"…","layout":"flow","template":"title", |
| +//! "background":{"kind":"theme","value":""}, |
| +//! "phone":"inherit","transition":"none", |
| +//! "notes":[[{"t":"Say hello"}]], |
| +//! "blocks":[{"id":"…","type":"heading","props":{"level":1}, |
| +//! "region":"heading","box":null,"z":0,"phone":{}, |
| +//! "text":[[{"t":"Hello","b":true}]]}]}], |
| +//! "assets":{"id":{"ref":"…","name":"","width":0,"height":0,"alt":""}}} |
| +//! ``` |
| +//! |
| +//! Text is an array of paragraphs, a paragraph an array of runs |
| +//! `{t, b?, i?, c?, h?, href?}` with false and absent marks omitted. A |
| +//! block's `text` key is present for text-bearing types (any block that |
| +//! carries a `text` fragment) and absent otherwise; a `table`'s cells |
| +//! appear as `props.cells`, a row-major array of texts. |
| +//! |
| +//! Every key has a default so a deck a peer left half-written still |
| +//! renders as something rather than failing to serialise: an absent |
| +//! `layout` is `flow`, an absent `template` is `blank`, an absent `z` |
| +//! is 0. Key order within `props`, `assets` and other free-form objects |
| +//! is sorted (serde_json's default map), so the output is deterministic |
| +//! and two documents with the same content render byte-identically — |
| +//! which is what the update round-trip tests compare. |
| + |
| +use crate::schema::{self, Paragraph}; |
| +use serde::Serialize; |
| +use serde_json::Value; |
| +use std::collections::BTreeMap; |
| +use yrs::{Array, Doc, Map, ReadTxn, Transact}; |
| + |
| +#[derive(Serialize)] |
| +struct View { |
| + meta: Meta, |
| + slides: Vec<Slide>, |
| + assets: BTreeMap<String, Value>, |
| +} |
| + |
| +#[derive(Serialize)] |
| +struct Meta { |
| + title: String, |
| + theme: String, |
| + accent: Option<String>, |
| + fonts: Option<Value>, |
| + aspect: String, |
| +} |
| + |
| +#[derive(Serialize)] |
| +struct Slide { |
| + id: String, |
| + layout: String, |
| + template: String, |
| + background: Value, |
| + phone: String, |
| + transition: String, |
| + notes: Vec<Paragraph>, |
| + blocks: Vec<Block>, |
| +} |
| + |
| +#[derive(Serialize)] |
| +struct Block { |
| + id: String, |
| + #[serde(rename = "type")] |
| + kind: String, |
| + props: Value, |
| + region: Option<String>, |
| + #[serde(rename = "box")] |
| + bx: Option<Value>, |
| + z: Value, |
| + phone: Value, |
| + #[serde(skip_serializing_if = "Option::is_none")] |
| + text: Option<Vec<Paragraph>>, |
| +} |
| + |
| +fn empty_object() -> Value { |
| + Value::Object(serde_json::Map::new()) |
| +} |
| + |
| +fn block_view<T: ReadTxn>(txn: &T, m: &yrs::MapRef) -> Block { |
| + let mut props = schema::get_object(txn, m, "props").unwrap_or_else(empty_object); |
| + if let Some(cells) = schema::get_array(txn, m, "cells") { |
| + let texts: Vec<Value> = cells |
| + .iter(txn) |
| + .map(|c| match c { |
| + yrs::Out::YXmlFragment(f) => { |
| + serde_json::to_value(schema::read_text(txn, &f)).unwrap_or_default() |
| + } |
| + other => schema::out_to_json(txn, &other), |
| + }) |
| + .collect(); |
| + if let Value::Object(p) = &mut props { |
| + p.insert("cells".into(), Value::Array(texts)); |
| + } |
| + } |
| + Block { |
| + id: schema::get_string(txn, m, "id").unwrap_or_default(), |
| + kind: schema::get_string(txn, m, "type").unwrap_or_default(), |
| + props, |
| + region: schema::get_string(txn, m, "region"), |
| + bx: schema::get_object(txn, m, "box"), |
| + z: schema::number_json(schema::get_number(txn, m, "z").unwrap_or(0.0)), |
| + phone: schema::get_object(txn, m, "phone").unwrap_or_else(empty_object), |
| + text: schema::get_fragment(txn, m, "text").map(|f| schema::read_text(txn, &f)), |
| + } |
| +} |
| + |
| +fn slide_view<T: ReadTxn>(txn: &T, m: &yrs::MapRef) -> Slide { |
| + let blocks = schema::get_array(txn, m, "blocks") |
| + .map(|a| { |
| + schema::maps_of(txn, &a) |
| + .iter() |
| + .map(|(_, b)| block_view(txn, b)) |
| + .collect() |
| + }) |
| + .unwrap_or_default(); |
| + Slide { |
| + id: schema::get_string(txn, m, "id").unwrap_or_default(), |
| + layout: schema::get_string(txn, m, "layout").unwrap_or_else(|| "flow".into()), |
| + template: schema::get_string(txn, m, "template") |
| + .unwrap_or_else(|| schema::DEFAULT_TEMPLATE.into()), |
| + background: schema::get_object(txn, m, "background") |
| + .unwrap_or_else(|| serde_json::json!({"kind": "theme", "value": ""})), |
| + phone: schema::get_string(txn, m, "phone").unwrap_or_else(|| "inherit".into()), |
| + transition: schema::get_string(txn, m, "transition").unwrap_or_else(|| "none".into()), |
| + notes: schema::get_fragment(txn, m, "notes") |
| + .map(|f| schema::read_text(txn, &f)) |
| + .unwrap_or_default(), |
| + blocks, |
| + } |
| +} |
| + |
| +/// The deck as JSON. Never empty: an untouched document renders as the |
| +/// defaults with no slides and no assets. |
| +pub fn view_json(doc: &Doc) -> String { |
| + let roots = schema::roots(doc); |
| + let txn = doc.transact(); |
| + let meta = Meta { |
| + title: schema::get_string(&txn, &roots.meta, "title").unwrap_or_default(), |
| + theme: schema::get_string(&txn, &roots.meta, "theme") |
| + .unwrap_or_else(|| schema::DEFAULT_THEME.into()), |
| + accent: schema::get_string(&txn, &roots.meta, "accent"), |
| + fonts: schema::get_object(&txn, &roots.meta, "fonts"), |
| + aspect: schema::get_string(&txn, &roots.meta, "aspect") |
| + .unwrap_or_else(|| schema::DEFAULT_ASPECT.into()), |
| + }; |
| + let slides = schema::maps_of(&txn, &roots.slides) |
| + .iter() |
| + .map(|(_, m)| slide_view(&txn, m)) |
| + .collect(); |
| + let assets = roots |
| + .assets |
| + .iter(&txn) |
| + .map(|(k, v)| (k.to_string(), schema::out_to_json(&txn, &v))) |
| + .collect(); |
| + let view = View { |
| + meta, |
| + slides, |
| + assets, |
| + }; |
| + // Serialising a struct of strings, numbers and already-built Values |
| + // cannot fail; the fallback is belt and braces, not a path. |
| + serde_json::to_string(&view).unwrap_or_else(|_| String::from("{}")) |
| +} |
diff --git a/core/tests/no_host_imports.rs b/core/tests/no_host_imports.rs| new file mode 100644 |
| index 0000000..027add4 |
| --- /dev/null |
| +++ b/core/tests/no_host_imports.rs |
| @@ -0,0 +1,56 @@ |
| +use std::path::Path; |
| + |
| +#[test] |
| +fn no_host_imports() { |
| + let wasm_path = "target/wasm32-unknown-unknown/release/slides_core.wasm"; |
| + |
| + // SLIDES_REQUIRE_WASM=1 is set by the CI gate, which always builds the |
| + // wasm artifact before running this test — so under the gate a missing |
| + // artifact is a real failure, not something to shrug off. Left unset, |
| + // a bare `cargo test` on a fresh clone (no wasm build yet) still skips, |
| + // so local iteration on the native tests isn't forced to build wasm |
| + // first. |
| + let require_wasm = std::env::var("SLIDES_REQUIRE_WASM").as_deref() == Ok("1"); |
| + |
| + if !Path::new(wasm_path).exists() { |
| + if require_wasm { |
| + panic!( |
| + "FAILED (SLIDES_REQUIRE_WASM=1): wasm artifact not found at {}. \ |
| + The gate must build it before running this test.", |
| + wasm_path |
| + ); |
| + } |
| + eprintln!( |
| + "SKIPPED (SLIDES_REQUIRE_WASM unset): wasm artifact not found at {}. \ |
| + Run ./scripts/build-wasm.sh first.", |
| + wasm_path |
| + ); |
| + return; |
| + } |
| + |
| + let wasm_bytes = std::fs::read(wasm_path).expect("Failed to read wasm file"); |
| + let parser = wasmparser::Parser::new(0); |
| + |
| + let mut imports = Vec::new(); |
| + |
| + for payload in parser.parse_all(&wasm_bytes) { |
| + let payload = payload.expect("Failed to parse wasm module"); |
| + |
| + if let wasmparser::Payload::ImportSection(reader) = payload { |
| + let import_reader = reader.into_imports(); |
| + for import_result in import_reader { |
| + let import = import_result.expect("Failed to parse import"); |
| + imports.push(format!("{}.{}", import.module, import.name)); |
| + } |
| + } |
| + } |
| + |
| + if !imports.is_empty() { |
| + panic!( |
| + "WASM module has host imports (breaks Go/wazero compatibility):\n {}", |
| + imports.join("\n ") |
| + ); |
| + } |
| + |
| + eprintln!("PASSED (checked {} bytes, 0 imports)", wasm_bytes.len()); |
| +} |
diff --git a/core/tests/ops.rs b/core/tests/ops.rs| new file mode 100644 |
| index 0000000..953c029 |
| --- /dev/null |
| +++ b/core/tests/ops.rs |
| @@ -0,0 +1,387 @@ |
| +//! The operations API end to end: a JSON batch in, a Yjs update out, and |
| +//! the view JSON as the witness of what changed. |
| + |
| +use slides_core::{apply_ops, new_doc, view_json}; |
| +use yrs::Doc; |
| + |
| +fn doc(n: u64) -> Doc { |
| + new_doc(&format!("test-guid-{n}"), n) |
| +} |
| + |
| +fn view(doc: &Doc) -> serde_json::Value { |
| + serde_json::from_str(&view_json(doc)).expect("view is JSON") |
| +} |
| + |
| +#[test] |
| +fn add_slide_then_view_lists_it() { |
| + let d = doc(1); |
| + let ops = r#"[ |
| + {"op":"set_meta","title":"T"}, |
| + {"op":"add_slide","id":"aaaaaaaaaaaa","layout":"flow","template":"title"} |
| + ]"#; |
| + apply_ops(&d, ops).expect("ops apply"); |
| + let v = view(&d); |
| + assert_eq!(v["meta"]["title"], "T"); |
| + assert_eq!(v["meta"]["theme"], "paper"); |
| + assert_eq!(v["meta"]["aspect"], "16:9"); |
| + let slides = v["slides"].as_array().expect("slides array"); |
| + assert_eq!(slides.len(), 1); |
| + assert_eq!(slides[0]["id"], "aaaaaaaaaaaa"); |
| + assert_eq!(slides[0]["template"], "title"); |
| + assert_eq!(slides[0]["layout"], "flow"); |
| + assert_eq!(slides[0]["blocks"], serde_json::json!([])); |
| + assert_eq!(slides[0]["notes"], serde_json::json!([])); |
| +} |
| + |
| +#[test] |
| +fn add_block_with_markdown_produces_runs() { |
| + let d = doc(2); |
| + let ops = r#"[ |
| + {"op":"add_slide","id":"aaaaaaaaaaaa","layout":"flow","template":"title"}, |
| + {"op":"add_block","slide":"aaaaaaaaaaaa","id":"bbbbbbbbbbbb","type":"heading", |
| + "props":{"level":1},"region":"heading","markdown":"Hello **world**"} |
| + ]"#; |
| + apply_ops(&d, ops).expect("ops apply"); |
| + let v = view(&d); |
| + let block = &v["slides"][0]["blocks"][0]; |
| + assert_eq!(block["id"], "bbbbbbbbbbbb"); |
| + assert_eq!(block["type"], "heading"); |
| + assert_eq!(block["props"]["level"], 1); |
| + assert_eq!(block["region"], "heading"); |
| + assert_eq!(block["box"], serde_json::Value::Null); |
| + assert_eq!( |
| + block["text"], |
| + serde_json::json!([[{"t": "Hello "}, {"t": "world", "b": true}]]) |
| + ); |
| +} |
| + |
| +#[test] |
| +fn ops_are_atomic() { |
| + let d = doc(3); |
| + let before = view_json(&d); |
| + let ops = r#"[ |
| + {"op":"add_slide","id":"aaaaaaaaaaaa","layout":"flow"}, |
| + {"op":"frobnicate","id":"aaaaaaaaaaaa"} |
| + ]"#; |
| + let err = apply_ops(&d, ops).expect_err("unknown op is an error"); |
| + assert!( |
| + err.to_string().contains("frobnicate"), |
| + "message names the op: {err}" |
| + ); |
| + assert_eq!(view_json(&d), before, "a failed batch applies nothing"); |
| + |
| + // A missing id is the same kind of failure: nothing applied. |
| + let ops = r#"[{"op":"add_slide","layout":"flow"}]"#; |
| + let err = apply_ops(&d, ops).expect_err("missing id is an error"); |
| + assert!( |
| + err.to_string().contains("id"), |
| + "message names the field: {err}" |
| + ); |
| + assert_eq!(view_json(&d), before); |
| + |
| + // A reference that only becomes wrong mid-batch is caught before |
| + // anything is written too: the first op is valid on its own. |
| + let ops = r#"[ |
| + {"op":"add_slide","id":"aaaaaaaaaaaa","layout":"flow"}, |
| + {"op":"remove_block","slide":"aaaaaaaaaaaa","id":"nope"} |
| + ]"#; |
| + apply_ops(&d, ops).expect_err("unknown block is an error"); |
| + assert_eq!(view_json(&d), before); |
| +} |
| + |
| +#[test] |
| +fn layout_switch_assigns_boxes() { |
| + let d = doc(4); |
| + let ops = r#"[ |
| + {"op":"add_slide","id":"aaaaaaaaaaaa","layout":"flow","template":"title-body"}, |
| + {"op":"add_block","slide":"aaaaaaaaaaaa","id":"bbbbbbbbbbbb","type":"heading", |
| + "props":{"level":1},"region":"heading","markdown":"Title"}, |
| + {"op":"add_block","slide":"aaaaaaaaaaaa","id":"cccccccccccc","type":"text", |
| + "props":{"size":"m"},"region":"body","markdown":"Body"} |
| + ]"#; |
| + apply_ops(&d, ops).expect("ops apply"); |
| + |
| + apply_ops( |
| + &d, |
| + r#"[{"op":"set_slide","id":"aaaaaaaaaaaa","layout":"canvas"}]"#, |
| + ) |
| + .expect("switch to canvas"); |
| + let v = view(&d); |
| + let slide = &v["slides"][0]; |
| + assert_eq!(slide["layout"], "canvas"); |
| + let blocks = slide["blocks"].as_array().unwrap(); |
| + assert_eq!(blocks.len(), 2); |
| + for b in blocks { |
| + assert!(b["box"].is_object(), "canvas block has a box: {b}"); |
| + assert!(b["region"].is_null(), "canvas block has no region: {b}"); |
| + } |
| + assert_eq!( |
| + blocks[0]["box"], |
| + serde_json::json!({"x":0.06,"y":0.08,"w":0.88,"h":0.18}) |
| + ); |
| + assert_eq!( |
| + blocks[1]["box"], |
| + serde_json::json!({"x":0.06,"y":0.30,"w":0.88,"h":0.62}) |
| + ); |
| + assert_eq!(blocks[0]["z"], 0); |
| + assert_eq!(blocks[1]["z"], 1); |
| + |
| + apply_ops( |
| + &d, |
| + r#"[{"op":"set_slide","id":"aaaaaaaaaaaa","layout":"flow"}]"#, |
| + ) |
| + .expect("switch back to flow"); |
| + let v = view(&d); |
| + let slide = &v["slides"][0]; |
| + assert_eq!(slide["layout"], "flow"); |
| + let blocks = slide["blocks"].as_array().unwrap(); |
| + assert_eq!(blocks.len(), 2); |
| + for b in blocks { |
| + assert!(b["box"].is_null(), "flow block has no box: {b}"); |
| + assert!(b["region"].is_string(), "flow block has a region: {b}"); |
| + } |
| + assert_eq!(blocks[0]["id"], "bbbbbbbbbbbb"); |
| + assert_eq!(blocks[0]["region"], "heading"); |
| + assert_eq!(blocks[1]["id"], "cccccccccccc"); |
| + assert_eq!(blocks[1]["region"], "body"); |
| +} |
| + |
| +#[test] |
| +fn update_roundtrip() { |
| + let a = doc(5); |
| + let b = doc(6); |
| + let ops = r##"[ |
| + {"op":"set_meta","title":"Roundtrip","theme":"ink","accent":"#ff0000"}, |
| + {"op":"add_slide","id":"aaaaaaaaaaaa","layout":"flow","template":"title-body"}, |
| + {"op":"add_block","slide":"aaaaaaaaaaaa","id":"bbbbbbbbbbbb","type":"heading", |
| + "props":{"level":1},"region":"heading","markdown":"Hello *there* [link](https://x.example)"}, |
| + {"op":"add_block","slide":"aaaaaaaaaaaa","id":"cccccccccccc","type":"list", |
| + "props":{"style":"bullet"},"region":"body","markdown":"- one\n- two ==hi=="}, |
| + {"op":"add_block","slide":"aaaaaaaaaaaa","id":"dddddddddddd","type":"table", |
| + "props":{"rows":2,"cols":2},"region":"body"}, |
| + {"op":"set_notes","id":"aaaaaaaaaaaa","markdown":"Say `hello`"}, |
| + {"op":"register_asset","id":"asset1","ref":"blobs/1","name":"pic.png","width":10,"height":20,"alt":"a picture"} |
| + ]"##; |
| + let out = apply_ops(&a, ops).expect("ops apply"); |
| + assert!(!out.is_empty(), "an applied batch yields an update"); |
| + slides_core::apply_update(&b, &out).expect("update applies to B"); |
| + assert_eq!(view_json(&a), view_json(&b)); |
| + |
| + // And the view really carries what was written. |
| + let v = view(&a); |
| + assert_eq!(v["meta"]["accent"], "#ff0000"); |
| + assert_eq!( |
| + v["slides"][0]["blocks"][0]["text"], |
| + serde_json::json!([[ |
| + {"t": "Hello "}, |
| + {"t": "there", "i": true}, |
| + {"t": " "}, |
| + {"t": "link", "href": "https://x.example"} |
| + ]]) |
| + ); |
| + assert_eq!( |
| + v["slides"][0]["blocks"][1]["text"], |
| + serde_json::json!([[{"t": "one"}], [{"t": "two "}, {"t": "hi", "h": true}]]) |
| + ); |
| + let cells = v["slides"][0]["blocks"][2]["props"]["cells"] |
| + .as_array() |
| + .unwrap(); |
| + assert_eq!(cells.len(), 4); |
| + assert_eq!( |
| + v["slides"][0]["notes"], |
| + serde_json::json!([[{"t": "Say "}, {"t": "hello", "c": true}]]) |
| + ); |
| + assert_eq!(v["assets"]["asset1"]["ref"], "blobs/1"); |
| + assert_eq!(v["assets"]["asset1"]["width"], 10); |
| +} |
| + |
| +/// The ops the tests above do not reach, each checked through the view |
| +/// and through a second document fed only the returned update. |
| +#[test] |
| +fn remaining_ops_roundtrip() { |
| + let a = doc(7); |
| + let b = doc(8); |
| + let mut updates = Vec::new(); |
| + let mut run = |ops: &str| { |
| + let out = apply_ops(&a, ops).unwrap_or_else(|e| panic!("{ops}: {e}")); |
| + updates.push(out); |
| + }; |
| + run(r##"[ |
| + {"op":"add_slide","id":"s1","layout":"flow","template":"title"}, |
| + {"op":"add_slide","id":"s2","layout":"canvas"}, |
| + {"op":"add_slide","id":"s0","layout":"flow","after":null}, |
| + {"op":"add_block","slide":"s1","id":"b1","type":"heading","markdown":"One"}, |
| + {"op":"add_block","slide":"s1","id":"b2","type":"code","props":{"language":"go"},"markdown":"x := 1\n\ny := **2**"}, |
| + {"op":"add_block","slide":"s1","id":"b3","type":"columns","props":{"count":2,"children":["b1","b2"]}}, |
| + {"op":"add_block","slide":"s1","id":"b0","type":"divider","after":null}, |
| + {"op":"add_block","slide":"s2","id":"c1","type":"text","markdown":"Canvas"}, |
| + {"op":"add_block","slide":"s2","id":"c2","type":"image","props":{"asset":"x","alt":"y"},"box":{"x":0.5,"y":0.5,"w":0.25,"h":0.25},"z":7} |
| + ]"##); |
| + let v = view(&a); |
| + let ids: Vec<_> = v["slides"] |
| + .as_array() |
| + .unwrap() |
| + .iter() |
| + .map(|s| s["id"].clone()) |
| + .collect(); |
| + assert_eq!( |
| + ids, |
| + serde_json::json!(["s0", "s1", "s2"]) |
| + .as_array() |
| + .unwrap() |
| + .clone() |
| + ); |
| + let s1 = &v["slides"][1]; |
| + let bids: Vec<_> = s1["blocks"] |
| + .as_array() |
| + .unwrap() |
| + .iter() |
| + .map(|b| b["id"].clone()) |
| + .collect(); |
| + assert_eq!(bids, vec!["b0", "b1", "b2", "b3"]); |
| + assert_eq!( |
| + s1["blocks"][1]["region"], "heading", |
| + "heading defaults to its region" |
| + ); |
| + assert_eq!( |
| + s1["blocks"][2]["text"], |
| + serde_json::json!([[{"t": "x := 1"}], [], [{"t": "y := **2**"}]]), |
| + "code is one plain paragraph per line" |
| + ); |
| + assert!( |
| + s1["blocks"][3].get("text").is_none(), |
| + "columns carry no text" |
| + ); |
| + let s2 = &v["slides"][2]; |
| + assert_eq!( |
| + s2["blocks"][0]["box"], |
| + serde_json::json!({"x":0.06,"y":0.30,"w":0.88,"h":0.62}) |
| + ); |
| + assert_eq!(s2["blocks"][0]["z"], 0); |
| + assert_eq!( |
| + s2["blocks"][1]["box"], |
| + serde_json::json!({"x":0.5,"y":0.5,"w":0.25,"h":0.25}) |
| + ); |
| + assert_eq!(s2["blocks"][1]["z"], 7); |
| + |
| + run(r##"[ |
| + {"op":"move_slide","id":"s2","after":null}, |
| + {"op":"move_block","slide":"s1","id":"b0","after":"b3"}, |
| + {"op":"duplicate_slide","id":"s1","new_id":"s1copy","new_block_ids":["d1","d2","d3","d4"]}, |
| + {"op":"set_slide","id":"s0","template":"section","background":{"kind":"color","value":"#123456"},"phone":"reflow","transition":"fade"}, |
| + {"op":"set_notes","id":"s0","markdown":"Notes\n\nMore *notes*"}, |
| + {"op":"set_meta","fonts":{"heading":"Inter","body":"Georgia"}}, |
| + {"op":"register_asset","id":"a1","ref":"r","name":"n","width":1,"height":2,"alt":"z"} |
| + ]"##); |
| + let v = view(&a); |
| + let ids: Vec<_> = v["slides"] |
| + .as_array() |
| + .unwrap() |
| + .iter() |
| + .map(|s| s["id"].clone()) |
| + .collect(); |
| + assert_eq!(ids, vec!["s2", "s0", "s1", "s1copy"]); |
| + let s1 = &v["slides"][2]; |
| + let bids: Vec<_> = s1["blocks"] |
| + .as_array() |
| + .unwrap() |
| + .iter() |
| + .map(|b| b["id"].clone()) |
| + .collect(); |
| + assert_eq!(bids, vec!["b1", "b2", "b3", "b0"]); |
| + let copy = &v["slides"][3]; |
| + let cids: Vec<_> = copy["blocks"] |
| + .as_array() |
| + .unwrap() |
| + .iter() |
| + .map(|b| b["id"].clone()) |
| + .collect(); |
| + assert_eq!(cids, vec!["d1", "d2", "d3", "d4"]); |
| + assert_eq!( |
| + copy["blocks"][0]["text"], |
| + serde_json::json!([[{"t": "One"}]]), |
| + "the copy keeps its text" |
| + ); |
| + assert_eq!( |
| + copy["blocks"][2]["props"]["children"], |
| + serde_json::json!(["d1", "d2"]), |
| + "columns children follow the new ids" |
| + ); |
| + let s0 = &v["slides"][1]; |
| + assert_eq!(s0["template"], "section"); |
| + assert_eq!( |
| + s0["background"], |
| + serde_json::json!({"kind":"color","value":"#123456"}) |
| + ); |
| + assert_eq!(s0["phone"], "reflow"); |
| + assert_eq!(s0["transition"], "fade"); |
| + assert_eq!( |
| + s0["notes"], |
| + serde_json::json!([[{"t":"Notes"}],[{"t":"More "},{"t":"notes","i":true}]]) |
| + ); |
| + assert_eq!( |
| + v["meta"]["fonts"], |
| + serde_json::json!({"heading":"Inter","body":"Georgia"}) |
| + ); |
| + |
| + run(r##"[ |
| + {"op":"add_block","slide":"s0","id":"t1","type":"table","props":{"rows":2,"cols":3,"header":true}}, |
| + {"op":"set_block","slide":"s0","id":"t1","props":{"rows":3,"header":null},"region":"aside"}, |
| + {"op":"set_text","slide":"s1","id":"b1","markdown":"Changed"}, |
| + {"op":"set_phone","slide":"s1","id":"b1","phone":{"hidden":true,"order":2,"size":"s"}}, |
| + {"op":"set_phone","slide":"s1","id":"b2","phone":{}}, |
| + {"op":"remove_block","slide":"s1","id":"b3"}, |
| + {"op":"remove_slide","id":"s1copy"}, |
| + {"op":"remove_asset","id":"a1"}, |
| + {"op":"set_meta","accent":null,"fonts":null} |
| + ]"##); |
| + let v = view(&a); |
| + let t1 = &v["slides"][1]["blocks"][0]; |
| + assert_eq!(t1["props"]["rows"], 3); |
| + assert_eq!(t1["props"]["cols"], 3); |
| + assert!(t1["props"].get("header").is_none(), "null removes a prop"); |
| + assert_eq!( |
| + t1["props"]["cells"].as_array().unwrap().len(), |
| + 9, |
| + "cells follow rows*cols" |
| + ); |
| + assert_eq!(t1["region"], "aside"); |
| + let s1 = &v["slides"][2]; |
| + assert_eq!( |
| + s1["blocks"][0]["text"], |
| + serde_json::json!([[{"t": "Changed"}]]) |
| + ); |
| + assert_eq!( |
| + s1["blocks"][0]["phone"], |
| + serde_json::json!({"hidden":true,"order":2,"size":"s"}) |
| + ); |
| + assert_eq!(s1["blocks"][1]["phone"], serde_json::json!({})); |
| + assert_eq!(s1["blocks"].as_array().unwrap().len(), 3); |
| + assert_eq!(v["slides"].as_array().unwrap().len(), 3); |
| + assert_eq!(v["assets"], serde_json::json!({})); |
| + assert_eq!(v["meta"]["accent"], serde_json::Value::Null); |
| + assert_eq!(v["meta"]["fonts"], serde_json::Value::Null); |
| + |
| + // Refusals the model catches. |
| + for bad in [ |
| + r#"[{"op":"add_slide","id":"s0","layout":"flow"}]"#, |
| + r#"[{"op":"add_slide","id":"new","layout":"sideways"}]"#, |
| + r#"[{"op":"set_slide","id":"s0","template":"nope"}]"#, |
| + r#"[{"op":"set_text","slide":"s0","id":"t1","markdown":"x"}]"#, |
| + r#"[{"op":"duplicate_slide","id":"s0","new_id":"n","new_block_ids":[]}]"#, |
| + r#"[{"op":"set_phone","slide":"s1","id":"b1","phone":{"colour":"red"}}]"#, |
| + r#"[{"op":"set_block","slide":"s0","id":"t1","props":{"rows":0}}]"#, |
| + r#"[{"op":"move_block","slide":"s1","id":"b1","after":"b1"}]"#, |
| + r#"[{"op":"remove_asset","id":"a1"}]"#, |
| + r#"{"op":"set_meta"}"#, |
| + ] { |
| + let before = view_json(&a); |
| + apply_ops(&a, bad).expect_err(bad); |
| + assert_eq!(view_json(&a), before, "{bad}"); |
| + } |
| + |
| + for u in &updates { |
| + slides_core::apply_update(&b, u).expect("update applies"); |
| + } |
| + assert_eq!(view_json(&a), view_json(&b)); |
| +} |
diff --git a/core/vendor/README.md b/core/vendor/README.md| new file mode 100644 |
| index 0000000..45248a5 |
| --- /dev/null |
| +++ b/core/vendor/README.md |
| @@ -0,0 +1,120 @@ |
| +# Vendored dependency patches |
| + |
| +## fastrand-2.5.0 |
| + |
| +### The problem |
| + |
| +`yrs` (pinned exact in `core/Cargo.toml`) depends on `fastrand` with its |
| +`js` feature enabled unconditionally, and fastrand's own manifest ties that |
| +feature, on `wasm32`/`wasm64`-`unknown`, to `getrandom` with the `wasm_js` |
| +feature forced on. That feature makes `getrandom` default to a |
| +`wasm-bindgen`-based backend which imports host JS functions |
| +(`crypto.getRandomValues`, etc.) — an import the Go/wazero host cannot |
| +satisfy, and the thing `core/tests/no_host_imports.rs` gates against. |
| + |
| +`yrs::Doc` calls into this path unconditionally on construction (even via |
| +`Doc::with_client_id`, which still generates a random document GUID via |
| +`fastrand::Rng::new()` -> `getrandom`). Worse: merely *referencing* |
| +`yrs::Doc` anywhere in `slides-core`'s public API — required for |
| +`apply_update`/`text_content`, since the test suite calls them from a |
| +separate compilation unit against the `rlib` — is enough for those |
| +`wasm-bindgen` exports to end up linked into the final `.wasm` module, even |
| +along code paths that are never reached at runtime. Reproduced in |
| +isolation (a scratch crate depending on nothing but `yrs = "=0.23.5"`, with |
| +a single `pub fn unused(_doc: &yrs::Doc) {}` and no method calls at all); |
| +see `task-3-report.md` in the plan directory for the full trace. |
| + |
| +### The fix, and why it needs vendoring |
| + |
| +`getrandom` supports exactly this scenario: selecting its `custom` backend |
| +via `--cfg getrandom_backend="custom"` (set in `slides/.cargo/ |
| +config.toml`, scoped to the `wasm32-unknown-unknown` target only) routes |
| +entropy requests to a symbol we provide ourselves |
| +(`__getrandom_v03_custom` in `core/src/lib.rs`, which — deliberately — |
| +fails rather than fabricating bytes; see the doc comment above it), with |
| +no JS dependency. |
| + |
| +That override alone doesn't compile, though. `getrandom-0.4.3`'s |
| +`src/error.rs:181-185` has a `Display`/`Debug` match arm: |
| + |
| +```rust |
| +#[cfg(all( |
| + feature = "wasm_js", |
| + target_arch = "wasm32", |
| + any(target_os = "unknown", target_os = "none") |
| +))] |
| +Error::WEB_CRYPTO => "Web Crypto API is unavailable", |
| +``` |
| + |
| +gated only on the `wasm_js` Cargo *feature* — not on which backend is |
| +actually *selected* via `--cfg getrandom_backend`. `Error::WEB_CRYPTO` |
| +itself is defined inside `src/backends/wasm_js.rs:71`, inside the `mod |
| +wasm_js;` declaration that a `getrandom_backend="custom"` override skips |
| +compiling entirely (`src/backends.rs`'s `cfg_if!` chain is mutually |
| +exclusive; `custom` is checked first and short-circuits the rest). Result, |
| +reproduced directly against the pinned version with nothing changed except |
| +adding the cfg override: |
| + |
| +``` |
| +error[E0599]: no associated function or constant named `WEB_CRYPTO` found for struct `error::Error` in the current scope |
| + --> .../getrandom-0.4.3/src/error.rs:185:20 |
| +``` |
| + |
| +As long as *anything* in the dependency graph requests fastrand's `js` |
| +feature on a wasm32/wasm64-unknown target — which `yrs` does, |
| +unconditionally, and which we cannot override, since Cargo features are |
| +additive across the whole graph and `slides-core` has no way to un-request a |
| +feature another crate requires — this is a hard compile error, regardless |
| +of which non-`wasm_js` backend `--cfg getrandom_backend` selects (`custom`, |
| +`unsupported`, etc. all hit the same match arm). This looks like a real |
| +gap in `getrandom` 0.4.3's own cfg-gating (the match arm's cfg should also |
| +require `not(getrandom_backend = "custom")` or equivalent, matching what |
| +the backend-selection `cfg_if!` already does) rather than anything specific |
| +to this project; `getrandom`'s own issue tracker has at least one adjacent, |
| +still-open report of the same problem area — a `custom`-backend override |
| +not fully insulating a build from a `wasm_js` feature forced on by another |
| +dependency in the graph — closed "not planned": |
| +<https://github.com/rust-random/getrandom/issues/633>. We did not find an |
| +issue number for this exact `E0599`/`WEB_CRYPTO` symptom; if one turns up, |
| +add it here. |
| + |
| +### What was changed |
| + |
| +This vendored copy of `fastrand-2.5.0` is the unmodified upstream Rust |
| +source (`src/`, `benches/`, `tests/`, licenses — byte-identical to the |
| +crates.io tarball); only `Cargo.toml` differs, by exactly one edit: the |
| +`wasm32`/`wasm64`-`unknown`-scoped `getrandom` dependency drops |
| +`features = ["wasm_js"]` (kept `optional = true`). Nothing in fastrand's |
| +actual logic changes — the `js` feature still exists, `fastrand::Rng::new()` |
| +still calls into `getrandom` the same way — only *which* `getrandom` |
| +backend ends up linked changes, and that's exactly what the |
| +`--cfg getrandom_backend="custom"` override in `.cargo/config.toml` is |
| +choosing on purpose. Wired in via `[patch.crates-io]` in `core/Cargo.toml`. |
| + |
| +### When this can be deleted |
| + |
| +Try removing the `[patch.crates-io]` block from `core/Cargo.toml` (and |
| +`vendor/fastrand-2.5.0/`) and running: |
| + |
| +``` |
| +cargo build --release --manifest-path core/Cargo.toml --target wasm32-unknown-unknown |
| +``` |
| + |
| +If that compiles cleanly (no `E0599`), delete the patch and this |
| +directory — that's the goal state. Concretely, that becomes true once any |
| +one of: |
| + |
| +1. A `getrandom` release fixes `error.rs`'s `WEB_CRYPTO` match arm to also |
| + respect `getrandom_backend` (e.g. adds `not(getrandom_backend = |
| + "custom")`), and `core/Cargo.toml`'s `getrandom = "=0.4.3"` pin is |
| + bumped to that version. |
| +2. A `fastrand` release stops forcing `wasm_js` onto its optional |
| + `getrandom` dependency (making it opt-in instead, which is what |
| + getrandom issue #633 above is effectively asking for), and `yrs`'s pin |
| + on it moves to that version. |
| +3. `yrs` stops requiring `fastrand`'s `js` feature at all for the |
| + wasm32-unknown-unknown target — unlikely while it still generates |
| + random document GUIDs internally, but would remove the whole problem. |
| + |
| +Verified as of this writing (2026-08-30) that the patch is still required: |
| +removing it and rebuilding reproduces the exact `E0599` above. |
diff --git a/core/vendor/fastrand-2.5.0/CHANGELOG.md b/core/vendor/fastrand-2.5.0/CHANGELOG.md| new file mode 100644 |
| index 0000000..455730f |
| --- /dev/null |
| +++ b/core/vendor/fastrand-2.5.0/CHANGELOG.md |
| @@ -0,0 +1,140 @@ |
| +# Version 2.5.0 |
| + |
| +- Update `getrandom` to 0.4. (#131) |
| + |
| +# Version 2.4.1 |
| + |
| +- Fix build failure with `js` feature. (#125) |
| + |
| +# Version 2.4.0 |
| + |
| +- Bump MSRV to 1.63. (#104) |
| +- Improve quality of f32/f64 generation. (#103) |
| +- Add `f{32,64}_inclusive` and `Rng::f{32,64}_inclusive`. (#103) |
| +- Make `Rng::with_seed` const. (#107) |
| +- Update `getrandom` to 0.3. (#104) |
| + |
| +# Version 2.3.0 |
| + |
| +- Accept `IntoIterator` in `choose_multiple` functions instead of just `Iterator`. (#92) |
| + |
| +# Version 2.2.0 |
| + |
| +- Expose missing `fill` method for the global RNG. (#90) |
| + |
| +# Version 2.1.1 |
| + |
| +- Remove support for 128-bit targets, as they are not supported by rustc yet. (#87) |
| + |
| +# Version 2.1.0 |
| + |
| +- Change the RNG algorithm and the way that the seed is computed. This will cause |
| + the algorithm to emit different constants for different seeds, hence the minor |
| + SemVer change. |
| + - Update to the final WyRand v4.2 constants for better entropy. (#82) |
| + - Remove an unnecessary seed modification. (#73) |
| + |
| +# Version 2.0.2 |
| + |
| +- Slight restructuring of the `with_seed` function. (#79) |
| + |
| +# Version 2.0.1 |
| + |
| +- Clarify documentation for the `fork()` method. (#62) |
| +- Mention `fastrand-contrib` in documentation. (#70) |
| + |
| +# Version 2.0.0 |
| + |
| +- **Breaking:** Remove interior mutability from `Rng`. (#47) |
| +- Add a `fork()` method. (#49) |
| +- Add a `no_std` mode. (#50) |
| +- Add an iterator selection function. (#51) |
| +- Add a `choose_multiple()` function for sampling several elements from an iterator. (#55) |
| +- Use the `getrandom` crate for seeding on WebAssembly targets if the `js` feature is enabled. (#60) |
| + |
| +# Version 1.9.0 |
| + |
| +- Add `Rng::fill()` (#35, #43) |
| +- Add `#[must_use]` to `Rng::with_seed()` (#46) |
| + |
| +# Version 1.8.0 |
| + |
| +- Add `get_seed()` and `Rng::get_seed()` (#33) |
| + |
| +# Version 1.7.0 |
| + |
| +- Add `char()` and `Rng::char()` (#25) |
| + |
| +# Version 1.6.0 |
| + |
| +- Implement `PartialEq` and `Eq` for `Rng` (#23) |
| + |
| +# Version 1.5.0 |
| + |
| +- Switch to Wyrand (#14) |
| + |
| +# Version 1.4.1 |
| + |
| +- Fix bug when generating a signed integer within a range (#16) |
| + |
| +# Version 1.4.0 |
| + |
| +- Add wasm support. |
| + |
| +# Version 1.3.5 |
| + |
| +- Reword docs. |
| +- Add `Rng::with_seed()`. |
| + |
| +# Version 1.3.4 |
| + |
| +- Implement `Clone` for `Rng`. |
| + |
| +# Version 1.3.3 |
| + |
| +- Forbid unsafe code. |
| + |
| +# Version 1.3.2 |
| + |
| +- Support older Rust versions. |
| + |
| +# Version 1.3.1 |
| + |
| +- Tweak Cargo keywords. |
| + |
| +# Version 1.3.0 |
| + |
| +- Add `f32()` and `f64()`. |
| +- Add `lowercase()`, `uppercase()`, `alphabetic()`, and `digit()`. |
| + |
| +# Version 1.2.4 |
| + |
| +- Switch to PCG XSH RR 64/32. |
| +- Fix a bug in `gen_mod_u128`. |
| +- Fix bias in ranges. |
| + |
| +# Version 1.2.3 |
| + |
| +- Support Rust 1.32.0 |
| + |
| +# Version 1.2.2 |
| + |
| +- Use `std::$t::MAX` rather than `$t::MAX` to support older Rust versions. |
| + |
| +# Version 1.2.1 |
| + |
| +- Inline all functions. |
| + |
| +# Version 1.2.0 |
| + |
| +- Add `Rng` struct. |
| + |
| +# Version 1.1.0 |
| + |
| +- Switch to PCG implementation. |
| +- Add `alphanumeric()`. |
| +- Add `seed()`. |
| + |
| +# Version 1.0.0 |
| + |
| +- Initial version |
diff --git a/core/vendor/fastrand-2.5.0/Cargo.toml b/core/vendor/fastrand-2.5.0/Cargo.toml| new file mode 100644 |
| index 0000000..e21cde2 |
| --- /dev/null |
| +++ b/core/vendor/fastrand-2.5.0/Cargo.toml |
| @@ -0,0 +1,96 @@ |
| +# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO |
| +# |
| +# When uploading crates to the registry Cargo will automatically |
| +# "normalize" Cargo.toml files for maximal compatibility |
| +# with all versions of Cargo and also rewrite `path` dependencies |
| +# to registry (e.g., crates.io) dependencies. |
| +# |
| +# If you are reading this file be aware that the original Cargo.toml |
| +# will likely look very different (and much more reasonable). |
| +# See Cargo.toml.orig for the original contents. |
| + |
| +[package] |
| +edition = "2018" |
| +rust-version = "1.63" |
| +name = "fastrand" |
| +version = "2.5.0" |
| +authors = ["Stjepan Glavina <stjepang@gmail.com>"] |
| +build = false |
| +exclude = ["/.*"] |
| +autolib = false |
| +autobins = false |
| +autoexamples = false |
| +autotests = false |
| +autobenches = false |
| +description = "A simple and fast random number generator" |
| +readme = "README.md" |
| +keywords = [ |
| + "simple", |
| + "fast", |
| + "rand", |
| + "random", |
| + "wyrand", |
| +] |
| +categories = ["algorithms"] |
| +license = "Apache-2.0 OR MIT" |
| +repository = "https://github.com/smol-rs/fastrand" |
| + |
| +[package.metadata.docs.rs] |
| +all-features = true |
| +rustdoc-args = [ |
| + "--cfg", |
| + "docsrs", |
| +] |
| + |
| +[features] |
| +alloc = [] |
| +default = ["std"] |
| +js = [ |
| + "std", |
| + "getrandom", |
| +] |
| +std = ["alloc"] |
| + |
| +[lib] |
| +name = "fastrand" |
| +path = "src/lib.rs" |
| + |
| +[[test]] |
| +name = "char" |
| +path = "tests/char.rs" |
| + |
| +[[test]] |
| +name = "smoke" |
| +path = "tests/smoke.rs" |
| + |
| +[[bench]] |
| +name = "bench" |
| +path = "benches/bench.rs" |
| + |
| +[dev-dependencies.getrandom] |
| +version = "0.4" |
| + |
| +[dev-dependencies.rand] |
| +version = "0.10" |
| + |
| +[dev-dependencies.wyhash] |
| +version = "0.6" |
| + |
| +# Vendored patch (see docs-wt-core/vendor/README.md): dropped the forced |
| +# "wasm_js" feature so that a `--cfg getrandom_backend="..."` override (used |
| +# by docs-core to avoid a wasm-bindgen/JS import dependency) doesn't hit |
| +# getrandom's `Error::WEB_CRYPTO` cfg mismatch (that item only exists when |
| +# the "wasm_js" backend module itself is compiled, which a cfg override to |
| +# a different backend skips, but `error.rs`'s Display impl references it |
| +# whenever the "wasm_js" *feature* is enabled, regardless of which backend |
| +# is actually selected). |
| +[target.'cfg(all(any(target_arch = "wasm32", target_arch = "wasm64"), target_os = "unknown"))'.dependencies.getrandom] |
| +version = "0.4" |
| +optional = true |
| + |
| +[target.'cfg(all(any(target_arch = "wasm32", target_arch = "wasm64"), target_os = "unknown"))'.dev-dependencies.getrandom] |
| +version = "0.4" |
| +features = ["wasm_js"] |
| + |
| +[target.'cfg(all(any(target_arch = "wasm32", target_arch = "wasm64"), target_os = "unknown"))'.dev-dependencies.wasm-bindgen-test] |
| +version = "0.3" |
diff --git a/core/vendor/fastrand-2.5.0/LICENSE-APACHE b/core/vendor/fastrand-2.5.0/LICENSE-APACHE| new file mode 100644 |
| index 0000000..16fe87b |
| --- /dev/null |
| +++ b/core/vendor/fastrand-2.5.0/LICENSE-APACHE |
| @@ -0,0 +1,201 @@ |
| + Apache License |
| + Version 2.0, January 2004 |
| + http://www.apache.org/licenses/ |
| + |
| +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION |
| + |
| +1. Definitions. |
| + |
| + "License" shall mean the terms and conditions for use, reproduction, |
| + and distribution as defined by Sections 1 through 9 of this document. |
| + |
| + "Licensor" shall mean the copyright owner or entity authorized by |
| + the copyright owner that is granting the License. |
| + |
| + "Legal Entity" shall mean the union of the acting entity and all |
| + other entities that control, are controlled by, or are under common |
| + control with that entity. For the purposes of this definition, |
| + "control" means (i) the power, direct or indirect, to cause the |
| + direction or management of such entity, whether by contract or |
| + otherwise, or (ii) ownership of fifty percent (50%) or more of the |
| + outstanding shares, or (iii) beneficial ownership of such entity. |
| + |
| + "You" (or "Your") shall mean an individual or Legal Entity |
| + exercising permissions granted by this License. |
| + |
| + "Source" form shall mean the preferred form for making modifications, |
| + including but not limited to software source code, documentation |
| + source, and configuration files. |
| + |
| + "Object" form shall mean any form resulting from mechanical |
| + transformation or translation of a Source form, including but |
| + not limited to compiled object code, generated documentation, |
| + and conversions to other media types. |
| + |
| + "Work" shall mean the work of authorship, whether in Source or |
| + Object form, made available under the License, as indicated by a |
| + copyright notice that is included in or attached to the work |
| + (an example is provided in the Appendix below). |
| + |
| + "Derivative Works" shall mean any work, whether in Source or Object |
| + form, that is based on (or derived from) the Work and for which the |
| + editorial revisions, annotations, elaborations, or other modifications |
| + represent, as a whole, an original work of authorship. For the purposes |
| + of this License, Derivative Works shall not include works that remain |
| + separable from, or merely link (or bind by name) to the interfaces of, |
| + the Work and Derivative Works thereof. |
| + |
| + "Contribution" shall mean any work of authorship, including |
| + the original version of the Work and any modifications or additions |
| + to that Work or Derivative Works thereof, that is intentionally |
| + submitted to Licensor for inclusion in the Work by the copyright owner |
| + or by an individual or Legal Entity authorized to submit on behalf of |
| + the copyright owner. For the purposes of this definition, "submitted" |
| + means any form of electronic, verbal, or written communication sent |
| + to the Licensor or its representatives, including but not limited to |
| + communication on electronic mailing lists, source code control systems, |
| + and issue tracking systems that are managed by, or on behalf of, the |
| + Licensor for the purpose of discussing and improving the Work, but |
| + excluding communication that is conspicuously marked or otherwise |
| + designated in writing by the copyright owner as "Not a Contribution." |
| + |
| + "Contributor" shall mean Licensor and any individual or Legal Entity |
| + on behalf of whom a Contribution has been received by Licensor and |
| + subsequently incorporated within the Work. |
| + |
| +2. Grant of Copyright License. Subject to the terms and conditions of |
| + this License, each Contributor hereby grants to You a perpetual, |
| + worldwide, non-exclusive, no-charge, royalty-free, irrevocable |
| + copyright license to reproduce, prepare Derivative Works of, |
| + publicly display, publicly perform, sublicense, and distribute the |
| + Work and such Derivative Works in Source or Object form. |
| + |
| +3. Grant of Patent License. Subject to the terms and conditions of |
| + this License, each Contributor hereby grants to You a perpetual, |
| + worldwide, non-exclusive, no-charge, royalty-free, irrevocable |
| + (except as stated in this section) patent license to make, have made, |
| + use, offer to sell, sell, import, and otherwise transfer the Work, |
| + where such license applies only to those patent claims licensable |
| + by such Contributor that are necessarily infringed by their |
| + Contribution(s) alone or by combination of their Contribution(s) |
| + with the Work to which such Contribution(s) was submitted. If You |
| + institute patent litigation against any entity (including a |
| + cross-claim or counterclaim in a lawsuit) alleging that the Work |
| + or a Contribution incorporated within the Work constitutes direct |
| + or contributory patent infringement, then any patent licenses |
| + granted to You under this License for that Work shall terminate |
| + as of the date such litigation is filed. |
| + |
| +4. Redistribution. You may reproduce and distribute copies of the |
| + Work or Derivative Works thereof in any medium, with or without |
| + modifications, and in Source or Object form, provided that You |
| + meet the following conditions: |
| + |
| + (a) You must give any other recipients of the Work or |
| + Derivative Works a copy of this License; and |
| + |
| + (b) You must cause any modified files to carry prominent notices |
| + stating that You changed the files; and |
| + |
| + (c) You must retain, in the Source form of any Derivative Works |
| + that You distribute, all copyright, patent, trademark, and |
| + attribution notices from the Source form of the Work, |
| + excluding those notices that do not pertain to any part of |
| + the Derivative Works; and |
| + |
| + (d) If the Work includes a "NOTICE" text file as part of its |
| + distribution, then any Derivative Works that You distribute must |
| + include a readable copy of the attribution notices contained |
| + within such NOTICE file, excluding those notices that do not |
| + pertain to any part of the Derivative Works, in at least one |
| + of the following places: within a NOTICE text file distributed |
| + as part of the Derivative Works; within the Source form or |
| + documentation, if provided along with the Derivative Works; or, |
| + within a display generated by the Derivative Works, if and |
| + wherever such third-party notices normally appear. The contents |
| + of the NOTICE file are for informational purposes only and |
| + do not modify the License. You may add Your own attribution |
| + notices within Derivative Works that You distribute, alongside |
| + or as an addendum to the NOTICE text from the Work, provided |
| + that such additional attribution notices cannot be construed |
| + as modifying the License. |
| + |
| + You may add Your own copyright statement to Your modifications and |
| + may provide additional or different license terms and conditions |
| + for use, reproduction, or distribution of Your modifications, or |
| + for any such Derivative Works as a whole, provided Your use, |
| + reproduction, and distribution of the Work otherwise complies with |
| + the conditions stated in this License. |
| + |
| +5. Submission of Contributions. Unless You explicitly state otherwise, |
| + any Contribution intentionally submitted for inclusion in the Work |
| + by You to the Licensor shall be under the terms and conditions of |
| + this License, without any additional terms or conditions. |
| + Notwithstanding the above, nothing herein shall supersede or modify |
| + the terms of any separate license agreement you may have executed |
| + with Licensor regarding such Contributions. |
| + |
| +6. Trademarks. This License does not grant permission to use the trade |
| + names, trademarks, service marks, or product names of the Licensor, |
| + except as required for reasonable and customary use in describing the |
| + origin of the Work and reproducing the content of the NOTICE file. |
| + |
| +7. Disclaimer of Warranty. Unless required by applicable law or |
| + agreed to in writing, Licensor provides the Work (and each |
| + Contributor provides its Contributions) on an "AS IS" BASIS, |
| + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or |
| + implied, including, without limitation, any warranties or conditions |
| + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A |
| + PARTICULAR PURPOSE. You are solely responsible for determining the |
| + appropriateness of using or redistributing the Work and assume any |
| + risks associated with Your exercise of permissions under this License. |
| + |
| +8. Limitation of Liability. In no event and under no legal theory, |
| + whether in tort (including negligence), contract, or otherwise, |
| + unless required by applicable law (such as deliberate and grossly |
| + negligent acts) or agreed to in writing, shall any Contributor be |
| + liable to You for damages, including any direct, indirect, special, |
| + incidental, or consequential damages of any character arising as a |
| + result of this License or out of the use or inability to use the |
| + Work (including but not limited to damages for loss of goodwill, |
| + work stoppage, computer failure or malfunction, or any and all |
| + other commercial damages or losses), even if such Contributor |
| + has been advised of the possibility of such damages. |
| + |
| +9. Accepting Warranty or Additional Liability. While redistributing |
| + the Work or Derivative Works thereof, You may choose to offer, |
| + and charge a fee for, acceptance of support, warranty, indemnity, |
| + or other liability obligations and/or rights consistent with this |
| + License. However, in accepting such obligations, You may act only |
| + on Your own behalf and on Your sole responsibility, not on behalf |
| + of any other Contributor, and only if You agree to indemnify, |
| + defend, and hold each Contributor harmless for any liability |
| + incurred by, or claims asserted against, such Contributor by reason |
| + of your accepting any such warranty or additional liability. |
| + |
| +END OF TERMS AND CONDITIONS |
| + |
| +APPENDIX: How to apply the Apache License to your work. |
| + |
| + To apply the Apache License to your work, attach the following |
| + boilerplate notice, with the fields enclosed by brackets "[]" |
| + replaced with your own identifying information. (Don't include |
| + the brackets!) The text should be enclosed in the appropriate |
| + comment syntax for the file format. We also recommend that a |
| + file or class name and description of purpose be included on the |
| + same "printed page" as the copyright notice for easier |
| + identification within third-party archives. |
| + |
| +Copyright [yyyy] [name of copyright owner] |
| + |
| +Licensed under the Apache License, Version 2.0 (the "License"); |
| +you may not use this file except in compliance with the License. |
| +You may obtain a copy of the License at |
| + |
| + http://www.apache.org/licenses/LICENSE-2.0 |
| + |
| +Unless required by applicable law or agreed to in writing, software |
| +distributed under the License is distributed on an "AS IS" BASIS, |
| +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| +See the License for the specific language governing permissions and |
| +limitations under the License. |
diff --git a/core/vendor/fastrand-2.5.0/LICENSE-MIT b/core/vendor/fastrand-2.5.0/LICENSE-MIT| new file mode 100644 |
| index 0000000..31aa793 |
| --- /dev/null |
| +++ b/core/vendor/fastrand-2.5.0/LICENSE-MIT |
| @@ -0,0 +1,23 @@ |
| +Permission is hereby granted, free of charge, to any |
| +person obtaining a copy of this software and associated |
| +documentation files (the "Software"), to deal in the |
| +Software without restriction, including without |
| +limitation the rights to use, copy, modify, merge, |
| +publish, distribute, sublicense, and/or sell copies of |
| +the Software, and to permit persons to whom the Software |
| +is furnished to do so, subject to the following |
| +conditions: |
| + |
| +The above copyright notice and this permission notice |
| +shall be included in all copies or substantial portions |
| +of the Software. |
| + |
| +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF |
| +ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED |
| +TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A |
| +PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT |
| +SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY |
| +CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION |
| +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR |
| +IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER |
| +DEALINGS IN THE SOFTWARE. |
diff --git a/core/vendor/fastrand-2.5.0/README.md b/core/vendor/fastrand-2.5.0/README.md| new file mode 100644 |
| index 0000000..f478b42 |
| --- /dev/null |
| +++ b/core/vendor/fastrand-2.5.0/README.md |
| @@ -0,0 +1,110 @@ |
| +# fastrand |
| + |
| +[]( |
| +https://github.com/smol-rs/fastrand/actions) |
| +[]( |
| +https://github.com/smol-rs/fastrand) |
| +[]( |
| +https://crates.io/crates/fastrand) |
| +[]( |
| +https://docs.rs/fastrand) |
| + |
| +A simple and fast random number generator. |
| + |
| +The implementation uses [Wyrand](https://github.com/wangyi-fudan/wyhash), a simple and fast |
| +generator but **not** cryptographically secure. |
| + |
| +## Examples |
| + |
| +Flip a coin: |
| + |
| +```rust |
| +if fastrand::bool() { |
| + println!("heads"); |
| +} else { |
| + println!("tails"); |
| +} |
| +``` |
| + |
| +Generate a random `i32`: |
| + |
| +```rust |
| +let num = fastrand::i32(..); |
| +``` |
| + |
| +Choose a random element in an array: |
| + |
| +```rust |
| +let v = vec![1, 2, 3, 4, 5]; |
| +let i = fastrand::usize(..v.len()); |
| +let elem = v[i]; |
| +``` |
| + |
| +Sample values from an array with `O(n)` complexity (`n` is the length of the array): |
| + |
| +```rust |
| +fastrand::choose_multiple([1, 4, 5], 2); |
| +fastrand::choose_multiple(0..20, 12); |
| +``` |
| + |
| +Shuffle an array: |
| + |
| +```rust |
| +let mut v = vec![1, 2, 3, 4, 5]; |
| +fastrand::shuffle(&mut v); |
| +``` |
| + |
| +Generate a random `Vec` or `String`: |
| + |
| +```rust |
| +use std::iter::repeat_with; |
| + |
| +let v: Vec<i32> = repeat_with(|| fastrand::i32(..)).take(10).collect(); |
| +let s: String = repeat_with(fastrand::alphanumeric).take(10).collect(); |
| +``` |
| + |
| +To get reproducible results on every run, initialize the generator with a seed: |
| + |
| +```rust |
| +// Pick an arbitrary number as seed. |
| +fastrand::seed(7); |
| + |
| +// Now this prints the same number on every run: |
| +println!("{}", fastrand::u32(..)); |
| +``` |
| + |
| +To be more efficient, create a new `Rng` instance instead of using the thread-local |
| +generator: |
| + |
| +```rust |
| +use std::iter::repeat_with; |
| + |
| +let mut rng = fastrand::Rng::new(); |
| +let mut bytes: Vec<u8> = repeat_with(|| rng.u8(..)).take(10_000).collect(); |
| +``` |
| + |
| +This crate aims to expose a core set of useful randomness primitives. For more niche algorithms, consider using the [`fastrand-contrib`] crate alongside this one. |
| + |
| +# Features |
| + |
| +- `std` (enabled by default): Enables the `std` library. This is required for the global |
| + generator and global entropy. Without this feature, [`Rng`] can only be instantiated using |
| + the [`with_seed`](https://docs.rs/fastrand/latest/fastrand/struct.Rng.html#method.with_seed) method. |
| +- `js`: Assumes that WebAssembly targets are being run in a JavaScript environment. |
| + |
| +[`fastrand-contrib`]: https://crates.io/crates/fastrand-contrib |
| + |
| +## License |
| + |
| +Licensed under either of |
| + |
| + * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or https://www.apache.org/licenses/LICENSE-2.0) |
| + * MIT license ([LICENSE-MIT](LICENSE-MIT) or https://opensource.org/license/mit) |
| + |
| +at your option. |
| + |
| +#### Contribution |
| + |
| +Unless you explicitly state otherwise, any contribution intentionally submitted |
| +for inclusion in the work by you, as defined in the Apache-2.0 license, shall be |
| +dual licensed as above, without any additional terms or conditions. |
diff --git a/core/vendor/fastrand-2.5.0/benches/bench.rs b/core/vendor/fastrand-2.5.0/benches/bench.rs| new file mode 100644 |
| index 0000000..4789e80 |
| --- /dev/null |
| +++ b/core/vendor/fastrand-2.5.0/benches/bench.rs |
| @@ -0,0 +1,141 @@ |
| +#![feature(test)] |
| + |
| +extern crate test; |
| + |
| +use rand::prelude::*; |
| +use test::Bencher; |
| + |
| +/// Exposes wyhash's RNG through the rand traits |
| +struct WyRng(u64); |
| + |
| +impl rand::TryRng for WyRng { |
| + type Error = std::convert::Infallible; |
| + |
| + fn try_next_u32(&mut self) -> Result<u32, Self::Error> { |
| + Ok(wyhash::wyrng(&mut self.0) as u32) |
| + } |
| + |
| + fn try_next_u64(&mut self) -> Result<u64, Self::Error> { |
| + Ok(wyhash::wyrng(&mut self.0)) |
| + } |
| + |
| + fn try_fill_bytes(&mut self, dst: &mut [u8]) -> Result<(), Self::Error> { |
| + for chunk in dst.chunks_mut(8) { |
| + let bytes = wyhash::wyrng(&mut self.0).to_le_bytes(); |
| + chunk.copy_from_slice(&bytes[..chunk.len()]); |
| + } |
| + Ok(()) |
| + } |
| +} |
| + |
| +fn wyrng() -> WyRng { |
| + WyRng(rand::random()) |
| +} |
| + |
| +#[bench] |
| +fn shuffle_wyhash(b: &mut Bencher) { |
| + let mut rng = wyrng(); |
| + let mut x = (0..100).collect::<Vec<usize>>(); |
| + b.iter(|| { |
| + x.shuffle(&mut rng); |
| + x[0] |
| + }) |
| +} |
| + |
| +#[bench] |
| +fn shuffle_fastrand(b: &mut Bencher) { |
| + let mut rng = fastrand::Rng::new(); |
| + let mut x = (0..100).collect::<Vec<usize>>(); |
| + b.iter(|| { |
| + rng.shuffle(&mut x); |
| + x[0] |
| + }) |
| +} |
| + |
| +#[bench] |
| +fn u8_wyhash(b: &mut Bencher) { |
| + let mut rng = wyrng(); |
| + b.iter(|| { |
| + let mut sum = 0u8; |
| + for _ in 0..10_000 { |
| + sum = sum.wrapping_add(rng.random::<u8>()); |
| + } |
| + sum |
| + }) |
| +} |
| + |
| +#[bench] |
| +fn u8_fastrand(b: &mut Bencher) { |
| + let mut rng = fastrand::Rng::new(); |
| + b.iter(|| { |
| + let mut sum = 0u8; |
| + for _ in 0..10_000 { |
| + sum = sum.wrapping_add(rng.u8(..)); |
| + } |
| + sum |
| + }) |
| +} |
| + |
| +#[bench] |
| +fn u32_wyhash(b: &mut Bencher) { |
| + let mut rng = wyrng(); |
| + b.iter(|| { |
| + let mut sum = 0u32; |
| + for _ in 0..10_000 { |
| + sum = sum.wrapping_add(rng.random::<u32>()); |
| + } |
| + sum |
| + }) |
| +} |
| + |
| +#[bench] |
| +fn u32_fastrand(b: &mut Bencher) { |
| + let mut rng = fastrand::Rng::new(); |
| + b.iter(|| { |
| + let mut sum = 0u32; |
| + for _ in 0..10_000 { |
| + sum = sum.wrapping_add(rng.u32(..)); |
| + } |
| + sum |
| + }) |
| +} |
| + |
| +#[bench] |
| +fn f32_fastrand(b: &mut Bencher) { |
| + let mut rng = fastrand::Rng::new(); |
| + b.iter(|| { |
| + // f32 sum unrolled 2x to hide f32-add latency. |
| + // |
| + // Optimal amount of unrolling is somewhat sensitive to CPU and algorithm. |
| + // Variously could be 2x, 3x, or 4x unrolling. On AArch64 and x86-64 on the |
| + // current algorithm, 2x seems to be optimal on this benchmark. |
| + let mut sum = 0.0; |
| + for _ in 0..5_000 { |
| + sum += rng.f32() + rng.f32(); |
| + } |
| + sum |
| + }) |
| +} |
| + |
| +#[bench] |
| +fn fill(b: &mut Bencher) { |
| + let mut rng = fastrand::Rng::new(); |
| + b.iter(|| { |
| + // Pick a size that isn't divisible by 8. |
| + let mut bytes = [0u8; 367]; |
| + rng.fill(&mut bytes); |
| + bytes |
| + }) |
| +} |
| + |
| +#[bench] |
| +fn fill_naive(b: &mut Bencher) { |
| + let mut rng = fastrand::Rng::new(); |
| + b.iter(|| { |
| + let mut bytes = [0u8; 367]; |
| + for item in &mut bytes { |
| + *item = rng.u8(..); |
| + } |
| + bytes |
| + }) |
| +} |
diff --git a/core/vendor/fastrand-2.5.0/src/global_rng.rs b/core/vendor/fastrand-2.5.0/src/global_rng.rs| new file mode 100644 |
| index 0000000..bfece0a |
| --- /dev/null |
| +++ b/core/vendor/fastrand-2.5.0/src/global_rng.rs |
| @@ -0,0 +1,234 @@ |
| +//! A global, thread-local random number generator. |
| + |
| +use crate::Rng; |
| + |
| +use std::cell::Cell; |
| +use std::ops::RangeBounds; |
| +use std::vec::Vec; |
| + |
| +// Chosen by a fair roll of the dice. |
| +const DEFAULT_RNG_SEED: u64 = 0xef6f79ed30ba75a; |
| + |
| +impl Default for Rng { |
| + /// Initialize the `Rng` from the system's random number generator. |
| + /// |
| + /// This is equivalent to [`Rng::new()`]. |
| + #[inline] |
| + fn default() -> Rng { |
| + Rng::new() |
| + } |
| +} |
| + |
| +impl Rng { |
| + /// Creates a new random number generator. |
| + #[inline] |
| + pub fn new() -> Rng { |
| + try_with_rng(Rng::fork).unwrap_or_else(|_| Rng::with_seed(0x4d595df4d0f33173)) |
| + } |
| +} |
| + |
| +std::thread_local! { |
| + static RNG: Cell<Rng> = Cell::new(Rng(random_seed().unwrap_or(DEFAULT_RNG_SEED))); |
| +} |
| + |
| +/// Run an operation with the current thread-local generator. |
| +#[inline] |
| +fn with_rng<R>(f: impl FnOnce(&mut Rng) -> R) -> R { |
| + RNG.with(|rng| { |
| + let current = rng.replace(Rng(0)); |
| + |
| + let mut restore = RestoreOnDrop { rng, current }; |
| + |
| + f(&mut restore.current) |
| + }) |
| +} |
| + |
| +/// Try to run an operation with the current thread-local generator. |
| +#[inline] |
| +fn try_with_rng<R>(f: impl FnOnce(&mut Rng) -> R) -> Result<R, std::thread::AccessError> { |
| + RNG.try_with(|rng| { |
| + let current = rng.replace(Rng(0)); |
| + |
| + let mut restore = RestoreOnDrop { rng, current }; |
| + |
| + f(&mut restore.current) |
| + }) |
| +} |
| + |
| +/// Make sure the original RNG is restored even on panic. |
| +struct RestoreOnDrop<'a> { |
| + rng: &'a Cell<Rng>, |
| + current: Rng, |
| +} |
| + |
| +impl Drop for RestoreOnDrop<'_> { |
| + fn drop(&mut self) { |
| + self.rng.set(Rng(self.current.0)); |
| + } |
| +} |
| + |
| +/// Initializes the thread-local generator with the given seed. |
| +#[inline] |
| +pub fn seed(seed: u64) { |
| + with_rng(|r| r.seed(seed)); |
| +} |
| + |
| +/// Gives back **current** seed that is being held by the thread-local generator. |
| +#[inline] |
| +pub fn get_seed() -> u64 { |
| + with_rng(|r| r.get_seed()) |
| +} |
| + |
| +/// Generates a random `bool`. |
| +#[inline] |
| +pub fn bool() -> bool { |
| + with_rng(|r| r.bool()) |
| +} |
| + |
| +/// Generates a random `char` in range a-z and A-Z. |
| +#[inline] |
| +pub fn alphabetic() -> char { |
| + with_rng(|r| r.alphabetic()) |
| +} |
| + |
| +/// Generates a random `char` in range a-z, A-Z and 0-9. |
| +#[inline] |
| +pub fn alphanumeric() -> char { |
| + with_rng(|r| r.alphanumeric()) |
| +} |
| + |
| +/// Generates a random `char` in range a-z. |
| +#[inline] |
| +pub fn lowercase() -> char { |
| + with_rng(|r| r.lowercase()) |
| +} |
| + |
| +/// Generates a random `char` in range A-Z. |
| +#[inline] |
| +pub fn uppercase() -> char { |
| + with_rng(|r| r.uppercase()) |
| +} |
| + |
| +/// Choose an item from an iterator at random. |
| +/// |
| +/// This function may have an unexpected result if the `len()` property of the |
| +/// iterator does not match the actual number of items in the iterator. If |
| +/// the iterator is empty, this returns `None`. |
| +#[inline] |
| +pub fn choice<I>(iter: I) -> Option<I::Item> |
| +where |
| + I: IntoIterator, |
| + I::IntoIter: ExactSizeIterator, |
| +{ |
| + with_rng(|r| r.choice(iter)) |
| +} |
| + |
| +/// Generates a random digit in the given `base`. |
| +/// |
| +/// Digits are represented by `char`s in range 0-9 and a-z. |
| +/// |
| +/// Panics if the base is zero or greater than 36. |
| +#[inline] |
| +pub fn digit(base: u32) -> char { |
| + with_rng(|r| r.digit(base)) |
| +} |
| + |
| +/// Shuffles a slice randomly. |
| +#[inline] |
| +pub fn shuffle<T>(slice: &mut [T]) { |
| + with_rng(|r| r.shuffle(slice)) |
| +} |
| + |
| +/// Fill a byte slice with random data. |
| +#[inline] |
| +pub fn fill(slice: &mut [u8]) { |
| + with_rng(|r| r.fill(slice)) |
| +} |
| + |
| +macro_rules! integer { |
| + ($t:tt, $doc:tt) => { |
| + #[doc = $doc] |
| + /// |
| + /// Panics if the range is empty. |
| + #[inline] |
| + pub fn $t(range: impl RangeBounds<$t>) -> $t { |
| + with_rng(|r| r.$t(range)) |
| + } |
| + }; |
| +} |
| + |
| +integer!(u8, "Generates a random `u8` in the given range."); |
| +integer!(i8, "Generates a random `i8` in the given range."); |
| +integer!(u16, "Generates a random `u16` in the given range."); |
| +integer!(i16, "Generates a random `i16` in the given range."); |
| +integer!(u32, "Generates a random `u32` in the given range."); |
| +integer!(i32, "Generates a random `i32` in the given range."); |
| +integer!(u64, "Generates a random `u64` in the given range."); |
| +integer!(i64, "Generates a random `i64` in the given range."); |
| +integer!(u128, "Generates a random `u128` in the given range."); |
| +integer!(i128, "Generates a random `i128` in the given range."); |
| +integer!(usize, "Generates a random `usize` in the given range."); |
| +integer!(isize, "Generates a random `isize` in the given range."); |
| +integer!(char, "Generates a random `char` in the given range."); |
| + |
| +/// Generates a random `f32` in range `0..1`. |
| +pub fn f32() -> f32 { |
| + with_rng(|r| r.f32()) |
| +} |
| + |
| +/// Generates a random `f32` in range `0..=1`. |
| +pub fn f32_inclusive() -> f32 { |
| + with_rng(|r| r.f32_inclusive()) |
| +} |
| + |
| +/// Generates a random `f64` in range `0..1`. |
| +pub fn f64() -> f64 { |
| + with_rng(|r| r.f64()) |
| +} |
| + |
| +/// Generates a random `f64` in range `0..=1`. |
| +pub fn f64_inclusive() -> f64 { |
| + with_rng(|r| r.f64_inclusive()) |
| +} |
| + |
| +/// Collects `amount` values at random from the iterable into a vector. |
| +pub fn choose_multiple<I: IntoIterator>(source: I, amount: usize) -> Vec<I::Item> { |
| + with_rng(|rng| rng.choose_multiple(source, amount)) |
| +} |
| + |
| +#[cfg(not(all( |
| + any(target_arch = "wasm32", target_arch = "wasm64"), |
| + target_os = "unknown" |
| +)))] |
| +fn random_seed() -> Option<u64> { |
| + use std::collections::hash_map::DefaultHasher; |
| + use std::hash::{Hash, Hasher}; |
| + use std::thread; |
| + use std::time::Instant; |
| + |
| + let mut hasher = DefaultHasher::new(); |
| + Instant::now().hash(&mut hasher); |
| + thread::current().id().hash(&mut hasher); |
| + Some(hasher.finish()) |
| +} |
| + |
| +#[cfg(all( |
| + any(target_arch = "wasm32", target_arch = "wasm64"), |
| + target_os = "unknown", |
| + feature = "js" |
| +))] |
| +fn random_seed() -> Option<u64> { |
| + // TODO(notgull): Failures should be logged somewhere. |
| + let mut seed = [0u8; 8]; |
| + getrandom::fill(&mut seed).ok()?; |
| + Some(u64::from_ne_bytes(seed)) |
| +} |
| + |
| +#[cfg(all( |
| + any(target_arch = "wasm32", target_arch = "wasm64"), |
| + target_os = "unknown", |
| + not(feature = "js") |
| +))] |
| +fn random_seed() -> Option<u64> { |
| + None |
| +} |
diff --git a/core/vendor/fastrand-2.5.0/src/lib.rs b/core/vendor/fastrand-2.5.0/src/lib.rs| new file mode 100644 |
| index 0000000..a753110 |
| --- /dev/null |
| +++ b/core/vendor/fastrand-2.5.0/src/lib.rs |
| @@ -0,0 +1,736 @@ |
| +//! A simple and fast random number generator. |
| +//! |
| +//! The implementation uses [Wyrand](https://github.com/wangyi-fudan/wyhash), a simple and fast |
| +//! generator but **not** cryptographically secure. |
| +//! |
| +//! # Examples |
| +//! |
| +//! Flip a coin: |
| +//! |
| +//! ``` |
| +//! if fastrand::bool() { |
| +//! println!("heads"); |
| +//! } else { |
| +//! println!("tails"); |
| +//! } |
| +//! ``` |
| +//! |
| +//! Generate a random `i32`: |
| +//! |
| +//! ``` |
| +//! let num = fastrand::i32(..); |
| +//! ``` |
| +//! |
| +//! Choose a random element in an array: |
| +//! |
| +//! ``` |
| +//! let v = vec![1, 2, 3, 4, 5]; |
| +//! let i = fastrand::usize(..v.len()); |
| +//! let elem = v[i]; |
| +//! ``` |
| +//! |
| +//! Sample values from an array with `O(n)` complexity (`n` is the length of the array): |
| +//! |
| +//! ``` |
| +//! fastrand::choose_multiple([1, 4, 5], 2); |
| +//! fastrand::choose_multiple(0..20, 12); |
| +//! ``` |
| +//! |
| +//! |
| +//! Shuffle an array: |
| +//! |
| +//! ``` |
| +//! let mut v = vec![1, 2, 3, 4, 5]; |
| +//! fastrand::shuffle(&mut v); |
| +//! ``` |
| +//! |
| +//! Generate a random [`Vec`] or [`String`](alloc::string::String): |
| +//! |
| +//! ``` |
| +//! use std::iter::repeat_with; |
| +//! |
| +//! let v: Vec<i32> = repeat_with(|| fastrand::i32(..)).take(10).collect(); |
| +//! let s: String = repeat_with(fastrand::alphanumeric).take(10).collect(); |
| +//! ``` |
| +//! |
| +//! To get reproducible results on every run, initialize the generator with a seed: |
| +//! |
| +//! ``` |
| +//! // Pick an arbitrary number as seed. |
| +//! fastrand::seed(7); |
| +//! |
| +//! // Now this prints the same number on every run: |
| +//! println!("{}", fastrand::u32(..)); |
| +//! ``` |
| +//! |
| +//! To be more efficient, create a new [`Rng`] instance instead of using the thread-local |
| +//! generator: |
| +//! |
| +//! ``` |
| +//! use std::iter::repeat_with; |
| +//! |
| +//! let mut rng = fastrand::Rng::new(); |
| +//! let mut bytes: Vec<u8> = repeat_with(|| rng.u8(..)).take(10_000).collect(); |
| +//! ``` |
| +//! |
| +//! This crate aims to expose a core set of useful randomness primitives. For more niche algorithms, |
| +//! consider using the [`fastrand-contrib`] crate alongside this one. |
| +//! |
| +//! # Features |
| +//! |
| +//! - `std` (enabled by default): Enables the `std` library. This is required for the global |
| +//! generator and global entropy. Without this feature, [`Rng`] can only be instantiated using |
| +//! the [`with_seed`](Rng::with_seed) method. |
| +//! - `js`: Assumes that WebAssembly targets are being run in a JavaScript environment. See the |
| +//! [WebAssembly Notes](#webassembly-notes) section for more information. |
| +//! |
| +//! # WebAssembly Notes |
| +//! |
| +//! For non-WASI WASM targets, there is additional subtlety to consider when utilizing the global RNG. |
| +//! By default, `std` targets will use entropy sources in the standard library to seed the global RNG. |
| +//! However, these sources are not available by default on WASM targets outside of WASI. |
| +//! |
| +//! If the `js` feature is enabled, this crate will assume that it is running in a JavaScript |
| +//! environment. At this point, the [`getrandom`] crate will be used in order to access the available |
| +//! entropy sources and seed the global RNG. If the `js` feature is not enabled, the global RNG will |
| +//! use a predefined seed. |
| +//! |
| +//! [`fastrand-contrib`]: https://crates.io/crates/fastrand-contrib |
| +//! [`getrandom`]: https://crates.io/crates/getrandom |
| + |
| +#![no_std] |
| +#![cfg_attr(docsrs, feature(doc_cfg))] |
| +#![forbid(unsafe_code)] |
| +#![warn(missing_docs, missing_debug_implementations, rust_2018_idioms)] |
| +#![doc( |
| + html_favicon_url = "https://raw.githubusercontent.com/smol-rs/smol/master/assets/images/logo_fullsize_transparent.png" |
| +)] |
| +#![doc( |
| + html_logo_url = "https://raw.githubusercontent.com/smol-rs/smol/master/assets/images/logo_fullsize_transparent.png" |
| +)] |
| + |
| +#[cfg(feature = "alloc")] |
| +extern crate alloc; |
| +#[cfg(feature = "std")] |
| +extern crate std; |
| + |
| +use core::convert::{TryFrom, TryInto}; |
| +use core::ops::{Bound, RangeBounds}; |
| + |
| +#[cfg(feature = "alloc")] |
| +use alloc::vec::Vec; |
| + |
| +#[cfg(feature = "std")] |
| +mod global_rng; |
| + |
| +#[cfg(feature = "std")] |
| +pub use global_rng::*; |
| + |
| +/// A random number generator. |
| +#[derive(Debug, PartialEq, Eq)] |
| +pub struct Rng(u64); |
| + |
| +impl Clone for Rng { |
| + /// Clones the generator by creating a new generator with the same seed. |
| + fn clone(&self) -> Rng { |
| + Rng::with_seed(self.0) |
| + } |
| +} |
| + |
| +impl Rng { |
| + /// Generates a random `u32`. |
| + #[inline] |
| + fn gen_u32(&mut self) -> u32 { |
| + self.gen_u64() as u32 |
| + } |
| + |
| + /// Generates a random `u64`. |
| + #[inline] |
| + fn gen_u64(&mut self) -> u64 { |
| + // Constants for WyRand taken from: https://github.com/wangyi-fudan/wyhash/blob/master/wyhash.h#L151 |
| + // Updated for the final v4.2 implementation with improved constants for better entropy output. |
| + const WY_CONST_0: u64 = 0x2d35_8dcc_aa6c_78a5; |
| + const WY_CONST_1: u64 = 0x8bb8_4b93_962e_acc9; |
| + |
| + let s = self.0.wrapping_add(WY_CONST_0); |
| + self.0 = s; |
| + let t = u128::from(s) * u128::from(s ^ WY_CONST_1); |
| + (t as u64) ^ (t >> 64) as u64 |
| + } |
| + |
| + /// Generates a random `u128`. |
| + #[inline] |
| + fn gen_u128(&mut self) -> u128 { |
| + (u128::from(self.gen_u64()) << 64) | u128::from(self.gen_u64()) |
| + } |
| + |
| + /// Generates a random `u32` in `0..n`. |
| + #[inline] |
| + fn gen_mod_u32(&mut self, n: u32) -> u32 { |
| + // Adapted from: https://lemire.me/blog/2016/06/30/fast-random-shuffling/ |
| + let mut r = self.gen_u32(); |
| + let mut hi = mul_high_u32(r, n); |
| + let mut lo = r.wrapping_mul(n); |
| + if lo < n { |
| + let t = n.wrapping_neg() % n; |
| + while lo < t { |
| + r = self.gen_u32(); |
| + hi = mul_high_u32(r, n); |
| + lo = r.wrapping_mul(n); |
| + } |
| + } |
| + hi |
| + } |
| + |
| + /// Generates a random `u64` in `0..n`. |
| + #[inline] |
| + fn gen_mod_u64(&mut self, n: u64) -> u64 { |
| + // Adapted from: https://lemire.me/blog/2016/06/30/fast-random-shuffling/ |
| + let mut r = self.gen_u64(); |
| + let mut hi = mul_high_u64(r, n); |
| + let mut lo = r.wrapping_mul(n); |
| + if lo < n { |
| + let t = n.wrapping_neg() % n; |
| + while lo < t { |
| + r = self.gen_u64(); |
| + hi = mul_high_u64(r, n); |
| + lo = r.wrapping_mul(n); |
| + } |
| + } |
| + hi |
| + } |
| + |
| + /// Generates a random `u128` in `0..n`. |
| + #[inline] |
| + fn gen_mod_u128(&mut self, n: u128) -> u128 { |
| + // Adapted from: https://lemire.me/blog/2016/06/30/fast-random-shuffling/ |
| + let mut r = self.gen_u128(); |
| + let mut hi = mul_high_u128(r, n); |
| + let mut lo = r.wrapping_mul(n); |
| + if lo < n { |
| + let t = n.wrapping_neg() % n; |
| + while lo < t { |
| + r = self.gen_u128(); |
| + hi = mul_high_u128(r, n); |
| + lo = r.wrapping_mul(n); |
| + } |
| + } |
| + hi |
| + } |
| +} |
| + |
| +/// Computes `(a * b) >> 32`. |
| +#[inline] |
| +fn mul_high_u32(a: u32, b: u32) -> u32 { |
| + (((a as u64) * (b as u64)) >> 32) as u32 |
| +} |
| + |
| +/// Computes `(a * b) >> 64`. |
| +#[inline] |
| +fn mul_high_u64(a: u64, b: u64) -> u64 { |
| + (((a as u128) * (b as u128)) >> 64) as u64 |
| +} |
| + |
| +/// Computes `(a * b) >> 128`. |
| +#[inline] |
| +fn mul_high_u128(a: u128, b: u128) -> u128 { |
| + // Adapted from: https://stackoverflow.com/a/28904636 |
| + let a_lo = a as u64 as u128; |
| + let a_hi = (a >> 64) as u64 as u128; |
| + let b_lo = b as u64 as u128; |
| + let b_hi = (b >> 64) as u64 as u128; |
| + let carry = (a_lo * b_lo) >> 64; |
| + let carry = ((a_hi * b_lo) as u64 as u128 + (a_lo * b_hi) as u64 as u128 + carry) >> 64; |
| + a_hi * b_hi + ((a_hi * b_lo) >> 64) + ((a_lo * b_hi) >> 64) + carry |
| +} |
| + |
| +macro_rules! rng_integer { |
| + ($t:tt, $unsigned_t:tt, $gen:tt, $mod:tt, $doc:tt) => { |
| + #[doc = $doc] |
| + /// |
| + /// Panics if the range is empty. |
| + #[inline] |
| + pub fn $t(&mut self, range: impl RangeBounds<$t>) -> $t { |
| + let panic_empty_range = || { |
| + panic!( |
| + "empty range: {:?}..{:?}", |
| + range.start_bound(), |
| + range.end_bound() |
| + ) |
| + }; |
| + |
| + let low = match range.start_bound() { |
| + Bound::Unbounded => $t::MIN, |
| + Bound::Included(&x) => x, |
| + Bound::Excluded(&x) => x.checked_add(1).unwrap_or_else(panic_empty_range), |
| + }; |
| + |
| + let high = match range.end_bound() { |
| + Bound::Unbounded => $t::MAX, |
| + Bound::Included(&x) => x, |
| + Bound::Excluded(&x) => x.checked_sub(1).unwrap_or_else(panic_empty_range), |
| + }; |
| + |
| + if low > high { |
| + panic_empty_range(); |
| + } |
| + |
| + if low == $t::MIN && high == $t::MAX { |
| + self.$gen() as $t |
| + } else { |
| + let len = high.wrapping_sub(low).wrapping_add(1); |
| + low.wrapping_add(self.$mod(len as $unsigned_t as _) as $t) |
| + } |
| + } |
| + }; |
| +} |
| + |
| +impl Rng { |
| + /// Creates a new random number generator with the initial seed. |
| + #[inline] |
| + #[must_use = "this creates a new instance of `Rng`; if you want to initialize the thread-local generator, use `fastrand::seed()` instead"] |
| + pub const fn with_seed(seed: u64) -> Self { |
| + Rng(seed) |
| + } |
| + |
| + /// Clones the generator by deterministically deriving a new generator based on the initial |
| + /// seed. |
| + /// |
| + /// This function can be used to create a new generator that is a "spinoff" of the old |
| + /// generator. The new generator will not produce the same sequence of values as the |
| + /// old generator. |
| + /// |
| + /// # Example |
| + /// |
| + /// ``` |
| + /// // Seed two generators equally, and clone both of them. |
| + /// let mut base1 = fastrand::Rng::with_seed(0x4d595df4d0f33173); |
| + /// base1.bool(); // Use the generator once. |
| + /// |
| + /// let mut base2 = fastrand::Rng::with_seed(0x4d595df4d0f33173); |
| + /// base2.bool(); // Use the generator once. |
| + /// |
| + /// let mut rng1 = base1.fork(); |
| + /// let mut rng2 = base2.fork(); |
| + /// |
| + /// println!("rng1 returns {}", rng1.u32(..)); |
| + /// println!("rng2 returns {}", rng2.u32(..)); |
| + /// ``` |
| + #[inline] |
| + #[must_use = "this creates a new instance of `Rng`"] |
| + pub fn fork(&mut self) -> Self { |
| + Rng::with_seed(self.gen_u64()) |
| + } |
| + |
| + /// Generates a random `char` in range a-z and A-Z. |
| + #[inline] |
| + pub fn alphabetic(&mut self) -> char { |
| + const CHARS: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"; |
| + *self.choice(CHARS).unwrap() as char |
| + } |
| + |
| + /// Generates a random `char` in range a-z, A-Z and 0-9. |
| + #[inline] |
| + pub fn alphanumeric(&mut self) -> char { |
| + const CHARS: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"; |
| + *self.choice(CHARS).unwrap() as char |
| + } |
| + |
| + /// Generates a random `bool`. |
| + #[inline] |
| + pub fn bool(&mut self) -> bool { |
| + self.u8(..) % 2 == 0 |
| + } |
| + |
| + /// Generates a random digit in the given `base`. |
| + /// |
| + /// Digits are represented by `char`s in range 0-9 and a-z. |
| + /// |
| + /// Panics if the base is zero or greater than 36. |
| + #[inline] |
| + pub fn digit(&mut self, base: u32) -> char { |
| + if base == 0 { |
| + panic!("base cannot be zero"); |
| + } |
| + if base > 36 { |
| + panic!("base cannot be larger than 36"); |
| + } |
| + let num = self.u8(..base as u8); |
| + if num < 10 { |
| + (b'0' + num) as char |
| + } else { |
| + (b'a' + num - 10) as char |
| + } |
| + } |
| + |
| + /// Generates a random `f32` in range `0..=1`. |
| + #[inline] |
| + pub fn f32_inclusive(&mut self) -> f32 { |
| + // Generate a number in 0..2^63 then convert to f32 and multiply by 2^(-63). |
| + // |
| + // Even though we're returning f32, we still generate u64 internally to make |
| + // it possible to return nonzero numbers as small as 2^(-63). If we only |
| + // generated u32 internally, the smallest nonzero number we could return |
| + // would be 2^(-32). |
| + // |
| + // The integer we generate is in 0..2^63 rather than 0..2^64 to improve speed |
| + // on x86-64, which has efficient i64->float conversion (cvtsi2ss) but for |
| + // which u64->float conversion must be implemented in software. |
| + // |
| + // There is still some remaining bias in the int-to-float conversion, because |
| + // nonzero numbers <=2^(-64) are never generated, even though they are |
| + // expressible in f32. However, at this point the bias in int-to-float conversion |
| + // is no larger than the bias in the underlying WyRand generator: since it only |
| + // has a 64-bit state, it necessarily already have biases of at least 2^(-64) |
| + // probability. |
| + // |
| + // See e.g. Section 3.1 of Thomas, David B., et al. "Gaussian random number generators, |
| + // https://www.doc.ic.ac.uk/~wl/papers/07/csur07dt.pdf, for background. |
| + const MUL: f32 = 1.0 / (1u64 << 63) as f32; |
| + (self.gen_u64() >> 1) as f32 * MUL |
| + } |
| + |
| + /// Generates a random `f32` in range `0..1`. |
| + /// |
| + /// Function `f32_inclusive()` is a little simpler and faster, so default |
| + /// to that if inclusive range is acceptable. |
| + #[inline] |
| + pub fn f32(&mut self) -> f32 { |
| + loop { |
| + let x = self.f32_inclusive(); |
| + if x < 1.0 { |
| + return x; |
| + } |
| + } |
| + } |
| + |
| + /// Generates a random `f64` in range `0..=1`. |
| + #[inline] |
| + pub fn f64_inclusive(&mut self) -> f64 { |
| + // See the comment in f32_inclusive() for more details. |
| + const MUL: f64 = 1.0 / (1u64 << 63) as f64; |
| + (self.gen_u64() >> 1) as f64 * MUL |
| + } |
| + |
| + /// Generates a random `f64` in range `0..1`. |
| + /// |
| + /// Function `f64_inclusive()` is a little simpler and faster, so default |
| + /// to that if inclusive range is acceptable. |
| + #[inline] |
| + pub fn f64(&mut self) -> f64 { |
| + loop { |
| + let x = self.f64_inclusive(); |
| + if x < 1.0 { |
| + return x; |
| + } |
| + } |
| + } |
| + |
| + /// Collects `amount` values at random from the iterable into a vector. |
| + /// |
| + /// The length of the returned vector equals `amount` unless the iterable |
| + /// contains insufficient elements, in which case it equals the number of |
| + /// elements available. |
| + /// |
| + /// Complexity is `O(n)` where `n` is the length of the iterable. |
| + #[cfg(feature = "alloc")] |
| + pub fn choose_multiple<I: IntoIterator>(&mut self, source: I, amount: usize) -> Vec<I::Item> { |
| + // Adapted from: https://docs.rs/rand/latest/rand/seq/trait.IteratorRandom.html#method.choose_multiple |
| + let mut reservoir = Vec::with_capacity(amount); |
| + let mut iter = source.into_iter(); |
| + |
| + reservoir.extend(iter.by_ref().take(amount)); |
| + |
| + // Continue unless the iterator was exhausted |
| + // |
| + // note: this prevents iterators that "restart" from causing problems. |
| + // If the iterator stops once, then so do we. |
| + if reservoir.len() == amount { |
| + for (i, elem) in iter.enumerate() { |
| + let end = i + 1 + amount; |
| + let k = self.usize(0..end); |
| + if let Some(slot) = reservoir.get_mut(k) { |
| + *slot = elem; |
| + } |
| + } |
| + } else { |
| + // If less than one third of the `Vec` was used, reallocate |
| + // so that the unused space is not wasted. There is a corner |
| + // case where `amount` was much less than `self.len()`. |
| + if reservoir.capacity() > 3 * reservoir.len() { |
| + reservoir.shrink_to_fit(); |
| + } |
| + } |
| + reservoir |
| + } |
| + |
| + rng_integer!( |
| + i8, |
| + u8, |
| + gen_u32, |
| + gen_mod_u32, |
| + "Generates a random `i8` in the given range." |
| + ); |
| + |
| + rng_integer!( |
| + i16, |
| + u16, |
| + gen_u32, |
| + gen_mod_u32, |
| + "Generates a random `i16` in the given range." |
| + ); |
| + |
| + rng_integer!( |
| + i32, |
| + u32, |
| + gen_u32, |
| + gen_mod_u32, |
| + "Generates a random `i32` in the given range." |
| + ); |
| + |
| + rng_integer!( |
| + i64, |
| + u64, |
| + gen_u64, |
| + gen_mod_u64, |
| + "Generates a random `i64` in the given range." |
| + ); |
| + |
| + rng_integer!( |
| + i128, |
| + u128, |
| + gen_u128, |
| + gen_mod_u128, |
| + "Generates a random `i128` in the given range." |
| + ); |
| + |
| + #[cfg(target_pointer_width = "16")] |
| + rng_integer!( |
| + isize, |
| + usize, |
| + gen_u32, |
| + gen_mod_u32, |
| + "Generates a random `isize` in the given range." |
| + ); |
| + #[cfg(target_pointer_width = "32")] |
| + rng_integer!( |
| + isize, |
| + usize, |
| + gen_u32, |
| + gen_mod_u32, |
| + "Generates a random `isize` in the given range." |
| + ); |
| + #[cfg(target_pointer_width = "64")] |
| + rng_integer!( |
| + isize, |
| + usize, |
| + gen_u64, |
| + gen_mod_u64, |
| + "Generates a random `isize` in the given range." |
| + ); |
| + |
| + /// Generates a random `char` in range a-z. |
| + #[inline] |
| + pub fn lowercase(&mut self) -> char { |
| + const CHARS: &[u8] = b"abcdefghijklmnopqrstuvwxyz"; |
| + *self.choice(CHARS).unwrap() as char |
| + } |
| + |
| + /// Initializes this generator with the given seed. |
| + #[inline] |
| + pub fn seed(&mut self, seed: u64) { |
| + self.0 = seed; |
| + } |
| + |
| + /// Gives back **current** seed that is being held by this generator. |
| + #[inline] |
| + pub fn get_seed(&self) -> u64 { |
| + self.0 |
| + } |
| + |
| + /// Choose an item from an iterator at random. |
| + /// |
| + /// This function may have an unexpected result if the `len()` property of the |
| + /// iterator does not match the actual number of items in the iterator. If |
| + /// the iterator is empty, this returns `None`. |
| + #[inline] |
| + pub fn choice<I>(&mut self, iter: I) -> Option<I::Item> |
| + where |
| + I: IntoIterator, |
| + I::IntoIter: ExactSizeIterator, |
| + { |
| + let mut iter = iter.into_iter(); |
| + |
| + // Get the item at a random index. |
| + let len = iter.len(); |
| + if len == 0 { |
| + return None; |
| + } |
| + let index = self.usize(0..len); |
| + |
| + iter.nth(index) |
| + } |
| + |
| + /// Shuffles a slice randomly. |
| + #[inline] |
| + pub fn shuffle<T>(&mut self, slice: &mut [T]) { |
| + for i in 1..slice.len() { |
| + slice.swap(i, self.usize(..=i)); |
| + } |
| + } |
| + |
| + /// Fill a byte slice with random data. |
| + #[inline] |
| + pub fn fill(&mut self, slice: &mut [u8]) { |
| + // We fill the slice by chunks of 8 bytes, or one block of |
| + // WyRand output per new state. |
| + let mut chunks = slice.chunks_exact_mut(core::mem::size_of::<u64>()); |
| + for chunk in chunks.by_ref() { |
| + let n = self.gen_u64().to_ne_bytes(); |
| + // Safe because the chunks are always 8 bytes exactly. |
| + chunk.copy_from_slice(&n); |
| + } |
| + |
| + let remainder = chunks.into_remainder(); |
| + |
| + // Any remainder will always be less than 8 bytes. |
| + if !remainder.is_empty() { |
| + // Generate one last block of 8 bytes of entropy |
| + let n = self.gen_u64().to_ne_bytes(); |
| + |
| + // Use the remaining length to copy from block |
| + remainder.copy_from_slice(&n[..remainder.len()]); |
| + } |
| + } |
| + |
| + rng_integer!( |
| + u8, |
| + u8, |
| + gen_u32, |
| + gen_mod_u32, |
| + "Generates a random `u8` in the given range." |
| + ); |
| + |
| + rng_integer!( |
| + u16, |
| + u16, |
| + gen_u32, |
| + gen_mod_u32, |
| + "Generates a random `u16` in the given range." |
| + ); |
| + |
| + rng_integer!( |
| + u32, |
| + u32, |
| + gen_u32, |
| + gen_mod_u32, |
| + "Generates a random `u32` in the given range." |
| + ); |
| + |
| + rng_integer!( |
| + u64, |
| + u64, |
| + gen_u64, |
| + gen_mod_u64, |
| + "Generates a random `u64` in the given range." |
| + ); |
| + |
| + rng_integer!( |
| + u128, |
| + u128, |
| + gen_u128, |
| + gen_mod_u128, |
| + "Generates a random `u128` in the given range." |
| + ); |
| + |
| + #[cfg(target_pointer_width = "16")] |
| + rng_integer!( |
| + usize, |
| + usize, |
| + gen_u32, |
| + gen_mod_u32, |
| + "Generates a random `usize` in the given range." |
| + ); |
| + #[cfg(target_pointer_width = "32")] |
| + rng_integer!( |
| + usize, |
| + usize, |
| + gen_u32, |
| + gen_mod_u32, |
| + "Generates a random `usize` in the given range." |
| + ); |
| + #[cfg(target_pointer_width = "64")] |
| + rng_integer!( |
| + usize, |
| + usize, |
| + gen_u64, |
| + gen_mod_u64, |
| + "Generates a random `usize` in the given range." |
| + ); |
| + |
| + /// Generates a random `char` in range A-Z. |
| + #[inline] |
| + pub fn uppercase(&mut self) -> char { |
| + const CHARS: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZ"; |
| + *self.choice(CHARS).unwrap() as char |
| + } |
| + |
| + /// Generates a random `char` in the given range. |
| + /// |
| + /// Panics if the range is empty. |
| + #[inline] |
| + pub fn char(&mut self, range: impl RangeBounds<char>) -> char { |
| + let panic_empty_range = || { |
| + panic!( |
| + "empty range: {:?}..{:?}", |
| + range.start_bound(), |
| + range.end_bound() |
| + ) |
| + }; |
| + |
| + let surrogate_start = 0xd800u32; |
| + let surrogate_len = 0x800u32; |
| + |
| + let low = match range.start_bound() { |
| + Bound::Unbounded => 0u8 as char, |
| + Bound::Included(&x) => x, |
| + Bound::Excluded(&x) => { |
| + let scalar = if x as u32 == surrogate_start - 1 { |
| + surrogate_start + surrogate_len |
| + } else { |
| + x as u32 + 1 |
| + }; |
| + char::try_from(scalar).unwrap_or_else(|_| panic_empty_range()) |
| + } |
| + }; |
| + |
| + let high = match range.end_bound() { |
| + Bound::Unbounded => core::char::MAX, |
| + Bound::Included(&x) => x, |
| + Bound::Excluded(&x) => { |
| + let scalar = if x as u32 == surrogate_start + surrogate_len { |
| + surrogate_start - 1 |
| + } else { |
| + (x as u32).wrapping_sub(1) |
| + }; |
| + char::try_from(scalar).unwrap_or_else(|_| panic_empty_range()) |
| + } |
| + }; |
| + |
| + if low > high { |
| + panic_empty_range(); |
| + } |
| + |
| + let gap = if (low as u32) < surrogate_start && (high as u32) >= surrogate_start { |
| + surrogate_len |
| + } else { |
| + 0 |
| + }; |
| + let range = high as u32 - low as u32 - gap; |
| + let mut val = self.u32(0..=range) + low as u32; |
| + if val >= surrogate_start { |
| + val += gap; |
| + } |
| + val.try_into().unwrap() |
| + } |
| +} |
diff --git a/core/vendor/fastrand-2.5.0/tests/char.rs b/core/vendor/fastrand-2.5.0/tests/char.rs| new file mode 100644 |
| index 0000000..0f48c57 |
| --- /dev/null |
| +++ b/core/vendor/fastrand-2.5.0/tests/char.rs |
| @@ -0,0 +1,44 @@ |
| +use std::convert::TryFrom; |
| +use std::ops::RangeBounds; |
| + |
| +fn test_char_coverage<R>(n: usize, range: R) |
| +where |
| + R: Iterator<Item = char> + RangeBounds<char> + Clone, |
| +{ |
| + use std::collections::HashSet; |
| + |
| + let all: HashSet<char> = range.clone().collect(); |
| + let mut covered = HashSet::new(); |
| + for _ in 0..n { |
| + let c = fastrand::char(range.clone()); |
| + assert!(all.contains(&c)); |
| + covered.insert(c); |
| + } |
| + assert_eq!(covered, all); |
| +} |
| + |
| +#[test] |
| +fn test_char() { |
| + // ASCII control chars. |
| + let nul = 0u8 as char; |
| + let soh = 1u8 as char; |
| + let stx = 2u8 as char; |
| + // Some undefined Hangul Jamo codepoints just before |
| + // the surrogate area. |
| + let last_jamo = char::try_from(0xd7ffu32).unwrap(); |
| + let penultimate_jamo = char::try_from(last_jamo as u32 - 1).unwrap(); |
| + // Private-use codepoints just after the surrogate area. |
| + let first_private = char::try_from(0xe000u32).unwrap(); |
| + let second_private = char::try_from(first_private as u32 + 1).unwrap(); |
| + // Private-use codepoints at the end of Unicode space. |
| + let last_private = std::char::MAX; |
| + let penultimate_private = char::try_from(last_private as u32 - 1).unwrap(); |
| + |
| + test_char_coverage(100, nul..stx); |
| + test_char_coverage(100, nul..=soh); |
| + |
| + test_char_coverage(400, penultimate_jamo..second_private); |
| + test_char_coverage(400, penultimate_jamo..=second_private); |
| + |
| + test_char_coverage(100, penultimate_private..=last_private); |
| +} |
diff --git a/core/vendor/fastrand-2.5.0/tests/smoke.rs b/core/vendor/fastrand-2.5.0/tests/smoke.rs| new file mode 100644 |
| index 0000000..ea200b9 |
| --- /dev/null |
| +++ b/core/vendor/fastrand-2.5.0/tests/smoke.rs |
| @@ -0,0 +1,357 @@ |
| +#[cfg(all(target_family = "wasm", not(target_os = "wasi")))] |
| +use wasm_bindgen_test::wasm_bindgen_test as test; |
| + |
| +#[cfg(all(target_family = "wasm", not(target_os = "wasi")))] |
| +wasm_bindgen_test::wasm_bindgen_test_configure!(run_in_browser); |
| + |
| +#[test] |
| +fn bool() { |
| + for x in &[false, true] { |
| + while fastrand::bool() != *x {} |
| + } |
| +} |
| + |
| +#[test] |
| +fn u8() { |
| + for x in 0..10 { |
| + while fastrand::u8(..10) != x {} |
| + } |
| + |
| + for x in 200..=u8::MAX { |
| + while fastrand::u8(200..) != x {} |
| + } |
| +} |
| + |
| +#[test] |
| +fn i8() { |
| + for x in -128..-120 { |
| + while fastrand::i8(..-120) != x {} |
| + } |
| + |
| + for x in 120..=127 { |
| + while fastrand::i8(120..) != x {} |
| + } |
| +} |
| + |
| +#[test] |
| +fn u32() { |
| + for n in 1u32..10_000 { |
| + let n = n.wrapping_mul(n); |
| + let n = n.wrapping_mul(n); |
| + if n != 0 { |
| + for _ in 0..1000 { |
| + assert!(fastrand::u32(..n) < n); |
| + } |
| + } |
| + } |
| +} |
| + |
| +#[test] |
| +fn u64() { |
| + for n in 1u64..10_000 { |
| + let n = n.wrapping_mul(n); |
| + let n = n.wrapping_mul(n); |
| + let n = n.wrapping_mul(n); |
| + if n != 0 { |
| + for _ in 0..1000 { |
| + assert!(fastrand::u64(..n) < n); |
| + } |
| + } |
| + } |
| +} |
| + |
| +#[test] |
| +fn u128() { |
| + for n in 1u128..10_000 { |
| + let n = n.wrapping_mul(n); |
| + let n = n.wrapping_mul(n); |
| + let n = n.wrapping_mul(n); |
| + let n = n.wrapping_mul(n); |
| + if n != 0 { |
| + for _ in 0..1000 { |
| + assert!(fastrand::u128(..n) < n); |
| + } |
| + } |
| + } |
| +} |
| + |
| +#[test] |
| +fn f32() { |
| + let mut r = fastrand::Rng::with_seed(0); |
| + let tiny = (-24.0f32).exp2(); |
| + let mut count_tiny_nonzero = 0; |
| + let mut count_top_half = 0; |
| + for _ in 0..100_000_000 { |
| + let x = r.f32(); |
| + assert!((0.0..1.0).contains(&x)); |
| + if x > 0.0 && x < tiny { |
| + count_tiny_nonzero += 1; |
| + } else if x > 0.5 { |
| + count_top_half += 1; |
| + } |
| + } |
| + assert!(count_top_half >= 49_000_000); |
| + assert!(count_tiny_nonzero > 0); |
| +} |
| + |
| +#[test] |
| +fn f32_inclusive() { |
| + let mut r = fastrand::Rng::with_seed(0); |
| + let tiny = (-24.0f32).exp2(); |
| + let mut count_top_half = 0; |
| + let mut count_tiny_nonzero = 0; |
| + let mut count_one = 0; |
| + for _ in 0..100_000_000 { |
| + let x = r.f32_inclusive(); |
| + assert!((0.0..=1.0).contains(&x)); |
| + if x == 1.0 { |
| + count_one += 1; |
| + } else if x > 0.5 { |
| + count_top_half += 1; |
| + } else if x > 0.0 && x < tiny { |
| + count_tiny_nonzero += 1; |
| + } |
| + } |
| + assert!(count_top_half >= 49_000_000); |
| + assert!(count_one > 0); |
| + assert!(count_tiny_nonzero > 0); |
| +} |
| + |
| +#[test] |
| +fn f64() { |
| + let mut r = fastrand::Rng::with_seed(0); |
| + let mut count_top_half = 0; |
| + for _ in 0..100_000_000 { |
| + let x = r.f64(); |
| + assert!((0.0..1.0).contains(&x)); |
| + if x > 0.5 { |
| + count_top_half += 1; |
| + } |
| + } |
| + assert!(count_top_half >= 49_000_000); |
| +} |
| + |
| +#[test] |
| +fn f64_inclusive() { |
| + let mut r = fastrand::Rng::with_seed(0); |
| + let mut count_top_half = 0; |
| + for _ in 0..100_000_000 { |
| + let x = r.f64_inclusive(); |
| + assert!((0.0..=1.0).contains(&x)); |
| + if x > 0.5 { |
| + count_top_half += 1; |
| + } |
| + } |
| + assert!(count_top_half >= 49_000_000); |
| +} |
| + |
| +#[test] |
| +fn digit() { |
| + for base in 1..36 { |
| + let result = fastrand::digit(base); |
| + assert!(result.is_ascii_digit() || result.is_ascii_lowercase()); |
| + } |
| +} |
| + |
| +#[test] |
| +fn global_rng_choice() { |
| + let items = [1, 4, 9, 5, 2, 3, 6, 7, 8, 0]; |
| + |
| + for item in &items { |
| + while fastrand::choice(&items).unwrap() != item {} |
| + } |
| +} |
| + |
| +#[test] |
| +fn global_rng_alphabetic() { |
| + for _ in 0..1000 { |
| + let result = fastrand::alphabetic(); |
| + assert!(result.is_ascii_alphabetic()) |
| + } |
| +} |
| + |
| +#[test] |
| +fn global_rng_lowercase() { |
| + for _ in 0..1000 { |
| + let result = fastrand::lowercase(); |
| + assert!(result.is_ascii_lowercase()) |
| + } |
| +} |
| + |
| +#[test] |
| +fn global_rng_uppercase() { |
| + for _ in 0..1000 { |
| + let result = fastrand::uppercase(); |
| + assert!(result.is_ascii_uppercase()) |
| + } |
| +} |
| + |
| +#[test] |
| +fn fill() { |
| + let mut r = fastrand::Rng::new(); |
| + let mut a = [0u8; 64]; |
| + let mut b = [0u8; 64]; |
| + |
| + r.fill(&mut a); |
| + r.fill(&mut b); |
| + |
| + assert_ne!(a, b); |
| + |
| + let mut a = [0u8; 9]; |
| + let mut b = [0u8; 9]; |
| + |
| + r.fill(&mut a); |
| + r.fill(&mut b); |
| + |
| + assert_ne!(a, b); |
| +} |
| + |
| +#[test] |
| +fn global_rng_fill() { |
| + let mut a = [0u8; 64]; |
| + let mut b = [0u8; 64]; |
| + |
| + fastrand::fill(&mut a); |
| + fastrand::fill(&mut b); |
| + |
| + assert_ne!(a, b); |
| +} |
| + |
| +#[test] |
| +fn rng() { |
| + let mut r = fastrand::Rng::new(); |
| + |
| + assert_ne!(r.u64(..), r.u64(..)); |
| + |
| + r.seed(7); |
| + let a = r.u64(..); |
| + r.seed(7); |
| + let b = r.u64(..); |
| + assert_eq!(a, b); |
| +} |
| + |
| +#[test] |
| +fn rng_init() { |
| + let mut a = fastrand::Rng::new(); |
| + let mut b = fastrand::Rng::new(); |
| + assert_ne!(a.u64(..), b.u64(..)); |
| + |
| + a.seed(7); |
| + b.seed(7); |
| + assert_eq!(a.u64(..), b.u64(..)); |
| +} |
| + |
| +#[test] |
| +fn rng_digit() { |
| + let mut rng = fastrand::Rng::new(); |
| + for base in 1..36 { |
| + let result = rng.digit(base); |
| + assert!(result.is_ascii_digit() || result.is_ascii_lowercase()); |
| + } |
| +} |
| + |
| +#[test] |
| +#[should_panic] |
| +fn rng_digit_panic_1() { |
| + let mut rng = fastrand::Rng::new(); |
| + let _result = rng.digit(0); |
| +} |
| + |
| +#[test] |
| +#[should_panic] |
| +fn rng_digit_panic_2() { |
| + let mut rng = fastrand::Rng::new(); |
| + let base = rng.u32(37..); |
| + let _result = rng.digit(base); |
| +} |
| + |
| +#[test] |
| +fn with_seed() { |
| + let mut a = fastrand::Rng::with_seed(7); |
| + let mut b = fastrand::Rng::new(); |
| + b.seed(7); |
| + assert_eq!(a.u64(..), b.u64(..)); |
| +} |
| + |
| +#[test] |
| +fn choose_multiple() { |
| + let mut a = fastrand::Rng::new(); |
| + let mut elements = (0..20).collect::<Vec<_>>(); |
| + |
| + while !elements.is_empty() { |
| + let chosen = a.choose_multiple(0..20, 5); |
| + for &x in &chosen { |
| + elements.retain(|&y| y != x); |
| + } |
| + } |
| + |
| + let empty_elements: Vec<i32> = Vec::new(); |
| + let empty_result = a.choose_multiple(empty_elements, 5); |
| + assert!(empty_result.is_empty()); |
| +} |
| + |
| +#[test] |
| +fn choice() { |
| + let items = [1, 4, 9, 5, 2, 3, 6, 7, 8, 0]; |
| + let mut r = fastrand::Rng::new(); |
| + |
| + for item in &items { |
| + while r.choice(&items).unwrap() != item {} |
| + } |
| +} |
| + |
| +#[test] |
| +fn choice_empty() { |
| + let mut rng = fastrand::Rng::new(); |
| + let data: Vec<i32> = Vec::new(); |
| + let result = rng.choice(data); |
| + assert!(result.is_none()); |
| +} |
| + |
| +#[test] |
| +fn lowercase() { |
| + let mut rng = fastrand::Rng::new(); |
| + for _ in 0..1000 { |
| + let result = rng.lowercase(); |
| + assert!(result.is_ascii_lowercase()) |
| + } |
| +} |
| + |
| +#[test] |
| +fn alphabetic() { |
| + let mut rng = fastrand::Rng::new(); |
| + for _ in 0..1000 { |
| + let result = rng.alphabetic(); |
| + assert!(result.is_ascii_alphabetic()) |
| + } |
| +} |
| + |
| +#[test] |
| +fn uppercase() { |
| + let mut rng = fastrand::Rng::new(); |
| + for _ in 0..1000 { |
| + let result = rng.uppercase(); |
| + assert!(result.is_ascii_uppercase()) |
| + } |
| +} |
| + |
| +#[test] |
| +#[should_panic] |
| +fn char_panic() { |
| + let mut rng = fastrand::Rng::new(); |
| + let _result = rng.char('z'..='a'); |
| +} |
| + |
| +#[test] |
| +fn char() { |
| + use core::ops::Bound; |
| + let mut rng = fastrand::Rng::new(); |
| + |
| + let result = rng.char(..); |
| + assert!(result >= 0 as char && result <= core::char::MAX); |
| + |
| + let result = rng.char((Bound::Excluded('0'), Bound::Excluded('9'))); |
| + for _ in 0..1000 { |
| + assert!(result > '0' && result < '9'); |
| + } |
| +} |
diff --git a/go.mod b/go.mod| index aa3f75a..c92f833 100644 |
| --- a/go.mod |
| +++ b/go.mod |
| @@ -6,6 +6,7 @@ require ( |
| amadan.net/rastrillo/rastrillo v0.26.0 |
| github.com/chromedp/chromedp v0.14.2 |
| github.com/go-chi/chi/v5 v5.3.2 |
| + github.com/tetratelabs/wazero v1.12.0 |
| gorm.io/gorm v1.31.2 |
| ) |
diff --git a/go.sum b/go.sum| index b4c751d..87e1671 100644 |
| --- a/go.sum |
| +++ b/go.sum |
| @@ -46,6 +46,8 @@ github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhA |
| github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0= |
| github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= |
| github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= |
| +github.com/tetratelabs/wazero v1.12.0 h1:DuWcpNu/FzgEXgGBDp8J1Spc+CWOvvtvVyjKlaZopYU= |
| +github.com/tetratelabs/wazero v1.12.0/go.mod h1:LvKtzl2RqO4gyF27BiXU+nKAjcV8f38U+kP/q2vgxh0= |
| golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ= |
| golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0= |
| golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM= |
diff --git a/internal/core/core.go b/internal/core/core.go| new file mode 100644 |
| index 0000000..1c1116a |
| --- /dev/null |
| +++ b/internal/core/core.go |
| @@ -0,0 +1,637 @@ |
| +// Package core hosts the Rust deck core as a WebAssembly module. |
| +// |
| +// The module is plain wasm32 with a C ABI and no wasm-bindgen, because |
| +// wazero provides no JS glue. Everything crosses the boundary as integers |
| +// and offsets into the module's linear memory. |
| +// |
| +// # Two failure modes the host must contain, not merely survive |
| +// |
| +// yrs decodes wire bytes with std::str::from_utf8_unchecked in places |
| +// (core/src/abi.rs's module doc comment), so a sufficiently corrupted |
| +// update can reach real undefined behaviour and trap the module. A wasm |
| +// trap does not unwind — the module's linear memory is left exactly as it |
| +// was at the moment of the trap, which for a thread-local RefCell borrow |
| +// guard means "permanently borrowed". The instance is not recoverable; |
| +// every Handle here is backed by exactly one module instance per document, |
| +// and a trap discards that instance rather than reusing it. |
| +// |
| +// Separately, a malformed update need not trap at all: Docs' fuzzing found |
| +// a 36-byte corrupted update that ran the apply export for 163 real CPU |
| +// seconds under an interpreter before returning an ordinary, correct |
| +// answer — not a hang, but on a server accepting updates from anyone with |
| +// a share link it is a denial-of-service vector no correctness test can |
| +// see. Every guest call here runs under a context.WithTimeout (default one |
| +// second, tunable per Runtime via WithGuestCallTimeout), and the Runtime is |
| +// built with WithCloseOnContextDone(true) so wazero actually terminates a |
| +// call that overruns its budget rather than merely making the caller stop |
| +// waiting. A deadline exceeded is treated exactly like a trap: the instance |
| +// is discarded and a fresh one is rebuilt. |
| +// |
| +// # Rebuilding is log replay |
| +// |
| +// Every Handle keeps, in memory, every update it has itself applied |
| +// successfully — the bytes given to Apply, and the bytes Ops produced — |
| +// up to maxRetainedUpdates, and a discarded instance is rebuilt by |
| +// replaying that log into a fresh document. Nothing in here is persisted; |
| +// starting a document from a stored snapshot (Handle.State's bytes) plus |
| +// the updates since is the caller's arrangement, exactly as in Docs, and |
| +// the cap on the in-memory log exists because within this package |
| +// "replay everything" is the only recovery there is. |
| +package core |
| + |
| +import ( |
| + "context" |
| + _ "embed" |
| + "errors" |
| + "fmt" |
| + "time" |
| + |
| + "github.com/tetratelabs/wazero" |
| + "github.com/tetratelabs/wazero/api" |
| +) |
| + |
| +//go:embed testdata/slides_core.wasm |
| +var moduleWASM []byte |
| + |
| +// abiVersion is the contract this host understands. A module reporting |
| +// anything else is refused at load, so a stale .wasm (see |
| +// scripts/build-wasm.sh and the copy into testdata) fails loudly rather |
| +// than behaving strangely. It matches core/src/lib.rs's |
| +// slides_core_abi_version, which explains when it must move. |
| +const abiVersion = 1 |
| + |
| +// ABIVersion is abiVersion, exported for callers outside this package that |
| +// need to know which ABI contract this host was built against. It is the |
| +// same constant, not a second number that could drift from it. |
| +const ABIVersion = abiVersion |
| + |
| +// defaultGuestCallTimeout bounds every single call into the wasm module, |
| +// unless overridden with WithGuestCallTimeout. One second keeps roughly |
| +// 1000x headroom over real decks (sub-millisecond) while cutting off |
| +// Docs' measured pathological input (~4.3s under wazero) at about a |
| +// quarter of its natural running time. |
| +const defaultGuestCallTimeout = 1 * time.Second |
| + |
| +// maxRetainedUpdates bounds how many previously-applied updates a Handle |
| +// will keep for the log-replay recovery described above. Once a |
| +// document's log is full, Apply and Ops refuse further work with |
| +// ErrUpdateLogFull rather than either growing the log without bound or |
| +// silently dropping old entries — dropping old entries would make a later |
| +// rebuild replay an incomplete history and produce silently wrong content, |
| +// which is worse than refusing. The real fix for a document that keeps |
| +// being edited past that point is a snapshot (Handle.State) persisted by |
| +// the caller and replayed from, not a bigger number here. |
| +const maxRetainedUpdates = 4096 |
| + |
| +// MaxRetainedUpdates is maxRetainedUpdates, exported for callers outside |
| +// this package that need the same cap. |
| +const MaxRetainedUpdates = maxRetainedUpdates |
| + |
| +// ErrUpdateLogFull is returned by Handle.Apply and Handle.Ops once a |
| +// document's retained update log has reached maxRetainedUpdates. See that |
| +// constant's doc comment. |
| +var ErrUpdateLogFull = errors.New("update log exceeds the retained limit; the document needs a snapshot before it can accept further updates") |
| + |
| +// Option configures a Runtime. See WithGuestCallTimeout. |
| +type Option func(*runtimeOptions) |
| + |
| +type runtimeOptions struct { |
| + callTimeout time.Duration |
| +} |
| + |
| +// WithGuestCallTimeout overrides defaultGuestCallTimeout for one Runtime. |
| +// Exists so the cap can be tuned (for a slower/faster production host, or |
| +// for a test that wants a tight budget without waiting out the default) in |
| +// a config value instead of a recompile. |
| +func WithGuestCallTimeout(d time.Duration) Option { |
| + return func(o *runtimeOptions) { o.callTimeout = d } |
| +} |
| + |
| +// Runtime owns the compiled module. Compiling is expensive and |
| +// instantiating is not, so one Runtime is shared and each document gets its |
| +// own instance (instances are not safe for concurrent use). |
| +type Runtime struct { |
| + rt wazero.Runtime |
| + compiled wazero.CompiledModule |
| + |
| + // callTimeout bounds every guest call made through this Runtime and |
| + // every Handle it creates. See defaultGuestCallTimeout and |
| + // WithGuestCallTimeout. |
| + callTimeout time.Duration |
| +} |
| + |
| +func New(ctx context.Context, opts ...Option) (*Runtime, error) { |
| + o := runtimeOptions{callTimeout: defaultGuestCallTimeout} |
| + for _, opt := range opts { |
| + opt(&o) |
| + } |
| + |
| + // WithCloseOnContextDone(true): without this, a context deadline or |
| + // cancellation on a guest call only makes the *caller* stop waiting — |
| + // the underlying goroutine executing wasm keeps running to completion. |
| + // With it, wazero itself terminates the call and closes the module |
| + // when the context is done, which is what turns a pathologically slow |
| + // input into a bounded failure instead of a resource leak. |
| + cfg := wazero.NewRuntimeConfig().WithCloseOnContextDone(true) |
| + rt := wazero.NewRuntimeWithConfig(ctx, cfg) |
| + compiled, err := rt.CompileModule(ctx, moduleWASM) |
| + if err != nil { |
| + rt.Close(ctx) |
| + return nil, fmt.Errorf("compiling core module: %w", err) |
| + } |
| + return &Runtime{rt: rt, compiled: compiled, callTimeout: o.callTimeout}, nil |
| +} |
| + |
| +func (r *Runtime) Close(ctx context.Context) error { return r.rt.Close(ctx) } |
| + |
| +// instantiate creates a fresh module instance and checks its reported ABI |
| +// version. Used both for a brand-new document and to rebuild after a |
| +// discarded (trapped or timed-out) instance. |
| +func (r *Runtime) instantiate(ctx context.Context) (api.Module, error) { |
| + cctx, cancel := context.WithTimeout(ctx, r.callTimeout) |
| + defer cancel() |
| + // Empty name: wazero treats an anonymous module (ModuleName == "") as |
| + // exempt from its store's name-uniqueness check, so instantiating many |
| + // of these concurrently — one per document, plus rebuilds — never |
| + // collides. |
| + mod, err := r.rt.InstantiateModule(cctx, r.compiled, wazero.NewModuleConfig().WithName("")) |
| + if err != nil { |
| + return nil, fmt.Errorf("instantiating core module: %w", err) |
| + } |
| + |
| + version, err := callGuest(ctx, r.callTimeout, mod.ExportedFunction("slides_core_abi_version")) |
| + if err != nil { |
| + mod.Close(ctx) |
| + return nil, fmt.Errorf("reading abi version: %w", err) |
| + } |
| + if err := checkABIVersion(uint32(version[0]), abiVersion); err != nil { |
| + mod.Close(ctx) |
| + return nil, err |
| + } |
| + return mod, nil |
| +} |
| + |
| +// checkABIVersion reports whether reported (what a module's |
| +// slides_core_abi_version export returned) matches want (abiVersion, the |
| +// contract this host understands). A pure function so the decision itself |
| +// is unit-testable without a compiled module standing in for every case. |
| +// |
| +// The error text names both numbers deliberately: whoever hits this in |
| +// production needs to know which side is stale (an old host against a |
| +// rebuilt module, or vice versa) without also reading wasm build logs. |
| +func checkABIVersion(reported, want uint32) error { |
| + if reported != want { |
| + return fmt.Errorf("core module reports ABI %d, host understands %d", reported, want) |
| + } |
| + return nil |
| +} |
| + |
| +// callGuest calls fn under the given timeout. A non-nil error here means |
| +// the call trapped, ran past its deadline (in which case wazero has |
| +// already closed the module — WithCloseOnContextDone(true)), or the |
| +// module was otherwise closed out from under it: in every case the |
| +// instance must be treated as poisoned, never merely retried. The error is |
| +// wrapped in trapFault so callers further up can tell it apart from an |
| +// ordinary clean rejection (a negative return code from the guest), which |
| +// does not poison anything — see isFatal. |
| +func callGuest(ctx context.Context, timeout time.Duration, fn api.Function, params ...uint64) ([]uint64, error) { |
| + cctx, cancel := context.WithTimeout(ctx, timeout) |
| + defer cancel() |
| + res, err := fn.Call(cctx, params...) |
| + if err != nil { |
| + return nil, &trapFault{err} |
| + } |
| + return res, nil |
| +} |
| + |
| +// trapFault marks an error as meaning the module instance itself is no |
| +// longer trustworthy: the wasm call that produced it did not return |
| +// normally at all (a trap, a deadline exceeded, or the module having |
| +// already been closed), as opposed to returning normally with a negative |
| +// application error code. Every error that originates from a wazero |
| +// api.Function.Call failing carries this; every error this package |
| +// constructs itself from a clean negative return code does not, so |
| +// isFatal (via errors.As, walking any fmt.Errorf("...: %w", err) wrapping) |
| +// tells the two apart. |
| +type trapFault struct{ err error } |
| + |
| +func (t *trapFault) Error() string { return t.err.Error() } |
| +func (t *trapFault) Unwrap() error { return t.err } |
| + |
| +// isFatal reports whether err means the module instance must be discarded |
| +// (a trap or a deadline exceeded reaching all the way from callGuest) as |
| +// opposed to an ordinary clean rejection, which leaves the instance fully |
| +// usable. |
| +func isFatal(err error) bool { |
| + var t *trapFault |
| + return errors.As(err, &t) |
| +} |
| + |
| +// RecoveredError wraps the error from a call whose instance trapped or |
| +// timed out but was then successfully discarded and rebuilt — so the |
| +// *document* is intact (every previously-applied update survived), even |
| +// though this specific call was not (its outcome is unknown; it is never |
| +// retried). Callers that only check "err != nil" cannot otherwise tell |
| +// that apart from an ordinary clean rejection (where the instance was |
| +// never touched) or from a rebuild that itself failed (a plain joined |
| +// error, where there is no usable instance at all). Use Recovered(err) |
| +// rather than a type assertion. |
| +type RecoveredError struct{ err error } |
| + |
| +func (e *RecoveredError) Error() string { return e.err.Error() } |
| +func (e *RecoveredError) Unwrap() error { return e.err } |
| + |
| +// Recovered reports whether err came from a call that trapped or timed |
| +// out but whose instance was successfully discarded and rebuilt — see |
| +// RecoveredError. |
| +func Recovered(err error) bool { |
| + var r *RecoveredError |
| + return errors.As(err, &r) |
| +} |
| + |
| +// RejectedError wraps the error from a call the guest cleanly refused — |
| +// a negative return code, such as malformed wire bytes yrs's decoder |
| +// catches, or an op batch the core would not apply — without the |
| +// instance ever being touched: the document's existing content is |
| +// completely unaffected, and the instance is exactly as usable afterward |
| +// as it was before the call. This is the ordinary, expected way bad input |
| +// is refused. For Ops the message is the core's own, verbatim, so a |
| +// caller can hand it back to whoever sent the batch. Use Rejected(err) |
| +// rather than a type assertion. |
| +type RejectedError struct{ err error } |
| + |
| +func (e *RejectedError) Error() string { return e.err.Error() } |
| +func (e *RejectedError) Unwrap() error { return e.err } |
| + |
| +// Rejected reports whether err came from a call that was cleanly refused |
| +// with the instance left fully usable — see RejectedError. |
| +func Rejected(err error) bool { |
| + var r *RejectedError |
| + return errors.As(err, &r) |
| +} |
| + |
| +// Handle is one document, backed by its own module instance. Not safe for |
| +// concurrent use; callers serialise access per document. |
| +// |
| +// applied records, in order, every update this handle has itself applied |
| +// successfully — the bytes given to Apply, and the bytes Ops produced — |
| +// up to maxRetainedUpdates. It exists solely so a trapped or timed-out |
| +// instance can be discarded and a fresh one rebuilt by replaying only the |
| +// updates already known to be good; the call that triggered the discard |
| +// is never replayed. |
| +type Handle struct { |
| + rt *Runtime |
| + guid string |
| + clientID uint64 |
| + |
| + mod api.Module |
| + handle uint64 |
| + |
| + applied [][]byte |
| +} |
| + |
| +// NewDoc creates a document. The GUID and client id are supplied by the |
| +// caller and never drawn from yrs: on wasm the CRDT library's entropy is a |
| +// constant seed, so a value it generates is identical in every process, and |
| +// a repeated client id or GUID corrupts the document. |
| +func (r *Runtime) NewDoc(ctx context.Context, guid string, clientID uint64) (*Handle, error) { |
| + mod, err := r.instantiate(ctx) |
| + if err != nil { |
| + return nil, err |
| + } |
| + |
| + handle := &Handle{rt: r, guid: guid, clientID: clientID, mod: mod} |
| + docHandle, err := createDoc(ctx, r.callTimeout, mod, guid, clientID) |
| + if err != nil { |
| + mod.Close(ctx) |
| + return nil, err |
| + } |
| + handle.handle = docHandle |
| + return handle, nil |
| +} |
| + |
| +// createDoc allocates the guid in mod's linear memory, calls |
| +// slides_core_doc_new, and frees the buffer. Shared between NewDoc and a |
| +// post-trap rebuild. |
| +func createDoc(ctx context.Context, timeout time.Duration, mod api.Module, guid string, clientID uint64) (uint64, error) { |
| + guidBytes := []byte(guid) |
| + ptr, err := allocIn(ctx, timeout, mod, uint64(len(guidBytes))) |
| + if err != nil { |
| + return 0, fmt.Errorf("alloc for guid: %w", err) |
| + } |
| + defer freeIn(ctx, timeout, mod, ptr, uint64(len(guidBytes))) |
| + |
| + if !mod.Memory().Write(uint32(ptr), guidBytes) { |
| + return 0, fmt.Errorf("writing guid at %d: out of range", ptr) |
| + } |
| + |
| + res, err := callGuest(ctx, timeout, mod.ExportedFunction("slides_core_doc_new"), ptr, uint64(len(guidBytes)), clientID) |
| + if err != nil { |
| + return 0, fmt.Errorf("doc_new: %w", err) |
| + } |
| + if res[0] == 0 { |
| + return 0, fmt.Errorf("doc_new returned a null handle") |
| + } |
| + return res[0], nil |
| +} |
| + |
| +// allocIn calls slides_core_alloc and checks the 0-on-failure sentinel |
| +// (including any length over isize::MAX — core/src/abi.rs's layout_for). |
| +func allocIn(ctx context.Context, timeout time.Duration, mod api.Module, length uint64) (uint64, error) { |
| + res, err := callGuest(ctx, timeout, mod.ExportedFunction("slides_core_alloc"), length) |
| + if err != nil { |
| + return 0, fmt.Errorf("alloc trapped or timed out: %w", err) |
| + } |
| + if res[0] == 0 { |
| + return 0, fmt.Errorf("alloc(%d) failed", length) |
| + } |
| + return res[0], nil |
| +} |
| + |
| +// freeIn best-efforts a free. A failure here (trap/timeout) is folded into |
| +// "the instance is now poisoned", which the caller's own callGuest error |
| +// (the one that actually matters) will already have triggered, so this |
| +// deliberately does not itself return an error to plumb through defers. |
| +func freeIn(ctx context.Context, timeout time.Duration, mod api.Module, ptr, length uint64) { |
| + _, _ = callGuest(ctx, timeout, mod.ExportedFunction("slides_core_free"), ptr, length) |
| +} |
| + |
| +func (h *Handle) Close(ctx context.Context) error { |
| + _, err := callGuest(ctx, h.rt.callTimeout, h.mod.ExportedFunction("slides_core_doc_free"), h.handle) |
| + if closeErr := h.mod.Close(ctx); err == nil { |
| + err = closeErr |
| + } |
| + return err |
| +} |
| + |
| +// rebuild discards h's current (poisoned) instance and replaces it with a |
| +// fresh one for the same document (guid, clientID), replaying every update |
| +// previously applied successfully. The call that triggered the discard is |
| +// not replayed — it is not in h.applied, since that log is only appended |
| +// to on a clean success. |
| +// |
| +// It deliberately does not run under ctx as given: ctx is the same context |
| +// whose deadline may be exactly what triggered this rebuild. Rebuilding |
| +// under an already-expired deadline would make recovery fail precisely in |
| +// the case it exists for, so this strips ctx's cancellation/deadline |
| +// (context.WithoutCancel) while keeping any values on it, and relies on |
| +// callTimeout to still bound each individual call made during the rebuild |
| +// itself. |
| +func (h *Handle) rebuild(ctx context.Context) error { |
| + rebuildCtx := context.WithoutCancel(ctx) |
| + timeout := h.rt.callTimeout |
| + |
| + // Best-effort: the old instance may already be closed (a timeout under |
| + // WithCloseOnContextDone(true) closes it as part of terminating the |
| + // call), so an error here is expected and not itself a failure. |
| + _ = h.mod.Close(rebuildCtx) |
| + |
| + mod, err := h.rt.instantiate(rebuildCtx) |
| + if err != nil { |
| + return fmt.Errorf("rebuilding instance: %w", err) |
| + } |
| + docHandle, err := createDoc(rebuildCtx, timeout, mod, h.guid, h.clientID) |
| + if err != nil { |
| + mod.Close(rebuildCtx) |
| + return fmt.Errorf("rebuilding instance: %w", err) |
| + } |
| + for i, update := range h.applied { |
| + if err := applyOn(rebuildCtx, timeout, mod, docHandle, update); err != nil { |
| + mod.Close(rebuildCtx) |
| + return fmt.Errorf("rebuilding instance: replaying update %d/%d, previously applied cleanly: %w", |
| + i+1, len(h.applied), err) |
| + } |
| + } |
| + h.mod = mod |
| + h.handle = docHandle |
| + return nil |
| +} |
| + |
| +// withBuffer copies payload into mod's linear memory, calls fn with the |
| +// buffer's (ptr, len), and frees the buffer afterwards. The alloc/write/ |
| +// call/free dance every export taking caller bytes needs, in one place. |
| +func withBuffer(ctx context.Context, timeout time.Duration, mod api.Module, payload []byte, fn func(ptr, length uint64) error) error { |
| + ptr, err := allocIn(ctx, timeout, mod, uint64(len(payload))) |
| + if err != nil { |
| + return fmt.Errorf("alloc: %w", err) |
| + } |
| + defer freeIn(ctx, timeout, mod, ptr, uint64(len(payload))) |
| + |
| + if !mod.Memory().Write(uint32(ptr), payload) { |
| + return fmt.Errorf("writing %d bytes at %d: out of range", len(payload), ptr) |
| + } |
| + return fn(ptr, uint64(len(payload))) |
| +} |
| + |
| +// applyOn runs slides_core_apply against a specific (mod, docHandle) pair — |
| +// factored out so both Handle.Apply and rebuild's replay share the same |
| +// logic. |
| +func applyOn(ctx context.Context, timeout time.Duration, mod api.Module, docHandle uint64, update []byte) error { |
| + return withBuffer(ctx, timeout, mod, update, func(ptr, length uint64) error { |
| + rc, err := callGuest(ctx, timeout, mod.ExportedFunction("slides_core_apply"), docHandle, ptr, length) |
| + if err != nil { |
| + return fmt.Errorf("apply trapped or timed out: %w", err) |
| + } |
| + if code := int32(rc[0]); code != 0 { |
| + return fmt.Errorf("apply rejected the update: code %d", code) |
| + } |
| + return nil |
| + }) |
| +} |
| + |
| +// retain records an update this handle applied cleanly, so a later |
| +// rebuild replays it. Copies: the caller may reuse its slice. |
| +func (h *Handle) retain(update []byte) { |
| + stored := make([]byte, len(update)) |
| + copy(stored, update) |
| + h.applied = append(h.applied, stored) |
| +} |
| + |
| +// contain turns an error from a guest call into what Apply and Ops |
| +// return: a trap or timeout discards and rebuilds the instance (a |
| +// *RecoveredError when that works, a joined error when it does not); |
| +// anything else is a *RejectedError, the instance untouched. |
| +func (h *Handle) contain(ctx context.Context, err error) error { |
| + if isFatal(err) { |
| + if rebuildErr := h.rebuild(ctx); rebuildErr != nil { |
| + return errors.Join(err, rebuildErr) |
| + } |
| + return &RecoveredError{err} |
| + } |
| + return &RejectedError{err} |
| +} |
| + |
| +// Apply applies update to the document. A clean rejection (a negative |
| +// error code from the guest, e.g. malformed wire bytes that yrs's decoder |
| +// catches) leaves the instance untouched and fully usable and comes back |
| +// as a *RejectedError (see Rejected). A trap or a deadline exceeded is |
| +// different: the instance is discarded and rebuilt from every update |
| +// applied here so far, and the triggering update is reported as an error |
| +// but never retried or logged. When that rebuild succeeds, the returned |
| +// error is a *RecoveredError (see Recovered); a rebuild that fails returns |
| +// a plain joined error instead — the one shape this method returns that is |
| +// neither, and the one a caller must not mistake for either: the document |
| +// itself may now be unusable, not merely this one update. |
| +// |
| +// Once the document's retained update log is full (maxRetainedUpdates), |
| +// Apply refuses any further update with ErrUpdateLogFull. |
| +func (h *Handle) Apply(ctx context.Context, update []byte) error { |
| + if len(h.applied) >= maxRetainedUpdates { |
| + return ErrUpdateLogFull |
| + } |
| + if err := applyOn(ctx, h.rt.callTimeout, h.mod, h.handle, update); err != nil { |
| + return h.contain(ctx, err) |
| + } |
| + h.retain(update) |
| + return nil |
| +} |
| + |
| +// Ops applies a JSON array of operations (spec §7.2, core/src/ops.rs) in |
| +// one transaction and returns the Yjs update that transaction produced — |
| +// the bytes to persist and broadcast exactly as a browser's own update |
| +// would be. The update is also retained for rebuilds, so the log stays |
| +// complete whichever way a document is edited. |
| +// |
| +// A batch the core refuses applies nothing and comes back as a |
| +// *RejectedError whose message is the core's own (which op, and why). The |
| +// trap and timeout handling is Apply's. |
| +func (h *Handle) Ops(ctx context.Context, ops []byte) ([]byte, error) { |
| + if len(h.applied) >= maxRetainedUpdates { |
| + return nil, ErrUpdateLogFull |
| + } |
| + timeout := h.rt.callTimeout |
| + var update []byte |
| + err := withBuffer(ctx, timeout, h.mod, ops, func(ptr, length uint64) error { |
| + rc, err := callGuest(ctx, timeout, h.mod.ExportedFunction("slides_core_ops"), h.handle, ptr, length) |
| + if err != nil { |
| + return fmt.Errorf("ops trapped or timed out: %w", err) |
| + } |
| + switch code := int32(rc[0]); code { |
| + case 0: |
| + update, err = readOn(ctx, timeout, h.mod, h.handle, "out") |
| + if err != nil { |
| + return fmt.Errorf("reading the ops update: %w", err) |
| + } |
| + return nil |
| + case errOps: |
| + msg, err := readOn(ctx, timeout, h.mod, h.handle, "err") |
| + if err != nil { |
| + return fmt.Errorf("ops refused the batch, and reading its message failed: %w", err) |
| + } |
| + return errors.New(string(msg)) |
| + default: |
| + return fmt.Errorf("ops failed: code %d", code) |
| + } |
| + }) |
| + if err != nil { |
| + return nil, h.contain(ctx, err) |
| + } |
| + h.retain(update) |
| + return update, nil |
| +} |
| + |
| +// errOps is core/src/abi.rs's ERR_OPS: the batch was refused and the |
| +// message is in the err read pair. |
| +const errOps = -6 |
| + |
| +// State returns the document's entire state as one Yjs update — what a |
| +// caller stores as a snapshot so a later replay can start from it instead |
| +// of from the whole log. Applying it to an empty document reproduces this |
| +// one. |
| +func (h *Handle) State(ctx context.Context) ([]byte, error) { |
| + return h.read(ctx, "state") |
| +} |
| + |
| +// JSON returns the deck's view (core/src/view.rs) — what the drive API |
| +// serves and what the renderer and exporters read. Always a complete JSON |
| +// object for a live handle, never empty. |
| +func (h *Handle) JSON(ctx context.Context) ([]byte, error) { |
| + return h.read(ctx, "json") |
| +} |
| + |
| +// read is the shared body of State and JSON: call the named len/read |
| +// pair, and if the call itself traps or times out, discard the instance, |
| +// rebuild it exactly as Apply does, and retry the read once against the |
| +// fresh one. Retrying rather than reporting an error is deliberate — the |
| +// content is available at that point, and returning nothing plus an error |
| +// would tell the caller the data is gone when it is not. |
| +func (h *Handle) read(ctx context.Context, renderer string) ([]byte, error) { |
| + b, err := readOn(ctx, h.rt.callTimeout, h.mod, h.handle, renderer) |
| + if err == nil { |
| + return b, nil |
| + } |
| + if !isFatal(err) { |
| + return nil, err |
| + } |
| + if rebuildErr := h.rebuild(ctx); rebuildErr != nil { |
| + return nil, errors.Join(err, rebuildErr) |
| + } |
| + b, err = readOn(ctx, h.rt.callTimeout, h.mod, h.handle, renderer) |
| + if err != nil { |
| + return nil, fmt.Errorf("reading %s from the rebuilt instance: %w", renderer, err) |
| + } |
| + return b, nil |
| +} |
| + |
| +// readOn performs one len-then-read dance against slides_core_<name>_len |
| +// and slides_core_<name>_read. name is always one of this package's own |
| +// literals ("state", "json", "out", "err"), never anything derived from a |
| +// request — the export names it builds are a closed set, and a name that |
| +// does not resolve to an exported function surfaces below as a nil |
| +// api.Function. |
| +func readOn(ctx context.Context, timeout time.Duration, mod api.Module, docHandle uint64, name string) ([]byte, error) { |
| + lenFn := mod.ExportedFunction("slides_core_" + name + "_len") |
| + readFn := mod.ExportedFunction("slides_core_" + name + "_read") |
| + if lenFn == nil || readFn == nil { |
| + // Unreachable while abiVersion matches — the version check at |
| + // load is what proves these exports exist — but a nil |
| + // api.Function panics inside callGuest rather than erroring, so |
| + // this turns a future ABI mistake into a returned error instead |
| + // of a panic in a request handler. |
| + return nil, fmt.Errorf("module exports no %q read pair", name) |
| + } |
| + |
| + lenRes, err := callGuest(ctx, timeout, lenFn, docHandle) |
| + if err != nil { |
| + return nil, fmt.Errorf("%s_len trapped or timed out: %w", name, err) |
| + } |
| + // The len functions return i32: negative is an error code, not a |
| + // length. Reading it as unsigned would turn ERR_NO_DOC into |
| + // 0xFFFFFFFF and then try to allocate four gigabytes. |
| + if code := int32(lenRes[0]); code < 0 { |
| + return nil, fmt.Errorf("%s_len failed: code %d", name, code) |
| + } |
| + n := lenRes[0] |
| + if n == 0 { |
| + return nil, nil |
| + } |
| + |
| + ptr, err := allocIn(ctx, timeout, mod, n) |
| + if err != nil { |
| + return nil, fmt.Errorf("alloc: %w", err) |
| + } |
| + defer freeIn(ctx, timeout, mod, ptr, n) |
| + |
| + rc, err := callGuest(ctx, timeout, readFn, docHandle, ptr, n) |
| + if err != nil { |
| + return nil, fmt.Errorf("%s_read trapped or timed out: %w", name, err) |
| + } |
| + written := int32(rc[0]) |
| + if written < 0 { |
| + return nil, fmt.Errorf("%s_read failed: code %d", name, written) |
| + } |
| + |
| + // The read functions return the byte count they wrote, guaranteed |
| + // only <= the len call's answer. Read exactly what the read call says |
| + // it wrote, never the buffer size, so the two can never disagree into |
| + // trailing uninitialised bytes. |
| + buf, ok := mod.Memory().Read(uint32(ptr), uint32(written)) |
| + if !ok { |
| + return nil, fmt.Errorf("reading %d bytes at %d: out of range", written, ptr) |
| + } |
| + // Copy: buf aliases the module's linear memory, which the deferred |
| + // free and any later guest call may reuse. |
| + out := make([]byte, len(buf)) |
| + copy(out, buf) |
| + return out, nil |
| +} |
diff --git a/internal/core/core_test.go b/internal/core/core_test.go| new file mode 100644 |
| index 0000000..2388cf8 |
| --- /dev/null |
| +++ b/internal/core/core_test.go |
| @@ -0,0 +1,199 @@ |
| +package core_test |
| + |
| +import ( |
| + "context" |
| + "encoding/json" |
| + "errors" |
| + "strings" |
| + "testing" |
| + |
| + "amadan.net/oficina/slides/internal/core" |
| +) |
| + |
| +func newRuntime(t *testing.T) (context.Context, *core.Runtime) { |
| + t.Helper() |
| + ctx := context.Background() |
| + rt, err := core.New(ctx) |
| + if err != nil { |
| + t.Fatalf("core.New: %v", err) |
| + } |
| + t.Cleanup(func() { rt.Close(ctx) }) |
| + return ctx, rt |
| +} |
| + |
| +func newDoc(t *testing.T, ctx context.Context, rt *core.Runtime, guid string, client uint64) *core.Handle { |
| + t.Helper() |
| + h, err := rt.NewDoc(ctx, guid, client) |
| + if err != nil { |
| + t.Fatalf("NewDoc(%q): %v", guid, err) |
| + } |
| + t.Cleanup(func() { h.Close(ctx) }) |
| + return h |
| +} |
| + |
| +const addSlide = `[ |
| + {"op":"set_meta","title":"Host test"}, |
| + {"op":"add_slide","id":"aaaaaaaaaaaa","layout":"flow","template":"title"}, |
| + {"op":"add_block","slide":"aaaaaaaaaaaa","id":"bbbbbbbbbbbb","type":"heading", |
| + "props":{"level":1},"region":"heading","markdown":"Hello **world**"} |
| +]` |
| + |
| +// TestOpsProduceUpdateAndView: an op batch yields a non-empty update, and |
| +// the view afterwards carries what the batch wrote. |
| +func TestOpsProduceUpdateAndView(t *testing.T) { |
| + ctx, rt := newRuntime(t) |
| + h := newDoc(t, ctx, rt, "ops-test", 1) |
| + |
| + update, err := h.Ops(ctx, []byte(addSlide)) |
| + if err != nil { |
| + t.Fatalf("Ops: %v", err) |
| + } |
| + if len(update) == 0 { |
| + t.Fatal("Ops returned an empty update") |
| + } |
| + |
| + view, err := h.JSON(ctx) |
| + if err != nil { |
| + t.Fatalf("JSON: %v", err) |
| + } |
| + if !strings.Contains(string(view), `"id":"aaaaaaaaaaaa"`) { |
| + t.Errorf("view lacks the slide id: %s", view) |
| + } |
| + var parsed struct { |
| + Meta struct { |
| + Title string `json:"title"` |
| + } `json:"meta"` |
| + Slides []struct { |
| + ID string `json:"id"` |
| + Blocks []struct { |
| + Text [][]struct { |
| + T string `json:"t"` |
| + B bool `json:"b"` |
| + } `json:"text"` |
| + } `json:"blocks"` |
| + } `json:"slides"` |
| + } |
| + if err := json.Unmarshal(view, &parsed); err != nil { |
| + t.Fatalf("view is not JSON: %v\n%s", err, view) |
| + } |
| + if parsed.Meta.Title != "Host test" || len(parsed.Slides) != 1 || parsed.Slides[0].ID != "aaaaaaaaaaaa" { |
| + t.Errorf("unexpected view: %s", view) |
| + } |
| + runs := parsed.Slides[0].Blocks[0].Text[0] |
| + if len(runs) != 2 || runs[0].T != "Hello " || runs[1].T != "world" || !runs[1].B { |
| + t.Errorf("unexpected runs: %+v", runs) |
| + } |
| + |
| + // The update the batch produced is a real Yjs update: a fresh |
| + // document fed only that reads the same. |
| + other := newDoc(t, ctx, rt, "ops-test-other", 2) |
| + if err := other.Apply(ctx, update); err != nil { |
| + t.Fatalf("Apply(update from Ops): %v", err) |
| + } |
| + otherView, err := other.JSON(ctx) |
| + if err != nil { |
| + t.Fatalf("JSON other: %v", err) |
| + } |
| + if string(otherView) != string(view) { |
| + t.Errorf("views differ after applying the ops update:\n%s\n%s", view, otherView) |
| + } |
| +} |
| + |
| +// TestApplyStateRoundtrip: State encodes the whole document as one |
| +// update; applying it to an empty document reproduces the view. |
| +func TestApplyStateRoundtrip(t *testing.T) { |
| + ctx, rt := newRuntime(t) |
| + a := newDoc(t, ctx, rt, "state-a", 1) |
| + if _, err := a.Ops(ctx, []byte(addSlide)); err != nil { |
| + t.Fatalf("Ops: %v", err) |
| + } |
| + state, err := a.State(ctx) |
| + if err != nil { |
| + t.Fatalf("State: %v", err) |
| + } |
| + if len(state) == 0 { |
| + t.Fatal("State returned no bytes") |
| + } |
| + |
| + b := newDoc(t, ctx, rt, "state-b", 2) |
| + if err := b.Apply(ctx, state); err != nil { |
| + t.Fatalf("Apply(state): %v", err) |
| + } |
| + av, err := a.JSON(ctx) |
| + if err != nil { |
| + t.Fatalf("JSON a: %v", err) |
| + } |
| + bv, err := b.JSON(ctx) |
| + if err != nil { |
| + t.Fatalf("JSON b: %v", err) |
| + } |
| + if string(av) != string(bv) { |
| + t.Errorf("views differ:\n%s\n%s", av, bv) |
| + } |
| +} |
| + |
| +// TestRejectedOpsSurfaceMessage: a refused batch comes back as a |
| +// *RejectedError carrying the core's message, applies nothing, and |
| +// leaves the instance usable. |
| +func TestRejectedOpsSurfaceMessage(t *testing.T) { |
| + ctx, rt := newRuntime(t) |
| + h := newDoc(t, ctx, rt, "reject-test", 1) |
| + before, err := h.JSON(ctx) |
| + if err != nil { |
| + t.Fatalf("JSON: %v", err) |
| + } |
| + |
| + _, err = h.Ops(ctx, []byte(`[{"op":"add_slide","id":"x","layout":"flow"},{"op":"frobnicate"}]`)) |
| + if err == nil { |
| + t.Fatal("expected an error for an unknown op") |
| + } |
| + var rejected *core.RejectedError |
| + if !errors.As(err, &rejected) { |
| + t.Fatalf("error is not a *RejectedError: %T %v", err, err) |
| + } |
| + if !core.Rejected(err) { |
| + t.Error("Rejected(err) is false") |
| + } |
| + if !strings.Contains(err.Error(), "frobnicate") { |
| + t.Errorf("message does not name the op: %v", err) |
| + } |
| + if core.Recovered(err) { |
| + t.Error("a clean refusal must not read as a recovery") |
| + } |
| + |
| + after, err := h.JSON(ctx) |
| + if err != nil { |
| + t.Fatalf("JSON after: %v", err) |
| + } |
| + if string(after) != string(before) { |
| + t.Errorf("a refused batch changed the document:\n%s\n%s", before, after) |
| + } |
| + |
| + // Malformed JSON and garbage bytes are refusals too, not traps. |
| + for _, bad := range []string{`not json`, `{"op":"set_meta"}`, "\xff\xfe"} { |
| + if _, err := h.Ops(ctx, []byte(bad)); !core.Rejected(err) { |
| + t.Errorf("Ops(%q): want a RejectedError, got %v", bad, err) |
| + } |
| + } |
| + |
| + // A garbage update is a clean rejection as well. |
| + if err := h.Apply(ctx, []byte{0xff, 0xff, 0xff, 0xff}); !core.Rejected(err) { |
| + t.Errorf("Apply(garbage): want a RejectedError, got %v", err) |
| + } |
| + |
| + // Still usable. |
| + if _, err := h.Ops(ctx, []byte(addSlide)); err != nil { |
| + t.Fatalf("instance unusable after refusals: %v", err) |
| + } |
| +} |
| + |
| +// TestABIVersion: NewDoc instantiates the embedded module and checks its |
| +// reported ABI version — this is the end-to-end proof that the module in |
| +// testdata and this host agree (core.New alone never reaches the check). |
| +func TestABIVersion(t *testing.T) { |
| + ctx, rt := newRuntime(t) |
| + if core.ABIVersion != 1 { |
| + t.Errorf("ABIVersion = %d, want 1", core.ABIVersion) |
| + } |
| + newDoc(t, ctx, rt, "abi-test", 1) |
| +} |
diff --git a/internal/core/testdata/slides_core.wasm b/internal/core/testdata/slides_core.wasm| new file mode 100755 |
| index 0000000..450ee08 |
| Binary files /dev/null and b/internal/core/testdata/slides_core.wasm differ |
diff --git a/rust-toolchain.toml b/rust-toolchain.toml| new file mode 100644 |
| index 0000000..ec92786 |
| --- /dev/null |
| +++ b/rust-toolchain.toml |
| @@ -0,0 +1,4 @@ |
| +[toolchain] |
| +channel = "1.98.0" |
| +components = ["rustfmt", "clippy"] |
| +targets = ["wasm32-unknown-unknown"] |
diff --git a/scripts/build-wasm.sh b/scripts/build-wasm.sh| new file mode 100755 |
| index 0000000..2e6976d |
| --- /dev/null |
| +++ b/scripts/build-wasm.sh |
| @@ -0,0 +1,43 @@ |
| +#!/usr/bin/env bash |
| +# Build the core crate for wazero. No wasm-bindgen, no wasm-pack: the Go host |
| +# provides no JS glue, so the module must be plain wasm32 with a C ABI. |
| +set -euo pipefail |
| +cd "$(dirname "$0")/.." |
| +# Lint before building: disallowed-methods (clippy.toml) enforces the |
| +# Doc-construction rule documented in core/src/lib.rs — Doc::new() and |
| +# Doc::with_client_id() both draw entropy this build has no safe source |
| +# for. A doc comment alone has been wrong, or gotten around, more than |
| +# once; this is the actual enforcement. |
| +cargo clippy --manifest-path core/Cargo.toml --all-targets -- -D warnings -D clippy::disallowed_methods |
| +# --lib only: the wasm module shipped to wazero is the library crate. Native |
| +# tooling under src/bin (e.g. emit_update, used from the host side of |
| +# wire-gate.sh) is never part of that artifact and does not define the |
| +# __getrandom_v03_custom symbol itself, so building it for wasm32 fails to |
| +# link -- it was never meant to run there. |
| +# |
| +# --remap-path-prefix (x2): rustc embeds the absolute source path of every |
| +# panic site (Location::caller()) into the binary. Left alone, that means |
| +# the registry path of every dependency (which contains $HOME, so it |
| +# leaks a username) and this worktree's own absolute path end up baked |
| +# into slides_core.wasm -- so the artifact only reproduces byte-for-byte on |
| +# the machine, and under the $HOME, that built it. The gate's `cmp` |
| +# against the committed copy then fails everywhere else and misreports |
| +# a stale artifact instead of a non-reproducible build. Remapping both |
| +# roots to fixed placeholders makes the build's output independent of |
| +# where it ran. |
| +# |
| +# Setting RUSTFLAGS here does not merge with .cargo/config.toml's |
| +# target.wasm32-unknown-unknown.rustflags -- an env RUSTFLAGS replaces a |
| +# config-file rustflags for the same target outright, it does not append |
| +# to it. That config entry (--cfg getrandom_backend="custom") is what |
| +# routes getrandom to our no-JS-import backend on wasm32; dropping it |
| +# breaks the build (getrandom falls back to a backend wasm32-unknown- |
| +# unknown doesn't support) rather than just losing the cfg silently. So |
| +# it has to be repeated here alongside the remap flags. |
| +registry_root="${CARGO_HOME:-$HOME/.cargo}/registry/src" |
| +workspace_root="$(pwd)" |
| +export RUSTFLAGS="--cfg getrandom_backend=\"custom\" --remap-path-prefix=${registry_root}=/cargo-registry-src --remap-path-prefix=${workspace_root}=/slides-core" |
| +cargo build --release --manifest-path core/Cargo.toml --target wasm32-unknown-unknown --locked --lib |
| +out="core/target/wasm32-unknown-unknown/release/slides_core.wasm" |
| +[ -f "$out" ] || { echo "build-wasm: no artifact at $out" >&2; exit 1; } |
| +echo "$out ($(wc -c < "$out") bytes)" |