Decisions

How do tests avoid "Text file busy"?

Programs are written by a child process (link_plugin::testing), rather than retried or run one test at a time: the race needs the test's own process to hold a program open for writing while another thread forks, and with a child writing it, it never does. Retries hide it and one-at-a-time slows the suite; neither removes it. lnk installs plugins the same way (tar writes them, then a rename).

Why does cargo test check the docs?

Because a wrong command in a doc is worse than none, and pull requests into dev get no CI. src/tests/e2e/tests/docs.rs fails on a broken link or anchor, on an lnk <group> <command> that neither the core's COMMANDS nor a plugin's command line has (hidden ones too, such as lnk agent send, which a harness runs), and on a ## Decisions section. It reads commands in code blocks and in inline code, and not the bare word lnk in a sentence, which as often says what lnk does ("lnk drops it") as gives a command. Inline code is checked only when it starts with lnk, so lnk-tunnel or a path is left alone. It reads every folder's docs, as the website does: a link that leaves its folder but as a page on www.local.link fails, and so does a link to a page or heading that doesn't exist, or a README.md that holds more than links. It reads the files as text, so it needs no built lnk and runs in a moment.

How do tests run cargo?

Through link_plugin::testing::cargo(), which leaves out the variables cargo sets for the test it runs (CARGO_MANIFEST_DIR, CARGO_PKG_*, ...). ring's build script reruns when CARGO_MANIFEST_DIR changes, so a cargo build run from a test with them recompiled ring, rustls and every Link crate above them, and the next cargo test recompiled them back: minutes and gigabytes on every run, with nothing changed. src/tests/e2e/tests/tooling.rs fails on any other way of running cargo.

How do tests get lnk and its plugins?

One cargo build --workspace --all-targets, once per test program (link_plugin::testing::programs; programs_in for a folder beside, which builds that folder's own). cargo test builds only the programs of packages that have integration tests, so the rest have to be built, and a build of some packages (-p) resolves their dependencies' features for those alone: a test's cargo build -p of the plugins it ran compiled regex, sha2 and WhatsApp's libraries a second time, 80 s and a copy of each, and the nine such builds in the folders' tests made a copy each. --all-targets takes in the tests' dev-dependencies, so it resolves features as cargo test does: after cargo test it compiles only the programs themselves (3 s), and after it, both are no-ops, so a test never runs an old build.

Why do the folders beside the core share a build folder?

Each has a .cargo/config.toml putting its build in ../target-beside, the folder CI's beside job builds them all into. They share most of their dependencies (link-plugin, tokio, axum, reqwest...), and each compiled them in a target/ of its own: measured on Oct 8 2026, Link harness compiled 283 crates and each adapter 151, 6 GB of target/ with Link's tools. Sharing, the adapters compiled 33, 1 and 9, and all five built in about 2 minutes, not 6. Cargo locks the build folder, so two of them never compile at once. Their programs stay in each folder's own target/, where tests and the release find them. The core keeps its own: it shares little with them and is cached on its own in CI.

Why are the end-to-end tests one program?

Each file in a tests/ folder is a program of its own, linked with all of its dependencies: link-e2e's 22 files were 22 links of about 40 MB on every build, and again after every edit to a crate they use. As modules of one program (src/tests/e2e/tests/main.rs) they link once, and run side by side; none changes the process's environment or working folder, which they now share.

Why are some programs built with test = false?

cargo test builds a test program for every library and program, and most plugins' main.rs hold no tests: 19 programs were built, linked and run to test nothing. Those are test = false, and src/tests/e2e/tests/tooling.rs fails if a test is added to one, which would otherwise never run.

Why does the licenses test download crates?

cargo metadata needs every platform's crates, macOS's and Windows's too, and a build fetches only this one's. The test runs cargo metadata --locked, which downloads the missing ones (about 20 MB, once) and reads them offline after, rather than failing on a fresh machine until someone runs cargo fetch.

How does the conformance kit see what a harness keeps?

Through its own model. Every harness keeps its threads its own way, in files the kit can't read, but every harness sends the thread to its model at the next turn. So the kit's model records what it's sent: a turn whose caller went before the answer is kept if the next message's request holds that answer, and a tool's answer reached the model if the model's last answer quotes it. The model finds the kit's message among the harness's own (instructions, the date, a runtime's context) by its text, not by its place, since harnesses add messages after it.

Why are the live tests a script, not part of cargo test?

They cost money, so a person or an agent runs them on purpose: src/tests/live/run.sh, one bash script that runs on a Mac (bash 3.2) and on Linux. cargo test stays free, and uses fakes for every cloud.

How do the live tests keep away from your own lnk?

A run has its own home and a made-up login, whose GitHub id tags its boxes (lnk-owner=github-<id>). It never touches your settings, and its resources are found by that tag afterwards. Its agent has a name of its own, because a background service is named after its agent and lives in your account, not the run's home.

What does a live test leave behind?

Nothing. A stage removes what it made, even when a check fails or it's interrupted. It then checks the account: nothing of the run is left, and what Link shares between boxes is as it was before.

How does a live run know what it cost?

It estimates from list prices, at the top of the script, and from how long each thing existed, per stage, in target/live/costs.tsv. The real bill comes by tag: AWS's through Cost Explorer (run.sh bill), Google Cloud's in its billing reports.

How do the live tests try this checkout's lnk on a box?

A box installs the latest release. --checkout builds this checkout for Linux, static, as a release does, and copies it onto the box after it starts, then checks the box's programs match the build. Changing how a box installs lnk for a test would test something other than what people get.

How does the agents stage move an agent with no model or chat?

It uses Link Harness with a made-up key for a hosted model. A hosted model is taken as answering, so the agent starts and moves. Nothing sends it a message, so the key is never used and no model is billed.

Why does Link build without debug info?

Debug info was most of target/: a full cargo test wrote 20 GB, its test programs 80% debug info, and filled CI's runner. Even line tables alone were half of each program. Without it, a panic still says the file and line it happened at; a backtrace (RUST_BACKTRACE=1) names the functions only. For a debugger, build with CARGO_PROFILE_DEV_DEBUG=true.

Where does CI's disk go?

GitHub's ubuntu-24.04 runner has a 72 GB disk, and its image fills about 58 GB of it with tools Link never uses (Android's SDK, .NET, Haskell, CodeQL), so a job starts with about 14 GB. Link's tests write to disk for three reasons, and only the first is big:

• Test programs are programs. Rust compiles each crate's tests, and each file in a tests/ folder, into an executable linked with all of its dependencies: about 110 of them, 0.9 GB. Clippy's own output is 0.6 GB more, and the dependencies' compiled libraries most of the rest.
• The end-to-end tests run the real lnk and its plugins, so they build them first (see above): the same build, once.
• Tests make folders to stand in for a home, an agent or a bucket, so none touches this machine's own: megabytes, removed after.

Measured from an empty target/ on Oct 5 2026 (clippy, then cargo test, as CI runs them): 2.5 GB after the build and 3.1 GB at the end. Before Oct 1 the same run wrote 20 GB, and filled the runner: debug info, tests that rebuilt the workspace with other features, and a build per package (the three entries above). CI's "disk" step prints the room left on every run, and its "room on disk" step removes the image's biggest unused tools first, about 25 GB more, so a build that grows fails a check, not the runner.

Why does CI pin its Rust version?

stable moves every six weeks, and each release can add clippy lints: 1.99's result_large_err failed checks on code no pull request had touched, and passed on any computer on an older Rust. The version also keys rust-cache, so every release missed the cache. RUST_TOOLCHAIN in checks.yml is the one Rust CI runs, and release.yml and relay.yml build with the same, so what's shipped is what was tested.

The repository's rust-toolchain.toml pins the same version for every folder on any computer, so a check passes or fails there as it does in CI: a computer whose default Rust was older passed what CI's clippy then failed. Move the file and the three workflows together, in a pull request of its own, with whatever the new clippy asks; link-core's tooling test fails when they differ.

What does CI cost, and why is it shaped this way?

On a private repository GitHub bills every job's minutes, rounded up, and a macOS minute is billed as ten Linux minutes. A release's two macOS builds (about 10 minutes each) are most of its bill; the checks (about 10 minutes, and again in release.yml and relay.yml), the Linux build (about 13) and the rest are under a quarter of it.

• Every job has timeout-minutes. GitHub's default is 6 hours: one hung macOS job would bill 3,600 minutes.
• A pull request's newer push cancels its older checks run.
• What's built beside the core is checked in one job, every Cargo workspace at the root but the core's, found rather than listed, in one target folder: measured on Oct 5 2026, the five then, their clippy and test builds took 107 s, not 252 s, one after another, and one runner starts, not five.
• Each harness adapter's real harness runs in a job of its own (on-the-wire in the root's checks.yml), side by side, on release pull requests only: each downloads a few hundred MB, Linux minutes are the cheap ones, and one job per adapter keeps one slow install from holding up the others. It never runs in the release's or the relay's own run, which call checks.yml too: the harnesses' installers run outside the sandbox, with dependency trees no one pins, and anything they run could upload an artifact into the run that signs, or a cache its builds restore.
• translate.yml looks for a finished batch once a day, not every 6 hours: a batch takes up to a day, and each look is a runner.
• The two macOS builds stay two jobs. Built after the other target in the same folder, a second target took 377 s, not 393 s: what they share (build scripts, macros) is small, so one job would only save a runner's start, and double the release's wait. Most of a release build is WhatsApp's library (waproto, wacore, whatsapp-rust); built at opt-level = 1 it saved 7% (388 s, not 418 s) for a lnk-whatsapp 13% bigger, so it stays at the release default.
• A push to main runs the checks again, though its pull request passed them: main also takes commits made on it directly, so what was tested isn't always what was merged.

Why doesn't CI run on a machine of Link's own?

A box from an image of our own, as a self-hosted runner, would save little and risk more:

• macOS is most of the bill, and a box is Linux in a cloud. macOS runs only on Apple's hardware, rented by the day at the least.
• On a public repository GitHub's runners are free, macOS's too.
• A self-hosted runner on a public repository runs anyone's pull request on that machine, which GitHub advises against.
• A runner costs while it waits, or needs starting and stopping for each job; a release's Linux jobs are about 40 minutes.

How do old CLIs keep working when the relay upgrades?

One relay that speaks a range of protocol versions, not one relay per version. Welcome tells the agent when a newer CLI exists, and the relay rejects an agent only below a minimum version, with an upgrade message. MessagePack with named fields lets fields be added without a version bump, so real breaks are rare, and several relays would add routing and operations work for nothing.

How large is a stream's window?

2 MiB when both sides support it, agreed in Hello and Welcome, and 640 KiB with older peers. A stream moves at most a window per round trip to the relay, so the old window held one download to about 13 MB/s at 50 ms. A larger window lets the relay hold more for a visitor who reads slowly, and its response budget (32 MiB per user) is fixed, so 2 MiB keeps room for 16 such streams per user where 8 MiB would leave 4.

How much of the response budget may a grown window take?

Credit fills at most three quarters of each cap, and a window grows only while a cap is under half, shrinking back as the visitor reads once it's past. Credit can't be taken back, but a new stream's first window comes with none, so a full budget could only drop it: one fast download that grew to fill its visitor's share cut every other response to that address short. The last quarter keeps room for first windows, and the shrinking gives a grown window's room back when others need it, so a full budget slows streams instead. Growing to half of a visitor's 16 MiB keeps one download near the 8 MiB window it reached before.

Where do secrets live on a Mac?

In the login Keychain (link_plugin::keychain): the relay token, agents' model keys and bot tokens, cloud accounts' secret keys, and the keys and encryption passwords lnk bucket writes, so they aren't in plain files that backups and copies of ~/.config carry. Linux and boxes have no Keychain and keep files. Callers keep secrets in memory as before and change only where they're saved.

How does Link reach the Keychain?

Apple's security tool, not a Keychain library: an item made by security trusts it, and it's the same program across lnk upgrades, so reading never asks. Link's programs aren't signed by Apple, so items they made themselves would ask after every upgrade. Secrets go to it on stdin, hex-encoded, and are read back to check.

How does rclone get secrets from the Keychain?

As RCLONE_CONFIG_<SECTION>_<KEY> variables for each run, which rclone reads like its config file's lines; rclone-keychain.toml lists which are in the Keychain. rclone's own config encryption would lock the whole file behind one password but Link edits the file itself.

How do you tell which lnk runs?

lnk --version says, after its version: the program's path, how it was installed (cargo, Homebrew, a build in target/, or installed), and any other lnk on the PATH. With two on the PATH, every command in a terminal starts with a note naming both. A release and a build from source report the same version, and in QA an old cargo install copy ran unnoticed for hours, beside the build being tested. The note isn't printed for a plugin running lnk (it has LNK_EXE), and the first line stays lnk <version>, which lnk upgrade reads.

Why do lnk's variables start with LNK_?

So one prefix finds them all, and a variable set by analogy works: LNK_AUTH was silently ignored while the tunnel read only LINK_AUTH. Four began with LINK_ (LINK_RELAY, LINK_TOKEN, LINK_AUTH, and the LINK_FILES given to adapters), and those names still work, since scripts and older adapters use them: lnk reads the LNK_ name first, and lnk-harness gives adapters both. The relay and its deploy scripts keep LINK_*, which running servers have set in /etc/link/; only LNK_RELEASES_URL, which lnk reads too, is accepted there as well.

What do lnk's colors mean?

Green is what you do next (a command to run, a step, a question), cyan is what Link is doing while it sets something up, yellow is a warning (a risk, a cost, something skipped), and red is an error; everything else keeps the terminal's own color, never a forced white, which vanishes on a light terminal. Red and yellow are what Rust's own tools use for errors and warnings, and cyan, cargo's color for progress, reads on a dark terminal where plain blue doesn't. Four colors, each with one meaning, so the line that needs you stands out. link_plugin::style holds them, so every plugin looks the same. It uses anstyle and anstream, clap's own, which drop the colors in a pipe or a file and follow NO_COLOR and CLICOLOR_FORCE: --json, and what one plugin reads from another, stay plain.

How does lnk lay out what it prints?

The same way in every command, so you know where to look: Link's progress flush left (a long flow's steps counted, [2/3] Setting up link...), then a blank line and the result indented two spaces, then what you do next last, under a bold Next, just above your prompt. Labels and a table's headers are bold, a style rather than a color, and columns are as wide as their widest cell. A check or a status carries a mark, ✓ ✗ or !, so its outcome is seen before it's read; the word stays beside it. link_plugin::style has the helpers (fields, a table, marks, steps, Next), so no plugin pads a column by hand. There are no spinners, which need another dependency and leave a messier terminal, and no wrapping to the terminal's width.

Where does a command's help live?

In one file per workspace, help.toml, not in doc comments. The core and its plugins share src/local/help.toml; Link Harness, Link's tools and each harness adapter keep their own at their folder's root, since every folder stands alone. Help spread over twenty cli.rs files went stale without anyone seeing it (commands renamed, flags gone, JSON fields added), and one file reads as the whole of lnk's help at once. The core's lnk --help reads its groups' lines from the same file, so it says what they say.

• Read at startup, only when shown (link_plugin::help): a command line is parsed first without help, and only when it asks for help or is wrong is the file parsed (about a millisecond) and put on clap's commands, with mut_arg and mut_subcommand. A build script or a proc macro would make it static, for code in every crate and no measurable gain.
• Read as clap reads a doc comment: paragraphs, the first the short help without its last period, so moving help out of doc comments changed no --help, but for a value's own help (lnk sandbox tool add --ask's all and none), which the argument's help now says.
• Checked both ways: a test fails on a command or an argument with no entry, help left in the code, and an entry naming nothing.
• The relay keeps its doc comments: it isn't lnk, and it depends only on link-protocol, so it can't read help with link-plugin.
• Programs with no clap (the harness adapters, Link Harness, Link Memory, run by lnk and never typed) list their commands from their help.toml for --help, and a test checks the list against the commands their main answers.