grenade closed issue #4 in lair/tireless

grenade commented on #4 in lair/tireless

Done in `98f193d`, together with #3. ## The spec gap Step 3 says "surface the new `ETag` so the caller can store it" — but `list_opted_in_issues` returned a bare `Vec<DiscoveredIssue>`, so…

Done in 98f193d, together with #3.

The spec gap

Step 3 says "surface the new ETag so the caller can store it" — but list_opted_in_issues returned a bare Vec<DiscoveredIssue>, so there was nowhere to put it. The only copy would have stayed inside the client, and a poller that looked correct would have re-fetched every repo in full on every tick, forever, with nothing to show it was happening.

It returns an IssuePage now, which also carries not_modified. That distinction turned out to matter more than the ETag: a 304 is not an empty repo. A caller that conflated them would read every quiet poll as "every issue disappeared" and abandon the jobs behind them. Carrying it in the type means a caller has to decide rather than assume, and there is a test pinning it.

Beyond the spec

Pull requests are filtered out. Gitea's type=issues parameter should exclude them, but an older server ignores it and returns both — and a pull request enqueued as an issue would be planned or implemented as though it were one. Cheap to guard, expensive to debug.

Retry-After is honoured but capped at 60s. A forge asking us to wait an hour would otherwise stall the whole poll tick for every other repo.

Backoff is jittered. N repos throttled at the same moment must not all retry at the same moment and throttle each other again.

Writes return an error naming the stage they land in, rather than a silent Ok. A no-op that succeeded would let stage 2 look finished while the forge saw nothing — and there is a test asserting they do not even reach the network.

Verified against a mock forge

Twelve tests, no database needed, so these run in the ordinary cargo test:

• a stored ETag is sent as If-None-Match, and 304 is reported as not-modified rather than as empty or as an error;
• a first poll sends no conditional header — a stale or empty If-None-Match could earn a 304 for a client that has never seen the issues;
• the returned ETag is surfaced for the next poll;
• 429 is retried and then given up on, with the attempt budget asserted — an unbounded retry would hold the tick open indefinitely;
• 503 is retried and a later success is returned;
• 404 is not retried, because it will not fix itself;
• pull requests are filtered out;
• labels and body survive the mapping, since the router reads one and the planner reads the other;
• every unimplemented write fails without touching the forge.

GitHubClient is deliberately still a stub. It is disabled in the shipped config, and an unused implementation is one more thing to keep working for nobody.

grenade commented on #3 in lair/tireless

Done in `98f193d`, together with #4. ## Two spec gaps, both only visible on contact **`enqueue(&[DiscoveredIssue])` cannot know a `JobKind`.** `DiscoveredIssue` carries labels but nothing says…

Done in 98f193d, together with #4.

Two spec gaps, both only visible on contact

enqueue(&[DiscoveredIssue]) cannot know a JobKind. DiscoveredIssue carries labels but nothing says which mode they imply, and the trait takes no protocol. The store now carries the LabelProtocol and core gained routing::job_kind_for.

That forced a decision the spec did not mention: what happens when an operator applies several mode labels, which is easy to do by accident. Precedence is discover, then plan, then implement — planning beats implementing because a plan produces the implementation children, so running it first loses nothing, while the reverse silently discards the decomposition that was also requested.

claim_next(worker, allowed_lanes) had no lane to filter on. The #2 schema stores kind and parent but not the routing result, and the labels that carry the tireless/agent:* override are not stored at all — so deriving a lane at claim time would mean a forge request per claim. Migration 0002_job_lane.sql adds a lane column, recorded at enqueue by a new routing::lane_for, which route now delegates to so the two cannot drift. There is a test asserting they agree across every combination.

The consequence is that the lane is a cache of operator intent, so refresh_lane exists for the case where someone adds tireless/agent:oc to something already queued. Without it the override would only work if applied before the poller first saw the issue.

The claim itself

One statement: a CTE takes the row lock with FOR UPDATE SKIP LOCKED, the update writes the claim. Select and update share a transaction without anyone managing one by hand.

Returning a job to pending clears the claim — not because the code remembers to, but because pending_holds_no_claim from #2 refuses the half-done row. That is what makes lease expiry safe: a buggy release path fails loudly instead of stranding an issue with a claim nobody holds.

renew_claim is extra, and needed: a run can outlast the 10-minute lease, and without renewal the sweeper would hand a still-running job to a second worker. It is guarded on claimed_by, so a worker cannot renew a claim it already lost.

Verified against Postgres 18

Fifteen database tests, covering the things that are the database's to get right:

• enqueue skips issues expressing no work (opted in with no mode label; mode label without opt-in — the state a discovery proposal sits in); ignores closed issues; and is a no-op on repeated polls, which matters because a repo polled every five minutes would otherwise accumulate a job every five minutes;
• claiming respects the allowed lanes, including the agent:oc override, and returns nothing when every lane is held;
• a claim records the worker, sets a lease, and counts the attempt at claim rather than at completion;
• an expired lease returns the job and clears the claim, and the retry is visible in the attempt count;
• a delivered job refuses to be transitioned — re-running creates a new job so history stays true (§4.3);
• renewal requires still holding the claim;
• relabelling a queued job re-routes it.

Enum values round-trip through serde rather than a hand-written match, so the schema's check constraints and the Rust types are provably the same vocabulary.

One test from #2 replaced

exactly_one_migration_ships_today failed the moment a second migration was legitimately added. A test that fails on correct behaviour teaches people to edit the assertion rather than think, so it now asserts what it was reaching for: versions unique and ascending, starting at 1.

grenade closed issue #3 in lair/tireless

grenade pushed 1 commit to lair/tireless:main

• 98f193d feat(data): implement JobStore and the Gitea read client

grenade commented on #2 in lair/tireless

Done in `6545d19`. ## The decision this issue asked for **Runtime queries, not `query!`.** No `.sqlx` offline metadata, no `DATABASE_URL` needed to build, no `cargo sqlx prepare --check` in…

Done in 6545d19.

The decision this issue asked for

Runtime queries, not query!. No .sqlx offline metadata, no DATABASE_URL needed to build, no cargo sqlx prepare --check in CI.

lairball — the other house project on this cluster — already does exactly this: 19 uses of sqlx::query(, zero macros, no .sqlx directory, no Postgres in CI. But the deciding argument is specific to tireless. Compile-time checking makes a database a build dependency: regenerating .sqlx after any query change needs a live Postgres. This crate is meant to be modified by a 27B model working unattended (design.md §2.4), and a build that fails without a database it cannot provision is a build that model cannot fix — its documented failure mode being to improvise.

The cost is real and named in store.rs: a malformed query is caught by a test against a real database rather than by cargo build. That is why the schema tests below exist.

What the schema does that isn't obvious

The live-issue index is partial, and getting it wrong is silent in both directions. A plain unique (forge, owner, repo, number) — which is what "enqueue is an upsert keyed on …" suggests — would forbid re-running a terminal job, and would forbid the discovery lane outright, since discovery recurs against one tracking issue on a cooldown (§2.6). Partial on the non-terminal states gives at most one live job per issue plus unlimited history.

The claim index orders by created_at alone, leaving kind as a filter. Leading with kind — which is what this issue's step 3 proposed — makes the planner sort every pending row on every claim, because LIMIT 1 can otherwise stop at the first match:

| index | buffers | plan | | --- | --- | --- | | (kind, created_at) | 720 | Bitmap scan → Sort 2503 rows | | (created_at) | 4 | Index scan, stops at row 1 |

The gap grows with the backlog rather than staying fixed. INCLUDE (kind) was measured too and dropped: FOR UPDATE visits the heap regardless.

A spelling that would have been permanent. rename_all = "snake_case" turns Forge::GitHub into git_hub — in the database, the JSON API and the generated TypeScript. Renamed to github while nothing is persisted and GitHub support is still disabled. There is a test.

Check constraints are tested against the domain enums. Adding a JobKind variant without a migration would otherwise fail as a constraint violation on the first job of that kind — in production, unattended. Now it fails cargo test.

Verified, not assumed

Against Postgres 18 (same major as the house cluster):

• migrations apply to an empty database, and a second run is a no-op — which matters because all three units migrate on start and a deploy restarts them together;
• all eight constraints reject what they should and admit what they should, including git_hub now being rejected;
• two concurrent claimers of one pending job produce exactly one winner, the loser skipping rather than blocking — the property §4.2 rests on.

The live tests are #[ignore]d rather than skipping on a missing variable, so a green cargo test never implies the schema was exercised. CLAUDE.md says how to run them, and that an applied migration must never be edited — sqlx keeps a checksum per version, so editing one makes every deployed database refuse to start.

Left alone

No JobStore impl (that is #3), no forge work, no reconciliation or lease-expiry behaviour — the columns exist, including labels_synced_at for the poller split in §6.4, but nothing reads them yet. No password field went near Database.

grenade pushed 1 commit to lair/tireless:main

• 6545d19 feat(data): add the initial schema and migrations

grenade closed issue #2 in lair/tireless

grenade pushed 1 commit to lair/tireless:main

• 9deeaaa feat(infra): provision the poller identity

grenade pushed 1 commit to lair/tireless:main

• 8586da2 fix(infra): grant the runner's account nothing; split labelling off

grenade closed pull request #12 in lair/tireless

grenade pushed 1 commit to lair/tireless:main

• b1e0177 feat(infra): provision the bot account, and move to a fork-based PR model

grenade closed pull request #11 in lair/tireless

grenade pushed 2 commits to lair/tireless:main

• ae285f5 docs: stage 0 complete — agent login done, all three units active
• bd65839 docs(infra): fix the agent login command — cd before npx

grenade commented on #9 in lair/tireless

Green: [run 8](https://git.lair.cafe/lair/tireless/actions/runs/8). ``` https://tireless.internal/v1/ready {"config":"ok","database":"not_implemented","forge":"not_implemented"} https://tireless.…

Green: run 8.

https://tireless.internal/v1/ready  {"config":"ok","database":"not_implemented","forge":"not_implemented"}
https://tireless.internal/          HTTP 200
tireless-api                        active
tireless-poller                     active
tireless-runner                     failed  (see below — this is correct)
tireless preflight                  Subscription / lair-helexa / Qwen3.6-27B (asserted non-Anthropic)

What was wrong

Eight faults. The two the issue predicted were both real, but neither was the one that would have cost the most to find later.

| # | Fault | Why it stayed hidden | | --- | --- | --- | | 1 | runs-on: fedora-43-rust is not a registered label | The job was never scheduled. Runs 2 and 3 have started_at: 1970-01-01 and are recorded as "cancelled", which tells you nothing | | 2 | Vhost bound :443, owned by the stream SNI router | nginx -t passes; the symptom is the wrong certificate on a working handshake | | 3 | Cert path was the host identity cert | Its SAN is bob's FQDN, so it fails only for a client verifying tireless.internal | | 4 | --rsync-path word-split by an unquoted variable | Loud, but only on the half of the deploy that used the variable | | 5 | restorecon on /var/lib/tireless, which cannot exist yet | The account that owns it is created later in the same deploy | | 6 | Config shipped 0640 root:root; services run as tireless | Compounded by --chmod being a no-op without -p, so the fix would have worked once and silently stopped | | 7 | API bound the clap default, ignoring [api] bind | The service looks perfectly healthy from bob — starts, logs "listening", answers a local curl. Only the proxy fails | | 8 | Health probe expanded $unit on the remote side | It ran systemctl is-active .service: checking nothing, and would have reported healthy regardless |

The artifact-action risk this issue led with turned out to be moot — build and deploy collapsed into one job, since the rust image already carries node, ssh and rsync, so there is nothing to hand between jobs.

Six of the eight were silent. That is the part worth carrying forward: where a probe exists, it must be able to fail. The health probe now runs from the proxy over the mesh rather than bob's loopback, so closing firewalld breaks it.

Faults 4–8 were found by exercising the deploy directly against the hosts as gitea_ci — every rsync destination, every sudo command, and the health probe step extracted from the YAML and run as a file — rather than at six minutes per attempt.

Provisioned

infra-setup.sh now does the ingress rather than describing it: mints the tireless.internal cert through the JWK provisioner (removing the credential even on failure), installs the vhost via sites-available + symlink, enables [email protected], and registers the split-horizon record on both routers — a record on one router NXDOMAINs at the other site. opn-cli has no reconfigure verb, so the apply is a direct API POST; without it the name resolves only whenever Unbound next happens to reload.

The served certificate was checked against disk by serial, per internal-tls.md §3. They match.

Remaining, and out of scope per this issue

tireless-runner is in failed because the interactive agent login has not been done as the service account (script/infra-setup.sh step 1). That is the invariant working — a runner with no credentials refuses to start rather than pretending — and it is why the deploy tolerates that one unit failing.

Note the acceptance criterion "all three units active" contradicted this issue's own Out of scope section, which said the runner is expected to refuse. The Out of scope text was right; the criterion was over-specified when I wrote it.

Also worth fixing, outside this repo

architecture/internal-tls.md §5 says hanzalova vhosts bind :443 directly. reverse-proxies.md §4 says both proxies use the 127.0.0.1:14443 tier and notes hanzalova gained its router on 2026-07-30; the live bench.internal.conf confirms it. Anyone following §5 gets fault #2 — which that same doc warns about in §3. Worth a correction in the architecture repo.

grenade pushed 1 commit to lair/tireless:main

• ffa2ad7 docs: record stage 0 as deployed, and what the first deploy cost

grenade closed issue #9 in lair/tireless

grenade pushed 1 commit to lair/tireless:main

• 8067cde fix(deploy): expand $unit locally in the health probe

grenade pushed 1 commit to lair/tireless:main

• 42582cb fix(api): bind the address from config, not a clap default

grenade pushed 1 commit to lair/tireless:main

• a5efa1e fix(deploy): make config readable by the service account, and assert modes

grenade pushed 2 commits to lair/tireless:main

• 267cb33 fix(deploy): quote --rsync-path, and let systemd own the state directory
• 2e3aff7 fix(infra): drop the proxy web root from bob's restorecon grant

grenade pushed 5 commits to lair/tireless:main

• 3c7d95e fix(deploy): correct the runner label, the vhost listen line and the cert paths
• 835e3e9 fix(deploy): put ingress on the proxy, and make the lint script runnable
• ddb2574 docs: state the purpose, the autonomy boundary and the dogfooding plan
• 581e6ae feat(discover): add the discovery lane and the autonomy boundary
• c49b531 feat(config): load and validate configuration in every binary

grenade opened issue #10 in lair/tireless

Test plan quality against a real issue, before stage 5 spends anything on it

grenade opened issue #9 in lair/tireless

Verify the deploy workflow end to end, before there is anything to lose

grenade opened issue #8 in lair/tireless

CI: fail the build when generated TypeScript bindings are stale

grenade opened issue #7 in lair/tireless

Dashboard: repos page

grenade opened issue #6 in lair/tireless

Repo CRUD over /v1

grenade opened issue #5 in lair/tireless

Poll loop: interval floor, per-repo jitter, quiet window

grenade opened issue #4 in lair/tireless

Gitea client: conditional issue listing

grenade opened issue #3 in lair/tireless

PgStore: implement JobStore with FOR UPDATE SKIP LOCKED

grenade opened issue #2 in lair/tireless

Postgres schema and migrations

grenade opened issue #1 in lair/tireless

Stage 1 — forge ingestion (read-only)

grenade pushed 1 commit to lair/tireless:main

• 21c35ff docs(prompt): record helexa#179 as settled; shift the oc risk to precedence

grenade pushed 1 commit to lair/tireless:main

• 7b8308d feat(prompt): make the plan handoff a versioned, validated contract

grenade pushed to lair/tireless:main

grenade created lair/tireless