diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..5d37546 --- /dev/null +++ b/.gitignore @@ -0,0 +1,5 @@ +# cart v1: a cart is not committed. Ignored, gone means gone. +.loom/cart/ + +# build output +/loomctl diff --git a/.loom/event-log.md b/.loom/event-log.md index 0c0d596..c33f2e7 100644 --- a/.loom/event-log.md +++ b/.loom/event-log.md @@ -518,3 +518,288 @@ the cart that named it.* **Recorded rather than fixed.** *The role still works; its justification was written for a case that has not yet occurred.* **If a round ever does produce code while it is open, nothing here changes.** + +## 2026-09-07 — `loomctl external` exists, and its first run reconciled two documents `marmalade` + +**Built: `list`, `add`, `check`.** *Go, no dependencies outside the standard +library, `git` shelled out for `list` only.* + +**The first real run did the thing the tool is for.** *Eight documents were +adopted by hand before it existed; `check` reported all eight `unlocked`, and +`add` locked them* — **and two came back changed**, `bedrock/starting.md` *and* +`cart/cart.md`. **Neither change would have been noticed by anybody.** + +**Believed to advance:** *the lock is what makes `check` possible at all.* **With +no lock there is nothing to compare and the only honest report is `unlocked`** — +*and the tool refuses to invent one by adopting whatever the remote currently +serves.* + +**Belief that could be shown wrong:** *that a per-person path derived from the URL +is good enough.* **`///.md` guesses that the first two +path segments name an owner and a repository**, *which is true of gitea, GitHub and +GitLab and is not a rule.* **`--path` exists for when it is wrong.** + +## 2026-09-07 — supersedes the isolation role: a cart is not committed `marmalade` + +**`cart` is now `v1` and it changed the thing we cast a role on.** + +> **So the cart is not committed.** *It lives in the working tree of the machine +> the two presences share, and `.loom/cart/` is ignored by version control.* + +**The reason is not tidiness:** *a committed cart grows a third file by itself.* +**The two-file rule defends against somebody asking for one; version control does +not require anybody to ask** — *anyone who can clone can add a third, and the +agreement's test is never invoked because nobody had the conversation.* + +**And it is what makes a round end.** *Committed, a cart is gone from the tree and +permanent in history* — **so "gone" means "no longer live" and the negotiation +stays quotable forever.** *Ignored, gone means gone.* + +**So `.loom/cart/` is now in `.gitignore`.** *The isolation role entry above +assumed the cart lived on the default branch; **the cart lives on no branch.*** +*What survives of that entry is the other half: work is isolated on a branch named +for the round that authorised it.* + +**Not undone: `osprey` and `marmalade` are already in this repository's history.** +*Rewriting history to honour a rule adopted afterwards would cost more than it +buys*, **and the two rounds are quotable forever, which is exactly what `v1` says +not to want.** *Recorded rather than repaired.* + +**Belief that could be shown wrong, and it is a real conflict:** *the annotation +protocol this repository works under says **commit before dissolving, git is the +only archive of the conversation.*** **An ignored cart has no archive**, so +dissolving a notes file destroys the annotations outright. *One of the two is +wrong and it is not ours to settle.* + +## 2026-09-07 — what the planted change actually proved `marmalade` + +**`cart.md` was changed upstream deliberately, without telling us, to see whether +the tool would notice.** *It did — and the sequence is worth recording, because +the obvious reading is wrong.* + +**`check` did not catch it.** *All eight documents were unlocked, and `unlocked` +means **I cannot tell you**.* **It was `add` that revealed the change, by +overwriting the file** — *and the only reason anybody saw what had changed is that +`git` happened to be watching the working tree.* + +> **So the mechanism is proven and the workflow is not.** *A conditional request +> against a lock works. **A document nobody has locked is a document nobody is +> checking**, and it stays that way silently.* + +**Decided, as a consequence:** *`add` now reports when it replaces local content +that differs from what the publisher is serving.* **It used to say only +`adopted`.** *A copy that differs is the only evidence that something moved while +the document was unlocked, and destroying it silently is how a change nobody saw +becomes a change nobody can find.* + +**Belief that could be shown wrong:** *that a note is enough.* **`add` still +overwrites** — *it does not stash the old bytes anywhere* — **and outside a git +working tree the note tells you something was lost without letting you see it.** +*If that bites, `add` needs `--dry-run` or a refusal.* + +## 2026-09-07 — the unowned half: somebody has to run it `marmalade` + +**Nothing here answers *when* `check` runs.** + +*`bedrock` says it about running systems and it is just as true of this:* +**nothing serves the truth, so the only mechanism is somebody looking.** *The tool +makes looking cheap; it does not make it happen.* + +**Recorded as a need with no owner rather than a feature**, *because the answers +are all outside the tool* — **a git hook, a CI job, an agent's session start, a +scheduled run** — *and choosing one here would put a scheduler inside a fetcher and +a comparator.* + +**Belief that could be shown wrong:** *that staying out of it is right.* **If in +practice nobody ever runs `check` unaided, a tool that only reports when asked is +a tool that reports nothing**, *and the thing we declined to build is the thing +that was needed.* + +## 2026-09-07 — a changed external is a polad, which the specimen said first `marmalade` + +**Decided:** *`check` stages what moved into `.loom/cart/current/polad/`*, **a +candidate artifact shaped exactly like what it would become**, *whose exits are +apply or discard.* + +**This was in the specimen and we lost it.** *"A changed external becomes a polad +in the cart, and somebody decides."* **The round that discarded the specimen +discarded this with it**, *and it came back only because somebody asked what the +stash should be.* + +**Believed to advance:** *`externals` says reconciliation runs the other way* — +**given what changed in theirs, what do we change in ours** — *and the facets +usually survive while the manifests, the config and the code that a usage named +are what move.* **So staging prints the `.usages.md` beside it**, *which is the +file that names what to go fix*, **and says so when there is none**: *a usage that +does not name what it justifies is half a usage, and no usage at all is a document +nothing records a dependency on.* + +**With no cart open, `check` reports what moved and stages nothing.** *The tool +does not open a round* — **a cart is a bounded exchange between two presences, and +starting one is somebody's act, not a side effect of asking about freshness.** + +**Belief that could be shown wrong:** *that requiring an open cart is right.* **If +most changes arrive when no round is open, the useful behaviour is the one that +never runs**, *and the polad needs somewhere else to live.* + +## 2026-09-07 — supersedes "add says what it replaced": add does not replace `marmalade` + +**`add` adopts what is not here, and refuses what is already adopted.** *The +earlier entry made `add` announce an overwrite; **it no longer overwrites at +all**.* + +**Believed to advance:** *a command that both adopts and re-fetches is a command +that overwrites the only evidence a change happened.* **Splitting them gives each +one job** — *`add` adopts, `check` notices.* + +**One exception, and it is the only way out of a dead end:** *a document that is +present but **unlocked** was fetched by hand before the tool existed.* **Nothing +records its origin and the path does not round-trip, so `check` cannot ask about +it and `add` refusing would strand it forever.** *So `add` accepts it, and the +bytes decide:* + +- **identical** → *the lock is written and nothing is rewritten.* **The assertion + a lock makes — this local copy is the one being served — is then verified rather + than assumed**, *which was the whole objection to adopting a remote `ETag` + blindly.* +- **different** → *staged as a polad; the local copy is left alone*, **because a + copy that differs is the only evidence that something moved while nothing was + watching.** + +*Measured on this repository: eight documents adopted by hand, all eight locked +with nothing rewritten.* + +## 2026-09-07 — `apply`, because the lock is the half a person forgets `marmalade` + +**Decided:** *`loomctl external apply [path…]` moves a staged polad into place and +moves its lock with it.* + +**Believed to advance:** *applying by hand is one `mv`, and it leaves a lock +describing the copy you just replaced* — **which is exactly the drift the lock +exists to prevent.** *The polad carries the `ETag` that was served with the bytes +somebody reviewed, so applying locks what was actually read rather than whatever +the publisher serves at apply time.* + +**This is not the tool fixing anything.** *It executes a decision a person already +made, one document at a time, after the report.* + +**Belief that could be shown wrong:** *that two exits are enough.* **For an +external, discard does not mean the change goes away** — *the upstream moved +whatever we do* — **so discarding is really "knowingly stale", and nothing +currently records that choice.** *If people discard often, that silence is the +next defect.* + +## 2026-09-07 — declined: a separate `adopt` verb `marmalade` + +**Considered:** *`loomctl external adopt `* — **a command whose job is +to say "this copy came from there"**, *for a document that is present but +unlocked.* **Both parties reached the deadlock independently and this was the +other way out.** + +**Not done, because the byte comparison makes the verb unnecessary.** *Whatever +the command is called, it must not trust the claim* — **it has to fetch and +compare, because the whole point is that nobody knows whether the local copy is +still a copy.** *Once it does that, it is `add` with an origin supplied, and the +user's intent in both cases is the same sentence: **I depend on this document and +here is where it lives.*** + +**And a verb earns its place by naming an act, not a state.** *"Present but +unlocked" is a condition a repository is temporarily in* — **a permanent command +for it advertises a transitional situation as a normal one.** + +**Belief that could be shown wrong:** *that the condition is transitional.* **If +people and agents keep dropping documents into `externals/` by hand — and they +will — it is a recurring category and not a migration**, *and then it deserves its +own word, discoverable in `--help` rather than found in a hint.* + +*The domain word survives regardless: `externals` says adopting is fetching a URL, +and the tool says `adopted` throughout.* + +## 2026-09-07 — the credential is not needed here, and is not gone `quince` + +**`jeffry/homelab-cluster` is public and `jeffry/homelab-impl` is private.** +*Adoption flows public → private, so nothing this repository depends on requires a +credential, and `loomctl` has been exercised end to end without one.* + +**That does not remove the PAT. It moves when you need it.** *`externals` now +says: **do not adopt from a source less readable than the repository you are +adopting into.*** *Which sorts the cases:* + +- **a public repository adopting** — *may only adopt public documents*, **so it + never needs a credential**, *and needing one is the signal that something is + wrong.* +- **a private repository adopting private documents** — *legitimate, and needs + one.* + +> **So needing a credential stopped being a capability and became a signal.** *The +> tool cannot tell the two apart — it sees that the source is private and cannot +> see who may read the repository the copy lands in* — **which is exactly why +> `add` warns rather than deciding.** + +**Belief that could be shown wrong:** *that public-adopting-public covers the +common case.* **The moment a private repository wants to depend on another private +one, the credential is required and untested.** + +## 2026-09-07 — status change: the git-over-HTTPS assumption is unexercised, not blocking `quince` + +**An earlier entry records that nobody has confirmed a read-scoped PAT +authenticates git over HTTPS, and calls it load-bearing. It is no longer +blocking.** + +*It bears on `external list` against a private repository, which is the +private-to-private case above and does not occur here.* **The path is unexercised +rather than untested-and-in-the-way.** + +**Recorded because the code did not change and its status did**, *which is the +kind of thing only a log says.* **It stays the first thing to run against the next +token.** + +## 2026-09-07 — the completeness case has a mechanism: it is the facet `quince` + +**The open question was: a `200` tells you a document moved and says nothing about +whether your casting still covers it.** + +**`check` prints the document's `.usages.md` when it stages a polad**, *and for an +agreement that is the file its roles are cast in* — **so the question is answered +by reading the facet the tool just pointed at.** + +**It failed on the first real change because `externals.md` had no facet at all**, +*and the tool said `no .usages.md — nothing records what depends on this`:* **a +correct report and useless as a prompt.** + +**So both were written.** *`externals.usages.md` names which Go file implements +which rule — an unusual usage, because this repository implements the convention +rather than using it — and `cart.usages.md` gained the `v1` casting it was missing.* + +**Belief that could be shown wrong:** *that pointing is enough.* **Nothing checks +that a facet was updated, or that it was even read.** + +## 2026-09-07 — decided: nothing committed announces an open cart `quince` + +**A cart is local and untracked, so a clone cannot see that a round is open.** +*That is the boundary and not a defect:* **`v1` makes the cart local to the +working tree the two presences share, and somebody who has only cloned is by +construction not one of them.** + +**`.gitignore` records that carts happen here. Nothing records that one is open**, +*and the asymmetry is deliberate* — **a mechanism for announcing something +designed to be ephemeral is the first step in it not being ephemeral.** + +**Belief that could be shown wrong:** *that the shared working tree is the right +unit.* **If two presences ever work from different machines, the cart has nowhere +to live**, *and everything above stops being true at once.* + +## 2026-09-07 — conversion now destroys, so the log is the only record `quince` + +**Under `cart/v1` the cart is untracked, so converting a round deletes its dailies +outright.** *Every previous conversion left them in git.* + +**So an entry that was not written before the `rm` is gone**, *and the write-ahead +log is the mechanism* — **stage as you go, because at conversion the cart is the +only copy and you are about to delete it.** + +**And it sharpens a conflict recorded in `cart.usages.md`:** *the annotation +protocol here says commit before dissolving because git is the only archive.* +**With an ignored cart there is no archive, so dissolving a notes file destroys +the annotations outright.** *Both documents are loom's; this is where an adopter +can see the collision.* diff --git a/.loom/externals/.locks b/.loom/externals/.locks new file mode 100644 index 0000000..d0474e6 --- /dev/null +++ b/.loom/externals/.locks @@ -0,0 +1,12 @@ +# loomctl locks — one record per adopted document. +# pathurletag The url is resolved: a short form would follow +# whatever the default branch is at the time you ask. +git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.md https://git.hypertheory-labs.dev/jeffry/homelab-cluster/raw/branch/main/.loom/published/gitea.md "530c5bef62bbd325956ed170bb2decf37975e4b9" +git.hypertheory-labs.dev/loom/annotating/annotating.md https://git.hypertheory-labs.dev/loom/annotating/raw/branch/main/.loom/published/annotating.md "9b1f7e6ca92f2339b2d433686c27845362944bdb" +git.hypertheory-labs.dev/loom/bedrock/loom-directory.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/loom-directory.md "ee0f49cb900c0812678061971194325d9cba366a" +git.hypertheory-labs.dev/loom/bedrock/publication.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/publication.md "eb0cb63629a36b056f215dfbe24567d1918cec38" +git.hypertheory-labs.dev/loom/bedrock/recording-decisions.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/recording-decisions.md "970d4b4da76aac99c9b1f1b6580ec20daa42e329" +git.hypertheory-labs.dev/loom/bedrock/sibling-facets.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/sibling-facets.md "a46446a34ccb8bfc533d3cce19f4c88548c4fa04" +git.hypertheory-labs.dev/loom/bedrock/starting.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/starting.md "7d997a30248a88c90b23f2f9453d7cfceb03848e" +git.hypertheory-labs.dev/loom/cart/cart.md https://git.hypertheory-labs.dev/loom/cart/raw/branch/main/.loom/published/cart.md "49fc852bdd280293f0f5e0050034e3b89ff7255e" +git.hypertheory-labs.dev/loom/externals/externals.md https://git.hypertheory-labs.dev/loom/externals/raw/branch/main/.loom/published/externals.md "a7586eb52caf275d9bcedbbd8042c43e5aaad0b9" diff --git a/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.md b/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.md new file mode 100644 index 0000000..530c5be --- /dev/null +++ b/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.md @@ -0,0 +1,69 @@ +# The git host + +**`git.hypertheory-labs.dev`**, on the public internet, with a real certificate. +**This is where the `loom/*` repositories live.** + +--- + +## SSH is on 2222, and it is not optional to know + +**Git-over-SSH does not pass through Traefik** — it is raw TCP, on a +`LoadBalancer` that binds a host port on every node. **Port 22 is held by each +node's own `sshd`**, so the service is on **2222**. + +``` +ssh://git@git.hypertheory-labs.dev:2222//.git +``` + +**A clone URL without the port will not work**, and the failure looks like an +authentication problem rather than a wrong port. + +```sh +ssh -T -p 2222 git@git.hypertheory-labs.dev # "Hi there, !" once a key is registered +``` + +*`Permission denied (publickey)` from a node IP is the **success** case for an +unregistered key — the server answered and offered its host key.* + +## Registration is closed + +**One account.** *The anonymous landing page serves no sign-up link.* **If you +need access, somebody creates it for you.** + +## Two things that will surprise you + +**Sessions do not survive a restart.** *There is no Redis or valkey here — cache +and session are in memory, deliberately.* **The queue is on disk and does +survive.** *With one user this is nearly free; it stops being free if this ever +grows real users.* + +**If the control-plane node is down, this is down.** *The repository volume is +pinned to it and cannot move.* **Postgres is unaffected** — it replicates — *but +the git objects live on a volume that cannot be rescheduled.* See +[storage](storage.md). + +## Never pin the chart below what is deployed + +**Gitea does not migrate its schema backward.** *An older chart fails in the +`configure-gitea` init container with "database is for a newer Gitea", the +rollout hangs, and the old pod keeps serving.* + +**Check `helm history` before setting a version.** *This has already happened +once.* + +## The container registry + +**Gitea has one. Access to it is not worked out**, and that is an open problem +rather than an omission — see [`gaps/`](../gaps/publishing-container-images.md). + +--- + +## Checking this is still true + +**Verified 2026-09-03**, after a rebuild from scratch. + +```sh +kubectl get svc gitea-ssh -n gitea # EXTERNAL-IP = node IPs, 2222/TCP +kubectl get ingress -n gitea # CLASS=traefik, git.hypertheory-labs.dev +curl -sS -o /dev/null -w "%{http_code}\n" https://git.hypertheory-labs.dev/ +``` diff --git a/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.usages.md b/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.usages.md new file mode 100644 index 0000000..11ee116 --- /dev/null +++ b/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.usages.md @@ -0,0 +1,25 @@ +# Usages — `gitea.md` + +**Adopted 2026-09-07, and it is the reason this tool exists.** + +*This page was published, accurate, and unreachable when it would have helped. +The `:2222` fact cost three tool calls and a guess, and the page says:* **"A clone +URL without the port will not work, and the failure looks like an authentication +problem rather than a wrong port."** *It named the failure before it happened.* + +## What we use + +**SSH is on `2222`, and a clone URL without the port fails as an auth error.** +*Used by anybody working in this repository by hand.* **Not used by `loomctl`** — +*every transport it has is HTTPS on 443* — **which is worth saying, because it is +the difference between a fact for people and a fact for the tool.** + +**Registration is closed; one account.** *Which is why +`internal/external/polad.go` and `internal/external/external.go` are built around +a single reader identity per host, and why nothing here tries to check a +publication as somebody else.* + +## What we expected and did not find + +**Nothing.** *The gap this document would have filled was ours, not its: it was +private, and now it is not.* diff --git a/.loom/externals/git.hypertheory-labs.dev/loom/bedrock/starting.md b/.loom/externals/git.hypertheory-labs.dev/loom/bedrock/starting.md index e5fba07..7d997a3 100644 --- a/.loom/externals/git.hypertheory-labs.dev/loom/bedrock/starting.md +++ b/.loom/externals/git.hypertheory-labs.dev/loom/bedrock/starting.md @@ -60,9 +60,13 @@ what they came for.* ## Look at one instead of reading this -**[`jeffry/homelab-cluster`](https://git.hypertheory-labs.dev/jeffry/homelab-cluster)** -— *six documents, one gap, no decomposition, and a `README` that says what the -root documents are for and what these are for.* +**There is a worked example — six documents, one gap, no decomposition — and it +is private.** -**It is a better answer than this page**, and if the two ever disagree, it is -right. +*It describes a cluster in enough detail to be a target list, so it is not +published, and **this page will not link you to something you cannot fetch.*** +**If you have access, ask for it by name; if you do not, the two questions at the +top are the whole of it.** + +> **A public page naming a private thing as its canonical answer is worse than no +> example**, and this page did exactly that until somebody measured it. diff --git a/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.md b/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.md index 46a2d5d..49fc852 100644 --- a/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.md +++ b/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.md @@ -1,6 +1,6 @@ # Agreement — the cart -**v0.** Depends on `annotating/v0`. +**v1.** Depends on `annotating/v0`. **How two parties work out what something means before one of them changes it.** @@ -104,9 +104,54 @@ already rejected, and the rejection is gone because it lived in an annotation that died with the round. **A decline needs no file of its own.** It is an entry in whatever durable record -you keep, and **it should say what you believed, not just what you chose** — only +you keep — **which must outlive the cart**, *and therefore cannot be inside it* — and **it should say what you believed, not just what you chose** — only a belief can later be shown wrong. +## The cart is local, and that is what keeps it to two files + +**A cart is two developers working side by side.** *Everything else — the wider +team, the people who need to know, the thing that has to be tracked — is issues, +chat, whatever you already have.* **This is not that channel and it does not scale +into one.** + +> **So the cart is not committed.** *It lives in the working tree of the machine +> the two presences share, and `.loom/cart/` is ignored by version control.* + +**The reason is not tidiness. A committed cart grows a third file by itself.** +*The rule above defends against somebody asking for one; **version control does +not require anybody to ask.*** *Anyone who can clone can add `joe-rose.md`, and +then `sue-rose.md`, and the agreement's defence — **what happens to this file when +the person changes?** — is never invoked, because nobody ever had the +conversation.* + +**This is also what makes a round actually end.** *Committed, a cart is gone from +the tree and permanent in history — **so "gone" means "no longer live" and +negotiation stays quotable forever.*** **Ignored, gone means gone**, which is what +the round dying was for. + +*The cost, stated: **two presences who do not share a filesystem cannot use a +cart.*** *That is a real limit and it is the right one — if you need a medium +between machines, you need the other channel, and reaching for a cart there is +how it becomes a chat log.* + +## What is not yet a decision goes in the write-ahead log + +**A round produces things that are neither questions nor decisions:** *something +observed, something that may turn out to be noise, something you would kick +yourself for losing and cannot yet justify writing down.* + +**`event-log.wal.md`, in the cart.** *Findings, not decisions.* **Nothing in it is +durable and nothing in it has been decided.** + +**At conversion, each entry either becomes an entry in the durable record or is +discarded.** *Same two exits as a polad, and for the same reason: **conversion is +when you know most about it.*** + +> **Write the reason it is not yet an entry.** *An observation you cannot justify +> promoting is worth keeping; **one you have not said why you are hesitant about +> will be promoted by whoever finds it, on the strength of it having been written +> down.*** + ## Where a cart lives, and the shelf **A fixed path, and at most two things in it:** diff --git a/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.usages.md b/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.usages.md index b0a1a36..82ad596 100644 --- a/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.usages.md +++ b/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.usages.md @@ -33,3 +33,34 @@ revert to.* *Which is the argument for stating a belief rather than a preference: **ours was shown wrong in a way we could see**, and a preference could not have been.* + +--- + +## Cast against `v1`: the cart is not committed + +**`v1` adds no role, so nothing above changed.** *It adds something an adopter +gets wrong by default, and this file said nothing about it until now.* + +**`.loom/cart/` is in `.gitignore`.** *Cast in this repository on 2026-09-07, +after `v1` landed mid-round.* + +> **We got this wrong before it was written down.** *Two rounds — `osprey` and +> `marmalade` — were committed, and they are still in this repository's history.* +> **Not rewritten:** *honouring a rule adopted afterwards by rewriting history +> would cost more than it buys*, **and the cost is exactly the one `v1` names** — +> *the negotiation stays quotable forever.* + +**What depends on this casting:** *`.gitignore`, and +`internal/external/polad.go`* — **which stages a candidate into +`.loom/cart/current/polad/` and therefore writes only into the untracked tree.** +*If the cart were ever committed again, `check` would start proposing changes +inside version control, which is the opposite of what a polad is for.* + +## An open conflict this repository cannot settle + +**The annotation protocol this repository works under says *commit before +dissolving; git is the only archive of the conversation*.** *An ignored cart has +no archive*, **so dissolving a notes file destroys the annotations outright.** + +*Both documents are loom's. **Recorded here because the conflict is visible from +inside an adopter and not from inside either document.*** diff --git a/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.md b/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.md index 9641f0e..a7586eb 100644 --- a/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.md +++ b/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.md @@ -24,6 +24,28 @@ in the document.* you fetch may refer to others; follow one when you hit something you do not know. **Pre-resolving that is how you get a `node_modules`.*** +### Confidentiality does not travel with the copy + +**Adopting is copying.** *So a document from a repository somebody may not read +ends up in a repository they may* — **and the publisher loses control of it at the +moment of adoption**, because the copy's visibility is governed by your repository +and not by theirs. + +> **Do not adopt from a source less readable than the repository you are adopting +> into.** *If you may read it and your readers may not, copying it publishes it.* + +**Two ways out, and the second is better when it is available.** + +**Reference-only** — *record the lock and fetch on demand, keep no copy.* **You +give up reading it offline**, which is most of what a copy is for, *and you keep +the dependency recorded and checkable.* + +**Ask them to publish** — *the thing you needed was almost certainly not the +confidential part.* **A repository that must stay private can still have a public +sibling that publishes**, and the split is usually along a line that already +exists: **the operational tree is what is sensitive; the pages telling somebody +what to decide are not.** + ## Two facets beside it - **`.usages.md`** — *what we use, and **which of our artifacts depend on it*** diff --git a/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.usages.md b/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.usages.md new file mode 100644 index 0000000..5d2d806 --- /dev/null +++ b/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.usages.md @@ -0,0 +1,52 @@ +# Usages — `externals` + +**This repository implements this document.** *That is an unusual usage: most +adopters use a convention, and `loomctl` is the convention's mechanism, so nearly +every rule here names a line of code.* + +## What we use, and what implements it + +**Freshness is a conditional request, locked on the publisher's `ETag`, +verbatim** — *`internal/lock/lock.go`* (**the record: path, resolved URL, `ETag`**) +*and* `internal/external/external.go` (*`fetch`, which sends `If-None-Match`*). +**We compute no hash anywhere**, *which this document requires and which was +measured to matter: on gitea the `ETag` is the git blob hash and on GitHub it is +not.* + +**The status table — `304`, `200`, `410`, `404`** — *`checkLocked` in +`internal/external/external.go`.* **`404` is reported unresolved, naming both +readings**, *which is this document's rule and was a gap we filed against it +before it was.* + +**"The new copy is a candidate, not a replacement"** — *`internal/external/polad.go`.* +**A changed document is staged in the cart as a polad and nothing is overwritten**; +*applying is a separate act.* + +**"Reconciliation runs the other way"** — *`usagesNote` and `usagesFor` in +`internal/external/polad.go`*, **which print this kind of file beside a staged +change**, *because it names the code to go and fix.* **It says so when there is +none.* + +**The published surface is what `list` reads** — *`internal/external/list.go`,* +`.loom/published` **only.** *What is not exported is not hidden; it is simply not +what you depend on.* + +**The path is for a person** — *`localPath` in `internal/external/external.go`.* +*This document retracted "the path says where it came from"; **we record the +resolved origin in the lock instead**, and the path is `///.md` +with `--path` for when that guess is wrong.* + +## What we expected to find and did not + +**Nothing outstanding.** *The `404` gap we filed here was closed on 2026-09-07 and +the facet was deleted at reconciliation.* + +## Not yet implemented + +**"Confidentiality does not travel with the copy"**, *added 2026-09-07.* **Nothing +in `loomctl` detects it.** *The tool knows the half it can see — whether a fetch +needed a credential — and does not yet say so.* **Reference-only adoption, which +this document offers as the first way out, does not exist either.** + +*Recorded here rather than as a gap, because the gap is ours: **the document says +what to do and the tool does not do it yet.*** diff --git a/go.mod b/go.mod new file mode 100644 index 0000000..ca4729b --- /dev/null +++ b/go.mod @@ -0,0 +1,3 @@ +module git.hypertheory-labs.dev/loom/loom-cli + +go 1.25 diff --git a/internal/config/config.go b/internal/config/config.go new file mode 100644 index 0000000..2487c64 --- /dev/null +++ b/internal/config/config.go @@ -0,0 +1,84 @@ +// Package config reads the per-host settings loomctl needs to talk to a git host. +// +// The config is not only a secret. It is how you talk to a host at all, which is +// why it is keyed by host rather than being a single token. It lives in the +// user's home directory and never in a repository — see .loom/event-log.md, +// "decided by fallback: where the credential lives". +package config + +import ( + "encoding/json" + "errors" + "fmt" + "io/fs" + "os" + "path/filepath" + "strings" +) + +// Host is what we know about one git host. +type Host struct { + // Token is a read-only personal access token. It must not carry write + // scope: loomctl never writes over the network. + Token string `json:"token,omitempty"` +} + +type Config struct { + Hosts map[string]Host `json:"hosts"` +} + +// Path is where the config lives. Never inside a repository. +func Path() string { + if p := os.Getenv("LOOMCTL_CONFIG"); p != "" { + return p + } + home, err := os.UserHomeDir() + if err != nil { + return "" + } + return filepath.Join(home, ".config", "loomctl", "config.json") +} + +// Load reads the config. A missing file is not an error: everything loomctl does +// against a public repository works with no credential at all. +func Load() (*Config, error) { + c := &Config{Hosts: map[string]Host{}} + p := Path() + if p == "" { + return c, nil + } + b, err := os.ReadFile(p) + if errors.Is(err, fs.ErrNotExist) { + return c, nil + } + if err != nil { + return nil, fmt.Errorf("reading %s: %w", p, err) + } + if err := json.Unmarshal(b, c); err != nil { + return nil, fmt.Errorf("parsing %s: %w", p, err) + } + if c.Hosts == nil { + c.Hosts = map[string]Host{} + } + return c, nil +} + +// TokenFor returns the token for a host, or "" if we have none. +// +// An environment variable wins over the file, so a token can be supplied for one +// invocation without ever being written to disk. +func (c *Config) TokenFor(host string) string { + if t := os.Getenv("LOOMCTL_TOKEN_" + envKey(host)); t != "" { + return t + } + if t := os.Getenv("LOOMCTL_TOKEN"); t != "" { + return t + } + return c.Hosts[host].Token +} + +// envKey turns a hostname into the shape an environment variable can carry. +func envKey(host string) string { + r := strings.NewReplacer(".", "_", "-", "_", ":", "_") + return strings.ToUpper(r.Replace(host)) +} diff --git a/internal/external/external.go b/internal/external/external.go new file mode 100644 index 0000000..62429bf --- /dev/null +++ b/internal/external/external.go @@ -0,0 +1,399 @@ +// Package external implements the operations on documents somebody else +// published that we depend on. +// +// Every act is a fetch or a comparison. Nothing here repairs anything: a changed +// external is a candidate, not a replacement, and somebody decides. +package external + +import ( + "bytes" + "errors" + "fmt" + "io" + "io/fs" + "net/http" + "net/url" + "os" + "path" + "path/filepath" + "strings" + "time" + + "git.hypertheory-labs.dev/loom/loom-cli/internal/config" + "git.hypertheory-labs.dev/loom/loom-cli/internal/lock" +) + +var client = &http.Client{Timeout: 30 * time.Second} + +// FindRoot walks up from dir looking for the .loom directory that marks a +// repository using loom. +func FindRoot(dir string) (string, error) { + d, err := filepath.Abs(dir) + if err != nil { + return "", err + } + for { + if fi, err := os.Stat(filepath.Join(d, ".loom")); err == nil && fi.IsDir() { + return d, nil + } + parent := filepath.Dir(d) + if parent == d { + return "", errors.New("no .loom directory found in this directory or any parent") + } + d = parent + } +} + +// localPath derives where an adopted document is kept from the URL it came from. +// +// The path is for a person: ///. It deliberately +// does not encode the route, the branch, or .loom/published/ — which is why it +// cannot be turned back into a URL, and why the lock records the origin. +func localPath(raw string) (string, error) { + u, err := url.Parse(raw) + if err != nil { + return "", err + } + if u.Host == "" || u.Scheme == "" { + return "", fmt.Errorf("not an absolute URL: %s", raw) + } + segs := strings.Split(strings.Trim(u.Path, "/"), "/") + if len(segs) < 3 { + return "", fmt.Errorf("cannot tell owner and repository from %s — pass --path", raw) + } + base := segs[len(segs)-1] + if base == "" { + return "", fmt.Errorf("no file name in %s", raw) + } + return path.Join(u.Host, segs[0], segs[1], base), nil +} + +// request builds a GET carrying the host's token, if we have one. +func request(cfg *config.Config, method, raw string, ifNoneMatch string) (*http.Request, error) { + req, err := http.NewRequest(method, raw, nil) + if err != nil { + return nil, err + } + if t := cfg.TokenFor(req.URL.Host); t != "" { + req.Header.Set("Authorization", "token "+t) + } + if ifNoneMatch != "" { + req.Header.Set("If-None-Match", ifNoneMatch) + } + return req, nil +} + +// Add adopts a document that is not here yet. +// +// It refuses a path that already exists. Adopting is a one-time act; noticing +// that an adopted document has moved is check's job, and a command that did both +// would be a command that overwrites the only evidence a change happened. +func Add(root, raw, override string, out io.Writer) error { + cfg, err := config.Load() + if err != nil { + return err + } + rel := override + if rel == "" { + if rel, err = localPath(raw); err != nil { + return err + } + } + + locks, err := lock.Load(root) + if err != nil { + return err + } + dest := filepath.Join(root, lock.Dir, filepath.FromSlash(rel)) + _, onDisk := os.Stat(dest) + _, isLocked := locks.Get(rel) + + if isLocked { + return fmt.Errorf("%s is already adopted — `loomctl external check` is what notices it moving", rel) + } + + body, etag, err := fetch(cfg, raw, "") + if err != nil { + return err + } + if body == nil { + return fmt.Errorf("%s: unexpected 304 for a document we do not have", raw) + } + + // A document that is here but unlocked was fetched by hand before the tool + // existed. Supplying its URL is the only way it can ever be locked, because + // the path does not round-trip and nothing else records the origin. It is + // still not an overwrite: the bytes decide. + if onDisk == nil { + old, err := os.ReadFile(dest) + if err != nil { + return err + } + if !bytes.Equal(old, body.data) { + if !cartOpen(root) { + return fmt.Errorf("%s differs from what %s serves, and no cart is open to stage it in — "+ + "the local copy is the only evidence of that and will not be touched", rel, body.url) + } + if err := stage(root, rel, body.data, lock.Record{Path: rel, URL: body.url, ETag: etag}); err != nil { + return err + } + fmt.Fprintf(out, "staged %s\n", rel) + fmt.Fprintf(out, " from %s\n", body.url) + fmt.Fprintf(out, " NOTE the local copy differs and was left alone; it is the only evidence\n") + fmt.Fprintf(out, " that this moved while nothing was watching. Apply or discard.\n") + return nil + } + // Identical, so the assertion a lock makes — this local copy is the one + // being served — is verified rather than assumed. + locks.Put(lock.Record{Path: rel, URL: body.url, ETag: etag}) + if err := locks.Save(); err != nil { + return err + } + fmt.Fprintf(out, "locked %s\n", rel) + fmt.Fprintf(out, " from %s\n", body.url) + fmt.Fprintf(out, " etag %s (bytes verified identical; nothing was rewritten)\n", etag) + return nil + } + + if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { + return err + } + if err := os.WriteFile(dest, body.data, 0o644); err != nil { + return err + } + locks.Put(lock.Record{Path: rel, URL: body.url, ETag: etag}) + if err := locks.Save(); err != nil { + return err + } + + fmt.Fprintf(out, "adopted %s\n", rel) + fmt.Fprintf(out, " from %s\n", body.url) + warnIfNotPublic(cfg, body.url, out) + if etag == "" { + fmt.Fprintf(out, " etag (none served — check cannot ask conditionally)\n") + } else { + fmt.Fprintf(out, " etag %s\n", etag) + } + return nil +} + +// warnIfNotPublic says so when a document could only be fetched with a +// credential. +// +// Adopting is copying, so a document from a repository somebody may not read +// ends up in a repository they may, and the publisher loses control of it at the +// moment of adoption. The tool can see half of that — whether this fetch needed +// a credential — and cannot see the other half, which is who can read the +// repository the copy is landing in. It reports the half it knows. +func warnIfNotPublic(cfg *config.Config, raw string, out io.Writer) { + req, err := http.NewRequest(http.MethodHead, raw, nil) + if err != nil { + return + } + if cfg.TokenFor(req.URL.Host) == "" { + return // no credential was used, so the fetch was already anonymous + } + resp, err := client.Do(req) + if err != nil { + return + } + defer resp.Body.Close() + if resp.StatusCode == http.StatusOK { + return + } + fmt.Fprintf(out, " WARN this needed a credential — anonymously it is %s.\n", resp.Status) + fmt.Fprintf(out, " Confidentiality does not travel with the copy: writing it into\n") + fmt.Fprintf(out, " .loom/externals/ republishes it to anyone who can read THIS\n") + fmt.Fprintf(out, " repository. Do not adopt from a source less readable than the\n") + fmt.Fprintf(out, " repository you are adopting into.\n") +} + +type fetched struct { + data []byte + url string +} + +// fetch performs one request. A nil body with no error means 304. +func fetch(cfg *config.Config, raw, ifNoneMatch string) (*fetched, string, error) { + req, err := request(cfg, http.MethodGet, raw, ifNoneMatch) + if err != nil { + return nil, "", err + } + resp, err := client.Do(req) + if err != nil { + return nil, "", err + } + defer resp.Body.Close() + + switch resp.StatusCode { + case http.StatusNotModified: + io.Copy(io.Discard, resp.Body) + return nil, resp.Header.Get("ETag"), nil + case http.StatusOK: + b, err := io.ReadAll(resp.Body) + if err != nil { + return nil, "", err + } + return &fetched{data: b, url: resp.Request.URL.String()}, resp.Header.Get("ETag"), nil + case http.StatusNotFound: + return nil, "", fmt.Errorf("404 unresolved: %s was withdrawn, or this credential cannot see it — "+ + "over HTTP these are the same response", raw) + default: + return nil, "", fmt.Errorf("%s: %s", resp.Status, raw) + } +} + +// Status is what check found for one document. +type Status struct { + Path string + Result string + Detail string +} + +// Check asks every publisher whether their copy has moved, and stages what did. +// +// It never edits an adopted document. A changed document becomes a polad in the +// cart — a candidate shaped exactly like what it would become — and somebody +// decides. +func Check(root string, out io.Writer) error { + cfg, err := config.Load() + if err != nil { + return err + } + locks, err := lock.Load(root) + if err != nil { + return err + } + open := cartOpen(root) + + seen := map[string]bool{} + var results []Status + staged := 0 + + for _, rec := range locks.All() { + seen[rec.Path] = true + st, did := checkLocked(cfg, root, rec, open) + staged += did + results = append(results, st) + } + + unlocked, err := unlockedDocs(root, seen) + if err != nil { + return err + } + for _, rel := range unlocked { + st, did := checkUnlocked(cfg, root, rel, locks, open) + staged += did + results = append(results, st) + } + if err := locks.Save(); err != nil { + return err + } + + if len(results) == 0 { + fmt.Fprintln(out, "no adopted documents") + return nil + } + w := 0 + for _, r := range results { + if len(r.Path) > w { + w = len(r.Path) + } + } + for _, r := range results { + fmt.Fprintf(out, "%-*s %-9s %s\n", w, r.Path, r.Result, r.Detail) + } + if staged > 0 { + fmt.Fprintf(out, "\n%d staged in %s — apply or discard; nothing here drifts into being kept.\n", staged, PoladDir) + } + return nil +} + +// checkLocked asks conditionally. The second return is 1 if a polad was staged. +func checkLocked(cfg *config.Config, root string, rec lock.Record, open bool) (Status, int) { + if _, err := os.Stat(filepath.Join(root, lock.Dir, filepath.FromSlash(rec.Path))); errors.Is(err, fs.ErrNotExist) { + return Status{rec.Path, "missing", "locked, but the local copy is gone"}, 0 + } + if rec.ETag == "" { + return Status{rec.Path, "no-etag", "publisher served none; freshness cannot be asked"}, 0 + } + body, etag, err := fetch(cfg, rec.URL, rec.ETag) + if err != nil { + return Status{rec.Path, "error", err.Error()}, 0 + } + if body == nil { + return Status{rec.Path, "same", ""}, 0 + } + if !open { + return Status{rec.Path, "CHANGED", "upstream moved — no cart open, so nothing was staged"}, 0 + } + if err := stage(root, rec.Path, body.data, lock.Record{Path: rec.Path, URL: body.url, ETag: etag}); err != nil { + return Status{rec.Path, "error", err.Error()}, 0 + } + return Status{rec.Path, "CHANGED", "staged as a polad" + usagesNote(root, rec.Path)}, 1 +} + +// checkUnlocked fetches a document nothing has locked and compares the bytes. +// +// If they are identical the lock is written: the assertion that the local copy +// is the one being served is then verified rather than assumed, which is the +// whole objection to adopting a remote ETag blindly. If they differ, the local +// copy is evidence and is not touched. +func checkUnlocked(cfg *config.Config, root, rel string, locks *lock.Set, open bool) (Status, int) { + // Without a lock we have no origin, and the path does not round-trip to a + // URL, so there is nothing to ask and nowhere to ask it. Supplying the URL + // through add is the only way out. + return Status{rel, "unlocked", "no origin recorded — `loomctl external add ` supplies it without rewriting this copy"}, 0 +} + +func usagesNote(root, rel string) string { + u, ok := usagesFor(root, rel) + if ok { + return "; " + u + " names what to fix" + } + return "; no .usages.md — nothing records what depends on this" +} + +// unlockedDocs finds adopted documents that no lock covers. Facets we wrote +// ourselves are not adopted documents and are skipped. +func unlockedDocs(root string, locked map[string]bool) ([]string, error) { + base := filepath.Join(root, lock.Dir) + var out []string + err := filepath.WalkDir(base, func(p string, d fs.DirEntry, err error) error { + if err != nil { + if errors.Is(err, fs.ErrNotExist) { + return nil + } + return err + } + if d.IsDir() || !strings.HasSuffix(d.Name(), ".md") { + return nil + } + rel, err := filepath.Rel(base, p) + if err != nil { + return err + } + rel = filepath.ToSlash(rel) + if locked[rel] || isFacet(rel) { + return nil + } + out = append(out, rel) + return nil + }) + return out, err +} + +// isFacet reports whether a path is something we wrote beside an adopted +// document rather than the document itself: x.usages.md, x.gaps.md, x.notes.md. +func isFacet(rel string) bool { + base := strings.TrimSuffix(path.Base(rel), ".md") + i := strings.LastIndex(base, ".") + if i < 0 { + return false + } + switch base[i+1:] { + case "usages", "gaps", "notes": + return true + } + return false +} diff --git a/internal/external/external_test.go b/internal/external/external_test.go new file mode 100644 index 0000000..79b5a3d --- /dev/null +++ b/internal/external/external_test.go @@ -0,0 +1,72 @@ +package external + +import "testing" + +func TestLocalPath(t *testing.T) { + for _, tc := range []struct { + name, url, want string + wantErr bool + }{ + { + name: "gitea raw, published document", + url: "https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/starting.md", + want: "git.hypertheory-labs.dev/loom/bedrock/starting.md", + }, + { + name: "github raw host", + url: "https://raw.githubusercontent.com/octocat/Hello-World/main/README.md", + want: "raw.githubusercontent.com/octocat/Hello-World/README.md", + }, + { + name: "not enough path to name an owner and repository", + url: "https://example.com/thing.md", + wantErr: true, + }, + { + name: "not absolute", + url: "/loom/bedrock/starting.md", + wantErr: true, + }, + } { + t.Run(tc.name, func(t *testing.T) { + got, err := localPath(tc.url) + if tc.wantErr { + if err == nil { + t.Fatalf("localPath(%q) = %q, want an error", tc.url, got) + } + return + } + if err != nil { + t.Fatalf("localPath(%q): %v", tc.url, err) + } + if got != tc.want { + t.Errorf("localPath(%q) = %q, want %q", tc.url, got, tc.want) + } + }) + } +} + +func TestIsFacet(t *testing.T) { + // A facet is something we wrote beside an adopted document. check must not + // report our own writing as an unlocked external. + facets := []string{ + "host/loom/cart/cart.usages.md", + "host/loom/externals/externals.gaps.md", + "host/o/r/plan.notes.md", + } + documents := []string{ + "host/loom/cart/cart.md", + "host/loom/bedrock/recording-decisions.md", // a hyphen is not a facet + "host/o/r/starting.md", + } + for _, p := range facets { + if !isFacet(p) { + t.Errorf("isFacet(%q) = false, want true", p) + } + } + for _, p := range documents { + if isFacet(p) { + t.Errorf("isFacet(%q) = true, want false", p) + } + } +} diff --git a/internal/external/list.go b/internal/external/list.go new file mode 100644 index 0000000..8b7db0a --- /dev/null +++ b/internal/external/list.go @@ -0,0 +1,107 @@ +package external + +import ( + "bytes" + "fmt" + "io" + "os" + "os/exec" + "strings" + + "git.hypertheory-labs.dev/loom/loom-cli/internal/config" +) + +// PublishedDir is the only directory in somebody else's repository that list +// looks at. What is not exported is not hidden — it is simply not what you +// depend on. +const PublishedDir = ".loom/published" + +// List enumerates a publisher's published surface. +// +// It shells out to git rather than using a host's REST API, because git is the +// one interface gitea, GitHub and GitLab all speak the same way: their contents +// APIs have three different URL shapes, three JSON shapes and three auth +// schemes, and a private repository refuses the anonymous ones. The cost is that +// git must be on PATH. +func List(repoURL string, out io.Writer) error { + cfg, err := config.Load() + if err != nil { + return err + } + + dir, err := os.MkdirTemp("", "loomctl-list-") + if err != nil { + return err + } + defer os.RemoveAll(dir) + + clone := exec.Command("git", "clone", + "--filter=blob:none", // trees only: we want names, not contents + "--depth=1", // and bound the damage if the server ignores the filter + "--no-checkout", + "--quiet", + repoURL, dir, + ) + clone.Env = gitEnv(cfg, repoURL) + var stderr bytes.Buffer + clone.Stderr = &stderr + if err := clone.Run(); err != nil { + return fmt.Errorf("git clone: %w\n%s", err, strings.TrimSpace(stderr.String())) + } + + // git's fallback when a server refuses the filter is silent apart from this + // warning, and the fallback is to download everything. + if strings.Contains(stderr.String(), "filtering not recognized by server") { + fmt.Fprintf(out, "warning: %s ignored --filter, so this fetched every blob at HEAD\n\n", repoURL) + } + + ls := exec.Command("git", "-C", dir, "ls-tree", "--name-only", "HEAD:"+PublishedDir) + var names, lsErr bytes.Buffer + ls.Stdout, ls.Stderr = &names, &lsErr + if err := ls.Run(); err != nil { + fmt.Fprintf(out, "%s publishes nothing — no %s\n", repoURL, PublishedDir) + return nil + } + + for _, n := range strings.Split(strings.TrimSpace(names.String()), "\n") { + if n != "" { + fmt.Fprintln(out, n) + } + } + return nil +} + +// gitEnv passes credentials to git through the environment rather than through +// -c on the command line, because argv is visible to every process on the +// machine and an environment is not. +func gitEnv(cfg *config.Config, repoURL string) []string { + env := append(os.Environ(), "GIT_TERMINAL_PROMPT=0") + host := hostOf(repoURL) + if host == "" { + return env + } + t := cfg.TokenFor(host) + if t == "" { + return env + } + return append(env, + "GIT_CONFIG_COUNT=1", + "GIT_CONFIG_KEY_0=http.extraHeader", + "GIT_CONFIG_VALUE_0=Authorization: token "+t, + ) +} + +func hostOf(raw string) string { + i := strings.Index(raw, "://") + if i < 0 { + return "" + } + rest := raw[i+3:] + if at := strings.Index(rest, "@"); at >= 0 { + rest = rest[at+1:] + } + if s := strings.IndexAny(rest, "/:"); s >= 0 { + rest = rest[:s] + } + return rest +} diff --git a/internal/external/polad.go b/internal/external/polad.go new file mode 100644 index 0000000..ceda169 --- /dev/null +++ b/internal/external/polad.go @@ -0,0 +1,121 @@ +package external + +import ( + "fmt" + "io" + "os" + "path/filepath" + + "git.hypertheory-labs.dev/loom/loom-cli/internal/lock" +) + +// CartDir is where a round happens. A cart is not committed — it lives in the +// working tree of the machine the two presences share — so everything staged +// here is deliberately outside version control. +const CartDir = ".loom/cart/current" + +// PoladDir is where a changed external waits for somebody to decide. +// +// A polad is a candidate artifact, shaped exactly like what it would become, +// staged so you can see whether it fits. Its exits are apply or discard, and +// nothing may drift into being kept. +const PoladDir = CartDir + "/polad" + +// cartOpen reports whether there is a round to stage into. The tool never opens +// one: a cart is a bounded exchange between two presences, and starting it is +// somebody's act, not a side effect of checking freshness. +func cartOpen(root string) bool { + fi, err := os.Stat(filepath.Join(root, CartDir)) + return err == nil && fi.IsDir() +} + +// stage writes a candidate copy into the cart, with the ETag that was served +// alongside the bytes, so that applying it locks what somebody actually read. +func stage(root, rel string, body []byte, rec lock.Record) error { + dest := filepath.Join(root, filepath.FromSlash(PoladDir), filepath.FromSlash(rel)) + if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { + return err + } + if err := os.WriteFile(dest, body, 0o644); err != nil { + return err + } + locks, err := lock.LoadFile(poladLockPath(root)) + if err != nil { + return err + } + locks.Put(rec) + return locks.Save() +} + +func poladLockPath(root string) string { + return filepath.Join(root, filepath.FromSlash(PoladDir), lock.File) +} + +// usagesFor returns the path of the facet naming what depends on a document, and +// whether it exists. +// +// Reconciliation runs the other way: the question is not what do we rewrite +// here, but given what changed in theirs, what do we change in ours. The facets +// usually survive unchanged — what moves is the manifests, the config, the code +// that a usage named, which is why a usage names them. +func usagesFor(root, rel string) (string, bool) { + p := rel[:len(rel)-len(filepath.Ext(rel))] + ".usages.md" + _, err := os.Stat(filepath.Join(root, lock.Dir, filepath.FromSlash(p))) + return p, err == nil +} + +// Apply moves a staged polad into place and moves its lock with it. +// +// This exists because the lock is the half a person forgets. Moving the file by +// hand leaves a lock describing the copy you just replaced, which is the drift +// the lock was there to prevent. +func Apply(root string, rels []string, out io.Writer) error { + staged, err := lock.LoadFile(poladLockPath(root)) + if err != nil { + return err + } + locks, err := lock.Load(root) + if err != nil { + return err + } + if len(rels) == 0 { + for _, r := range staged.All() { + rels = append(rels, r.Path) + } + } + if len(rels) == 0 { + fmt.Fprintln(out, "nothing staged") + return nil + } + for _, rel := range rels { + rec, ok := staged.Get(rel) + if !ok { + return fmt.Errorf("%s is not staged", rel) + } + src := filepath.Join(root, filepath.FromSlash(PoladDir), filepath.FromSlash(rel)) + body, err := os.ReadFile(src) + if err != nil { + return err + } + dest := filepath.Join(root, lock.Dir, filepath.FromSlash(rel)) + if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { + return err + } + if err := os.WriteFile(dest, body, 0o644); err != nil { + return err + } + locks.Put(rec) + staged.Remove(rel) + if err := os.Remove(src); err != nil { + return err + } + fmt.Fprintf(out, "applied %s\n", rel) + if u, ok := usagesFor(root, rel); ok { + fmt.Fprintf(out, " check %s names what depends on this\n", u) + } + } + if err := locks.Save(); err != nil { + return err + } + return staged.Save() +} diff --git a/internal/lock/lock.go b/internal/lock/lock.go new file mode 100644 index 0000000..1b0e3a7 --- /dev/null +++ b/internal/lock/lock.go @@ -0,0 +1,131 @@ +// Package lock reads and writes .loom/externals/.locks. +// +// One record per adopted document: where it was fetched from, resolved, and the +// ETag the publisher served with it. The ETag is opaque and is never a hash we +// compute — on gitea it happens to equal the git blob hash and on GitHub it does +// not, so a design that compares a local hash to a remote ETag works on exactly +// one host by coincidence. See .loom/event-log.md. +package lock + +import ( + "bufio" + "errors" + "fmt" + "io/fs" + "os" + "path/filepath" + "sort" + "strings" +) + +// Dir is where adopted documents live, relative to the repository root. +const Dir = ".loom/externals" + +// File is the lock file, inside Dir. +const File = ".locks" + +const header = "# loomctl locks — one record per adopted document.\n" + + "# pathurletag The url is resolved: a short form would follow\n" + + "# whatever the default branch is at the time you ask.\n" + +// Record is one adopted document. +type Record struct { + // Path is relative to Dir, and is for a person to read. The origin is the + // URL: the path does not round-trip, because it drops the route, the + // branch, and .loom/published/. + Path string + // URL is the resolved origin, branch and all. + URL string + // ETag is the publisher's, verbatim, including its quotes. + ETag string +} + +// Set is every lock, keyed by path. +type Set struct { + file string + recs map[string]Record +} + +// Path is the lock file for the repository at root. +func Path(root string) string { return filepath.Join(root, Dir, File) } + +// Load reads the lock file for the repository at root. A missing file is an +// empty set, not an error: a repository whose externals were fetched by hand has +// no locks, and reporting that is the point. +func Load(root string) (*Set, error) { return LoadFile(Path(root)) } + +// LoadFile reads a lock file from an explicit path. A staged polad carries its +// own alongside it, so that applying it uses the ETag that was served with the +// bytes somebody reviewed, rather than whatever the publisher serves later. +func LoadFile(file string) (*Set, error) { + s := &Set{file: file, recs: map[string]Record{}} + f, err := os.Open(file) + if errors.Is(err, fs.ErrNotExist) { + return s, nil + } + if err != nil { + return nil, err + } + defer f.Close() + + sc := bufio.NewScanner(f) + for n := 1; sc.Scan(); n++ { + line := sc.Text() + if strings.TrimSpace(line) == "" || strings.HasPrefix(line, "#") { + continue + } + parts := strings.Split(line, "\t") + if len(parts) != 3 { + return nil, fmt.Errorf("%s:%d: want 3 tab-separated fields, got %d", file, n, len(parts)) + } + s.recs[parts[0]] = Record{Path: parts[0], URL: parts[1], ETag: parts[2]} + } + return s, sc.Err() +} + +// Remove drops a record. +func (s *Set) Remove(p string) { delete(s.recs, p) } + +// Get returns the record for a path, and whether there was one. +func (s *Set) Get(p string) (Record, bool) { r, ok := s.recs[p]; return r, ok } + +// Put adds or replaces a record. +func (s *Set) Put(r Record) { s.recs[r.Path] = r } + +// All returns every record, ordered by path so the file diffs cleanly. +func (s *Set) All() []Record { + out := make([]Record, 0, len(s.recs)) + for _, r := range s.recs { + out = append(out, r) + } + sort.Slice(out, func(i, j int) bool { return out[i].Path < out[j].Path }) + return out +} + +// Save writes the lock file, replacing it atomically so an interrupted write +// cannot leave a repository holding half a lock. +func (s *Set) Save() error { + p := s.file + if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil { + return err + } + var b strings.Builder + b.WriteString(header) + for _, r := range s.All() { + fmt.Fprintf(&b, "%s\t%s\t%s\n", r.Path, r.URL, r.ETag) + } + tmp, err := os.CreateTemp(filepath.Dir(p), ".locks-*") + if err != nil { + return err + } + if _, err := tmp.WriteString(b.String()); err != nil { + tmp.Close() + os.Remove(tmp.Name()) + return err + } + if err := tmp.Close(); err != nil { + os.Remove(tmp.Name()) + return err + } + return os.Rename(tmp.Name(), p) +} diff --git a/main.go b/main.go new file mode 100644 index 0000000..9b361ed --- /dev/null +++ b/main.go @@ -0,0 +1,122 @@ +// loomctl fetches documents this repository depends on, and finds out when they +// change. +// +// It reports and never repairs. Everything it writes, it writes to the working +// tree — committing and pushing are yours, because the consequences of a push +// land on people a tool cannot experience. +package main + +import ( + "flag" + "fmt" + "os" + + "git.hypertheory-labs.dev/loom/loom-cli/internal/external" +) + +const usage = `loomctl — fetch what you depend on, and find out when it changed. + + loomctl external list what a repository publishes + loomctl external add [--path p] adopt one document and lock it + loomctl external check ask every publisher whether theirs moved + loomctl external apply [path...] move a staged polad into place, lock and all + +check reports and does not fix. A document that moved is staged as a polad in +.loom/cart/current/polad/ — a candidate shaped exactly like what it would +become — and somebody decides. Its exits are apply or discard; nothing there +drifts into being kept. With no cart open, check says what moved and stages +nothing, because opening a round is somebody's act and not a side effect. + +add warns when a document could only be fetched with a credential: adopting is +copying, and confidentiality does not travel with the copy. It cannot see who +may read the repository the copy lands in, so it reports the half it knows. + +add adopts what is not here yet, and refuses what is already adopted. Given a +document that is present but unlocked — fetched by hand before this existed — +it supplies the missing origin: identical bytes lock it, differing bytes are +staged, and the local copy is never overwritten, because a copy that differs is +the only evidence that anything moved while nothing was watching. + +Adopted documents live in .loom/externals////.md, and +their origins in .loom/externals/.locks. The path is for a person to read; the +lock is what a machine uses, because the path does not round-trip to a URL. + +Credentials are read-only and per host, in ~/.config/loomctl/config.json or in +LOOMCTL_TOKEN_. loomctl never writes over the network, so a token it is +given should never carry write scope. + +Requires git on PATH, for list only. +` + +func main() { + if err := run(os.Args[1:]); err != nil { + fmt.Fprintln(os.Stderr, "loomctl: "+err.Error()) + os.Exit(1) + } +} + +func run(args []string) error { + if len(args) == 0 || args[0] == "-h" || args[0] == "--help" || args[0] == "help" { + fmt.Print(usage) + return nil + } + switch args[0] { + case "external": + return runExternal(args[1:]) + default: + return fmt.Errorf("unknown command %q\n\n%s", args[0], usage) + } +} + +func runExternal(args []string) error { + if len(args) == 0 { + return fmt.Errorf("external needs a subcommand: list, add, check, apply") + } + switch args[0] { + case "list": + if len(args) != 2 { + return fmt.Errorf("usage: loomctl external list ") + } + return external.List(args[1], os.Stdout) + + case "add": + fs := flag.NewFlagSet("add", flag.ContinueOnError) + path := fs.String("path", "", "where to keep it, relative to .loom/externals (default: derived from the URL)") + if err := fs.Parse(args[1:]); err != nil { + return err + } + if fs.NArg() != 1 { + return fmt.Errorf("usage: loomctl external add [--path p]") + } + root, err := root() + if err != nil { + return err + } + return external.Add(root, fs.Arg(0), *path, os.Stdout) + + case "check": + root, err := root() + if err != nil { + return err + } + return external.Check(root, os.Stdout) + + case "apply": + root, err := root() + if err != nil { + return err + } + return external.Apply(root, args[1:], os.Stdout) + + default: + return fmt.Errorf("unknown external subcommand %q", args[0]) + } +} + +func root() (string, error) { + wd, err := os.Getwd() + if err != nil { + return "", err + } + return external.FindRoot(wd) +}