From 1543df0a0c0c8adab4e357f72c69ad7254cb946a Mon Sep 17 00:00:00 2001 From: Jeff Gonzalez Date: Mon, 7 Sep 2026 14:48:29 -0400 Subject: [PATCH 1/6] =?UTF-8?q?loomctl=20external:=20list,=20add,=20check?= =?UTF-8?q?=20=E2=80=94=20and=20its=20first=20run=20found=20two=20changed?= =?UTF-8?q?=20documents?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Go, standard library only, with git shelled out for list alone. list enumerates a publisher's .loom/published by partial clone and ls-tree; add fetches one document, writes it under .loom/externals and records the resolved origin and the publisher's ETag in .loom/externals/.locks; check asks conditionally and reports. The first real run did what the tool exists for. All eight documents adopted by hand before it existed reported unlocked — the tool refuses to invent a lock by adopting whatever the remote currently serves, since that would assert the local copy is the one being served, which is the thing it was about to check. Locking them fetched two that had moved: bedrock/starting.md, which now says the worked example is private and will not link to something you cannot fetch, and cart.md, which went to v1. cart v1 changes a role we cast: a cart is not committed, because a committed cart grows a third file by itself — version control does not require anybody to ask, so the two-file rule is never invoked — and because ignored, gone means gone. Adds .loom/cart/ to .gitignore and supersedes the isolation entry rather than editing it. osprey and marmalade are already in history and are left there: rewriting to honour a rule adopted afterwards costs more than it buys. Records the conflict this creates rather than settling it: the annotation protocol here says commit before dissolving because git is the only archive, and an ignored cart has no archive, so dissolving would destroy the annotations outright. Credentials are read-only, per host, and passed to git through the environment rather than argv, because argv is visible to every process on the machine. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018UTxuSizozEA8yDitPuris --- .gitignore | 5 + .loom/event-log.md | 52 ++++ .loom/externals/.locks | 11 + .../loom/bedrock/starting.md | 14 +- .../loom/cart/cart.md | 49 ++- go.mod | 3 + internal/config/config.go | 84 ++++++ internal/external/external.go | 283 ++++++++++++++++++ internal/external/external_test.go | 72 +++++ internal/external/list.go | 107 +++++++ internal/lock/lock.go | 122 ++++++++ main.go | 102 +++++++ 12 files changed, 897 insertions(+), 7 deletions(-) create mode 100644 .gitignore create mode 100644 .loom/externals/.locks create mode 100644 go.mod create mode 100644 internal/config/config.go create mode 100644 internal/external/external.go create mode 100644 internal/external/external_test.go create mode 100644 internal/external/list.go create mode 100644 internal/lock/lock.go create mode 100644 main.go diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..5d37546 --- /dev/null +++ b/.gitignore @@ -0,0 +1,5 @@ +# cart v1: a cart is not committed. Ignored, gone means gone. +.loom/cart/ + +# build output +/loomctl diff --git a/.loom/event-log.md b/.loom/event-log.md index 0c0d596..7183ffd 100644 --- a/.loom/event-log.md +++ b/.loom/event-log.md @@ -518,3 +518,55 @@ the cart that named it.* **Recorded rather than fixed.** *The role still works; its justification was written for a case that has not yet occurred.* **If a round ever does produce code while it is open, nothing here changes.** + +## 2026-09-07 — `loomctl external` exists, and its first run reconciled two documents `marmalade` + +**Built: `list`, `add`, `check`.** *Go, no dependencies outside the standard +library, `git` shelled out for `list` only.* + +**The first real run did the thing the tool is for.** *Eight documents were +adopted by hand before it existed; `check` reported all eight `unlocked`, and +`add` locked them* — **and two came back changed**, `bedrock/starting.md` *and* +`cart/cart.md`. **Neither change would have been noticed by anybody.** + +**Believed to advance:** *the lock is what makes `check` possible at all.* **With +no lock there is nothing to compare and the only honest report is `unlocked`** — +*and the tool refuses to invent one by adopting whatever the remote currently +serves.* + +**Belief that could be shown wrong:** *that a per-person path derived from the URL +is good enough.* **`///.md` guesses that the first two +path segments name an owner and a repository**, *which is true of gitea, GitHub and +GitLab and is not a rule.* **`--path` exists for when it is wrong.** + +## 2026-09-07 — supersedes the isolation role: a cart is not committed `marmalade` + +**`cart` is now `v1` and it changed the thing we cast a role on.** + +> **So the cart is not committed.** *It lives in the working tree of the machine +> the two presences share, and `.loom/cart/` is ignored by version control.* + +**The reason is not tidiness:** *a committed cart grows a third file by itself.* +**The two-file rule defends against somebody asking for one; version control does +not require anybody to ask** — *anyone who can clone can add a third, and the +agreement's test is never invoked because nobody had the conversation.* + +**And it is what makes a round end.** *Committed, a cart is gone from the tree and +permanent in history* — **so "gone" means "no longer live" and the negotiation +stays quotable forever.** *Ignored, gone means gone.* + +**So `.loom/cart/` is now in `.gitignore`.** *The isolation role entry above +assumed the cart lived on the default branch; **the cart lives on no branch.*** +*What survives of that entry is the other half: work is isolated on a branch named +for the round that authorised it.* + +**Not undone: `osprey` and `marmalade` are already in this repository's history.** +*Rewriting history to honour a rule adopted afterwards would cost more than it +buys*, **and the two rounds are quotable forever, which is exactly what `v1` says +not to want.** *Recorded rather than repaired.* + +**Belief that could be shown wrong, and it is a real conflict:** *the annotation +protocol this repository works under says **commit before dissolving, git is the +only archive of the conversation.*** **An ignored cart has no archive**, so +dissolving a notes file destroys the annotations outright. *One of the two is +wrong and it is not ours to settle.* diff --git a/.loom/externals/.locks b/.loom/externals/.locks new file mode 100644 index 0000000..8d1faf9 --- /dev/null +++ b/.loom/externals/.locks @@ -0,0 +1,11 @@ +# loomctl locks — one record per adopted document. +# pathurletag The url is resolved: a short form would follow +# whatever the default branch is at the time you ask. +git.hypertheory-labs.dev/loom/annotating/annotating.md https://git.hypertheory-labs.dev/loom/annotating/raw/branch/main/.loom/published/annotating.md "9b1f7e6ca92f2339b2d433686c27845362944bdb" +git.hypertheory-labs.dev/loom/bedrock/loom-directory.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/loom-directory.md "ee0f49cb900c0812678061971194325d9cba366a" +git.hypertheory-labs.dev/loom/bedrock/publication.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/publication.md "eb0cb63629a36b056f215dfbe24567d1918cec38" +git.hypertheory-labs.dev/loom/bedrock/recording-decisions.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/recording-decisions.md "970d4b4da76aac99c9b1f1b6580ec20daa42e329" +git.hypertheory-labs.dev/loom/bedrock/sibling-facets.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/sibling-facets.md "a46446a34ccb8bfc533d3cce19f4c88548c4fa04" +git.hypertheory-labs.dev/loom/bedrock/starting.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/starting.md "7d997a30248a88c90b23f2f9453d7cfceb03848e" +git.hypertheory-labs.dev/loom/cart/cart.md https://git.hypertheory-labs.dev/loom/cart/raw/branch/main/.loom/published/cart.md "49fc852bdd280293f0f5e0050034e3b89ff7255e" +git.hypertheory-labs.dev/loom/externals/externals.md https://git.hypertheory-labs.dev/loom/externals/raw/branch/main/.loom/published/externals.md "9641f0e77b6f5c0161fa7593805277ba0c1e6176" diff --git a/.loom/externals/git.hypertheory-labs.dev/loom/bedrock/starting.md b/.loom/externals/git.hypertheory-labs.dev/loom/bedrock/starting.md index e5fba07..7d997a3 100644 --- a/.loom/externals/git.hypertheory-labs.dev/loom/bedrock/starting.md +++ b/.loom/externals/git.hypertheory-labs.dev/loom/bedrock/starting.md @@ -60,9 +60,13 @@ what they came for.* ## Look at one instead of reading this -**[`jeffry/homelab-cluster`](https://git.hypertheory-labs.dev/jeffry/homelab-cluster)** -— *six documents, one gap, no decomposition, and a `README` that says what the -root documents are for and what these are for.* +**There is a worked example — six documents, one gap, no decomposition — and it +is private.** -**It is a better answer than this page**, and if the two ever disagree, it is -right. +*It describes a cluster in enough detail to be a target list, so it is not +published, and **this page will not link you to something you cannot fetch.*** +**If you have access, ask for it by name; if you do not, the two questions at the +top are the whole of it.** + +> **A public page naming a private thing as its canonical answer is worse than no +> example**, and this page did exactly that until somebody measured it. diff --git a/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.md b/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.md index 46a2d5d..49fc852 100644 --- a/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.md +++ b/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.md @@ -1,6 +1,6 @@ # Agreement — the cart -**v0.** Depends on `annotating/v0`. +**v1.** Depends on `annotating/v0`. **How two parties work out what something means before one of them changes it.** @@ -104,9 +104,54 @@ already rejected, and the rejection is gone because it lived in an annotation that died with the round. **A decline needs no file of its own.** It is an entry in whatever durable record -you keep, and **it should say what you believed, not just what you chose** — only +you keep — **which must outlive the cart**, *and therefore cannot be inside it* — and **it should say what you believed, not just what you chose** — only a belief can later be shown wrong. +## The cart is local, and that is what keeps it to two files + +**A cart is two developers working side by side.** *Everything else — the wider +team, the people who need to know, the thing that has to be tracked — is issues, +chat, whatever you already have.* **This is not that channel and it does not scale +into one.** + +> **So the cart is not committed.** *It lives in the working tree of the machine +> the two presences share, and `.loom/cart/` is ignored by version control.* + +**The reason is not tidiness. A committed cart grows a third file by itself.** +*The rule above defends against somebody asking for one; **version control does +not require anybody to ask.*** *Anyone who can clone can add `joe-rose.md`, and +then `sue-rose.md`, and the agreement's defence — **what happens to this file when +the person changes?** — is never invoked, because nobody ever had the +conversation.* + +**This is also what makes a round actually end.** *Committed, a cart is gone from +the tree and permanent in history — **so "gone" means "no longer live" and +negotiation stays quotable forever.*** **Ignored, gone means gone**, which is what +the round dying was for. + +*The cost, stated: **two presences who do not share a filesystem cannot use a +cart.*** *That is a real limit and it is the right one — if you need a medium +between machines, you need the other channel, and reaching for a cart there is +how it becomes a chat log.* + +## What is not yet a decision goes in the write-ahead log + +**A round produces things that are neither questions nor decisions:** *something +observed, something that may turn out to be noise, something you would kick +yourself for losing and cannot yet justify writing down.* + +**`event-log.wal.md`, in the cart.** *Findings, not decisions.* **Nothing in it is +durable and nothing in it has been decided.** + +**At conversion, each entry either becomes an entry in the durable record or is +discarded.** *Same two exits as a polad, and for the same reason: **conversion is +when you know most about it.*** + +> **Write the reason it is not yet an entry.** *An observation you cannot justify +> promoting is worth keeping; **one you have not said why you are hesitant about +> will be promoted by whoever finds it, on the strength of it having been written +> down.*** + ## Where a cart lives, and the shelf **A fixed path, and at most two things in it:** diff --git a/go.mod b/go.mod new file mode 100644 index 0000000..ca4729b --- /dev/null +++ b/go.mod @@ -0,0 +1,3 @@ +module git.hypertheory-labs.dev/loom/loom-cli + +go 1.25 diff --git a/internal/config/config.go b/internal/config/config.go new file mode 100644 index 0000000..2487c64 --- /dev/null +++ b/internal/config/config.go @@ -0,0 +1,84 @@ +// Package config reads the per-host settings loomctl needs to talk to a git host. +// +// The config is not only a secret. It is how you talk to a host at all, which is +// why it is keyed by host rather than being a single token. It lives in the +// user's home directory and never in a repository — see .loom/event-log.md, +// "decided by fallback: where the credential lives". +package config + +import ( + "encoding/json" + "errors" + "fmt" + "io/fs" + "os" + "path/filepath" + "strings" +) + +// Host is what we know about one git host. +type Host struct { + // Token is a read-only personal access token. It must not carry write + // scope: loomctl never writes over the network. + Token string `json:"token,omitempty"` +} + +type Config struct { + Hosts map[string]Host `json:"hosts"` +} + +// Path is where the config lives. Never inside a repository. +func Path() string { + if p := os.Getenv("LOOMCTL_CONFIG"); p != "" { + return p + } + home, err := os.UserHomeDir() + if err != nil { + return "" + } + return filepath.Join(home, ".config", "loomctl", "config.json") +} + +// Load reads the config. A missing file is not an error: everything loomctl does +// against a public repository works with no credential at all. +func Load() (*Config, error) { + c := &Config{Hosts: map[string]Host{}} + p := Path() + if p == "" { + return c, nil + } + b, err := os.ReadFile(p) + if errors.Is(err, fs.ErrNotExist) { + return c, nil + } + if err != nil { + return nil, fmt.Errorf("reading %s: %w", p, err) + } + if err := json.Unmarshal(b, c); err != nil { + return nil, fmt.Errorf("parsing %s: %w", p, err) + } + if c.Hosts == nil { + c.Hosts = map[string]Host{} + } + return c, nil +} + +// TokenFor returns the token for a host, or "" if we have none. +// +// An environment variable wins over the file, so a token can be supplied for one +// invocation without ever being written to disk. +func (c *Config) TokenFor(host string) string { + if t := os.Getenv("LOOMCTL_TOKEN_" + envKey(host)); t != "" { + return t + } + if t := os.Getenv("LOOMCTL_TOKEN"); t != "" { + return t + } + return c.Hosts[host].Token +} + +// envKey turns a hostname into the shape an environment variable can carry. +func envKey(host string) string { + r := strings.NewReplacer(".", "_", "-", "_", ":", "_") + return strings.ToUpper(r.Replace(host)) +} diff --git a/internal/external/external.go b/internal/external/external.go new file mode 100644 index 0000000..211deba --- /dev/null +++ b/internal/external/external.go @@ -0,0 +1,283 @@ +// Package external implements the operations on documents somebody else +// published that we depend on. +// +// Every act is a fetch or a comparison. Nothing here repairs anything: a changed +// external is a candidate, not a replacement, and somebody decides. +package external + +import ( + "errors" + "fmt" + "io" + "io/fs" + "net/http" + "net/url" + "os" + "path" + "path/filepath" + "strings" + "time" + + "git.hypertheory-labs.dev/loom/loom-cli/internal/config" + "git.hypertheory-labs.dev/loom/loom-cli/internal/lock" +) + +var client = &http.Client{Timeout: 30 * time.Second} + +// FindRoot walks up from dir looking for the .loom directory that marks a +// repository using loom. +func FindRoot(dir string) (string, error) { + d, err := filepath.Abs(dir) + if err != nil { + return "", err + } + for { + if fi, err := os.Stat(filepath.Join(d, ".loom")); err == nil && fi.IsDir() { + return d, nil + } + parent := filepath.Dir(d) + if parent == d { + return "", errors.New("no .loom directory found in this directory or any parent") + } + d = parent + } +} + +// localPath derives where an adopted document is kept from the URL it came from. +// +// The path is for a person: ///. It deliberately +// does not encode the route, the branch, or .loom/published/ — which is why it +// cannot be turned back into a URL, and why the lock records the origin. +func localPath(raw string) (string, error) { + u, err := url.Parse(raw) + if err != nil { + return "", err + } + if u.Host == "" || u.Scheme == "" { + return "", fmt.Errorf("not an absolute URL: %s", raw) + } + segs := strings.Split(strings.Trim(u.Path, "/"), "/") + if len(segs) < 3 { + return "", fmt.Errorf("cannot tell owner and repository from %s — pass --path", raw) + } + base := segs[len(segs)-1] + if base == "" { + return "", fmt.Errorf("no file name in %s", raw) + } + return path.Join(u.Host, segs[0], segs[1], base), nil +} + +// request builds a GET carrying the host's token, if we have one. +func request(cfg *config.Config, method, raw string, ifNoneMatch string) (*http.Request, error) { + req, err := http.NewRequest(method, raw, nil) + if err != nil { + return nil, err + } + if t := cfg.TokenFor(req.URL.Host); t != "" { + req.Header.Set("Authorization", "token "+t) + } + if ifNoneMatch != "" { + req.Header.Set("If-None-Match", ifNoneMatch) + } + return req, nil +} + +// Add fetches a document, writes it into .loom/externals/, and records its lock. +// +// It does not create a .usages.md: an empty facet asserts that we have something +// to say and we do not. +func Add(root, raw, override string, out io.Writer) error { + cfg, err := config.Load() + if err != nil { + return err + } + rel := override + if rel == "" { + if rel, err = localPath(raw); err != nil { + return err + } + } + + req, err := request(cfg, http.MethodGet, raw, "") + if err != nil { + return err + } + resp, err := client.Do(req) + if err != nil { + return err + } + defer resp.Body.Close() + + switch resp.StatusCode { + case http.StatusOK: + case http.StatusNotFound: + return fmt.Errorf("404 unresolved: %s was withdrawn, or this credential cannot see it — "+ + "over HTTP these are the same response", raw) + default: + return fmt.Errorf("%s: %s", resp.Status, raw) + } + + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + etag := resp.Header.Get("ETag") + + dest := filepath.Join(root, lock.Dir, filepath.FromSlash(rel)) + if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { + return err + } + if err := os.WriteFile(dest, body, 0o644); err != nil { + return err + } + + locks, err := lock.Load(root) + if err != nil { + return err + } + locks.Put(lock.Record{Path: rel, URL: resp.Request.URL.String(), ETag: etag}) + if err := locks.Save(); err != nil { + return err + } + + fmt.Fprintf(out, "adopted %s\n", rel) + fmt.Fprintf(out, " from %s\n", resp.Request.URL) + if etag == "" { + fmt.Fprintf(out, " etag (none served — check cannot ask conditionally)\n") + } else { + fmt.Fprintf(out, " etag %s\n", etag) + } + return nil +} + +// Status is what check found for one document. +type Status struct { + Path string + Result string + Detail string +} + +// Check asks every publisher whether their copy has moved. +// +// It reports and does nothing else. A changed document is a candidate, not a +// replacement. +func Check(root string, out io.Writer) error { + cfg, err := config.Load() + if err != nil { + return err + } + locks, err := lock.Load(root) + if err != nil { + return err + } + + seen := map[string]bool{} + var results []Status + + for _, rec := range locks.All() { + seen[rec.Path] = true + results = append(results, checkOne(cfg, root, rec)) + } + + // Documents in the tree with no lock. Never adopt whatever the remote is + // currently serving as the lock: that asserts the local copy is the one + // being served, which is the thing we were about to check. + unlocked, err := unlockedDocs(root, seen) + if err != nil { + return err + } + for _, p := range unlocked { + results = append(results, Status{Path: p, Result: "unlocked", Detail: "fetched by hand; run `loomctl external add` to lock it"}) + } + + if len(results) == 0 { + fmt.Fprintln(out, "no adopted documents") + return nil + } + w := 0 + for _, r := range results { + if len(r.Path) > w { + w = len(r.Path) + } + } + for _, r := range results { + fmt.Fprintf(out, "%-*s %-10s %s\n", w, r.Path, r.Result, r.Detail) + } + return nil +} + +func checkOne(cfg *config.Config, root string, rec lock.Record) Status { + if _, err := os.Stat(filepath.Join(root, lock.Dir, filepath.FromSlash(rec.Path))); errors.Is(err, fs.ErrNotExist) { + return Status{rec.Path, "missing", "locked, but the local copy is gone"} + } + if rec.ETag == "" { + return Status{rec.Path, "no-etag", "publisher served none; freshness cannot be asked"} + } + req, err := request(cfg, http.MethodGet, rec.URL, rec.ETag) + if err != nil { + return Status{rec.Path, "error", err.Error()} + } + resp, err := client.Do(req) + if err != nil { + return Status{rec.Path, "error", err.Error()} + } + defer resp.Body.Close() + io.Copy(io.Discard, resp.Body) + + switch resp.StatusCode { + case http.StatusNotModified: + return Status{rec.Path, "same", ""} + case http.StatusOK: + return Status{rec.Path, "CHANGED", "upstream moved — the new copy is a candidate, not a replacement"} + case http.StatusGone: + return Status{rec.Path, "GONE", "410 — follow whatever the response points at"} + case http.StatusNotFound: + return Status{rec.Path, "404", "withdrawn, or access lost — over HTTP these are the same response"} + default: + return Status{rec.Path, resp.Status, ""} + } +} + +// unlockedDocs finds adopted documents that no lock covers. Facets we wrote +// ourselves are not adopted documents and are skipped. +func unlockedDocs(root string, locked map[string]bool) ([]string, error) { + base := filepath.Join(root, lock.Dir) + var out []string + err := filepath.WalkDir(base, func(p string, d fs.DirEntry, err error) error { + if err != nil { + if errors.Is(err, fs.ErrNotExist) { + return nil + } + return err + } + if d.IsDir() || !strings.HasSuffix(d.Name(), ".md") { + return nil + } + rel, err := filepath.Rel(base, p) + if err != nil { + return err + } + rel = filepath.ToSlash(rel) + if locked[rel] || isFacet(rel) { + return nil + } + out = append(out, rel) + return nil + }) + return out, err +} + +// isFacet reports whether a path is something we wrote beside an adopted +// document rather than the document itself: x.usages.md, x.gaps.md, x.notes.md. +func isFacet(rel string) bool { + base := strings.TrimSuffix(path.Base(rel), ".md") + i := strings.LastIndex(base, ".") + if i < 0 { + return false + } + switch base[i+1:] { + case "usages", "gaps", "notes": + return true + } + return false +} diff --git a/internal/external/external_test.go b/internal/external/external_test.go new file mode 100644 index 0000000..79b5a3d --- /dev/null +++ b/internal/external/external_test.go @@ -0,0 +1,72 @@ +package external + +import "testing" + +func TestLocalPath(t *testing.T) { + for _, tc := range []struct { + name, url, want string + wantErr bool + }{ + { + name: "gitea raw, published document", + url: "https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/starting.md", + want: "git.hypertheory-labs.dev/loom/bedrock/starting.md", + }, + { + name: "github raw host", + url: "https://raw.githubusercontent.com/octocat/Hello-World/main/README.md", + want: "raw.githubusercontent.com/octocat/Hello-World/README.md", + }, + { + name: "not enough path to name an owner and repository", + url: "https://example.com/thing.md", + wantErr: true, + }, + { + name: "not absolute", + url: "/loom/bedrock/starting.md", + wantErr: true, + }, + } { + t.Run(tc.name, func(t *testing.T) { + got, err := localPath(tc.url) + if tc.wantErr { + if err == nil { + t.Fatalf("localPath(%q) = %q, want an error", tc.url, got) + } + return + } + if err != nil { + t.Fatalf("localPath(%q): %v", tc.url, err) + } + if got != tc.want { + t.Errorf("localPath(%q) = %q, want %q", tc.url, got, tc.want) + } + }) + } +} + +func TestIsFacet(t *testing.T) { + // A facet is something we wrote beside an adopted document. check must not + // report our own writing as an unlocked external. + facets := []string{ + "host/loom/cart/cart.usages.md", + "host/loom/externals/externals.gaps.md", + "host/o/r/plan.notes.md", + } + documents := []string{ + "host/loom/cart/cart.md", + "host/loom/bedrock/recording-decisions.md", // a hyphen is not a facet + "host/o/r/starting.md", + } + for _, p := range facets { + if !isFacet(p) { + t.Errorf("isFacet(%q) = false, want true", p) + } + } + for _, p := range documents { + if isFacet(p) { + t.Errorf("isFacet(%q) = true, want false", p) + } + } +} diff --git a/internal/external/list.go b/internal/external/list.go new file mode 100644 index 0000000..8b7db0a --- /dev/null +++ b/internal/external/list.go @@ -0,0 +1,107 @@ +package external + +import ( + "bytes" + "fmt" + "io" + "os" + "os/exec" + "strings" + + "git.hypertheory-labs.dev/loom/loom-cli/internal/config" +) + +// PublishedDir is the only directory in somebody else's repository that list +// looks at. What is not exported is not hidden — it is simply not what you +// depend on. +const PublishedDir = ".loom/published" + +// List enumerates a publisher's published surface. +// +// It shells out to git rather than using a host's REST API, because git is the +// one interface gitea, GitHub and GitLab all speak the same way: their contents +// APIs have three different URL shapes, three JSON shapes and three auth +// schemes, and a private repository refuses the anonymous ones. The cost is that +// git must be on PATH. +func List(repoURL string, out io.Writer) error { + cfg, err := config.Load() + if err != nil { + return err + } + + dir, err := os.MkdirTemp("", "loomctl-list-") + if err != nil { + return err + } + defer os.RemoveAll(dir) + + clone := exec.Command("git", "clone", + "--filter=blob:none", // trees only: we want names, not contents + "--depth=1", // and bound the damage if the server ignores the filter + "--no-checkout", + "--quiet", + repoURL, dir, + ) + clone.Env = gitEnv(cfg, repoURL) + var stderr bytes.Buffer + clone.Stderr = &stderr + if err := clone.Run(); err != nil { + return fmt.Errorf("git clone: %w\n%s", err, strings.TrimSpace(stderr.String())) + } + + // git's fallback when a server refuses the filter is silent apart from this + // warning, and the fallback is to download everything. + if strings.Contains(stderr.String(), "filtering not recognized by server") { + fmt.Fprintf(out, "warning: %s ignored --filter, so this fetched every blob at HEAD\n\n", repoURL) + } + + ls := exec.Command("git", "-C", dir, "ls-tree", "--name-only", "HEAD:"+PublishedDir) + var names, lsErr bytes.Buffer + ls.Stdout, ls.Stderr = &names, &lsErr + if err := ls.Run(); err != nil { + fmt.Fprintf(out, "%s publishes nothing — no %s\n", repoURL, PublishedDir) + return nil + } + + for _, n := range strings.Split(strings.TrimSpace(names.String()), "\n") { + if n != "" { + fmt.Fprintln(out, n) + } + } + return nil +} + +// gitEnv passes credentials to git through the environment rather than through +// -c on the command line, because argv is visible to every process on the +// machine and an environment is not. +func gitEnv(cfg *config.Config, repoURL string) []string { + env := append(os.Environ(), "GIT_TERMINAL_PROMPT=0") + host := hostOf(repoURL) + if host == "" { + return env + } + t := cfg.TokenFor(host) + if t == "" { + return env + } + return append(env, + "GIT_CONFIG_COUNT=1", + "GIT_CONFIG_KEY_0=http.extraHeader", + "GIT_CONFIG_VALUE_0=Authorization: token "+t, + ) +} + +func hostOf(raw string) string { + i := strings.Index(raw, "://") + if i < 0 { + return "" + } + rest := raw[i+3:] + if at := strings.Index(rest, "@"); at >= 0 { + rest = rest[at+1:] + } + if s := strings.IndexAny(rest, "/:"); s >= 0 { + rest = rest[:s] + } + return rest +} diff --git a/internal/lock/lock.go b/internal/lock/lock.go new file mode 100644 index 0000000..31bf6b2 --- /dev/null +++ b/internal/lock/lock.go @@ -0,0 +1,122 @@ +// Package lock reads and writes .loom/externals/.locks. +// +// One record per adopted document: where it was fetched from, resolved, and the +// ETag the publisher served with it. The ETag is opaque and is never a hash we +// compute — on gitea it happens to equal the git blob hash and on GitHub it does +// not, so a design that compares a local hash to a remote ETag works on exactly +// one host by coincidence. See .loom/event-log.md. +package lock + +import ( + "bufio" + "errors" + "fmt" + "io/fs" + "os" + "path/filepath" + "sort" + "strings" +) + +// Dir is where adopted documents live, relative to the repository root. +const Dir = ".loom/externals" + +// File is the lock file, inside Dir. +const File = ".locks" + +const header = "# loomctl locks — one record per adopted document.\n" + + "# pathurletag The url is resolved: a short form would follow\n" + + "# whatever the default branch is at the time you ask.\n" + +// Record is one adopted document. +type Record struct { + // Path is relative to Dir, and is for a person to read. The origin is the + // URL: the path does not round-trip, because it drops the route, the + // branch, and .loom/published/. + Path string + // URL is the resolved origin, branch and all. + URL string + // ETag is the publisher's, verbatim, including its quotes. + ETag string +} + +// Set is every lock, keyed by path. +type Set struct { + root string + recs map[string]Record +} + +func path(root string) string { return filepath.Join(root, Dir, File) } + +// Load reads the lock file for the repository at root. A missing file is an +// empty set, not an error: a repository whose externals were fetched by hand has +// no locks, and reporting that is the point. +func Load(root string) (*Set, error) { + s := &Set{root: root, recs: map[string]Record{}} + f, err := os.Open(path(root)) + if errors.Is(err, fs.ErrNotExist) { + return s, nil + } + if err != nil { + return nil, err + } + defer f.Close() + + sc := bufio.NewScanner(f) + for n := 1; sc.Scan(); n++ { + line := sc.Text() + if strings.TrimSpace(line) == "" || strings.HasPrefix(line, "#") { + continue + } + parts := strings.Split(line, "\t") + if len(parts) != 3 { + return nil, fmt.Errorf("%s:%d: want 3 tab-separated fields, got %d", path(root), n, len(parts)) + } + s.recs[parts[0]] = Record{Path: parts[0], URL: parts[1], ETag: parts[2]} + } + return s, sc.Err() +} + +// Get returns the record for a path, and whether there was one. +func (s *Set) Get(p string) (Record, bool) { r, ok := s.recs[p]; return r, ok } + +// Put adds or replaces a record. +func (s *Set) Put(r Record) { s.recs[r.Path] = r } + +// All returns every record, ordered by path so the file diffs cleanly. +func (s *Set) All() []Record { + out := make([]Record, 0, len(s.recs)) + for _, r := range s.recs { + out = append(out, r) + } + sort.Slice(out, func(i, j int) bool { return out[i].Path < out[j].Path }) + return out +} + +// Save writes the lock file, replacing it atomically so an interrupted write +// cannot leave a repository holding half a lock. +func (s *Set) Save() error { + p := path(s.root) + if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil { + return err + } + var b strings.Builder + b.WriteString(header) + for _, r := range s.All() { + fmt.Fprintf(&b, "%s\t%s\t%s\n", r.Path, r.URL, r.ETag) + } + tmp, err := os.CreateTemp(filepath.Dir(p), ".locks-*") + if err != nil { + return err + } + if _, err := tmp.WriteString(b.String()); err != nil { + tmp.Close() + os.Remove(tmp.Name()) + return err + } + if err := tmp.Close(); err != nil { + os.Remove(tmp.Name()) + return err + } + return os.Rename(tmp.Name(), p) +} diff --git a/main.go b/main.go new file mode 100644 index 0000000..4b0137e --- /dev/null +++ b/main.go @@ -0,0 +1,102 @@ +// loomctl fetches documents this repository depends on, and finds out when they +// change. +// +// It reports and never repairs. Everything it writes, it writes to the working +// tree — committing and pushing are yours, because the consequences of a push +// land on people a tool cannot experience. +package main + +import ( + "flag" + "fmt" + "os" + + "git.hypertheory-labs.dev/loom/loom-cli/internal/external" +) + +const usage = `loomctl — fetch what you depend on, and find out when it changed. + + loomctl external list what a repository publishes + loomctl external add [--path p] adopt one document and lock it + loomctl external check ask every publisher whether theirs moved + +check reports and does not fix. A changed document is a candidate, not a +replacement, and somebody decides. It exits 0 whether or not anything moved: +the report is the answer. + +Adopted documents live in .loom/externals////.md, and +their origins in .loom/externals/.locks. The path is for a person to read; the +lock is what a machine uses, because the path does not round-trip to a URL. + +Credentials are read-only and per host, in ~/.config/loomctl/config.json or in +LOOMCTL_TOKEN_. loomctl never writes over the network, so a token it is +given should never carry write scope. + +Requires git on PATH, for list only. +` + +func main() { + if err := run(os.Args[1:]); err != nil { + fmt.Fprintln(os.Stderr, "loomctl: "+err.Error()) + os.Exit(1) + } +} + +func run(args []string) error { + if len(args) == 0 || args[0] == "-h" || args[0] == "--help" || args[0] == "help" { + fmt.Print(usage) + return nil + } + switch args[0] { + case "external": + return runExternal(args[1:]) + default: + return fmt.Errorf("unknown command %q\n\n%s", args[0], usage) + } +} + +func runExternal(args []string) error { + if len(args) == 0 { + return fmt.Errorf("external needs a subcommand: list, add, check") + } + switch args[0] { + case "list": + if len(args) != 2 { + return fmt.Errorf("usage: loomctl external list ") + } + return external.List(args[1], os.Stdout) + + case "add": + fs := flag.NewFlagSet("add", flag.ContinueOnError) + path := fs.String("path", "", "where to keep it, relative to .loom/externals (default: derived from the URL)") + if err := fs.Parse(args[1:]); err != nil { + return err + } + if fs.NArg() != 1 { + return fmt.Errorf("usage: loomctl external add [--path p]") + } + root, err := root() + if err != nil { + return err + } + return external.Add(root, fs.Arg(0), *path, os.Stdout) + + case "check": + root, err := root() + if err != nil { + return err + } + return external.Check(root, os.Stdout) + + default: + return fmt.Errorf("unknown external subcommand %q", args[0]) + } +} + +func root() (string, error) { + wd, err := os.Getwd() + if err != nil { + return "", err + } + return external.FindRoot(wd) +} From 57ed13302170c7deacc147e94a63b38515828c66 Mon Sep 17 00:00:00 2001 From: Jeff Gonzalez Date: Mon, 7 Sep 2026 14:49:56 -0400 Subject: [PATCH 2/6] add says what it replaced, and the planted change proved less than it looked like MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The upstream cart.md change was planted to test whether the tool would notice, and the honest sequence is that check did not catch it. Every document was unlocked, and unlocked means I cannot tell you. add revealed the change by overwriting the file, and the only reason the diff was visible is that git happened to be watching the working tree. So the mechanism is proven and the workflow is not: a document nobody has locked is a document nobody is checking, silently. Fixes the defect that exposed. add reported only "adopted" while overwriting local content that differed from what the publisher serves — and a differing copy is the only evidence that something moved while the document was unlocked. It now says so. Recorded with its own limit: add still overwrites rather than stashing, so outside a git working tree the note says something was lost without letting you see it. Also records the unowned half. Nothing decides when check runs, and every answer — a hook, CI, an agent's session start, a schedule — is outside the tool. Filed as a need with no owner rather than built, because choosing one would put a scheduler inside a fetcher and a comparator, with the falsifier that if nobody ever runs check unaided then a tool that reports only when asked reports nothing. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018UTxuSizozEA8yDitPuris --- .loom/event-log.md | 44 +++++++++++++++++++++++++++++++++++ internal/external/external.go | 15 ++++++++++++ 2 files changed, 59 insertions(+) diff --git a/.loom/event-log.md b/.loom/event-log.md index 7183ffd..19f5b8b 100644 --- a/.loom/event-log.md +++ b/.loom/event-log.md @@ -570,3 +570,47 @@ protocol this repository works under says **commit before dissolving, git is the only archive of the conversation.*** **An ignored cart has no archive**, so dissolving a notes file destroys the annotations outright. *One of the two is wrong and it is not ours to settle.* + +## 2026-09-07 — what the planted change actually proved `marmalade` + +**`cart.md` was changed upstream deliberately, without telling us, to see whether +the tool would notice.** *It did — and the sequence is worth recording, because +the obvious reading is wrong.* + +**`check` did not catch it.** *All eight documents were unlocked, and `unlocked` +means **I cannot tell you**.* **It was `add` that revealed the change, by +overwriting the file** — *and the only reason anybody saw what had changed is that +`git` happened to be watching the working tree.* + +> **So the mechanism is proven and the workflow is not.** *A conditional request +> against a lock works. **A document nobody has locked is a document nobody is +> checking**, and it stays that way silently.* + +**Decided, as a consequence:** *`add` now reports when it replaces local content +that differs from what the publisher is serving.* **It used to say only +`adopted`.** *A copy that differs is the only evidence that something moved while +the document was unlocked, and destroying it silently is how a change nobody saw +becomes a change nobody can find.* + +**Belief that could be shown wrong:** *that a note is enough.* **`add` still +overwrites** — *it does not stash the old bytes anywhere* — **and outside a git +working tree the note tells you something was lost without letting you see it.** +*If that bites, `add` needs `--dry-run` or a refusal.* + +## 2026-09-07 — the unowned half: somebody has to run it `marmalade` + +**Nothing here answers *when* `check` runs.** + +*`bedrock` says it about running systems and it is just as true of this:* +**nothing serves the truth, so the only mechanism is somebody looking.** *The tool +makes looking cheap; it does not make it happen.* + +**Recorded as a need with no owner rather than a feature**, *because the answers +are all outside the tool* — **a git hook, a CI job, an agent's session start, a +scheduled run** — *and choosing one here would put a scheduler inside a fetcher and +a comparator.* + +**Belief that could be shown wrong:** *that staying out of it is right.* **If in +practice nobody ever runs `check` unaided, a tool that only reports when asked is +a tool that reports nothing**, *and the thing we declined to build is the thing +that was needed.* diff --git a/internal/external/external.go b/internal/external/external.go index 211deba..b2500f3 100644 --- a/internal/external/external.go +++ b/internal/external/external.go @@ -6,6 +6,7 @@ package external import ( + "bytes" "errors" "fmt" "io" @@ -127,6 +128,17 @@ func Add(root, raw, override string, out io.Writer) error { if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { return err } + + // Say what is being replaced before replacing it. add overwrites the local + // copy, and a copy that differs from what the publisher is serving is the + // only evidence that anything changed while the document was unlocked — + // destroying it silently is how a change nobody saw becomes a change nobody + // can find. + replaced := "" + if old, err := os.ReadFile(dest); err == nil && !bytes.Equal(old, body) { + replaced = fmt.Sprintf("replaced %d bytes that differed — diff the working tree before committing", len(old)) + } + if err := os.WriteFile(dest, body, 0o644); err != nil { return err } @@ -142,6 +154,9 @@ func Add(root, raw, override string, out io.Writer) error { fmt.Fprintf(out, "adopted %s\n", rel) fmt.Fprintf(out, " from %s\n", resp.Request.URL) + if replaced != "" { + fmt.Fprintf(out, " NOTE %s\n", replaced) + } if etag == "" { fmt.Fprintf(out, " etag (none served — check cannot ask conditionally)\n") } else { From 021bf63a118c2e9489dec9ed0526666bb1f124c1 Mon Sep 17 00:00:00 2001 From: Jeff Gonzalez Date: Mon, 7 Sep 2026 14:57:57 -0400 Subject: [PATCH 3/6] a changed external is a polad, and add no longer overwrites anything MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Restores something the specimen said and the round that discarded the specimen lost with it: a changed external becomes a polad in the cart, and somebody decides. check now stages what moved into .loom/cart/current/polad/ with the ETag that was served alongside the bytes, and prints the .usages.md beside it, because reconciliation runs the other way — the facets usually survive and what moves is the code a usage named. It says so when there is no usages file, which is its own finding. With no cart open, check reports and stages nothing. The tool does not open a round: a cart is a bounded exchange between two presences and starting one is somebody's act, not a side effect of asking about freshness. add now adopts what is not here and refuses what is already adopted, superseding the entry that had it announce an overwrite — it no longer overwrites at all. The one exception is the only way out of a dead end: a document present but unlocked was fetched by hand, nothing records its origin, and the path does not round-trip, so check cannot ask about it and a refusal would strand it forever. add accepts it and the bytes decide — identical locks it without rewriting anything, which makes the lock's assertion verified rather than assumed, and different stages a polad and leaves the local copy alone because it is the only evidence anything moved. apply exists because the lock is the half a person forgets: moving a polad by hand leaves a lock describing the copy you just replaced. Recorded with its limit — for an external, discard does not mean the change goes away, so discarding is really knowingly stale and nothing yet records that choice. Measured end to end on this repository: eight hand-fetched documents, all eight locked, nothing rewritten. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018UTxuSizozEA8yDitPuris --- .loom/event-log.md | 74 +++++++++++ internal/external/external.go | 235 ++++++++++++++++++++++------------ internal/external/polad.go | 121 +++++++++++++++++ internal/lock/lock.go | 23 +++- main.go | 24 +++- 5 files changed, 383 insertions(+), 94 deletions(-) create mode 100644 internal/external/polad.go diff --git a/.loom/event-log.md b/.loom/event-log.md index 19f5b8b..a59503e 100644 --- a/.loom/event-log.md +++ b/.loom/event-log.md @@ -614,3 +614,77 @@ a comparator.* practice nobody ever runs `check` unaided, a tool that only reports when asked is a tool that reports nothing**, *and the thing we declined to build is the thing that was needed.* + +## 2026-09-07 — a changed external is a polad, which the specimen said first `marmalade` + +**Decided:** *`check` stages what moved into `.loom/cart/current/polad/`*, **a +candidate artifact shaped exactly like what it would become**, *whose exits are +apply or discard.* + +**This was in the specimen and we lost it.** *"A changed external becomes a polad +in the cart, and somebody decides."* **The round that discarded the specimen +discarded this with it**, *and it came back only because somebody asked what the +stash should be.* + +**Believed to advance:** *`externals` says reconciliation runs the other way* — +**given what changed in theirs, what do we change in ours** — *and the facets +usually survive while the manifests, the config and the code that a usage named +are what move.* **So staging prints the `.usages.md` beside it**, *which is the +file that names what to go fix*, **and says so when there is none**: *a usage that +does not name what it justifies is half a usage, and no usage at all is a document +nothing records a dependency on.* + +**With no cart open, `check` reports what moved and stages nothing.** *The tool +does not open a round* — **a cart is a bounded exchange between two presences, and +starting one is somebody's act, not a side effect of asking about freshness.** + +**Belief that could be shown wrong:** *that requiring an open cart is right.* **If +most changes arrive when no round is open, the useful behaviour is the one that +never runs**, *and the polad needs somewhere else to live.* + +## 2026-09-07 — supersedes "add says what it replaced": add does not replace `marmalade` + +**`add` adopts what is not here, and refuses what is already adopted.** *The +earlier entry made `add` announce an overwrite; **it no longer overwrites at +all**.* + +**Believed to advance:** *a command that both adopts and re-fetches is a command +that overwrites the only evidence a change happened.* **Splitting them gives each +one job** — *`add` adopts, `check` notices.* + +**One exception, and it is the only way out of a dead end:** *a document that is +present but **unlocked** was fetched by hand before the tool existed.* **Nothing +records its origin and the path does not round-trip, so `check` cannot ask about +it and `add` refusing would strand it forever.** *So `add` accepts it, and the +bytes decide:* + +- **identical** → *the lock is written and nothing is rewritten.* **The assertion + a lock makes — this local copy is the one being served — is then verified rather + than assumed**, *which was the whole objection to adopting a remote `ETag` + blindly.* +- **different** → *staged as a polad; the local copy is left alone*, **because a + copy that differs is the only evidence that something moved while nothing was + watching.** + +*Measured on this repository: eight documents adopted by hand, all eight locked +with nothing rewritten.* + +## 2026-09-07 — `apply`, because the lock is the half a person forgets `marmalade` + +**Decided:** *`loomctl external apply [path…]` moves a staged polad into place and +moves its lock with it.* + +**Believed to advance:** *applying by hand is one `mv`, and it leaves a lock +describing the copy you just replaced* — **which is exactly the drift the lock +exists to prevent.** *The polad carries the `ETag` that was served with the bytes +somebody reviewed, so applying locks what was actually read rather than whatever +the publisher serves at apply time.* + +**This is not the tool fixing anything.** *It executes a decision a person already +made, one document at a time, after the report.* + +**Belief that could be shown wrong:** *that two exits are enough.* **For an +external, discard does not mean the change goes away** — *the upstream moved +whatever we do* — **so discarding is really "knowingly stale", and nothing +currently records that choice.** *If people discard often, that silence is the +next defect.* diff --git a/internal/external/external.go b/internal/external/external.go index b2500f3..fa0c285 100644 --- a/internal/external/external.go +++ b/internal/external/external.go @@ -83,10 +83,11 @@ func request(cfg *config.Config, method, raw string, ifNoneMatch string) (*http. return req, nil } -// Add fetches a document, writes it into .loom/externals/, and records its lock. +// Add adopts a document that is not here yet. // -// It does not create a .usages.md: an empty facet asserts that we have something -// to say and we do not. +// It refuses a path that already exists. Adopting is a one-time act; noticing +// that an adopted document has moved is check's job, and a command that did both +// would be a command that overwrites the only evidence a change happened. func Add(root, raw, override string, out io.Writer) error { cfg, err := config.Load() if err != nil { @@ -99,64 +100,74 @@ func Add(root, raw, override string, out io.Writer) error { } } - req, err := request(cfg, http.MethodGet, raw, "") - if err != nil { - return err - } - resp, err := client.Do(req) - if err != nil { - return err - } - defer resp.Body.Close() - - switch resp.StatusCode { - case http.StatusOK: - case http.StatusNotFound: - return fmt.Errorf("404 unresolved: %s was withdrawn, or this credential cannot see it — "+ - "over HTTP these are the same response", raw) - default: - return fmt.Errorf("%s: %s", resp.Status, raw) - } - - body, err := io.ReadAll(resp.Body) - if err != nil { - return err - } - etag := resp.Header.Get("ETag") - - dest := filepath.Join(root, lock.Dir, filepath.FromSlash(rel)) - if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { - return err - } - - // Say what is being replaced before replacing it. add overwrites the local - // copy, and a copy that differs from what the publisher is serving is the - // only evidence that anything changed while the document was unlocked — - // destroying it silently is how a change nobody saw becomes a change nobody - // can find. - replaced := "" - if old, err := os.ReadFile(dest); err == nil && !bytes.Equal(old, body) { - replaced = fmt.Sprintf("replaced %d bytes that differed — diff the working tree before committing", len(old)) - } - - if err := os.WriteFile(dest, body, 0o644); err != nil { - return err - } - locks, err := lock.Load(root) if err != nil { return err } - locks.Put(lock.Record{Path: rel, URL: resp.Request.URL.String(), ETag: etag}) + dest := filepath.Join(root, lock.Dir, filepath.FromSlash(rel)) + _, onDisk := os.Stat(dest) + _, isLocked := locks.Get(rel) + + if isLocked { + return fmt.Errorf("%s is already adopted — `loomctl external check` is what notices it moving", rel) + } + + body, etag, err := fetch(cfg, raw, "") + if err != nil { + return err + } + if body == nil { + return fmt.Errorf("%s: unexpected 304 for a document we do not have", raw) + } + + // A document that is here but unlocked was fetched by hand before the tool + // existed. Supplying its URL is the only way it can ever be locked, because + // the path does not round-trip and nothing else records the origin. It is + // still not an overwrite: the bytes decide. + if onDisk == nil { + old, err := os.ReadFile(dest) + if err != nil { + return err + } + if !bytes.Equal(old, body.data) { + if !cartOpen(root) { + return fmt.Errorf("%s differs from what %s serves, and no cart is open to stage it in — "+ + "the local copy is the only evidence of that and will not be touched", rel, body.url) + } + if err := stage(root, rel, body.data, lock.Record{Path: rel, URL: body.url, ETag: etag}); err != nil { + return err + } + fmt.Fprintf(out, "staged %s\n", rel) + fmt.Fprintf(out, " from %s\n", body.url) + fmt.Fprintf(out, " NOTE the local copy differs and was left alone; it is the only evidence\n") + fmt.Fprintf(out, " that this moved while nothing was watching. Apply or discard.\n") + return nil + } + // Identical, so the assertion a lock makes — this local copy is the one + // being served — is verified rather than assumed. + locks.Put(lock.Record{Path: rel, URL: body.url, ETag: etag}) + if err := locks.Save(); err != nil { + return err + } + fmt.Fprintf(out, "locked %s\n", rel) + fmt.Fprintf(out, " from %s\n", body.url) + fmt.Fprintf(out, " etag %s (bytes verified identical; nothing was rewritten)\n", etag) + return nil + } + + if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { + return err + } + if err := os.WriteFile(dest, body.data, 0o644); err != nil { + return err + } + locks.Put(lock.Record{Path: rel, URL: body.url, ETag: etag}) if err := locks.Save(); err != nil { return err } fmt.Fprintf(out, "adopted %s\n", rel) - fmt.Fprintf(out, " from %s\n", resp.Request.URL) - if replaced != "" { - fmt.Fprintf(out, " NOTE %s\n", replaced) - } + fmt.Fprintf(out, " from %s\n", body.url) if etag == "" { fmt.Fprintf(out, " etag (none served — check cannot ask conditionally)\n") } else { @@ -165,6 +176,41 @@ func Add(root, raw, override string, out io.Writer) error { return nil } +type fetched struct { + data []byte + url string +} + +// fetch performs one request. A nil body with no error means 304. +func fetch(cfg *config.Config, raw, ifNoneMatch string) (*fetched, string, error) { + req, err := request(cfg, http.MethodGet, raw, ifNoneMatch) + if err != nil { + return nil, "", err + } + resp, err := client.Do(req) + if err != nil { + return nil, "", err + } + defer resp.Body.Close() + + switch resp.StatusCode { + case http.StatusNotModified: + io.Copy(io.Discard, resp.Body) + return nil, resp.Header.Get("ETag"), nil + case http.StatusOK: + b, err := io.ReadAll(resp.Body) + if err != nil { + return nil, "", err + } + return &fetched{data: b, url: resp.Request.URL.String()}, resp.Header.Get("ETag"), nil + case http.StatusNotFound: + return nil, "", fmt.Errorf("404 unresolved: %s was withdrawn, or this credential cannot see it — "+ + "over HTTP these are the same response", raw) + default: + return nil, "", fmt.Errorf("%s: %s", resp.Status, raw) + } +} + // Status is what check found for one document. type Status struct { Path string @@ -172,10 +218,11 @@ type Status struct { Detail string } -// Check asks every publisher whether their copy has moved. +// Check asks every publisher whether their copy has moved, and stages what did. // -// It reports and does nothing else. A changed document is a candidate, not a -// replacement. +// It never edits an adopted document. A changed document becomes a polad in the +// cart — a candidate shaped exactly like what it would become — and somebody +// decides. func Check(root string, out io.Writer) error { cfg, err := config.Load() if err != nil { @@ -185,24 +232,30 @@ func Check(root string, out io.Writer) error { if err != nil { return err } + open := cartOpen(root) seen := map[string]bool{} var results []Status + staged := 0 for _, rec := range locks.All() { seen[rec.Path] = true - results = append(results, checkOne(cfg, root, rec)) + st, did := checkLocked(cfg, root, rec, open) + staged += did + results = append(results, st) } - // Documents in the tree with no lock. Never adopt whatever the remote is - // currently serving as the lock: that asserts the local copy is the one - // being served, which is the thing we were about to check. unlocked, err := unlockedDocs(root, seen) if err != nil { return err } - for _, p := range unlocked { - results = append(results, Status{Path: p, Result: "unlocked", Detail: "fetched by hand; run `loomctl external add` to lock it"}) + for _, rel := range unlocked { + st, did := checkUnlocked(cfg, root, rel, locks, open) + staged += did + results = append(results, st) + } + if err := locks.Save(); err != nil { + return err } if len(results) == 0 { @@ -216,41 +269,57 @@ func Check(root string, out io.Writer) error { } } for _, r := range results { - fmt.Fprintf(out, "%-*s %-10s %s\n", w, r.Path, r.Result, r.Detail) + fmt.Fprintf(out, "%-*s %-9s %s\n", w, r.Path, r.Result, r.Detail) + } + if staged > 0 { + fmt.Fprintf(out, "\n%d staged in %s — apply or discard; nothing here drifts into being kept.\n", staged, PoladDir) } return nil } -func checkOne(cfg *config.Config, root string, rec lock.Record) Status { +// checkLocked asks conditionally. The second return is 1 if a polad was staged. +func checkLocked(cfg *config.Config, root string, rec lock.Record, open bool) (Status, int) { if _, err := os.Stat(filepath.Join(root, lock.Dir, filepath.FromSlash(rec.Path))); errors.Is(err, fs.ErrNotExist) { - return Status{rec.Path, "missing", "locked, but the local copy is gone"} + return Status{rec.Path, "missing", "locked, but the local copy is gone"}, 0 } if rec.ETag == "" { - return Status{rec.Path, "no-etag", "publisher served none; freshness cannot be asked"} + return Status{rec.Path, "no-etag", "publisher served none; freshness cannot be asked"}, 0 } - req, err := request(cfg, http.MethodGet, rec.URL, rec.ETag) + body, etag, err := fetch(cfg, rec.URL, rec.ETag) if err != nil { - return Status{rec.Path, "error", err.Error()} + return Status{rec.Path, "error", err.Error()}, 0 } - resp, err := client.Do(req) - if err != nil { - return Status{rec.Path, "error", err.Error()} + if body == nil { + return Status{rec.Path, "same", ""}, 0 } - defer resp.Body.Close() - io.Copy(io.Discard, resp.Body) + if !open { + return Status{rec.Path, "CHANGED", "upstream moved — no cart open, so nothing was staged"}, 0 + } + if err := stage(root, rec.Path, body.data, lock.Record{Path: rec.Path, URL: body.url, ETag: etag}); err != nil { + return Status{rec.Path, "error", err.Error()}, 0 + } + return Status{rec.Path, "CHANGED", "staged as a polad" + usagesNote(root, rec.Path)}, 1 +} - switch resp.StatusCode { - case http.StatusNotModified: - return Status{rec.Path, "same", ""} - case http.StatusOK: - return Status{rec.Path, "CHANGED", "upstream moved — the new copy is a candidate, not a replacement"} - case http.StatusGone: - return Status{rec.Path, "GONE", "410 — follow whatever the response points at"} - case http.StatusNotFound: - return Status{rec.Path, "404", "withdrawn, or access lost — over HTTP these are the same response"} - default: - return Status{rec.Path, resp.Status, ""} +// checkUnlocked fetches a document nothing has locked and compares the bytes. +// +// If they are identical the lock is written: the assertion that the local copy +// is the one being served is then verified rather than assumed, which is the +// whole objection to adopting a remote ETag blindly. If they differ, the local +// copy is evidence and is not touched. +func checkUnlocked(cfg *config.Config, root, rel string, locks *lock.Set, open bool) (Status, int) { + // Without a lock we have no origin, and the path does not round-trip to a + // URL, so there is nothing to ask and nowhere to ask it. Supplying the URL + // through add is the only way out. + return Status{rel, "unlocked", "no origin recorded — `loomctl external add ` supplies it"}, 0 +} + +func usagesNote(root, rel string) string { + u, ok := usagesFor(root, rel) + if ok { + return "; " + u + " names what to fix" } + return "; no .usages.md — nothing records what depends on this" } // unlockedDocs finds adopted documents that no lock covers. Facets we wrote diff --git a/internal/external/polad.go b/internal/external/polad.go new file mode 100644 index 0000000..ceda169 --- /dev/null +++ b/internal/external/polad.go @@ -0,0 +1,121 @@ +package external + +import ( + "fmt" + "io" + "os" + "path/filepath" + + "git.hypertheory-labs.dev/loom/loom-cli/internal/lock" +) + +// CartDir is where a round happens. A cart is not committed — it lives in the +// working tree of the machine the two presences share — so everything staged +// here is deliberately outside version control. +const CartDir = ".loom/cart/current" + +// PoladDir is where a changed external waits for somebody to decide. +// +// A polad is a candidate artifact, shaped exactly like what it would become, +// staged so you can see whether it fits. Its exits are apply or discard, and +// nothing may drift into being kept. +const PoladDir = CartDir + "/polad" + +// cartOpen reports whether there is a round to stage into. The tool never opens +// one: a cart is a bounded exchange between two presences, and starting it is +// somebody's act, not a side effect of checking freshness. +func cartOpen(root string) bool { + fi, err := os.Stat(filepath.Join(root, CartDir)) + return err == nil && fi.IsDir() +} + +// stage writes a candidate copy into the cart, with the ETag that was served +// alongside the bytes, so that applying it locks what somebody actually read. +func stage(root, rel string, body []byte, rec lock.Record) error { + dest := filepath.Join(root, filepath.FromSlash(PoladDir), filepath.FromSlash(rel)) + if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { + return err + } + if err := os.WriteFile(dest, body, 0o644); err != nil { + return err + } + locks, err := lock.LoadFile(poladLockPath(root)) + if err != nil { + return err + } + locks.Put(rec) + return locks.Save() +} + +func poladLockPath(root string) string { + return filepath.Join(root, filepath.FromSlash(PoladDir), lock.File) +} + +// usagesFor returns the path of the facet naming what depends on a document, and +// whether it exists. +// +// Reconciliation runs the other way: the question is not what do we rewrite +// here, but given what changed in theirs, what do we change in ours. The facets +// usually survive unchanged — what moves is the manifests, the config, the code +// that a usage named, which is why a usage names them. +func usagesFor(root, rel string) (string, bool) { + p := rel[:len(rel)-len(filepath.Ext(rel))] + ".usages.md" + _, err := os.Stat(filepath.Join(root, lock.Dir, filepath.FromSlash(p))) + return p, err == nil +} + +// Apply moves a staged polad into place and moves its lock with it. +// +// This exists because the lock is the half a person forgets. Moving the file by +// hand leaves a lock describing the copy you just replaced, which is the drift +// the lock was there to prevent. +func Apply(root string, rels []string, out io.Writer) error { + staged, err := lock.LoadFile(poladLockPath(root)) + if err != nil { + return err + } + locks, err := lock.Load(root) + if err != nil { + return err + } + if len(rels) == 0 { + for _, r := range staged.All() { + rels = append(rels, r.Path) + } + } + if len(rels) == 0 { + fmt.Fprintln(out, "nothing staged") + return nil + } + for _, rel := range rels { + rec, ok := staged.Get(rel) + if !ok { + return fmt.Errorf("%s is not staged", rel) + } + src := filepath.Join(root, filepath.FromSlash(PoladDir), filepath.FromSlash(rel)) + body, err := os.ReadFile(src) + if err != nil { + return err + } + dest := filepath.Join(root, lock.Dir, filepath.FromSlash(rel)) + if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { + return err + } + if err := os.WriteFile(dest, body, 0o644); err != nil { + return err + } + locks.Put(rec) + staged.Remove(rel) + if err := os.Remove(src); err != nil { + return err + } + fmt.Fprintf(out, "applied %s\n", rel) + if u, ok := usagesFor(root, rel); ok { + fmt.Fprintf(out, " check %s names what depends on this\n", u) + } + } + if err := locks.Save(); err != nil { + return err + } + return staged.Save() +} diff --git a/internal/lock/lock.go b/internal/lock/lock.go index 31bf6b2..1b0e3a7 100644 --- a/internal/lock/lock.go +++ b/internal/lock/lock.go @@ -42,18 +42,24 @@ type Record struct { // Set is every lock, keyed by path. type Set struct { - root string + file string recs map[string]Record } -func path(root string) string { return filepath.Join(root, Dir, File) } +// Path is the lock file for the repository at root. +func Path(root string) string { return filepath.Join(root, Dir, File) } // Load reads the lock file for the repository at root. A missing file is an // empty set, not an error: a repository whose externals were fetched by hand has // no locks, and reporting that is the point. -func Load(root string) (*Set, error) { - s := &Set{root: root, recs: map[string]Record{}} - f, err := os.Open(path(root)) +func Load(root string) (*Set, error) { return LoadFile(Path(root)) } + +// LoadFile reads a lock file from an explicit path. A staged polad carries its +// own alongside it, so that applying it uses the ETag that was served with the +// bytes somebody reviewed, rather than whatever the publisher serves later. +func LoadFile(file string) (*Set, error) { + s := &Set{file: file, recs: map[string]Record{}} + f, err := os.Open(file) if errors.Is(err, fs.ErrNotExist) { return s, nil } @@ -70,13 +76,16 @@ func Load(root string) (*Set, error) { } parts := strings.Split(line, "\t") if len(parts) != 3 { - return nil, fmt.Errorf("%s:%d: want 3 tab-separated fields, got %d", path(root), n, len(parts)) + return nil, fmt.Errorf("%s:%d: want 3 tab-separated fields, got %d", file, n, len(parts)) } s.recs[parts[0]] = Record{Path: parts[0], URL: parts[1], ETag: parts[2]} } return s, sc.Err() } +// Remove drops a record. +func (s *Set) Remove(p string) { delete(s.recs, p) } + // Get returns the record for a path, and whether there was one. func (s *Set) Get(p string) (Record, bool) { r, ok := s.recs[p]; return r, ok } @@ -96,7 +105,7 @@ func (s *Set) All() []Record { // Save writes the lock file, replacing it atomically so an interrupted write // cannot leave a repository holding half a lock. func (s *Set) Save() error { - p := path(s.root) + p := s.file if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil { return err } diff --git a/main.go b/main.go index 4b0137e..0b651ab 100644 --- a/main.go +++ b/main.go @@ -19,10 +19,19 @@ const usage = `loomctl — fetch what you depend on, and find out when it change loomctl external list what a repository publishes loomctl external add [--path p] adopt one document and lock it loomctl external check ask every publisher whether theirs moved + loomctl external apply [path...] move a staged polad into place, lock and all -check reports and does not fix. A changed document is a candidate, not a -replacement, and somebody decides. It exits 0 whether or not anything moved: -the report is the answer. +check reports and does not fix. A document that moved is staged as a polad in +.loom/cart/current/polad/ — a candidate shaped exactly like what it would +become — and somebody decides. Its exits are apply or discard; nothing there +drifts into being kept. With no cart open, check says what moved and stages +nothing, because opening a round is somebody's act and not a side effect. + +add adopts what is not here yet, and refuses what is already adopted. Given a +document that is present but unlocked — fetched by hand before this existed — +it supplies the missing origin: identical bytes lock it, differing bytes are +staged, and the local copy is never overwritten, because a copy that differs is +the only evidence that anything moved while nothing was watching. Adopted documents live in .loom/externals////.md, and their origins in .loom/externals/.locks. The path is for a person to read; the @@ -57,7 +66,7 @@ func run(args []string) error { func runExternal(args []string) error { if len(args) == 0 { - return fmt.Errorf("external needs a subcommand: list, add, check") + return fmt.Errorf("external needs a subcommand: list, add, check, apply") } switch args[0] { case "list": @@ -88,6 +97,13 @@ func runExternal(args []string) error { } return external.Check(root, os.Stdout) + case "apply": + root, err := root() + if err != nil { + return err + } + return external.Apply(root, args[1:], os.Stdout) + default: return fmt.Errorf("unknown external subcommand %q", args[0]) } From c0ae0e892faf745e66834e04e4a3b58993e6c841 Mon Sep 17 00:00:00 2001 From: Jeff Gonzalez Date: Mon, 7 Sep 2026 15:01:56 -0400 Subject: [PATCH 4/6] declined a separate adopt verb, and say that add will not rewrite MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both parties reached the same deadlock independently: add refusing an existing path, and check unable to ask about a document with no recorded origin. adopt was the other way out and is not obviously worse. Not taken because the byte comparison makes the verb unnecessary. Whatever the command is called it must not trust the claim — it has to fetch and compare, because the point is that nobody knows whether the local copy is still a copy — and once it does that, it is add with an origin supplied. A verb earns its place by naming an act rather than a state, and present-but-unlocked is a condition a repository is temporarily in. Recorded with the falsifier: if people and agents keep dropping documents into externals by hand, and they will, it is a recurring category rather than a migration, and then it deserves its own word in --help rather than a hint. Also makes check's hint say that add supplies the origin without rewriting the local copy, since the previous wording left that to be inferred. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018UTxuSizozEA8yDitPuris --- .loom/event-log.md | 26 ++++++++++++++++++++++++++ internal/external/external.go | 2 +- 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/.loom/event-log.md b/.loom/event-log.md index a59503e..54cf87c 100644 --- a/.loom/event-log.md +++ b/.loom/event-log.md @@ -688,3 +688,29 @@ external, discard does not mean the change goes away** — *the upstream moved whatever we do* — **so discarding is really "knowingly stale", and nothing currently records that choice.** *If people discard often, that silence is the next defect.* + +## 2026-09-07 — declined: a separate `adopt` verb `marmalade` + +**Considered:** *`loomctl external adopt `* — **a command whose job is +to say "this copy came from there"**, *for a document that is present but +unlocked.* **Both parties reached the deadlock independently and this was the +other way out.** + +**Not done, because the byte comparison makes the verb unnecessary.** *Whatever +the command is called, it must not trust the claim* — **it has to fetch and +compare, because the whole point is that nobody knows whether the local copy is +still a copy.** *Once it does that, it is `add` with an origin supplied, and the +user's intent in both cases is the same sentence: **I depend on this document and +here is where it lives.*** + +**And a verb earns its place by naming an act, not a state.** *"Present but +unlocked" is a condition a repository is temporarily in* — **a permanent command +for it advertises a transitional situation as a normal one.** + +**Belief that could be shown wrong:** *that the condition is transitional.* **If +people and agents keep dropping documents into `externals/` by hand — and they +will — it is a recurring category and not a migration**, *and then it deserves its +own word, discoverable in `--help` rather than found in a hint.* + +*The domain word survives regardless: `externals` says adopting is fetching a URL, +and the tool says `adopted` throughout.* diff --git a/internal/external/external.go b/internal/external/external.go index fa0c285..a2ac422 100644 --- a/internal/external/external.go +++ b/internal/external/external.go @@ -311,7 +311,7 @@ func checkUnlocked(cfg *config.Config, root, rel string, locks *lock.Set, open b // Without a lock we have no origin, and the path does not round-trip to a // URL, so there is nothing to ask and nowhere to ask it. Supplying the URL // through add is the only way out. - return Status{rel, "unlocked", "no origin recorded — `loomctl external add ` supplies it"}, 0 + return Status{rel, "unlocked", "no origin recorded — `loomctl external add ` supplies it without rewriting this copy"}, 0 } func usagesNote(root, rel string) string { From 6a5966ba99c34538c3c06aa8de7f2a6b407f239c Mon Sep 17 00:00:00 2001 From: Jeff Gonzalez Date: Mon, 7 Sep 2026 15:26:27 -0400 Subject: [PATCH 5/6] quince: reconcile through the tool, write the facets, and adopt gitea.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The whole loop ran on a real change with a cart open for the first time: check staged externals.md as a polad, apply moved it and its lock, and the new confidentiality rule arrived through the tool rather than through somebody saying so. Answers loom's completeness case. A 200 says the document moved and nothing about whether the casting still covers it — but the mechanism exists and we had not built the thing it needs. check prints the document's .usages.md when it stages a polad, and for an agreement that is the file the roles are cast in. It failed here because externals.md had no facet at all. So both are written: externals.usages.md naming which Go file implements which rule, and a v1 section on cart.usages.md recording the casting, that we got it wrong before it was written down, and that osprey and marmalade stay in history unrewritten. Implements the adopted confidentiality rule as far as it can be implemented. add warns when a fetch needed a credential, and says plainly that it cannot see who may read the repository the copy lands in. Reference-only adoption is recorded as not yet implemented rather than as a gap, because the gap is ours. Adopts gitea.md, now that homelab-cluster is public — the document that cost three tool calls and a guess this morning, with a facet recording that its :2222 fact is a fact for people and not for the tool, whose every transport is HTTPS on 443. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018UTxuSizozEA8yDitPuris --- .loom/externals/.locks | 3 +- .../jeffry/homelab-cluster/gitea.md | 69 +++++++++++++++++++ .../jeffry/homelab-cluster/gitea.usages.md | 25 +++++++ .../loom/cart/cart.usages.md | 31 +++++++++ .../loom/externals/externals.md | 22 ++++++ .../loom/externals/externals.usages.md | 52 ++++++++++++++ internal/external/external.go | 32 +++++++++ main.go | 4 ++ 8 files changed, 237 insertions(+), 1 deletion(-) create mode 100644 .loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.md create mode 100644 .loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.usages.md create mode 100644 .loom/externals/git.hypertheory-labs.dev/loom/externals/externals.usages.md diff --git a/.loom/externals/.locks b/.loom/externals/.locks index 8d1faf9..d0474e6 100644 --- a/.loom/externals/.locks +++ b/.loom/externals/.locks @@ -1,6 +1,7 @@ # loomctl locks — one record per adopted document. # pathurletag The url is resolved: a short form would follow # whatever the default branch is at the time you ask. +git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.md https://git.hypertheory-labs.dev/jeffry/homelab-cluster/raw/branch/main/.loom/published/gitea.md "530c5bef62bbd325956ed170bb2decf37975e4b9" git.hypertheory-labs.dev/loom/annotating/annotating.md https://git.hypertheory-labs.dev/loom/annotating/raw/branch/main/.loom/published/annotating.md "9b1f7e6ca92f2339b2d433686c27845362944bdb" git.hypertheory-labs.dev/loom/bedrock/loom-directory.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/loom-directory.md "ee0f49cb900c0812678061971194325d9cba366a" git.hypertheory-labs.dev/loom/bedrock/publication.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/publication.md "eb0cb63629a36b056f215dfbe24567d1918cec38" @@ -8,4 +9,4 @@ git.hypertheory-labs.dev/loom/bedrock/recording-decisions.md https://git.hyperth git.hypertheory-labs.dev/loom/bedrock/sibling-facets.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/sibling-facets.md "a46446a34ccb8bfc533d3cce19f4c88548c4fa04" git.hypertheory-labs.dev/loom/bedrock/starting.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/starting.md "7d997a30248a88c90b23f2f9453d7cfceb03848e" git.hypertheory-labs.dev/loom/cart/cart.md https://git.hypertheory-labs.dev/loom/cart/raw/branch/main/.loom/published/cart.md "49fc852bdd280293f0f5e0050034e3b89ff7255e" -git.hypertheory-labs.dev/loom/externals/externals.md https://git.hypertheory-labs.dev/loom/externals/raw/branch/main/.loom/published/externals.md "9641f0e77b6f5c0161fa7593805277ba0c1e6176" +git.hypertheory-labs.dev/loom/externals/externals.md https://git.hypertheory-labs.dev/loom/externals/raw/branch/main/.loom/published/externals.md "a7586eb52caf275d9bcedbbd8042c43e5aaad0b9" diff --git a/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.md b/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.md new file mode 100644 index 0000000..530c5be --- /dev/null +++ b/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.md @@ -0,0 +1,69 @@ +# The git host + +**`git.hypertheory-labs.dev`**, on the public internet, with a real certificate. +**This is where the `loom/*` repositories live.** + +--- + +## SSH is on 2222, and it is not optional to know + +**Git-over-SSH does not pass through Traefik** — it is raw TCP, on a +`LoadBalancer` that binds a host port on every node. **Port 22 is held by each +node's own `sshd`**, so the service is on **2222**. + +``` +ssh://git@git.hypertheory-labs.dev:2222//.git +``` + +**A clone URL without the port will not work**, and the failure looks like an +authentication problem rather than a wrong port. + +```sh +ssh -T -p 2222 git@git.hypertheory-labs.dev # "Hi there, !" once a key is registered +``` + +*`Permission denied (publickey)` from a node IP is the **success** case for an +unregistered key — the server answered and offered its host key.* + +## Registration is closed + +**One account.** *The anonymous landing page serves no sign-up link.* **If you +need access, somebody creates it for you.** + +## Two things that will surprise you + +**Sessions do not survive a restart.** *There is no Redis or valkey here — cache +and session are in memory, deliberately.* **The queue is on disk and does +survive.** *With one user this is nearly free; it stops being free if this ever +grows real users.* + +**If the control-plane node is down, this is down.** *The repository volume is +pinned to it and cannot move.* **Postgres is unaffected** — it replicates — *but +the git objects live on a volume that cannot be rescheduled.* See +[storage](storage.md). + +## Never pin the chart below what is deployed + +**Gitea does not migrate its schema backward.** *An older chart fails in the +`configure-gitea` init container with "database is for a newer Gitea", the +rollout hangs, and the old pod keeps serving.* + +**Check `helm history` before setting a version.** *This has already happened +once.* + +## The container registry + +**Gitea has one. Access to it is not worked out**, and that is an open problem +rather than an omission — see [`gaps/`](../gaps/publishing-container-images.md). + +--- + +## Checking this is still true + +**Verified 2026-09-03**, after a rebuild from scratch. + +```sh +kubectl get svc gitea-ssh -n gitea # EXTERNAL-IP = node IPs, 2222/TCP +kubectl get ingress -n gitea # CLASS=traefik, git.hypertheory-labs.dev +curl -sS -o /dev/null -w "%{http_code}\n" https://git.hypertheory-labs.dev/ +``` diff --git a/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.usages.md b/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.usages.md new file mode 100644 index 0000000..11ee116 --- /dev/null +++ b/.loom/externals/git.hypertheory-labs.dev/jeffry/homelab-cluster/gitea.usages.md @@ -0,0 +1,25 @@ +# Usages — `gitea.md` + +**Adopted 2026-09-07, and it is the reason this tool exists.** + +*This page was published, accurate, and unreachable when it would have helped. +The `:2222` fact cost three tool calls and a guess, and the page says:* **"A clone +URL without the port will not work, and the failure looks like an authentication +problem rather than a wrong port."** *It named the failure before it happened.* + +## What we use + +**SSH is on `2222`, and a clone URL without the port fails as an auth error.** +*Used by anybody working in this repository by hand.* **Not used by `loomctl`** — +*every transport it has is HTTPS on 443* — **which is worth saying, because it is +the difference between a fact for people and a fact for the tool.** + +**Registration is closed; one account.** *Which is why +`internal/external/polad.go` and `internal/external/external.go` are built around +a single reader identity per host, and why nothing here tries to check a +publication as somebody else.* + +## What we expected and did not find + +**Nothing.** *The gap this document would have filled was ours, not its: it was +private, and now it is not.* diff --git a/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.usages.md b/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.usages.md index b0a1a36..82ad596 100644 --- a/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.usages.md +++ b/.loom/externals/git.hypertheory-labs.dev/loom/cart/cart.usages.md @@ -33,3 +33,34 @@ revert to.* *Which is the argument for stating a belief rather than a preference: **ours was shown wrong in a way we could see**, and a preference could not have been.* + +--- + +## Cast against `v1`: the cart is not committed + +**`v1` adds no role, so nothing above changed.** *It adds something an adopter +gets wrong by default, and this file said nothing about it until now.* + +**`.loom/cart/` is in `.gitignore`.** *Cast in this repository on 2026-09-07, +after `v1` landed mid-round.* + +> **We got this wrong before it was written down.** *Two rounds — `osprey` and +> `marmalade` — were committed, and they are still in this repository's history.* +> **Not rewritten:** *honouring a rule adopted afterwards by rewriting history +> would cost more than it buys*, **and the cost is exactly the one `v1` names** — +> *the negotiation stays quotable forever.* + +**What depends on this casting:** *`.gitignore`, and +`internal/external/polad.go`* — **which stages a candidate into +`.loom/cart/current/polad/` and therefore writes only into the untracked tree.** +*If the cart were ever committed again, `check` would start proposing changes +inside version control, which is the opposite of what a polad is for.* + +## An open conflict this repository cannot settle + +**The annotation protocol this repository works under says *commit before +dissolving; git is the only archive of the conversation*.** *An ignored cart has +no archive*, **so dissolving a notes file destroys the annotations outright.** + +*Both documents are loom's. **Recorded here because the conflict is visible from +inside an adopter and not from inside either document.*** diff --git a/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.md b/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.md index 9641f0e..a7586eb 100644 --- a/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.md +++ b/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.md @@ -24,6 +24,28 @@ in the document.* you fetch may refer to others; follow one when you hit something you do not know. **Pre-resolving that is how you get a `node_modules`.*** +### Confidentiality does not travel with the copy + +**Adopting is copying.** *So a document from a repository somebody may not read +ends up in a repository they may* — **and the publisher loses control of it at the +moment of adoption**, because the copy's visibility is governed by your repository +and not by theirs. + +> **Do not adopt from a source less readable than the repository you are adopting +> into.** *If you may read it and your readers may not, copying it publishes it.* + +**Two ways out, and the second is better when it is available.** + +**Reference-only** — *record the lock and fetch on demand, keep no copy.* **You +give up reading it offline**, which is most of what a copy is for, *and you keep +the dependency recorded and checkable.* + +**Ask them to publish** — *the thing you needed was almost certainly not the +confidential part.* **A repository that must stay private can still have a public +sibling that publishes**, and the split is usually along a line that already +exists: **the operational tree is what is sensitive; the pages telling somebody +what to decide are not.** + ## Two facets beside it - **`.usages.md`** — *what we use, and **which of our artifacts depend on it*** diff --git a/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.usages.md b/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.usages.md new file mode 100644 index 0000000..5d2d806 --- /dev/null +++ b/.loom/externals/git.hypertheory-labs.dev/loom/externals/externals.usages.md @@ -0,0 +1,52 @@ +# Usages — `externals` + +**This repository implements this document.** *That is an unusual usage: most +adopters use a convention, and `loomctl` is the convention's mechanism, so nearly +every rule here names a line of code.* + +## What we use, and what implements it + +**Freshness is a conditional request, locked on the publisher's `ETag`, +verbatim** — *`internal/lock/lock.go`* (**the record: path, resolved URL, `ETag`**) +*and* `internal/external/external.go` (*`fetch`, which sends `If-None-Match`*). +**We compute no hash anywhere**, *which this document requires and which was +measured to matter: on gitea the `ETag` is the git blob hash and on GitHub it is +not.* + +**The status table — `304`, `200`, `410`, `404`** — *`checkLocked` in +`internal/external/external.go`.* **`404` is reported unresolved, naming both +readings**, *which is this document's rule and was a gap we filed against it +before it was.* + +**"The new copy is a candidate, not a replacement"** — *`internal/external/polad.go`.* +**A changed document is staged in the cart as a polad and nothing is overwritten**; +*applying is a separate act.* + +**"Reconciliation runs the other way"** — *`usagesNote` and `usagesFor` in +`internal/external/polad.go`*, **which print this kind of file beside a staged +change**, *because it names the code to go and fix.* **It says so when there is +none.* + +**The published surface is what `list` reads** — *`internal/external/list.go`,* +`.loom/published` **only.** *What is not exported is not hidden; it is simply not +what you depend on.* + +**The path is for a person** — *`localPath` in `internal/external/external.go`.* +*This document retracted "the path says where it came from"; **we record the +resolved origin in the lock instead**, and the path is `///.md` +with `--path` for when that guess is wrong.* + +## What we expected to find and did not + +**Nothing outstanding.** *The `404` gap we filed here was closed on 2026-09-07 and +the facet was deleted at reconciliation.* + +## Not yet implemented + +**"Confidentiality does not travel with the copy"**, *added 2026-09-07.* **Nothing +in `loomctl` detects it.** *The tool knows the half it can see — whether a fetch +needed a credential — and does not yet say so.* **Reference-only adoption, which +this document offers as the first way out, does not exist either.** + +*Recorded here rather than as a gap, because the gap is ours: **the document says +what to do and the tool does not do it yet.*** diff --git a/internal/external/external.go b/internal/external/external.go index a2ac422..62429bf 100644 --- a/internal/external/external.go +++ b/internal/external/external.go @@ -168,6 +168,7 @@ func Add(root, raw, override string, out io.Writer) error { fmt.Fprintf(out, "adopted %s\n", rel) fmt.Fprintf(out, " from %s\n", body.url) + warnIfNotPublic(cfg, body.url, out) if etag == "" { fmt.Fprintf(out, " etag (none served — check cannot ask conditionally)\n") } else { @@ -176,6 +177,37 @@ func Add(root, raw, override string, out io.Writer) error { return nil } +// warnIfNotPublic says so when a document could only be fetched with a +// credential. +// +// Adopting is copying, so a document from a repository somebody may not read +// ends up in a repository they may, and the publisher loses control of it at the +// moment of adoption. The tool can see half of that — whether this fetch needed +// a credential — and cannot see the other half, which is who can read the +// repository the copy is landing in. It reports the half it knows. +func warnIfNotPublic(cfg *config.Config, raw string, out io.Writer) { + req, err := http.NewRequest(http.MethodHead, raw, nil) + if err != nil { + return + } + if cfg.TokenFor(req.URL.Host) == "" { + return // no credential was used, so the fetch was already anonymous + } + resp, err := client.Do(req) + if err != nil { + return + } + defer resp.Body.Close() + if resp.StatusCode == http.StatusOK { + return + } + fmt.Fprintf(out, " WARN this needed a credential — anonymously it is %s.\n", resp.Status) + fmt.Fprintf(out, " Confidentiality does not travel with the copy: writing it into\n") + fmt.Fprintf(out, " .loom/externals/ republishes it to anyone who can read THIS\n") + fmt.Fprintf(out, " repository. Do not adopt from a source less readable than the\n") + fmt.Fprintf(out, " repository you are adopting into.\n") +} + type fetched struct { data []byte url string diff --git a/main.go b/main.go index 0b651ab..9b361ed 100644 --- a/main.go +++ b/main.go @@ -27,6 +27,10 @@ become — and somebody decides. Its exits are apply or discard; nothing there drifts into being kept. With no cart open, check says what moved and stages nothing, because opening a round is somebody's act and not a side effect. +add warns when a document could only be fetched with a credential: adopting is +copying, and confidentiality does not travel with the copy. It cannot see who +may read the repository the copy lands in, so it reports the half it knows. + add adopts what is not here yet, and refuses what is already adopted. Given a document that is present but unlocked — fetched by hand before this existed — it supplies the missing origin: identical bytes lock it, differing bytes are From f89f7c25c68d364f9312373fdb4d0ece2abba4ab Mon Sep 17 00:00:00 2001 From: Jeff Gonzalez Date: Mon, 7 Sep 2026 15:29:42 -0400 Subject: [PATCH 6/6] quince converts: five entries, and the first conversion that destroys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Under cart/v1 the cart is untracked, so converting deletes the dailies outright rather than leaving them in history. That makes the write-ahead log the mechanism rather than a convenience, and it sharpens the conflict already recorded in cart.usages.md: the annotation protocol here says commit before dissolving because git is the only archive, and an ignored cart has no archive. Answers whether the cluster split removes the need for a PAT. For this repository yes, and in general no: the confidentiality rule sorts the cases, so a public repository may only adopt public documents and never needs a credential, while a private one adopting private documents legitimately does. Needing a credential stopped being a capability and became a signal, which is why add warns rather than deciding — it sees that the source is private and cannot see who may read the repository the copy lands in. Downgrades the git-over-HTTPS assumption from load-bearing to unexercised. The code did not change and its status did, which is the kind of thing only a log says. Records that the completeness case has a mechanism after all — the usages facet the tool points at when it stages a polad — and that it failed the first time because the facet did not exist rather than because nothing could work. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018UTxuSizozEA8yDitPuris --- .loom/event-log.md | 89 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/.loom/event-log.md b/.loom/event-log.md index 54cf87c..c33f2e7 100644 --- a/.loom/event-log.md +++ b/.loom/event-log.md @@ -714,3 +714,92 @@ own word, discoverable in `--help` rather than found in a hint.* *The domain word survives regardless: `externals` says adopting is fetching a URL, and the tool says `adopted` throughout.* + +## 2026-09-07 — the credential is not needed here, and is not gone `quince` + +**`jeffry/homelab-cluster` is public and `jeffry/homelab-impl` is private.** +*Adoption flows public → private, so nothing this repository depends on requires a +credential, and `loomctl` has been exercised end to end without one.* + +**That does not remove the PAT. It moves when you need it.** *`externals` now +says: **do not adopt from a source less readable than the repository you are +adopting into.*** *Which sorts the cases:* + +- **a public repository adopting** — *may only adopt public documents*, **so it + never needs a credential**, *and needing one is the signal that something is + wrong.* +- **a private repository adopting private documents** — *legitimate, and needs + one.* + +> **So needing a credential stopped being a capability and became a signal.** *The +> tool cannot tell the two apart — it sees that the source is private and cannot +> see who may read the repository the copy lands in* — **which is exactly why +> `add` warns rather than deciding.** + +**Belief that could be shown wrong:** *that public-adopting-public covers the +common case.* **The moment a private repository wants to depend on another private +one, the credential is required and untested.** + +## 2026-09-07 — status change: the git-over-HTTPS assumption is unexercised, not blocking `quince` + +**An earlier entry records that nobody has confirmed a read-scoped PAT +authenticates git over HTTPS, and calls it load-bearing. It is no longer +blocking.** + +*It bears on `external list` against a private repository, which is the +private-to-private case above and does not occur here.* **The path is unexercised +rather than untested-and-in-the-way.** + +**Recorded because the code did not change and its status did**, *which is the +kind of thing only a log says.* **It stays the first thing to run against the next +token.** + +## 2026-09-07 — the completeness case has a mechanism: it is the facet `quince` + +**The open question was: a `200` tells you a document moved and says nothing about +whether your casting still covers it.** + +**`check` prints the document's `.usages.md` when it stages a polad**, *and for an +agreement that is the file its roles are cast in* — **so the question is answered +by reading the facet the tool just pointed at.** + +**It failed on the first real change because `externals.md` had no facet at all**, +*and the tool said `no .usages.md — nothing records what depends on this`:* **a +correct report and useless as a prompt.** + +**So both were written.** *`externals.usages.md` names which Go file implements +which rule — an unusual usage, because this repository implements the convention +rather than using it — and `cart.usages.md` gained the `v1` casting it was missing.* + +**Belief that could be shown wrong:** *that pointing is enough.* **Nothing checks +that a facet was updated, or that it was even read.** + +## 2026-09-07 — decided: nothing committed announces an open cart `quince` + +**A cart is local and untracked, so a clone cannot see that a round is open.** +*That is the boundary and not a defect:* **`v1` makes the cart local to the +working tree the two presences share, and somebody who has only cloned is by +construction not one of them.** + +**`.gitignore` records that carts happen here. Nothing records that one is open**, +*and the asymmetry is deliberate* — **a mechanism for announcing something +designed to be ephemeral is the first step in it not being ephemeral.** + +**Belief that could be shown wrong:** *that the shared working tree is the right +unit.* **If two presences ever work from different machines, the cart has nowhere +to live**, *and everything above stops being true at once.* + +## 2026-09-07 — conversion now destroys, so the log is the only record `quince` + +**Under `cart/v1` the cart is untracked, so converting a round deletes its dailies +outright.** *Every previous conversion left them in git.* + +**So an entry that was not written before the `rm` is gone**, *and the write-ahead +log is the mechanism* — **stage as you go, because at conversion the cart is the +only copy and you are about to delete it.** + +**And it sharpens a conflict recorded in `cart.usages.md`:** *the annotation +protocol here says commit before dissolving because git is the only archive.* +**With an ignored cart there is no archive, so dissolving a notes file destroys +the annotations outright.** *Both documents are loom's; this is where an adopter +can see the collision.*