loomctl external: list, add, check — and its first run found two changed documents
Go, standard library only, with git shelled out for list alone. list enumerates a publisher's .loom/published by partial clone and ls-tree; add fetches one document, writes it under .loom/externals and records the resolved origin and the publisher's ETag in .loom/externals/.locks; check asks conditionally and reports. The first real run did what the tool exists for. All eight documents adopted by hand before it existed reported unlocked — the tool refuses to invent a lock by adopting whatever the remote currently serves, since that would assert the local copy is the one being served, which is the thing it was about to check. Locking them fetched two that had moved: bedrock/starting.md, which now says the worked example is private and will not link to something you cannot fetch, and cart.md, which went to v1. cart v1 changes a role we cast: a cart is not committed, because a committed cart grows a third file by itself — version control does not require anybody to ask, so the two-file rule is never invoked — and because ignored, gone means gone. Adds .loom/cart/ to .gitignore and supersedes the isolation entry rather than editing it. osprey and marmalade are already in history and are left there: rewriting to honour a rule adopted afterwards costs more than it buys. Records the conflict this creates rather than settling it: the annotation protocol here says commit before dissolving because git is the only archive, and an ignored cart has no archive, so dissolving would destroy the annotations outright. Credentials are read-only, per host, and passed to git through the environment rather than argv, because argv is visible to every process on the machine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018UTxuSizozEA8yDitPuris
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,5 @@
|
||||
# cart v1: a cart is not committed. Ignored, gone means gone.
|
||||
.loom/cart/
|
||||
|
||||
# build output
|
||||
/loomctl
|
||||
@@ -518,3 +518,55 @@ the cart that named it.*
|
||||
**Recorded rather than fixed.** *The role still works; its justification was
|
||||
written for a case that has not yet occurred.* **If a round ever does produce code
|
||||
while it is open, nothing here changes.**
|
||||
|
||||
## 2026-09-07 — `loomctl external` exists, and its first run reconciled two documents `marmalade`
|
||||
|
||||
**Built: `list`, `add`, `check`.** *Go, no dependencies outside the standard
|
||||
library, `git` shelled out for `list` only.*
|
||||
|
||||
**The first real run did the thing the tool is for.** *Eight documents were
|
||||
adopted by hand before it existed; `check` reported all eight `unlocked`, and
|
||||
`add` locked them* — **and two came back changed**, `bedrock/starting.md` *and*
|
||||
`cart/cart.md`. **Neither change would have been noticed by anybody.**
|
||||
|
||||
**Believed to advance:** *the lock is what makes `check` possible at all.* **With
|
||||
no lock there is nothing to compare and the only honest report is `unlocked`** —
|
||||
*and the tool refuses to invent one by adopting whatever the remote currently
|
||||
serves.*
|
||||
|
||||
**Belief that could be shown wrong:** *that a per-person path derived from the URL
|
||||
is good enough.* **`<host>/<owner>/<repo>/<name>.md` guesses that the first two
|
||||
path segments name an owner and a repository**, *which is true of gitea, GitHub and
|
||||
GitLab and is not a rule.* **`--path` exists for when it is wrong.**
|
||||
|
||||
## 2026-09-07 — supersedes the isolation role: a cart is not committed `marmalade`
|
||||
|
||||
**`cart` is now `v1` and it changed the thing we cast a role on.**
|
||||
|
||||
> **So the cart is not committed.** *It lives in the working tree of the machine
|
||||
> the two presences share, and `.loom/cart/` is ignored by version control.*
|
||||
|
||||
**The reason is not tidiness:** *a committed cart grows a third file by itself.*
|
||||
**The two-file rule defends against somebody asking for one; version control does
|
||||
not require anybody to ask** — *anyone who can clone can add a third, and the
|
||||
agreement's test is never invoked because nobody had the conversation.*
|
||||
|
||||
**And it is what makes a round end.** *Committed, a cart is gone from the tree and
|
||||
permanent in history* — **so "gone" means "no longer live" and the negotiation
|
||||
stays quotable forever.** *Ignored, gone means gone.*
|
||||
|
||||
**So `.loom/cart/` is now in `.gitignore`.** *The isolation role entry above
|
||||
assumed the cart lived on the default branch; **the cart lives on no branch.***
|
||||
*What survives of that entry is the other half: work is isolated on a branch named
|
||||
for the round that authorised it.*
|
||||
|
||||
**Not undone: `osprey` and `marmalade` are already in this repository's history.**
|
||||
*Rewriting history to honour a rule adopted afterwards would cost more than it
|
||||
buys*, **and the two rounds are quotable forever, which is exactly what `v1` says
|
||||
not to want.** *Recorded rather than repaired.*
|
||||
|
||||
**Belief that could be shown wrong, and it is a real conflict:** *the annotation
|
||||
protocol this repository works under says **commit before dissolving, git is the
|
||||
only archive of the conversation.*** **An ignored cart has no archive**, so
|
||||
dissolving a notes file destroys the annotations outright. *One of the two is
|
||||
wrong and it is not ours to settle.*
|
||||
|
||||
Vendored
+11
@@ -0,0 +1,11 @@
|
||||
# loomctl locks — one record per adopted document.
|
||||
# path<TAB>url<TAB>etag The url is resolved: a short form would follow
|
||||
# whatever the default branch is at the time you ask.
|
||||
git.hypertheory-labs.dev/loom/annotating/annotating.md https://git.hypertheory-labs.dev/loom/annotating/raw/branch/main/.loom/published/annotating.md "9b1f7e6ca92f2339b2d433686c27845362944bdb"
|
||||
git.hypertheory-labs.dev/loom/bedrock/loom-directory.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/loom-directory.md "ee0f49cb900c0812678061971194325d9cba366a"
|
||||
git.hypertheory-labs.dev/loom/bedrock/publication.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/publication.md "eb0cb63629a36b056f215dfbe24567d1918cec38"
|
||||
git.hypertheory-labs.dev/loom/bedrock/recording-decisions.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/recording-decisions.md "970d4b4da76aac99c9b1f1b6580ec20daa42e329"
|
||||
git.hypertheory-labs.dev/loom/bedrock/sibling-facets.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/sibling-facets.md "a46446a34ccb8bfc533d3cce19f4c88548c4fa04"
|
||||
git.hypertheory-labs.dev/loom/bedrock/starting.md https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/starting.md "7d997a30248a88c90b23f2f9453d7cfceb03848e"
|
||||
git.hypertheory-labs.dev/loom/cart/cart.md https://git.hypertheory-labs.dev/loom/cart/raw/branch/main/.loom/published/cart.md "49fc852bdd280293f0f5e0050034e3b89ff7255e"
|
||||
git.hypertheory-labs.dev/loom/externals/externals.md https://git.hypertheory-labs.dev/loom/externals/raw/branch/main/.loom/published/externals.md "9641f0e77b6f5c0161fa7593805277ba0c1e6176"
|
||||
@@ -60,9 +60,13 @@ what they came for.*
|
||||
|
||||
## Look at one instead of reading this
|
||||
|
||||
**[`jeffry/homelab-cluster`](https://git.hypertheory-labs.dev/jeffry/homelab-cluster)**
|
||||
— *six documents, one gap, no decomposition, and a `README` that says what the
|
||||
root documents are for and what these are for.*
|
||||
**There is a worked example — six documents, one gap, no decomposition — and it
|
||||
is private.**
|
||||
|
||||
**It is a better answer than this page**, and if the two ever disagree, it is
|
||||
right.
|
||||
*It describes a cluster in enough detail to be a target list, so it is not
|
||||
published, and **this page will not link you to something you cannot fetch.***
|
||||
**If you have access, ask for it by name; if you do not, the two questions at the
|
||||
top are the whole of it.**
|
||||
|
||||
> **A public page naming a private thing as its canonical answer is worse than no
|
||||
> example**, and this page did exactly that until somebody measured it.
|
||||
|
||||
+47
-2
@@ -1,6 +1,6 @@
|
||||
# Agreement — the cart
|
||||
|
||||
**v0.** Depends on `annotating/v0`.
|
||||
**v1.** Depends on `annotating/v0`.
|
||||
|
||||
**How two parties work out what something means before one of them changes it.**
|
||||
|
||||
@@ -104,9 +104,54 @@ already rejected, and the rejection is gone because it lived in an annotation
|
||||
that died with the round.
|
||||
|
||||
**A decline needs no file of its own.** It is an entry in whatever durable record
|
||||
you keep, and **it should say what you believed, not just what you chose** — only
|
||||
you keep — **which must outlive the cart**, *and therefore cannot be inside it* — and **it should say what you believed, not just what you chose** — only
|
||||
a belief can later be shown wrong.
|
||||
|
||||
## The cart is local, and that is what keeps it to two files
|
||||
|
||||
**A cart is two developers working side by side.** *Everything else — the wider
|
||||
team, the people who need to know, the thing that has to be tracked — is issues,
|
||||
chat, whatever you already have.* **This is not that channel and it does not scale
|
||||
into one.**
|
||||
|
||||
> **So the cart is not committed.** *It lives in the working tree of the machine
|
||||
> the two presences share, and `.loom/cart/` is ignored by version control.*
|
||||
|
||||
**The reason is not tidiness. A committed cart grows a third file by itself.**
|
||||
*The rule above defends against somebody asking for one; **version control does
|
||||
not require anybody to ask.*** *Anyone who can clone can add `joe-rose.md`, and
|
||||
then `sue-rose.md`, and the agreement's defence — **what happens to this file when
|
||||
the person changes?** — is never invoked, because nobody ever had the
|
||||
conversation.*
|
||||
|
||||
**This is also what makes a round actually end.** *Committed, a cart is gone from
|
||||
the tree and permanent in history — **so "gone" means "no longer live" and
|
||||
negotiation stays quotable forever.*** **Ignored, gone means gone**, which is what
|
||||
the round dying was for.
|
||||
|
||||
*The cost, stated: **two presences who do not share a filesystem cannot use a
|
||||
cart.*** *That is a real limit and it is the right one — if you need a medium
|
||||
between machines, you need the other channel, and reaching for a cart there is
|
||||
how it becomes a chat log.*
|
||||
|
||||
## What is not yet a decision goes in the write-ahead log
|
||||
|
||||
**A round produces things that are neither questions nor decisions:** *something
|
||||
observed, something that may turn out to be noise, something you would kick
|
||||
yourself for losing and cannot yet justify writing down.*
|
||||
|
||||
**`event-log.wal.md`, in the cart.** *Findings, not decisions.* **Nothing in it is
|
||||
durable and nothing in it has been decided.**
|
||||
|
||||
**At conversion, each entry either becomes an entry in the durable record or is
|
||||
discarded.** *Same two exits as a polad, and for the same reason: **conversion is
|
||||
when you know most about it.***
|
||||
|
||||
> **Write the reason it is not yet an entry.** *An observation you cannot justify
|
||||
> promoting is worth keeping; **one you have not said why you are hesitant about
|
||||
> will be promoted by whoever finds it, on the strength of it having been written
|
||||
> down.***
|
||||
|
||||
## Where a cart lives, and the shelf
|
||||
|
||||
**A fixed path, and at most two things in it:**
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
// Package config reads the per-host settings loomctl needs to talk to a git host.
|
||||
//
|
||||
// The config is not only a secret. It is how you talk to a host at all, which is
|
||||
// why it is keyed by host rather than being a single token. It lives in the
|
||||
// user's home directory and never in a repository — see .loom/event-log.md,
|
||||
// "decided by fallback: where the credential lives".
|
||||
package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Host is what we know about one git host.
|
||||
type Host struct {
|
||||
// Token is a read-only personal access token. It must not carry write
|
||||
// scope: loomctl never writes over the network.
|
||||
Token string `json:"token,omitempty"`
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
Hosts map[string]Host `json:"hosts"`
|
||||
}
|
||||
|
||||
// Path is where the config lives. Never inside a repository.
|
||||
func Path() string {
|
||||
if p := os.Getenv("LOOMCTL_CONFIG"); p != "" {
|
||||
return p
|
||||
}
|
||||
home, err := os.UserHomeDir()
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
return filepath.Join(home, ".config", "loomctl", "config.json")
|
||||
}
|
||||
|
||||
// Load reads the config. A missing file is not an error: everything loomctl does
|
||||
// against a public repository works with no credential at all.
|
||||
func Load() (*Config, error) {
|
||||
c := &Config{Hosts: map[string]Host{}}
|
||||
p := Path()
|
||||
if p == "" {
|
||||
return c, nil
|
||||
}
|
||||
b, err := os.ReadFile(p)
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return c, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading %s: %w", p, err)
|
||||
}
|
||||
if err := json.Unmarshal(b, c); err != nil {
|
||||
return nil, fmt.Errorf("parsing %s: %w", p, err)
|
||||
}
|
||||
if c.Hosts == nil {
|
||||
c.Hosts = map[string]Host{}
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// TokenFor returns the token for a host, or "" if we have none.
|
||||
//
|
||||
// An environment variable wins over the file, so a token can be supplied for one
|
||||
// invocation without ever being written to disk.
|
||||
func (c *Config) TokenFor(host string) string {
|
||||
if t := os.Getenv("LOOMCTL_TOKEN_" + envKey(host)); t != "" {
|
||||
return t
|
||||
}
|
||||
if t := os.Getenv("LOOMCTL_TOKEN"); t != "" {
|
||||
return t
|
||||
}
|
||||
return c.Hosts[host].Token
|
||||
}
|
||||
|
||||
// envKey turns a hostname into the shape an environment variable can carry.
|
||||
func envKey(host string) string {
|
||||
r := strings.NewReplacer(".", "_", "-", "_", ":", "_")
|
||||
return strings.ToUpper(r.Replace(host))
|
||||
}
|
||||
Vendored
+283
@@ -0,0 +1,283 @@
|
||||
// Package external implements the operations on documents somebody else
|
||||
// published that we depend on.
|
||||
//
|
||||
// Every act is a fetch or a comparison. Nothing here repairs anything: a changed
|
||||
// external is a candidate, not a replacement, and somebody decides.
|
||||
package external
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.hypertheory-labs.dev/loom/loom-cli/internal/config"
|
||||
"git.hypertheory-labs.dev/loom/loom-cli/internal/lock"
|
||||
)
|
||||
|
||||
var client = &http.Client{Timeout: 30 * time.Second}
|
||||
|
||||
// FindRoot walks up from dir looking for the .loom directory that marks a
|
||||
// repository using loom.
|
||||
func FindRoot(dir string) (string, error) {
|
||||
d, err := filepath.Abs(dir)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
for {
|
||||
if fi, err := os.Stat(filepath.Join(d, ".loom")); err == nil && fi.IsDir() {
|
||||
return d, nil
|
||||
}
|
||||
parent := filepath.Dir(d)
|
||||
if parent == d {
|
||||
return "", errors.New("no .loom directory found in this directory or any parent")
|
||||
}
|
||||
d = parent
|
||||
}
|
||||
}
|
||||
|
||||
// localPath derives where an adopted document is kept from the URL it came from.
|
||||
//
|
||||
// The path is for a person: <host>/<owner>/<repo>/<basename>. It deliberately
|
||||
// does not encode the route, the branch, or .loom/published/ — which is why it
|
||||
// cannot be turned back into a URL, and why the lock records the origin.
|
||||
func localPath(raw string) (string, error) {
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if u.Host == "" || u.Scheme == "" {
|
||||
return "", fmt.Errorf("not an absolute URL: %s", raw)
|
||||
}
|
||||
segs := strings.Split(strings.Trim(u.Path, "/"), "/")
|
||||
if len(segs) < 3 {
|
||||
return "", fmt.Errorf("cannot tell owner and repository from %s — pass --path", raw)
|
||||
}
|
||||
base := segs[len(segs)-1]
|
||||
if base == "" {
|
||||
return "", fmt.Errorf("no file name in %s", raw)
|
||||
}
|
||||
return path.Join(u.Host, segs[0], segs[1], base), nil
|
||||
}
|
||||
|
||||
// request builds a GET carrying the host's token, if we have one.
|
||||
func request(cfg *config.Config, method, raw string, ifNoneMatch string) (*http.Request, error) {
|
||||
req, err := http.NewRequest(method, raw, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if t := cfg.TokenFor(req.URL.Host); t != "" {
|
||||
req.Header.Set("Authorization", "token "+t)
|
||||
}
|
||||
if ifNoneMatch != "" {
|
||||
req.Header.Set("If-None-Match", ifNoneMatch)
|
||||
}
|
||||
return req, nil
|
||||
}
|
||||
|
||||
// Add fetches a document, writes it into .loom/externals/, and records its lock.
|
||||
//
|
||||
// It does not create a .usages.md: an empty facet asserts that we have something
|
||||
// to say and we do not.
|
||||
func Add(root, raw, override string, out io.Writer) error {
|
||||
cfg, err := config.Load()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rel := override
|
||||
if rel == "" {
|
||||
if rel, err = localPath(raw); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
req, err := request(cfg, http.MethodGet, raw, "")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
switch resp.StatusCode {
|
||||
case http.StatusOK:
|
||||
case http.StatusNotFound:
|
||||
return fmt.Errorf("404 unresolved: %s was withdrawn, or this credential cannot see it — "+
|
||||
"over HTTP these are the same response", raw)
|
||||
default:
|
||||
return fmt.Errorf("%s: %s", resp.Status, raw)
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
etag := resp.Header.Get("ETag")
|
||||
|
||||
dest := filepath.Join(root, lock.Dir, filepath.FromSlash(rel))
|
||||
if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.WriteFile(dest, body, 0o644); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
locks, err := lock.Load(root)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
locks.Put(lock.Record{Path: rel, URL: resp.Request.URL.String(), ETag: etag})
|
||||
if err := locks.Save(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fmt.Fprintf(out, "adopted %s\n", rel)
|
||||
fmt.Fprintf(out, " from %s\n", resp.Request.URL)
|
||||
if etag == "" {
|
||||
fmt.Fprintf(out, " etag (none served — check cannot ask conditionally)\n")
|
||||
} else {
|
||||
fmt.Fprintf(out, " etag %s\n", etag)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Status is what check found for one document.
|
||||
type Status struct {
|
||||
Path string
|
||||
Result string
|
||||
Detail string
|
||||
}
|
||||
|
||||
// Check asks every publisher whether their copy has moved.
|
||||
//
|
||||
// It reports and does nothing else. A changed document is a candidate, not a
|
||||
// replacement.
|
||||
func Check(root string, out io.Writer) error {
|
||||
cfg, err := config.Load()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
locks, err := lock.Load(root)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
seen := map[string]bool{}
|
||||
var results []Status
|
||||
|
||||
for _, rec := range locks.All() {
|
||||
seen[rec.Path] = true
|
||||
results = append(results, checkOne(cfg, root, rec))
|
||||
}
|
||||
|
||||
// Documents in the tree with no lock. Never adopt whatever the remote is
|
||||
// currently serving as the lock: that asserts the local copy is the one
|
||||
// being served, which is the thing we were about to check.
|
||||
unlocked, err := unlockedDocs(root, seen)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, p := range unlocked {
|
||||
results = append(results, Status{Path: p, Result: "unlocked", Detail: "fetched by hand; run `loomctl external add` to lock it"})
|
||||
}
|
||||
|
||||
if len(results) == 0 {
|
||||
fmt.Fprintln(out, "no adopted documents")
|
||||
return nil
|
||||
}
|
||||
w := 0
|
||||
for _, r := range results {
|
||||
if len(r.Path) > w {
|
||||
w = len(r.Path)
|
||||
}
|
||||
}
|
||||
for _, r := range results {
|
||||
fmt.Fprintf(out, "%-*s %-10s %s\n", w, r.Path, r.Result, r.Detail)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func checkOne(cfg *config.Config, root string, rec lock.Record) Status {
|
||||
if _, err := os.Stat(filepath.Join(root, lock.Dir, filepath.FromSlash(rec.Path))); errors.Is(err, fs.ErrNotExist) {
|
||||
return Status{rec.Path, "missing", "locked, but the local copy is gone"}
|
||||
}
|
||||
if rec.ETag == "" {
|
||||
return Status{rec.Path, "no-etag", "publisher served none; freshness cannot be asked"}
|
||||
}
|
||||
req, err := request(cfg, http.MethodGet, rec.URL, rec.ETag)
|
||||
if err != nil {
|
||||
return Status{rec.Path, "error", err.Error()}
|
||||
}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return Status{rec.Path, "error", err.Error()}
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
io.Copy(io.Discard, resp.Body)
|
||||
|
||||
switch resp.StatusCode {
|
||||
case http.StatusNotModified:
|
||||
return Status{rec.Path, "same", ""}
|
||||
case http.StatusOK:
|
||||
return Status{rec.Path, "CHANGED", "upstream moved — the new copy is a candidate, not a replacement"}
|
||||
case http.StatusGone:
|
||||
return Status{rec.Path, "GONE", "410 — follow whatever the response points at"}
|
||||
case http.StatusNotFound:
|
||||
return Status{rec.Path, "404", "withdrawn, or access lost — over HTTP these are the same response"}
|
||||
default:
|
||||
return Status{rec.Path, resp.Status, ""}
|
||||
}
|
||||
}
|
||||
|
||||
// unlockedDocs finds adopted documents that no lock covers. Facets we wrote
|
||||
// ourselves are not adopted documents and are skipped.
|
||||
func unlockedDocs(root string, locked map[string]bool) ([]string, error) {
|
||||
base := filepath.Join(root, lock.Dir)
|
||||
var out []string
|
||||
err := filepath.WalkDir(base, func(p string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
if d.IsDir() || !strings.HasSuffix(d.Name(), ".md") {
|
||||
return nil
|
||||
}
|
||||
rel, err := filepath.Rel(base, p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rel = filepath.ToSlash(rel)
|
||||
if locked[rel] || isFacet(rel) {
|
||||
return nil
|
||||
}
|
||||
out = append(out, rel)
|
||||
return nil
|
||||
})
|
||||
return out, err
|
||||
}
|
||||
|
||||
// isFacet reports whether a path is something we wrote beside an adopted
|
||||
// document rather than the document itself: x.usages.md, x.gaps.md, x.notes.md.
|
||||
func isFacet(rel string) bool {
|
||||
base := strings.TrimSuffix(path.Base(rel), ".md")
|
||||
i := strings.LastIndex(base, ".")
|
||||
if i < 0 {
|
||||
return false
|
||||
}
|
||||
switch base[i+1:] {
|
||||
case "usages", "gaps", "notes":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
Vendored
+72
@@ -0,0 +1,72 @@
|
||||
package external
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestLocalPath(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name, url, want string
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "gitea raw, published document",
|
||||
url: "https://git.hypertheory-labs.dev/loom/bedrock/raw/branch/main/.loom/published/starting.md",
|
||||
want: "git.hypertheory-labs.dev/loom/bedrock/starting.md",
|
||||
},
|
||||
{
|
||||
name: "github raw host",
|
||||
url: "https://raw.githubusercontent.com/octocat/Hello-World/main/README.md",
|
||||
want: "raw.githubusercontent.com/octocat/Hello-World/README.md",
|
||||
},
|
||||
{
|
||||
name: "not enough path to name an owner and repository",
|
||||
url: "https://example.com/thing.md",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "not absolute",
|
||||
url: "/loom/bedrock/starting.md",
|
||||
wantErr: true,
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got, err := localPath(tc.url)
|
||||
if tc.wantErr {
|
||||
if err == nil {
|
||||
t.Fatalf("localPath(%q) = %q, want an error", tc.url, got)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("localPath(%q): %v", tc.url, err)
|
||||
}
|
||||
if got != tc.want {
|
||||
t.Errorf("localPath(%q) = %q, want %q", tc.url, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsFacet(t *testing.T) {
|
||||
// A facet is something we wrote beside an adopted document. check must not
|
||||
// report our own writing as an unlocked external.
|
||||
facets := []string{
|
||||
"host/loom/cart/cart.usages.md",
|
||||
"host/loom/externals/externals.gaps.md",
|
||||
"host/o/r/plan.notes.md",
|
||||
}
|
||||
documents := []string{
|
||||
"host/loom/cart/cart.md",
|
||||
"host/loom/bedrock/recording-decisions.md", // a hyphen is not a facet
|
||||
"host/o/r/starting.md",
|
||||
}
|
||||
for _, p := range facets {
|
||||
if !isFacet(p) {
|
||||
t.Errorf("isFacet(%q) = false, want true", p)
|
||||
}
|
||||
}
|
||||
for _, p := range documents {
|
||||
if isFacet(p) {
|
||||
t.Errorf("isFacet(%q) = true, want false", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
Vendored
+107
@@ -0,0 +1,107 @@
|
||||
package external
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
|
||||
"git.hypertheory-labs.dev/loom/loom-cli/internal/config"
|
||||
)
|
||||
|
||||
// PublishedDir is the only directory in somebody else's repository that list
|
||||
// looks at. What is not exported is not hidden — it is simply not what you
|
||||
// depend on.
|
||||
const PublishedDir = ".loom/published"
|
||||
|
||||
// List enumerates a publisher's published surface.
|
||||
//
|
||||
// It shells out to git rather than using a host's REST API, because git is the
|
||||
// one interface gitea, GitHub and GitLab all speak the same way: their contents
|
||||
// APIs have three different URL shapes, three JSON shapes and three auth
|
||||
// schemes, and a private repository refuses the anonymous ones. The cost is that
|
||||
// git must be on PATH.
|
||||
func List(repoURL string, out io.Writer) error {
|
||||
cfg, err := config.Load()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
dir, err := os.MkdirTemp("", "loomctl-list-")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.RemoveAll(dir)
|
||||
|
||||
clone := exec.Command("git", "clone",
|
||||
"--filter=blob:none", // trees only: we want names, not contents
|
||||
"--depth=1", // and bound the damage if the server ignores the filter
|
||||
"--no-checkout",
|
||||
"--quiet",
|
||||
repoURL, dir,
|
||||
)
|
||||
clone.Env = gitEnv(cfg, repoURL)
|
||||
var stderr bytes.Buffer
|
||||
clone.Stderr = &stderr
|
||||
if err := clone.Run(); err != nil {
|
||||
return fmt.Errorf("git clone: %w\n%s", err, strings.TrimSpace(stderr.String()))
|
||||
}
|
||||
|
||||
// git's fallback when a server refuses the filter is silent apart from this
|
||||
// warning, and the fallback is to download everything.
|
||||
if strings.Contains(stderr.String(), "filtering not recognized by server") {
|
||||
fmt.Fprintf(out, "warning: %s ignored --filter, so this fetched every blob at HEAD\n\n", repoURL)
|
||||
}
|
||||
|
||||
ls := exec.Command("git", "-C", dir, "ls-tree", "--name-only", "HEAD:"+PublishedDir)
|
||||
var names, lsErr bytes.Buffer
|
||||
ls.Stdout, ls.Stderr = &names, &lsErr
|
||||
if err := ls.Run(); err != nil {
|
||||
fmt.Fprintf(out, "%s publishes nothing — no %s\n", repoURL, PublishedDir)
|
||||
return nil
|
||||
}
|
||||
|
||||
for _, n := range strings.Split(strings.TrimSpace(names.String()), "\n") {
|
||||
if n != "" {
|
||||
fmt.Fprintln(out, n)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// gitEnv passes credentials to git through the environment rather than through
|
||||
// -c on the command line, because argv is visible to every process on the
|
||||
// machine and an environment is not.
|
||||
func gitEnv(cfg *config.Config, repoURL string) []string {
|
||||
env := append(os.Environ(), "GIT_TERMINAL_PROMPT=0")
|
||||
host := hostOf(repoURL)
|
||||
if host == "" {
|
||||
return env
|
||||
}
|
||||
t := cfg.TokenFor(host)
|
||||
if t == "" {
|
||||
return env
|
||||
}
|
||||
return append(env,
|
||||
"GIT_CONFIG_COUNT=1",
|
||||
"GIT_CONFIG_KEY_0=http.extraHeader",
|
||||
"GIT_CONFIG_VALUE_0=Authorization: token "+t,
|
||||
)
|
||||
}
|
||||
|
||||
func hostOf(raw string) string {
|
||||
i := strings.Index(raw, "://")
|
||||
if i < 0 {
|
||||
return ""
|
||||
}
|
||||
rest := raw[i+3:]
|
||||
if at := strings.Index(rest, "@"); at >= 0 {
|
||||
rest = rest[at+1:]
|
||||
}
|
||||
if s := strings.IndexAny(rest, "/:"); s >= 0 {
|
||||
rest = rest[:s]
|
||||
}
|
||||
return rest
|
||||
}
|
||||
@@ -0,0 +1,122 @@
|
||||
// Package lock reads and writes .loom/externals/.locks.
|
||||
//
|
||||
// One record per adopted document: where it was fetched from, resolved, and the
|
||||
// ETag the publisher served with it. The ETag is opaque and is never a hash we
|
||||
// compute — on gitea it happens to equal the git blob hash and on GitHub it does
|
||||
// not, so a design that compares a local hash to a remote ETag works on exactly
|
||||
// one host by coincidence. See .loom/event-log.md.
|
||||
package lock
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Dir is where adopted documents live, relative to the repository root.
|
||||
const Dir = ".loom/externals"
|
||||
|
||||
// File is the lock file, inside Dir.
|
||||
const File = ".locks"
|
||||
|
||||
const header = "# loomctl locks — one record per adopted document.\n" +
|
||||
"# path<TAB>url<TAB>etag The url is resolved: a short form would follow\n" +
|
||||
"# whatever the default branch is at the time you ask.\n"
|
||||
|
||||
// Record is one adopted document.
|
||||
type Record struct {
|
||||
// Path is relative to Dir, and is for a person to read. The origin is the
|
||||
// URL: the path does not round-trip, because it drops the route, the
|
||||
// branch, and .loom/published/.
|
||||
Path string
|
||||
// URL is the resolved origin, branch and all.
|
||||
URL string
|
||||
// ETag is the publisher's, verbatim, including its quotes.
|
||||
ETag string
|
||||
}
|
||||
|
||||
// Set is every lock, keyed by path.
|
||||
type Set struct {
|
||||
root string
|
||||
recs map[string]Record
|
||||
}
|
||||
|
||||
func path(root string) string { return filepath.Join(root, Dir, File) }
|
||||
|
||||
// Load reads the lock file for the repository at root. A missing file is an
|
||||
// empty set, not an error: a repository whose externals were fetched by hand has
|
||||
// no locks, and reporting that is the point.
|
||||
func Load(root string) (*Set, error) {
|
||||
s := &Set{root: root, recs: map[string]Record{}}
|
||||
f, err := os.Open(path(root))
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return s, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
sc := bufio.NewScanner(f)
|
||||
for n := 1; sc.Scan(); n++ {
|
||||
line := sc.Text()
|
||||
if strings.TrimSpace(line) == "" || strings.HasPrefix(line, "#") {
|
||||
continue
|
||||
}
|
||||
parts := strings.Split(line, "\t")
|
||||
if len(parts) != 3 {
|
||||
return nil, fmt.Errorf("%s:%d: want 3 tab-separated fields, got %d", path(root), n, len(parts))
|
||||
}
|
||||
s.recs[parts[0]] = Record{Path: parts[0], URL: parts[1], ETag: parts[2]}
|
||||
}
|
||||
return s, sc.Err()
|
||||
}
|
||||
|
||||
// Get returns the record for a path, and whether there was one.
|
||||
func (s *Set) Get(p string) (Record, bool) { r, ok := s.recs[p]; return r, ok }
|
||||
|
||||
// Put adds or replaces a record.
|
||||
func (s *Set) Put(r Record) { s.recs[r.Path] = r }
|
||||
|
||||
// All returns every record, ordered by path so the file diffs cleanly.
|
||||
func (s *Set) All() []Record {
|
||||
out := make([]Record, 0, len(s.recs))
|
||||
for _, r := range s.recs {
|
||||
out = append(out, r)
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].Path < out[j].Path })
|
||||
return out
|
||||
}
|
||||
|
||||
// Save writes the lock file, replacing it atomically so an interrupted write
|
||||
// cannot leave a repository holding half a lock.
|
||||
func (s *Set) Save() error {
|
||||
p := path(s.root)
|
||||
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
var b strings.Builder
|
||||
b.WriteString(header)
|
||||
for _, r := range s.All() {
|
||||
fmt.Fprintf(&b, "%s\t%s\t%s\n", r.Path, r.URL, r.ETag)
|
||||
}
|
||||
tmp, err := os.CreateTemp(filepath.Dir(p), ".locks-*")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tmp.WriteString(b.String()); err != nil {
|
||||
tmp.Close()
|
||||
os.Remove(tmp.Name())
|
||||
return err
|
||||
}
|
||||
if err := tmp.Close(); err != nil {
|
||||
os.Remove(tmp.Name())
|
||||
return err
|
||||
}
|
||||
return os.Rename(tmp.Name(), p)
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
// loomctl fetches documents this repository depends on, and finds out when they
|
||||
// change.
|
||||
//
|
||||
// It reports and never repairs. Everything it writes, it writes to the working
|
||||
// tree — committing and pushing are yours, because the consequences of a push
|
||||
// land on people a tool cannot experience.
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"git.hypertheory-labs.dev/loom/loom-cli/internal/external"
|
||||
)
|
||||
|
||||
const usage = `loomctl — fetch what you depend on, and find out when it changed.
|
||||
|
||||
loomctl external list <repo-url> what a repository publishes
|
||||
loomctl external add <url> [--path p] adopt one document and lock it
|
||||
loomctl external check ask every publisher whether theirs moved
|
||||
|
||||
check reports and does not fix. A changed document is a candidate, not a
|
||||
replacement, and somebody decides. It exits 0 whether or not anything moved:
|
||||
the report is the answer.
|
||||
|
||||
Adopted documents live in .loom/externals/<host>/<owner>/<repo>/<name>.md, and
|
||||
their origins in .loom/externals/.locks. The path is for a person to read; the
|
||||
lock is what a machine uses, because the path does not round-trip to a URL.
|
||||
|
||||
Credentials are read-only and per host, in ~/.config/loomctl/config.json or in
|
||||
LOOMCTL_TOKEN_<HOST>. loomctl never writes over the network, so a token it is
|
||||
given should never carry write scope.
|
||||
|
||||
Requires git on PATH, for list only.
|
||||
`
|
||||
|
||||
func main() {
|
||||
if err := run(os.Args[1:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "loomctl: "+err.Error())
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func run(args []string) error {
|
||||
if len(args) == 0 || args[0] == "-h" || args[0] == "--help" || args[0] == "help" {
|
||||
fmt.Print(usage)
|
||||
return nil
|
||||
}
|
||||
switch args[0] {
|
||||
case "external":
|
||||
return runExternal(args[1:])
|
||||
default:
|
||||
return fmt.Errorf("unknown command %q\n\n%s", args[0], usage)
|
||||
}
|
||||
}
|
||||
|
||||
func runExternal(args []string) error {
|
||||
if len(args) == 0 {
|
||||
return fmt.Errorf("external needs a subcommand: list, add, check")
|
||||
}
|
||||
switch args[0] {
|
||||
case "list":
|
||||
if len(args) != 2 {
|
||||
return fmt.Errorf("usage: loomctl external list <repo-url>")
|
||||
}
|
||||
return external.List(args[1], os.Stdout)
|
||||
|
||||
case "add":
|
||||
fs := flag.NewFlagSet("add", flag.ContinueOnError)
|
||||
path := fs.String("path", "", "where to keep it, relative to .loom/externals (default: derived from the URL)")
|
||||
if err := fs.Parse(args[1:]); err != nil {
|
||||
return err
|
||||
}
|
||||
if fs.NArg() != 1 {
|
||||
return fmt.Errorf("usage: loomctl external add <url> [--path p]")
|
||||
}
|
||||
root, err := root()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return external.Add(root, fs.Arg(0), *path, os.Stdout)
|
||||
|
||||
case "check":
|
||||
root, err := root()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return external.Check(root, os.Stdout)
|
||||
|
||||
default:
|
||||
return fmt.Errorf("unknown external subcommand %q", args[0])
|
||||
}
|
||||
}
|
||||
|
||||
func root() (string, error) {
|
||||
wd, err := os.Getwd()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return external.FindRoot(wd)
|
||||
}
|
||||
Reference in New Issue
Block a user