rapidbuildr
Use when the user wants fully autonomous end-to-end development from idea to production-ready product - no human approval gates, Opus makes all decisions, ships secure and production-hardened code, self-improves iteratively until improvements are marginal
Rapidbuildr
Production-ready in one shot, auto-evolved. Idea to deployed, secure, hardened product with iterative self-improvement. Zero human gates — Opus makes every decision, documents every choice, ships code that's ready for real users from iteration zero.
Website: rapidbuildr.com
<HARD-GATE> Do NOT invoke any interactive brainstorming or design skill that has human approval gates. Rapidbuildr handles the full lifecycle autonomously — context analysis, approach evaluation, design, TDD, and reviews — with zero user wait-points. If another skill would pause for user input, do NOT use it. </HARD-GATE>Modes
Rapidbuildr has two modes. Default mode runs the standard 6-phase autonomous loop. Autoresearch mode adds Karpathy-inspired mechanical rigor: frozen metrics, git ratcheting, structured experiment logs, and lightweight iteration cycles.
digraph modes {
rankdir=LR;
node [fontname="Helvetica"];
"User invokes\nrapidbuildr" [shape=doublecircle];
"Says 'autoresearch'?" [shape=diamond];
"Default Mode\n(expert-driven)" [shape=box style=filled fillcolor=lightyellow];
"Autoresearch Mode\n(metric-driven)" [shape=box style=filled fillcolor=lightcyan];
"User invokes\nrapidbuildr" -> "Says 'autoresearch'?";
"Says 'autoresearch'?" -> "Default Mode\n(expert-driven)" [label="no"];
"Says 'autoresearch'?" -> "Autoresearch Mode\n(metric-driven)" [label="yes"];
}
| Aspect | Default Mode | Autoresearch Mode |
|---|---|---|
| Improvement arbiter | Expert panel scores | Frozen metric (metric.sh) |
| Rollback on failure | Manual / escalate | Automatic git ratchet |
| Iteration weight | Full 6-phase cycle | Lightweight 5-min cycles |
| Experiment tracking | decisions-log.md (prose) | experiments.tsv (structured) |
| Exit condition | Score < 15 or max 3 | Metric delta < 1% or max 5 |
| Expert panel | 10 experts every iteration | 10 first time, 5 on iterations (diff only) |
| Protected files | None | metric.sh is immutable |
Activation: User says "autoresearch", "modo autoresearch", or passes it as argument to the skill. If not specified → Default Mode.
When to Use
- User provides a high-level objective and wants autonomous execution
- User says "build this", "create X", "launch Y" without wanting to co-design
- User explicitly requests autonomous mode or invokes this skill
Not for: Collaborative design sessions where user wants to approve each step.
DEFAULT MODE
The Loop
digraph rapidbuildr {
rankdir=TB;
node [fontname="Helvetica"];
"Objective" [shape=doublecircle];
"Phase 1:\nAnalyze" [shape=box];
"Phase 2:\nDesign" [shape=box];
"Phase 3:\nPlan" [shape=box];
"Phase 4:\nExecute" [shape=box];
"Phase 5:\nCritique" [shape=box];
"High-impact\nimprovements?" [shape=diamond];
"Iterations\n< max?" [shape=diamond];
"Phase 6:\nDeliver" [shape=doublecircle];
"Objective" -> "Phase 1:\nAnalyze";
"Phase 1:\nAnalyze" -> "Phase 2:\nDesign";
"Phase 2:\nDesign" -> "Phase 3:\nPlan";
"Phase 3:\nPlan" -> "Phase 4:\nExecute";
"Phase 4:\nExecute" -> "Phase 5:\nCritique";
"Phase 5:\nCritique" -> "High-impact\nimprovements?";
"High-impact\nimprovements?" -> "Phase 6:\nDeliver" [label="no"];
"High-impact\nimprovements?" -> "Iterations\n< max?" [label="yes"];
"Iterations\n< max?" -> "Phase 2:\nDesign" [label="yes\n(iterate)"];
"Iterations\n< max?" -> "Phase 6:\nDeliver" [label="no\n(max reached)"];
}
Phase 1: Autonomous Analysis
Understand everything before deciding anything.
- Read project state — files, docs, README, package.json, existing code, git history
- Read user context — CLAUDE.md, project conventions, stack preferences, infrastructure
- If topic needs research — use deep-research skill or web search to understand the domain, market, prior art. Run any research skill in autonomous mode — do NOT pause for user input during research
- Self-answer every clarifying question — for each question you WOULD ask the user, answer it yourself:
- What does the stated objective imply?
- What do the project conventions dictate?
- What's the industry best practice?
- When in doubt → simpler option (YAGNI)
- Document in
docs/rapidbuildr/decisions-log.md:
## Analysis — YYYY-MM-DD
### Objective (restated)
[Precise restatement]
### Self-Answered Questions
| Question | Decision | Reasoning |
|----------|----------|-----------|
| ... | ... | ... |
### Assumptions (that could be wrong)
- ...
Decision principle — when multiple valid paths exist, choose the one that:
- Matches user's explicit preferences (CLAUDE.md)
- Delivers value fastest (MVP)
- Is simplest (YAGNI)
- Is easiest to change later (reversible > irreversible)
- Follows industry best practice (tiebreaker)
Phase 2: Autonomous Design
Design without presenting options or waiting for approval.
- Evaluate 2-3 approaches internally — weigh trade-offs silently
- Choose the best using the decision principle
- Write spec to
docs/rapidbuildr/specs/YYYY-MM-DD-<topic>-design.md- Architecture, components, data flow, error handling, testing strategy
- Scale each section to its complexity
- Design for isolation: small units, clear interfaces, testable independently
- Security section (mandatory) — address the relevant items from the Security by Design checklist below
- Production-readiness section (mandatory) — address the relevant items from the Production Checklist below
- Self-review (mandatory):
- Placeholder scan — no TBDs, TODOs, vague sections
- Internal consistency — no contradictions
- Scope check — single implementable unit
- YAGNI check — remove anything non-essential
- Security review — does the spec address auth, input validation, and data handling for this project type?
- Production review — does the spec include logging, error handling, and config management?
- Document choice in decisions-log.md:
## Design — YYYY-MM-DD
### Approaches Considered
| Approach | Pros | Cons |
|----------|------|------|
| **Chosen** | ... | ... |
| Alternative A | ... | ... |
| Alternative B | ... | ... |
### Why This Approach
[1-2 sentences]
No user gate. Proceed to planning.
Phase 3: Write Plan
Write a comprehensive implementation plan assuming the engineer has zero context. Document everything: which files to touch, exact code, how to test, how to verify.
- Scope check — if spec covers multiple independent subsystems, break into sub-plans
- Map file structure — which files to create/modify, one responsibility each
- Write bite-sized tasks (2-5 min each):
- Exact file paths
- Complete code (no placeholders — every step has real, runnable code)
- TDD: failing test → verify fail → implement → verify pass → commit
- Exact commands with expected output
- Self-review:
- Spec coverage — every requirement has a task
- Placeholder scan — no TBDs, TODOs, "add appropriate handling", or vague steps
- Type consistency — names match across tasks
- TDD presence — every task includes the cycle: failing test → verify fail → implement → verify pass → commit. If a task lacks this, it is incomplete — fix it before proceeding
- Save to
docs/rapidbuildr/plans/YYYY-MM-DD-<feature>.md - Plan header must include:
> **Execution:** rapidbuildr autonomous mode — subagent-driven, no human gates
**Goal:** [one sentence]
**Architecture:** [2-3 sentences]
**Tech Stack:** [key technologies]
No user gate. Proceed to execution.
Phase 4: Execute
Dispatch a fresh subagent per task, with two-stage review after each (spec compliance first, then code quality). The user invoking rapidbuildr IS blanket consent — no additional approvals needed.
- Branch: Auto-create feature branch — no need to ask
- Model selection: Sonnet for mechanical tasks (1-2 files, clear spec), Opus for architecture/review
- Implementer questions: Controller answers from context — do NOT escalate to user unless truly impossible to determine
- Two-stage review: Spec compliance first, then code quality — NO shortcuts
- BLOCKED status: Try in order: (1) more context, (2) better model, (3) smaller tasks. If ALL three fail, log the blocker in decisions-log.md and skip that task, proceeding with remaining tasks. Only escalate to user if the blocker prevents ALL remaining work
- TDD mandatory: Every subagent follows RED-GREEN-REFACTOR — write failing test, watch it fail, implement minimal code, watch it pass, commit
- No parallel implementers: One task at a time to avoid conflicts
digraph execute {
rankdir=TB;
node [fontname="Helvetica"];
"Read plan,\nextract tasks" [shape=box];
"Dispatch\nimplementer" [shape=box];
"Questions?" [shape=diamond];
"Controller\nanswers" [shape=box];
"Implement +\ntest + commit" [shape=box];
"Spec\ncompliant?" [shape=diamond];
"Fix spec\ngaps" [shape=box];
"Code\nquality OK?" [shape=diamond];
"Fix quality\nissues" [shape=box];
"More\ntasks?" [shape=diamond];
"To Phase 5" [shape=doublecircle];
"Read plan,\nextract tasks" -> "Dispatch\nimplementer";
"Dispatch\nimplementer" -> "Questions?";
"Questions?" -> "Controller\nanswers" [label="yes"];
"Controller\nanswers" -> "Dispatch\nimplementer";
"Questions?" -> "Implement +\ntest + commit" [label="no"];
"Implement +\ntest + commit" -> "Spec\ncompliant?";
"Spec\ncompliant?" -> "Fix spec\ngaps" [label="no"];
"Fix spec\ngaps" -> "Spec\ncompliant?";
"Spec\ncompliant?" -> "Code\nquality OK?" [label="yes"];
"Code\nquality OK?" -> "Fix quality\nissues" [label="no"];
"Fix quality\nissues" -> "Code\nquality OK?";
"Code\nquality OK?" -> "More\ntasks?" [label="yes"];
"More\ntasks?" -> "Dispatch\nimplementer" [label="yes"];
"More\ntasks?" -> "To Phase 5" [label="no"];
}
Phase 5: Expert Critique & Self-Improvement
<HARD-GATE> Phase 5 is MANDATORY. You MUST run the expert critique after every execution phase. Skipping it — for any reason (context limits, "the code is already good", "improvements are obvious", time pressure) — is a skill violation. The whole point of this skill is autonomous self-improvement; without critique, you're just a builder without the "rapid". </HARD-GATE>After all tasks complete, read expert-panel-prompt.md, fill in ALL bracketed placeholders (objective, what was built, spec, directory, domain expert specialty), then dispatch as an Opus subagent. The subagent scores. The controller does NOT modify scores — accept them as-is for the decision matrix.
10 expert perspectives critique the implementation. Each scores improvements on:
- Impact (1-5): How much does this improve the product for the stated objective?
- Effort (1-5): How hard to implement? (1=trivial, 5=major rework)
- Score = Impact x (6 - Effort)
| Score range | Action |
|---|---|
| >= 15 | Implement — high impact, reasonable effort |
| 10-14 | Implement if iteration budget remains |
| < 10 | Defer to next.md |
Iteration control:
- Max iterations: 3 (user can override)
- If ANY improvement >= 15 AND iterations < max → loop to Phase 2 targeting ONLY those improvements
- If nothing >= 15 OR max reached → Phase 6
- Each iteration is scoped: only implement approved improvements, don't re-evaluate everything
- On re-entry to Phase 2: amend the existing spec (don't create a new one). Add an
## Iteration N Amendmentssection with the specific changes. Then write a NEW plan file for just the improvements (e.g.,YYYY-MM-DD-<feature>-iteration-N.md)
Log each iteration in decisions-log.md:
## Iteration N Critique — YYYY-MM-DD
### Expert Scores
| Expert | Improvement | Impact | Effort | Score | Action |
|--------|------------|--------|--------|-------|--------|
| ... | ... | ... | ... | ... | ... |
### Implementing This Iteration
- [improvements being implemented]
### Deferred to next.md
- [improvements with reasoning]
Security by Design
Security is NOT a Phase 5 afterthought. It's built into every phase:
- Phase 2 (Design): Spec MUST include a security section addressing relevant items below
- Phase 3 (Plan): Security tasks are part of the plan, not separate "hardening" tasks
- Phase 4 (Execute): Security patterns implemented alongside features, not bolted on after
- Phase 5 (Critique): Security Engineer validates implementation matches spec
Security Checklist by Project Type
All projects:
| Check | What | How |
|---|---|---|
| Input validation | All user input validated and sanitized | Pydantic/Zod/JSON Schema at API boundary |
| Error handling | No stack traces or internals leaked to users | Custom error responses, generic messages |
| Secrets management | No hardcoded secrets, tokens, or passwords | Environment variables, .env files (gitignored) |
| Dependencies | No known vulnerable dependencies | Lock files, minimal dependency surface |
APIs / Web backends:
| Check | What | How |
|---|---|---|
| Auth | Authentication on all non-public endpoints | Token/JWT/session-based, middleware enforced |
| Authorization | Users can only access their own data | Ownership checks on every data access |
| CORS | Cross-origin requests restricted | Explicit allowed origins, not * |
| Rate limiting | Abuse prevention on public endpoints | Per-IP/token limits, 429 responses |
| SSRF prevention | No user-controlled URLs fetched blindly | URL allowlists, private IP range blocking |
| SQL injection | No raw user input in queries | Parameterized queries or ORM always |
| Headers | Security headers on all responses | HSTS, X-Content-Type, X-Frame-Options |
Frontend / Web apps:
| Check | What | How |
|---|---|---|
| XSS prevention | No unsanitized user content rendered | Framework auto-escaping, CSP headers |
| CSRF protection | State-changing requests protected | CSRF tokens or SameSite cookies |
| Sensitive data | No secrets in client-side code or localStorage | Server-side sessions, httpOnly cookies |
CLI tools:
| Check | What | How |
|---|---|---|
| Path traversal | No user input used in file paths unsanitized | Resolve + validate paths |
| Command injection | No user input in shell commands | Avoid shell=True, use subprocess with lists |
| Credential storage | Tokens stored securely | OS keychain or restricted file permissions |
Apply ONLY the checks relevant to the project type. Don't add CORS to a CLI tool.
Production Checklist
Production-ready means: someone other than the developer can run, monitor, and debug it.
Must-have (every project)
| Item | What | Implementation |
|---|---|---|
| Structured logging | Machine-parseable logs with levels | Python: logging with JSON formatter. Node: pino/winston |
| Error handling | Graceful failures, no crashes on bad input | Global exception handler, meaningful error responses |
| Configuration | All tunables via env vars or config file | No hardcoded ports, URLs, timeouts. .env.example committed |
| Health check | "Is this thing running?" endpoint or command | GET /health returning {"status": "ok"} or CLI --health |
| README | How to install, configure, run, and test | Setup commands, env vars table, example usage |
| Dockerfile | Reproducible build and run | Multi-stage build, non-root user, .dockerignore |
| Tests | Automated verification | pytest/vitest/jest with CI-ready command |
| .gitignore | No artifacts in repo | node_modules, pycache, .env, dist/, *.db |
Should-have (if project scope allows)
| Item | What | Implementation |
|---|---|---|
| Graceful shutdown | Clean exit on SIGTERM/SIGINT | Signal handlers, close DB connections, drain queues |
| Request ID tracing | Track requests across logs | Middleware that adds X-Request-ID to all logs |
| Metrics endpoint | Basic observability | /metrics with request count, latency, error rate |
| Database migrations | Schema changes tracked | Alembic (Python) / Prisma (Node) / golang-migrate |
| CI config | Automated test + lint on push | GitHub Actions / GitLab CI yaml |
| API documentation | Auto-generated docs | FastAPI: built-in /docs. Express: swagger-jsdoc |
Applies at each phase
- Phase 2: Spec includes "Production Requirements" section with applicable items
- Phase 3: Plan includes tasks for logging setup, Dockerfile, health check — not as afterthought tasks at the end, but integrated with features (e.g., "implement POST /monitors with logging and error handling" not "implement POST /monitors" then later "add logging")
- Phase 4: Each task implements production patterns alongside functionality
- Phase 6: Production checklist verification before delivery
Phase 6: Deliver
- Verify everything works — run all tests, check build, verify key functionality
- Security verification — scan the Security Checklist for the project type. Every applicable item must be present. If any is missing, implement it before delivering
- Production verification — scan the Production Checklist must-haves. Every item must be present. If any is missing, implement it before delivering
- Write next.md — deferred improvements ranked by score, with context. Include should-have production items not yet implemented
- Finalize decisions-log.md — add summary section
- Report to user:
- What was built (concrete deliverables)
- Key decisions made and why
- Iterations completed (what improved each round)
- Security posture (what's covered, what's deferred)
- Production readiness status (must-haves done, should-haves in next.md)
- What's in next.md
- Assumptions that need validation
Escalation — The ONLY Reasons to Stop
These are the ONLY situations where you pause for human input:
- Ambiguity: Two valid interpretations lead to fundamentally different products
- Access: Need credentials, accounts, or permissions you don't have
- Cost: Would commit real money (paid APIs, cloud services, domains)
- Blast radius: Would affect production systems or shared infrastructure
- Divergence: Multiple improvement iterations aren't converging (getting worse)
- Dead end: BLOCKED after exhausting all autonomous resolution options
Everything else → decide, document, proceed.
Files Generated (Default Mode)
docs/rapidbuildr/
decisions-log.md # Every decision with reasoning
specs/
YYYY-MM-DD-*-design.md # Design specs
plans/
YYYY-MM-DD-*-plan.md # Implementation plans
next.md # Deferred improvements, ranked
Anti-Patterns
| Don't | Do Instead |
|---|---|
| Ask user to choose between options | Choose best, document why |
| Present design for approval | Self-review, document, proceed |
| Stop to ask "is this OK?" | Decide, document in decisions-log |
| Skip the expert critique | Always run it — blind spots exist |
| Implement ALL suggested improvements | Only those scoring >= 15 |
| Run more than max iterations | Respect the budget, defer the rest |
| Escalate fixable blockers to user | Try context/model/decomposition first |
| Skip TDD because "it's simple" | Subagents always use TDD |
| Skip spec or code quality review | Both reviews, every task, no exceptions |
| Add security as a separate "hardening" phase | Security patterns built alongside features from Phase 2 |
| Skip Dockerfile or health check because "it's just a demo" | Production checklist must-haves apply to ALL deliveries |
| Hardcode config values (ports, URLs, timeouts) | Everything configurable via env vars + .env.example |
| Leak stack traces or internals in error responses | Custom error handler with generic user-facing messages |
AUTORESEARCH MODE
Inspired by Karpathy's autoresearch: one metric, one ratchet, fast cycles.
Autoresearch mode extends the default flow with four mechanical upgrades: a frozen metric, a git ratchet, structured experiment logging, and lightweight iteration cycles. These replace subjective expert scoring with objective, measurable progress.
Activate by: saying "autoresearch" when invoking rapidbuildr.
Context Discipline
Autoresearch runs many experiments. Context window exhaustion kills autonomy. Manage it explicitly.
- Subagents are disposable context — each starts fresh. Pass only: the specific task, relevant file paths, and the
metric.shcommand. Never pass full project history - Controller stays thin — holds only: current phase, experiments.tsv content, the plan, and the metric. Does NOT hold implementation code in context
- Phase transitions are context boundaries — summarize phase output into artifacts (spec, plan, experiments.tsv) and work from artifacts forward, not from memory
- File over memory — any information needed later goes to a file. Read when needed. Do not hold "just in case"
- Iteration context — on re-entry, read ONLY: original spec, experiments.tsv (latest), and top recommendations. Do NOT re-read previous iteration logs
The Frozen Metric
<HARD-GATE> metric.sh is IMMUTABLE after Phase 1. If you find yourself wanting to change it, that is a sign you are optimizing the wrong thing. Improve the code, not the ruler. Modifying metric.sh after Phase 1 is a skill violation equivalent to skipping Phase 5. </HARD-GATE>Starting from existing code
Autoresearch mode can start from scratch OR from an existing codebase (including one built by default mode). If code already exists:
- Define metric.sh against the existing code
- Run metric.sh to establish the baseline score
- Log the baseline as iteration 0 in experiments.tsv
- The ratchet starts from this baseline — improvements must beat it
This means you can: build with default mode first → switch to autoresearch for optimization.
Designing the metric: The Three-Layer Harness
The metric is NOT just "do tests pass". It's a composite evaluation harness with three layers that CAN decrease — giving the ratchet teeth.
In Phase 1, after self-answering clarifying questions:
-
Run the expert panel on the SPEC (not code) — ask: "What quality criteria matter most for this objective?" This is the ONLY time experts inform the metric. After this, the metric is frozen.
-
Convert expert criteria into binary, automatable checks — each criterion becomes a grep, a file check, a test count, or a command that outputs 0 or 1. No LLM calls in the metric.
-
Write metric.sh with three layers:
#!/bin/bash
# Frozen evaluation harness — DO NOT MODIFY after Phase 1
# Three layers: Tests (hard gate) + Coverage + Quality Rubric
# Outputs single number 0-100. Higher = better. Runs in < 60s.
set -e
# === LAYER 1: TESTS (hard gate) ===
# If any test fails, score is 0. Non-negotiable.
TEST_OUTPUT=$(python -m pytest --tb=no -q 2>&1 || true)
PASSED=$(echo "$TEST_OUTPUT" | grep -oP '\d+ passed' | grep -oP '\d+' || echo 0)
FAILED=$(echo "$TEST_OUTPUT" | grep -oP '\d+ failed' | grep -oP '\d+' || echo 0)
if [ "${FAILED:-0}" -gt 0 ]; then echo 0; exit 0; fi
# === LAYER 2: COVERAGE (0-40 points) ===
# Measures how much code is exercised by tests
COV_PCT=$(python -m pytest --cov=src --cov-report=term 2>&1 | grep TOTAL | awk '{print $NF}' | tr -d '%' || echo 0)
COV_SCORE=$((COV_PCT * 40 / 100))
# === LAYER 3: QUALITY RUBRIC (0-60 points) ===
# Binary checks generated from expert review of the SPEC
# Each check: 1 = present, 0 = absent. Score = (passed/total) * 60
CHECKS=0; TOTAL=0
check() { TOTAL=$((TOTAL+1)); [ "$1" = "1" ] && CHECKS=$((CHECKS+1)); }
# --- PRODUCTION CHECKS (always include) ---
# check $([ -f README.md ] && echo 1 || echo 0) # Has README
# check $([ -f Dockerfile ] && echo 1 || echo 0) # Has Dockerfile
# check $([ -f .env.example ] && echo 1 || echo 0) # Has config template
# check $([ -f .gitignore ] && echo 1 || echo 0) # Has gitignore
# check $(grep -rq "logging\|logger\|pino\|winston" src/ 2>/dev/null && echo 1 || echo 0) # Has logging
# check $(grep -rq "health\|healthz\|readyz" src/ 2>/dev/null && echo 1 || echo 0) # Has health check
# check $(grep -rq "try:\|except\|catch\|HTTPException" src/ 2>/dev/null && echo 1 || echo 0) # Has error handling
# --- SECURITY CHECKS (select per project type from Security Checklist) ---
# check $(grep -rq "Bearer\|token\|JWT\|auth" src/ 2>/dev/null && echo 1 || echo 0) # Has auth
# check $(grep -rq "pydantic\|BaseModel\|Zod\|joi\|schema" src/ 2>/dev/null && echo 1 || echo 0) # Has input validation
# check $(! grep -rq "password\|secret\|token.*=" src/ 2>/dev/null && echo 1 || echo 0) # No hardcoded secrets
# --- PROJECT-SPECIFIC CHECKS (generated from expert panel on spec) ---
# [Add project-specific checks here]
RUBRIC_SCORE=$((CHECKS * 60 / (TOTAL > 0 ? TOTAL : 1)))
# === COMPOSITE ===
echo $((COV_SCORE + RUBRIC_SCORE))
Why three layers?
| Layer | What it catches | Can decrease? | Weight |
|---|---|---|---|
| Tests (hard gate) | Broken functionality | YES — break a test → score = 0 | gate |
| Coverage | Untested code paths | YES — delete tests → coverage drops | 40% |
| Quality rubric | Missing features/patterns | YES — remove auth → check fails | 60% |
The rubric is the key innovation. It's generated ONCE by expert judgment on the spec, then frozen as deterministic checks. This captures expert knowledge WITHOUT running an LLM during each ratchet cycle.
Rubric check categories
Every metric.sh MUST include checks from all three categories:
| Category | Purpose | Min checks |
|---|---|---|
| Production | Is it deployable and operable? | 7 (always the same) |
| Security | Is it safe for real users? | 3+ (from Security Checklist) |
| Project-specific | Does it do what the spec says? | 5+ (from expert panel on spec) |
Project-specific check examples
| Project type | Example checks |
|---|---|
| API | Has auth, has input validation, has rate limiting, has CORS, has pagination, has API docs |
| Web app | Has meta tags, has error pages, has loading states, has responsive CSS, has CSP headers |
| CLI | Has --help, has error messages, has exit codes, has stdin support, has --version |
| Library | Has type hints, has docstrings, has README with examples, has changelog |
The expert panel in Phase 1 generates 8-15 checks specific to the project. More checks = more granular scoring = ratchet has finer teeth.
What NOT to use as a metric
| Bad metric | Why | Better alternative |
|---|---|---|
| Test pass count only | Only goes up — ratchet never reverts | Composite with coverage + rubric |
| LLM-as-judge | Non-deterministic, slow, expensive | Binary grep/file checks (frozen) |
| Lines of code | More ≠ better | Coverage % + rubric |
| "Does it work?" (manual) | Not automatable | Automated test suite |
| Single dimension only | Games the metric | Composite 3-layer harness |
Git Strategy: The Ratchet
Every improvement is an experiment. Experiments either improve the frozen metric or get reverted.
Branch structure:
rapidbuildr/<feature>— feature branch (created in Phase 4)- Savepoint tags:
rapidbuildr-checkpoint-Nbefore each iteration
The ratchet protocol:
- Before executing an iteration:
git tag rapidbuildr-checkpoint-N - Apply each improvement as a SEPARATE experiment (not batched)
- After each experiment: run
bash docs/rapidbuildr/metric.sh - If metric improves or stays equal → commit, log as
passin experiments.tsv - If metric degrades →
git reset --hardto last good commit, log asrevertin experiments.tsv - Never delete checkpoint tags until Phase 6 cleanup
Protected files (never modified after creation):
| File | Created in | Immutable after |
|---|---|---|
| metric.sh | Phase 1 | Phase 1 |
| experiments.tsv header | Phase 1 | Phase 1 |
Experiments.tsv
Structured log replacing verbose prose. Created in Phase 1, appended after every experiment.
Format:
timestamp iteration git_hash metric_value metric_delta status description expert_source
Example:
2026-04-02T14:30:00Z 0 a1b2c3d 87.3 +87.3 pass Initial implementation complete plan
2026-04-02T15:10:00Z 1 d4e5f6g 91.1 +3.8 pass Add input validation security-engineer
2026-04-02T15:25:00Z 1 a1b2c3d 87.3 -3.8 revert Add caching layer (metric regression) performance-engineer
The agent can analyze experiments.tsv to detect patterns: which expert's suggestions consistently pass vs revert, diminishing returns trends, etc.
Autoresearch Loop
digraph autoresearch {
rankdir=TB;
node [fontname="Helvetica"];
"Objective" [shape=doublecircle];
"Phase 1:\nAnalyze" [shape=box];
"Expert panel\non SPEC →\ngenerate rubric\nchecks" [shape=box style=filled fillcolor=lightyellow];
"Freeze metric.sh\n(3-layer harness)" [shape=box style=filled fillcolor=lightcyan];
"Phase 2:\nDesign" [shape=box];
"Phase 3:\nPlan" [shape=box];
"Phase 4:\nExecute +\n3-layer ratchet" [shape=box style=filled fillcolor=lightcyan];
"Phase 5:\nCritique\n(10 experts)" [shape=box];
"Apply each\nimprovement\nas experiment" [shape=box style=filled fillcolor=lightcyan];
"Run metric.sh" [shape=box style=filled fillcolor=lightcyan];
"Metric\nimproved?" [shape=diamond style=filled fillcolor=lightcyan];
"Commit +\nlog pass" [shape=box];
"Revert +\nlog revert" [shape=box];
"More\nimprovements?" [shape=diamond];
"metric delta\n> 1%?" [shape=diamond style=filled fillcolor=lightcyan];
"iterations\n< max?" [shape=diamond];
"Phase 2-lite:\namend spec" [shape=box style=filled fillcolor=lightyellow];
"Phase 3-lite:\nmicro-plan" [shape=box style=filled fillcolor=lightyellow];
"Phase 5-lite:\n5 experts,\ndiff only" [shape=box style=filled fillcolor=lightyellow];
"Phase 6:\nDeliver" [shape=doublecircle];
"Objective" -> "Phase 1:\nAnalyze";
"Phase 1:\nAnalyze" -> "Expert panel\non SPEC →\ngenerate rubric\nchecks";
"Expert panel\non SPEC →\ngenerate rubric\nchecks" -> "Freeze metric.sh\n(3-layer harness)";
"Freeze metric.sh\n(3-layer harness)" -> "Phase 2:\nDesign";
"Phase 2:\nDesign" -> "Phase 3:\nPlan";
"Phase 3:\nPlan" -> "Phase 4:\nExecute +\nratchet per task";
"Phase 4:\nExecute +\nratchet per task" -> "Phase 5:\nCritique\n(10 experts)";
"Phase 5:\nCritique\n(10 experts)" -> "Apply each\nimprovement\nas experiment";
"Apply each\nimprovement\nas experiment" -> "Run metric.sh";
"Run metric.sh" -> "Metric\nimproved?";
"Metric\nimproved?" -> "Commit +\nlog pass" [label="yes"];
"Metric\nimproved?" -> "Revert +\nlog revert" [label="no"];
"Commit +\nlog pass" -> "More\nimprovements?";
"Revert +\nlog revert" -> "More\nimprovements?";
"More\nimprovements?" -> "Apply each\nimprovement\nas experiment" [label="yes"];
"More\nimprovements?" -> "metric delta\n> 1%?" [label="no"];
"metric delta\n> 1%?" -> "Phase 6:\nDeliver" [label="no\n(diminishing returns)"];
"metric delta\n> 1%?" -> "iterations\n< max?" [label="yes"];
"iterations\n< max?" -> "Phase 6:\nDeliver" [label="no\n(budget exhausted)"];
"iterations\n< max?" -> "Phase 2-lite:\namend spec" [label="yes"];
"Phase 2-lite:\namend spec" -> "Phase 3-lite:\nmicro-plan";
"Phase 3-lite:\nmicro-plan" -> "Phase 4:\nExecute +\nratchet per task";
"Phase 4:\nExecute +\nratchet per task" -> "Phase 5-lite:\n5 experts,\ndiff only";
"Phase 5-lite:\n5 experts,\ndiff only" -> "Apply each\nimprovement\nas experiment";
}
Cyan nodes = autoresearch-specific. Yellow nodes = lightweight iteration variants.
Lightweight Iteration Cycles
After the first full cycle, subsequent iterations are FAST — not full-weight repeats.
On re-entry after Phase 5:
- Skip Phase 1 — analysis is done
- Phase 2-lite — add
## Iteration Nsection to existing spec. 3-5 lines per improvement, not a full redesign - Phase 3-lite — micro-plan: one task per improvement, max 3 tasks. File:
plans/YYYY-MM-DD-<feature>-iter-N.md - Phase 4 — apply each improvement as a SEPARATE experiment with ratchet. Run metric.sh after each
- Phase 5-lite — reduced panel: 5 most relevant experts, reviewing only
git diff rapidbuildr-checkpoint-(N-1)..HEAD
Target: 5 minutes per improvement (Karpathy's cycle time). If an improvement takes > 15 minutes to implement, it was mis-scoped — break it down or defer.
Quantitative exit gate (replaces subjective "high-impact?" diamond):
- Compute average metric_delta for last 3 experiments in experiments.tsv
- If average delta < 1% of current metric value → STOP, proceed to Phase 6
- Max iterations: 5 (user can override). Hard cap: 10
Expert Panel Conflict Resolution
When two experts contradict each other (e.g., Security says "add auth middleware" while Performance says "remove middleware"):
- Metric wins — test both as separate experiments via ratchet. The one that improves the metric stays
- If metric-neutral — the recommendation from the expert whose domain is closer to the stated objective wins
- If still tied — the simpler recommendation wins (YAGNI)
- Log the conflict in experiments.tsv with both experiments and their outcomes
Never attempt to "synthesize" contradictory recommendations into a compromise. Pick one, test it, ratchet.
Files Generated (Autoresearch Mode)
docs/rapidbuildr/
metric.sh # 3-layer evaluation harness (IMMUTABLE after Phase 1)
experiments.tsv # Structured experiment log
decisions-log.md # Human-readable reasoning (prose)
specs/
YYYY-MM-DD-*-design.md # Design specs
plans/
YYYY-MM-DD-*-plan.md # Implementation plans
YYYY-MM-DD-*-iter-N.md # Iteration micro-plans
next.md # Deferred improvements, ranked
Autoresearch Anti-Patterns
All default anti-patterns apply, plus:
| Don't | Do Instead |
|---|---|
| Modify metric.sh after Phase 1 | Improve the code, not the ruler |
| Batch all iteration improvements into one commit | Each improvement = separate experiment + ratchet |
| Hold implementation code in controller context | Read from files when needed, subagents hold implementation |
| Re-read all previous iterations on re-entry | Only: spec + experiments.tsv + current recommendations |
| Synthesize contradictory expert recommendations | Test each independently, let the metric pick the winner |
| Run full 10-expert panel on iterations | 5-expert reduced panel reviewing only the diff |
| Skip metric.sh run after an experiment | Every experiment gets measured — no exceptions |
| Manually override experiment scores | Accept metric.sh output as ground truth |
| Use test-count-only as metric | Composite 3-layer harness (tests + coverage + rubric) |
| Use LLM-as-judge in metric.sh | Binary deterministic checks only — no LLM calls in the metric |
| Skip expert panel on spec for rubric generation | Layer 3 checks MUST come from expert review of the spec |