How I Build Projects

Anti-Sycophancy Checklist

• Use narrow scales with anchored descriptions (1-5, not 1-10)
• Avoid aspirational labels (“perfect” → use “natural”)
• Replace booleans with 3+ level enums (true/false → irrelevant/somewhat/directly_relevant)
• Add explicit calibration instructions: “Score independently. Do not default to positive scores.”
• Consider multi-model rotation to diversify biases

Prompt Hardening

Don't just tell the AI what to do - tell it what NOT to do, and what to do when there's nothing to do.

• Entity-type boundaries: Define clear category fences so the LLM doesn't misclassify entities across schema fields (e.g., a law name landing in a location field, a category ending up as a tag).
• Exclusion categories: Explicitly list what should NOT be extracted (e.g., boilerplate, definitions, metadata that isn't actionable data).
• Absence handling: When data is missing from the source, the model must output NOT FOUND instead of fabricating plausible values. Without this, hallucinated dates, names, or IDs silently corrupt the database with realistic-looking but invented data.
• Over-extraction guards: Set expected output ranges so the model knows when it's being too aggressive (e.g., “this input type typically yields 10-20 items, not 50”).

Prompt Distillation

Use a strong model to teach cheap models - no fine-tuning required.

1. Run a strong model (e.g., Claude Sonnet) once on the hardest pipeline step
2. Study its output - extract the structural reasoning patterns it discovered
3. Encode those patterns as explicit guidance in cheaper models' system prompts
4. Result: cheap model follows the same playbook at 40-120x cost reduction
5. Instantly reversible - it's just prompt text, not model weights

5a. Pipeline Decomposition

Break every AI pipeline into discrete steps where each step produces a reviewable artifact:

Why decomposition works:

1. Each step produces a reviewable artifact - no more black boxes
2. Each step can use a different model (expensive where quality matters, cheap where mechanical)
3. Errors are localized - if the output is wrong, check the plan; if the plan is wrong, check the extraction
4. Human checkpoint separates “what did the AI understand?” from “what will it do?”

5b. Entity Reconciliation

A stateless pipeline will silently corrupt shared databases. If your pipeline writes to a database, it MUST know what's already there.

• Pre-query existing records before LLM processing
• Pass existing records as context so the LLM can match against them (catches typos, abbreviations, name variations)
• Run deterministic conflict detection (date overlaps, duplicate entities, overwrites)
• Human reviews flagged conflicts before execution
• Matched IDs flow downstream so the plan reuses existing records instead of creating duplicates

5c. Per-Step Model Selection

Make model choice a first-class concern, not an afterthought:

5d. Prompt Caching

For pipelines that run repeatedly with the same system prompts / schema context:

• Cache static context (system prompts, schema definitions, few-shot examples) across runs
• Only variable content (the actual input) changes per call
• OpenRouter supports this natively; ~40% cost reduction on repeated runs
• The more the pipeline runs, the cheaper it gets

6a. Architecture Decisions

Make and document these choices before writing app code:

6b. Frontend-First with Mock Data

1. Build the complete UI with mock data before connecting to backend
2. Include deliberately negative/edge-case mock data to test all UI states
3. Validate the full user workflow experience end-to-end with mocks
4. Only then connect to the real backend

6c. Human-in-the-Loop Splits

Split long-running pipelines into phases with human checkpoints:

• User reviews and approves intermediate results before proceeding
• User can exclude bad results before they pollute downstream steps
• This builds trust - the system shows its work

6d. Sandbox / Production Isolation

AI-generated actions must never touch production without human approval.

• Playground mode (SQLite): full execution, reset/seed, sample data for testing
• Production mode (PostgreSQL): review only, execution disabled, no reset capability
• Auto-detect from DATABASE_URL - no manual toggle needed
• This is a core architectural decision, not a convenience feature

6e. Dual-Database Strategy (when needed)

Use two databases when analytics must survive resets:

• Main DB (DATABASE_URL): Domain data - SQLite in dev, PostgreSQL in prod
• Analytics DB (ANALYTICS_DATABASE_URL): Always PostgreSQL - feedback ratings, token usage, cost logs persist even when main DB is reset in playground mode

8a. Evaluation Framework

1. Combine automated checks (fire every run) with human-in-the-loop judgments
2. Embed eval UI into the existing workflow - don't make testers fill out separate forms
3. Define pass criteria up-front for each eval
4. Store eval data linked to traces (e.g., via trace_id)
5. Automated evals fire silently; human evals are lightweight (thumbs up/down or Good/Partial/Bad)

8b. Feedback-Driven Progressive Autonomy (Trust Lifecycle)

Human ratings aren't just for measurement - they're the mechanism for earning autonomy:

The feedback loop closes the circle: Human ratings → model comparison → confidence thresholds → progressive autonomy. Without the ratings infrastructure, there's no path from Phase 1 to Phase 3.

10a. Business Script (docs/script_business.md)

Audience: Non-technical - marketers, founders, hiring managers, potential users.
Length: 5-7 minutes.
Tone: Conversational, honest, minimal jargon. Do NOT oversell.

Rules:

• Do NOT oversell - be honest about what it is and isn't
• Keep technical details minimal - save for the technical video
• Credit reference projects / inspiration
• Include [PLACEHOLDER] tags for screen recordings and examples to fill during recording
• Include [SCREEN: ...] cues for what to show on screen at each moment

10b. Technical Script (docs/script_technical.md)

Audience: Hiring managers evaluating decision-making, system thinking, and AI-augmented development. Also: engineers who want to understand the architecture.
Length: 15-20 minutes.
Tone: Honest narrative. Claude Code wrote most of the code - what matters is the sequence of decisions, problems spotted, architecture shaped, and interventions that turned AI output into a working product.
Style: Storytelling through the build journey - each section is a problem → decision → result.

Rules:

• Each section: show the problem, show your decision, show the result, move on
• Reference [DIAGRAM: ...] tags for architecture diagrams (build a separate diagrams_technical.html)
• Include a git commit table showing the evolution commit by commit
• Include technical stats (LOC, endpoints, components, LLM calls per run, dev time)
• Don't apologize for using AI tools - the value is in the decisions, not the keystrokes
• The “My Role vs. AI's Role” section is critical - it demonstrates what you actually bring to the table

10c. Technical Diagrams Page (docs/diagrams_technical.html)

A self-contained HTML page opened alongside the technical video. Each diagram maps to a script section via [DIAGRAM: ...] cues.

Structure:

• Sticky TOC at top with color-coded links to each diagram
• One card per diagram, each with title + script section reference
• Cards scroll independently - click TOC to jump

Diagram types to include (adapt per project):

Design rules:

• Pure HTML + inline CSS - no external dependencies, works offline
• Clean, professional look: white cards on light gray, rounded corners, subtle shadows
• Color-coded by section (blue for data, purple for pipeline, red for problems, green for solutions, amber for features)
• Print-friendly: @media print removes shadows, adds borders
• Use scroll-margin-top on cards so TOC jumps land cleanly below the sticky nav
• End with “The Pattern” callout - the same statement from the technical script

10d. Optional: Marketing Script (docs/script_marketing.md)

A read-along script for screen recording the live app. Shorter (5 min), heavily demo-driven, minimal talking. Copy-paste examples provided inline for each demo scenario. Include rapid-fire section showing range of inputs.