Validate and audit quality of AI agent slash commands (.claude/commands/*.md) — checks YAML frontmatter, description length, jargon detection, directive language, and duplicate detection. Use when creating, editing, or auditing project commands and skills. Run proactively after adding new commands.

Skill Quality Evaluator

Validates .claude/commands/*.md files for quality, triggering accuracy, and best practices from Phil Schmid's (Google DeepMind) eval harness methodology.

What it checks

YAML frontmatter — every command must start with --- block containing description:
Description length — minimum 40 characters (short descriptions don't trigger well)
Jargon detection — flags internal terms that confuse users (dogfood, --dangerously, etc.)
Directive language — descriptions must contain action verbs (Run, Fix, Scan, Extract, etc.) because passive descriptions don't trigger Claude's skill routing
Duplicate detection — similar commands must have distinct descriptions (first 30 chars normalized)

How to run

Execute the bundled script — it requires zero dependencies (pure Node.js):

node .claude/skills/skill-quality-eval/scripts/run-eval.js [commands-dir]

Default commands-dir: .claude/commands (relative to cwd)
Pass an absolute path to scan a different directory

The script outputs a structured report with PASS/FAIL per file, summary stats, and specific fix suggestions.

When a check fails

Check	How to fix
Missing frontmatter	Add `---\ndescription: "..."\n---` at top of file
Description too short	Expand to 40+ chars — explain WHEN to use and WHAT it does
Jargon detected	Replace internal terms with user-facing language
No directive verb	Add action verbs: "Run...", "Fix...", "Scan...", "Use when..."
Duplicate description	Make the first 30 chars unique between similar commands

Customization

Edit scripts/run-eval.js to add:

Custom jargon patterns (line ~15, JARGON_PATTERNS array)
Additional directive verbs (line ~30, DIRECTIVE_PATTERNS array)
Differentiation pairs (line ~50, DIFF_PAIRS array)

Integration with CI

The script exits with code 1 if any check fails, making it suitable for pre-commit hooks or CI pipelines:

node .claude/skills/skill-quality-eval/scripts/run-eval.js || echo "Fix command quality issues!"

Background

Based on Phil Schmid's (Google DeepMind) eval harness methodology. See references/phil-schmid-eval-harness.md for the full framework and best practices.