Evaluation
Score the system, not the vibes.
Benchmarks target attention allocation, plan feasibility, critique recall, and provenance completeness. Leaderboard chat scores are out of scope.
Suites
Named suites ship with CELESTRA research lineage and an open harness — do not invent marketing numbers.
Attention
Budget adherence under competing goals.
Decomposition
Task-graph fidelity vs gold hierarchies.
Critique
Recall of seeded material faults before accept.
Provenance
Percent of answers with resolvable citations.
Reporting
Results publish as versioned artifacts — reproducible inputs, pinned stack versions, no silent prompt edits.
Format
JSON + human summary in Observatory.
Compare
Diff runs across AetherOS versions.
Private
Enterprise suites stay in-tenant.