ANSWER-ONLY BOUNDARY
Grade the response.
- Returned answer
- Reference comparison
- Answer grade
03 / evidence over impressions
Private Evaluation
An answer-only benchmark can grade the response while missing what the agent did. Environment-grounded evaluation records actions and checks their consequences against explicit rules and task predicates.
Private Evaluation starts with your agent interface, authorized environment and evaluation question. We agree held-out tasks, allowed tools, run conditions, failure scenarios and reporting boundaries before execution.
Current GameOps evidence demonstrates dataset verification, not measured performance of a customer agent. Commercial measurement capabilities are configured and validated against the agreed environment before a score is reported.
No real chatbot evaluations have been approved for publication yet.
Inspect the three offers ↗ · WFRB v0 methodology & results ↗ · Explore the design partner program ↗
ANSWER-ONLY BOUNDARY
ENVIRONMENT-GROUNDED BOUNDARY
A comparison of instrument boundaries, not an executed customer benchmark.
| Question | Answer-only benchmark | Environment-grounded evaluation |
|---|---|---|
| What is checked? | The returned answer against a reference or rubric. | Actions, transitions and final state against environment rules. |
| What may be missed? | Invalid actions or side effects behind a convincing response. | Behavior outside the instrumented environment and defined checks. |
| What is retained? | An answer and its grading evidence. | A replayable trajectory, verifier outcome and scoped provenance. |
These dimensions are not a claim that a customer agent has already been scored.
| Dimension | Current evidence / proposed measurement | Boundary |
|---|---|---|
| Task success | Canonical episode final-state predicates. | Dataset verification today; agent success rate requires actual runs. |
| Action validity | Invalid moves and argument constraints in controlled fixtures. | Validated only for the implemented action schema. |
| Trajectory efficiency | Agreed step, time or cost budgets. | Configurable; no existing efficiency score claimed. |
| Failure handling | Recorded failure types and failure states. | Live tool failures require environment-specific instrumentation. |
| Recovery | Replay checked recovery trajectories. | Agent recovery measurements require controlled failure trials. |
| Constraint compliance | Explicit environment rule checks. | Client constraints need implementation and negative tests. |
| State integrity | Initial/final states and retained failure state. | External side effects require an agreed observation boundary. |
| Reproducibility | Reset, replay and provenance gates. | External services may remain nondeterministic. |