A human example
An agent says a release is live, but the output only confirms a successful build.
What the caller supplies
The caller supplies the exact claim and current build and deployment evidence.
What happens next
Jev returns separate support and contradiction signals. The agent checks missing evidence before stating success.
Illustrative example, not a recorded result.
Potential value: high
A short check can catch premature completion claims in recurring agent workflows.
Evidence confidence: moderate
Our small diagnostic and a controlled SRE study support testing the pattern. Both expose errors; neither establishes general reliability.
The rating describes support for this claim. It is separate from Jev's returned probability. How we assign ratings.
Evidence, including disagreement
- Batching two evidence checks. first-party. Fewer calls and tokens, with one extra error in this small test.
- Separate signals from a final verdict. reported. Question design helped, but a simple rule was a strong baseline.
- Choose diagnostic tests and review repair evidence. reported. More attempts passed in one small study; some incidents regressed.
The next test
60 pre-labeled claims covering stale outputs, partial completion, missing evidence and explicit contradictions.
Compare against
- Parse structured status fields
- Direct agent review
- Separate Jev questions
Measure
- False confirmations
- Review rate
- Final response accuracy
- Additional latency
Decision after the test
Do not default to automatic acceptance. A useful adviser must reduce false confirmations without hiding uncertain cases.
The report will retain inputs, question versions, every attempt and failure examples. We will update the confidence rating after reviewing the result.
Use a related Engine recipe
Recipes are implementation starting points. Their presence does not mean the protocol above has passed.