Evidence confidence
moderate. A described comparison supports a bounded conclusion. Workload transfer and independent reproduction remain unresolved.
What was observed
Help a coding agent investigate and repair a controlled incident.
Ten SREGym-Lite problems, five attempts per condition. The same Luna agent gets optional Jev planning and submission-review tools.
Baseline
The same agent without Jev tools.
Finding
The authors report 24/50 successful attempts with Jev versus 20/50 without it. Two problems regressed.
- Observed passes: 24/50 with Jev; 20/50 without Jev.
What the result does not establish
A small experiment, without a diagnosis-time comparison. Jev sometimes accepted temporary recovery as a durable repair and could not rank an absent correct hypothesis.
What we would test in System One
Evaluate diagnostic candidates and repair invariants separately. Keep executable checks and the agent responsible for the repair.
This recommendation is our interpretation of the study. Related research does not establish the quality of every Engine recipe.
Primary sources
Read this record in System One Bench. Source commits are pinned where available. Review dates describe our inspection, not the original run date.
Metric definitions and review method · Submit a correction or new result