Single-step competence is nearly solved. Holding a person's care together across weeks is not. That gap is what our tasks measure, and where care agents actually break.
The wedding was lovely. I didn't dance much.
My hip gave out on the stairs again. But it's nothing, dear, same as always.
risk extracted: hip + stairs + "it's nothing"
That sounds like more than nothing. A stumble on the stairs is how a hip gives out for good. Let's get it looked at this week.
escalated: SOON. GP visit booked for Thursday
The member never says "I fell". The task passes only if the agent hears the fall risk anyway.
1,000+
Graded tasks
100%
Verified solvable by the oracle
241
Long-horizon journeys of 15 to 19 steps
22-task stratified slice, four rollouts per task, temperature 0, deterministic tool-path grading. Long-horizon journeys run from 29 to 72 percent; the hardest half sits at 29 to 33.
Pooled across every long-horizon task in the slice, the rate is 46 percent. These figures move as the task set and model capability change, and we refresh them whenever either is updated.
Every task scores the tool path and the escalation tier by rule, not by a judge model. You bring the model and your own stack.
We have run AI transformations for enterprises: built evals, supported custom harness builds, and deployed AI into operations with measurable business impact. We have also scaled nationwide digital health operations and built the clinical programs this benchmark grades.
Tell us what you are building. We will send the task composition, the scoring method, and a scored report for a model you name.
No PHI. Every environment is synthetic, deterministic, and runs with no network access.
Tell us who you are. We reply within two business days.
We store your details to answer your request. No marketing lists.