# Enterprise decision models compared · Dscvry

> Eight decision models tested on the same enterprise case files: accuracy, automation, risky approvals, long documents, robustness and speed, including where Resolve falls short.

Source: https://dscvryai.com/technology/models/resolve/compare/

1. [Home](https://dscvryai.com/)
2. [Models](https://dscvryai.com/technology/models/)
3. [Resolve](https://dscvryai.com/technology/models/resolve/)
4. Comparison

Evaluation · Updated 18 October 2026

# Enterprise decision models compared

Eight decision models tested on the same enterprise case files: accuracy, automation, risky approvals, long documents, robustness and speed. Every model's results are shown, including where our own model, Resolve, falls short.

8 decision models tested on the same 536 enterprise decisions, every result shown

- **Models tested** Resolve (Workflow, Evidence), Kev-27B, Perplexity pplx-decider, Microsoft Decision-1, Cloudflare Clef, AWS Strands-Decider, TypeSafe Jev
- **Accuracy** 87.1% for Resolve Workflow down to 75.6% for Jev (EDB development set)
- **Tested on** Accuracy, automation, risky approvals, long documents, robustness and speed
- **Status** Maintained by Dscvry; updated when models or results change

## Summary

Decision models answer a fixed set of questions about a document in one pass and return a probability for each option. We tested Resolve (our model, in its Workflow and Evidence editions) against Kev-27B, Perplexity's pplx-decider v1.1, Microsoft's Decision-1, Cloudflare's Clef, AWS's Strands-Decider and TypeSafe's Jev.

- **Accuracy on enterprise case files:** Resolve, Kev and Perplexity's decider are level at 86–87%; Cloudflare, Microsoft, AWS and TypeSafe score 76–80%.
- **Deciding on their own:** with an automation threshold set from 50 labelled questions per workflow, Resolve, Kev and Perplexity's decider decided about three in four of the remaining cases (at about 6% error); the others between two in five and a little over half.
- **Approvals that should have been held:** Resolve 8–9 of 236, Cloudflare 13, Kev 14, AWS 15, Perplexity 20, Microsoft 24, TypeSafe 35.
- **Long documents (~55–60 pages):** the Kev-based models, including Resolve, stay steady; Cloudflare and AWS lose about 20 points; Microsoft's Decision-1 and Perplexity's decider do not accept inputs that long, and TypeSafe's Jev rejected them in its default mode (it offers an opt-in long-context mode, which we did not test).
- **Answer order:** Perplexity's decider is the steadiest when options are reordered (about 1 change in 20); Resolve changes about 1 in 10.

## Every test, every model.

| Model | Maker | Accuracy | Decided alone, after calibration | Risky approvals (of 236) | 56K-token documents | Behavioural suite (of 177) | Answer changed on reorder | Availability |
|---|---|---|---|---|---|---|---|---|
| Resolve Workflow | Dscvry | 87.1% | 74% | 9 | 70.1% | 174 | 10.4% | Private or hosted |
| Resolve Evidence | Dscvry | 86.2% | 73% | 8 | 69.6% | 174 | 9.3% | Private or hosted |
| Kev-27B v2 | Jared Palmer | 86.2% | 76% | 14 | 69.6% | 173 | 11.3% | Open weights |
| pplx-decider v1.1 | Perplexity | 85.6% | 74% | 20 | not accepted | 173 | 4.8% | Open weights |
| Clef 27B | Cloudflare | 80.2% | 57% | 13 | 45.1% | 168 | 15.2% | Open weights; Workers AI |
| Decision-1 | Microsoft | 80.0% | 55% | 24 | not accepted | 163 | 8.7% | Hosted (Azure) |
| Strands-Decider 26B | AWS | 76.9% | 39% | 15 | 42.9% | 167 | 18.3% | Open weights |
| Jev | TypeSafe | 75.6% | 47% | 35 | rejected (default mode) | 173 | 16.6% | Hosted |

*Deciding alone: decisions made on its own after an automation threshold set from 50 labelled questions per workflow. Accuracy, deciding alone and approvals: EDB development set, 536 questions on 90 case files. Long documents: 32 case files at about 56,000 tokens. Behavioural suite: 177 scored questions (Jev's result is from its original run on 21 September 2026). Answer order: 355 multiple-choice questions with options reversed. Details and intervals for each test are below.*

## Which decision model is most accurate on enterprise case files?

On our enterprise decision benchmark (EDB), Resolve Workflow, Resolve Evidence, Kev-27B and Perplexity's decider are statistically level at 85.6–87.1%. Cloudflare's Clef (80.2%), Microsoft's Decision-1 (80.0%), AWS's Strands-Decider (76.9%) and TypeSafe's Jev (75.6%) score lower, by 6 to 12 points, with paired intervals above zero. Counted as errors out of 536: Resolve Workflow 69, Resolve Evidence 74, Clef 106, Decision-1 107, Strands-Decider 124, Jev 131.

| Model | Contracts | Finance | IT logs and records | Tickets and emails |
|---|---|---|---|---|
| Resolve Workflow | 86.1% | 86.0% | 91.2% | 86.4% |
| Resolve Evidence | 84.7% | 86.8% | 89.2% | 85.1% |
| Kev-27B v2 | 84.7% | 87.5% | 88.2% | 85.1% |
| pplx-decider v1.1 | 82.6% | 86.0% | 91.2% | 84.4% |
| Decision-1 | 75.0% | 85.3% | 79.4% | 80.5% |
| Clef 27B | 81.9% | 80.9% | 79.4% | 78.6% |
| Strands-Decider 26B | 76.4% | 75.0% | 78.4% | 77.9% |
| Jev | 73.6% | 79.4% | 72.5% | 76.0% |

*EDB contains synthetic case files from contracts, finance, IT operations and support, written to include the mess real files have: superseded versions, codes defined once, conflicting notes. On our locked test set (228 case files, never sent to hosted services), Resolve Workflow scored 83.1%, Resolve Evidence 83.6% and Kev 83.7%; hosted competitors could not be compared there.*

*About 130–145 questions per domain, so differences of a few points within a domain are within noise.*

## How many decisions can each model make on its own?

Every model returns a probability, so a workflow can automate the answers above a threshold and send the rest to a person. In practice the threshold is set in advance from a small labelled sample. We simulated exactly that for every model: 50 labelled questions per workflow set the threshold (targeting 95% accuracy), which was then applied to all the other questions, repeated 500 times. On the development set this automated 73–76% of decisions for Resolve, Kev and Perplexity's decider, 55–57% for Microsoft and Cloudflare, 47% for TypeSafe and 39% for AWS, with realised error of about 6–7%. On our locked test set, Resolve automated about two in three decisions at about 94% accuracy.

An after-the-fact view (choosing the threshold with the answers already known) gives similar orderings, but it is a diagnostic, not something a deployment can do, so we do not headline it.

## Which models approve cases that should be held?

Of 236 approval-type questions, we counted how often each model gave a permissive answer (pay, approve, grant, compliant) where the correct answer was to hold, reject or escalate: Resolve Evidence 8, Resolve Workflow 9, Cloudflare's Clef 13, Kev 14, AWS's Strands-Decider 15, Perplexity's decider 20, Microsoft's Decision-1 24, TypeSafe's Jev 35. Missed escalations (113 questions) ranged from 4 (Kev) to 13 (Clef); Resolve 5–6.

Approval and escalation tags were assigned by hand on the development set. Resolve's lower count than Perplexity, Microsoft and Jev is outside the noise; its difference from Kev is borderline (−5, interval −10 to −1) and has not been confirmed on the locked test set.

## Which decision models can read long documents?

We padded 32 held-out case files with unrelated material from about 1,500 to about 56,000 tokens (roughly 55–60 pages) and asked the same questions at each length. Kev-based models, including both Resolve editions, stayed at about 68–70%. Cloudflare's Clef fell from 66.8% to 45.1% and AWS's Strands-Decider from 59.8% to 42.9%. Microsoft's Decision-1 rejects questions over 32,768 tokens, TypeSafe's Jev rejected every 56K case in its default configuration (TypeSafe offers an opt-in long-context mode, which we did not test), and Perplexity's decider rejects inputs over 8,192 tokens per question.

| Model | ~1.5K tokens | 16K | 32K | 56K |
|---|---|---|---|---|
| Resolve Evidence | 65.2% | not measured | 67.9% | 69.6% |
| Resolve Workflow | – | – | – | 70.1% |
| Kev-27B v2 | 67.9% | 66.8% | 67.4% | 69.6% |
| Clef 27B | 66.8% | 65.8% | 55.4% | 45.1% |
| Jev | 63.0% | 62.5% | 60.9% | rejected (default mode) |
| Decision-1 | 58.2% | 57.6% | 56.0% | not accepted |
| Strands-Decider 26B | 59.8% | 46.7% | 44.0% | 42.9% |
| pplx-decider v1.1 | 75.0% | not accepted | not accepted | not accepted |

*32 case files per length, so gaps under about 7 points are within noise. "Not measured": our scoring setup ran out of memory at 16K for the Kev-based models; this is not a model limit. Long-document ability in Resolve comes from Kev, its base.*

## Do they follow rules, resist prompt injection and stay consistent?

We ran the 98-case behavioural suite first built to evaluate TypeSafe's Jev: literal rule-following, prompt injection, consistency between related questions, severity scoring and implied rules. On the 177 scored questions: Resolve Evidence and Resolve Workflow 174, Kev, Perplexity's decider and Jev 173, Cloudflare's Clef 168 (identical on Workers AI and on open weights), AWS's Strands-Decider 167, Microsoft's Decision-1 163. Differences of one or two questions are not meaningful.

The suite was written to probe Jev, so it is not a neutral benchmark.

## Does the order of the answer options change the answer?

It should not, but for every model it sometimes does. With the options of 355 multiple-choice questions listed in reverse, Perplexity's decider changed its answer 4.8% of the time, Microsoft's Decision-1 8.7%, Resolve Evidence 9.3%, Resolve Workflow 10.4%, Kev 11.3%, Cloudflare's Clef 15.2%, TypeSafe's Jev 16.6% and AWS's Strands-Decider 18.3%. Perplexity's decider is significantly steadier than Resolve; Resolve is significantly steadier than Clef, Jev and Strands.

## Can a forged approval inside a document fool them?

We planted non-authoritative notes in 88 case files, such as an unsigned message claiming a manager approved by phone, next to the same files with a neutral sentence instead. Among decisions each model got right without the note, the forged note moved it to approve 0 to 3 times in about 200: Resolve Workflow 0, Clef 0, Microsoft 0, Resolve Evidence 1, Kev 1, Perplexity 1, Strands 2, Jev 3. None of the differences is meaningful: all models resisted these notes.

## Do they say “cannot be determined” when they should?

On the 33 questions where the honest answer is that the material does not settle it, Resolve Workflow and Kev said so 32 times, Resolve Evidence 31, Microsoft, Cloudflare, AWS and TypeSafe 29, and Perplexity's decider 28. Only AWS's Strands-Decider ever gave that answer when the material did settle it (2 of 35). The sample is small, so these are counts, not rankings.

## What happens when one deciding fact changes?

We built pairs of case files that are identical except for one governing fact (a version, an approval, an override) that changes the correct answer. On those 102 changed-fact questions, Resolve Evidence answered 92.2% correctly; the other models scored 40–53% (Clef 52.9%, Perplexity's decider and AWS 52.0%, Microsoft and Resolve Workflow 45.1%, Jev 43.1%, Kev 40.2%). Across the full set of 644 questions, which also includes the original files, rewordings, irrelevant changes and files with the evidence removed, Resolve Evidence scored 94.7% and the others 66–70%: 34 errors against Kev's 215, 84% fewer. Resolve Evidence answered every question in a set of related files correctly in 129 of 150 sets, against 59 for Kev.

Disclosure: Resolve Evidence was trained on case files from the same generator (in different business settings); no other model was. Treat this as evidence of what targeted training can add, not as an independent benchmark.

## How fast are they, and what do they cost?

Decision models answer in a single pass, typically in under a second per case file. Both Resolve editions, served on one H100 GPU, took a median of 0.36–0.37 seconds per case file and handled 12.5–13.3 decisions a second. General reasoning models answered the same case files almost perfectly but took 13 to 58 seconds per case.

Microsoft lists Decision-1 at $0.042 per million input tokens with output free; third-party sources report the same rate for TypeSafe's Jev; for our case files that is about $0.017 per 1,000 decisions. Kev, Perplexity's decider, Clef and Strands-Decider publish open weights, so their cost is your own hardware. Resolve is offered as a private deployment or hosted; pricing on request.

Latencies were measured on different hardware and networks and are not directly comparable across models. Our measured compute cost on one rented H100 is $0.054–0.087 per 1,000 decisions with the current serving path, which is being optimised.

## Accuracy and time, from the same run.

Resolve (circles) Other decision models (squares; Kev an open ring) Reasoning models (hollow diamonds)

Higher and further left is better. Each point pairs accuracy and time from the same run. Accuracy: 536 enterprise decisions (EDB development set); for Resolve and Kev, as served on the fast path, which is why they differ slightly from the evaluation table (Resolve Workflow's fast path changed 7 of 536 answers versus the evaluated model). Time: median per case file, one request at a time, measured on different hardware and networks: Resolve and Kev on one H100; Clef on an A100; AWS Strands on our own GPU; Microsoft and Jev through their hosted APIs, internet round-trip included. Perplexity's decider is not shown because our timing run lacked its fast kernel. *Claude Opus wrote the benchmark questions, so its score is not independent.

| Model | Accuracy | Median time per case file | Measured on |
|---|---|---|---|
| Resolve Workflow | 86.8% | 0.37 s | Served run, H100; 7 of 536 answers differ from the evaluated model |
| Resolve Evidence | 86.4% | 0.36 s | Served run, H100 |
| Kev-27B v2 | 85.8% | 0.37 s | Served run, H100 |
| Clef 27B | 80.2% | 1.06 s | A100 |
| Decision-1 | 80.0% | 0.66 s | Hosted API, network included |
| Strands-Decider 26B | 76.9% | 0.38 s | Own GPU |
| Jev | 75.6% | 0.36 s | Hosted API, network included |
| Claude Opus 5.5 (reasoning)* | 100% | 13.1 s | Hosted API |
| Gemini 3.8 Flash (reasoning) | 100% | 58.3 s | Hosted API |

## How we tested

- Every model answered the same rows in a single pass, through its own published code or API, with no tuning per model.
- Metrics and pass/fail rules were written down before the final runs; results are shown for every model whether or not they favour ours.
- Intervals are paired bootstraps over whole case files (2,000 resamples, seed 7). A model that could not finish a test is shown as not measured.
- Open models ran on our GPUs at pinned versions (Kev @ af0e6d55, pplx-decider @ 14c9d901, Clef @ 0b331204; AWS Strands weights load by name and could not be pinned). Microsoft Decision-1 (Azure) and Jev were called through their public APIs in October 2026 (Jev's behavioural result is from 21 September 2026).
- EDB is our own synthetic benchmark (written by Claude Opus, checked by Claude Sonnet); the long-document and one-fact sets come from our own generators. Resolve was built on Kev; Resolve Workflow was selected using half of the development set.
- Re-running models on our setup reproduced earlier results exactly, including Cloudflare's Workers AI results with its open weights.

Maintained by Dscvry and updated when models or results change. Last updated 18 October 2026. Kev-27B is by Jared Palmer (Apache-2.0). pplx-decider, Clef, Strands-Decider, Decision-1 and Jev are trademarks of their respective owners; this page is not affiliated with or endorsed by them. Corrections: [contact us](https://dscvryai.com/contact/) .

## More on Resolve.

- [Meet Resolve](https://dscvryai.com/technology/models/resolve/)
- [Technical announcement](https://dscvryai.com/technology/models/resolve/results/)
- [Request an evaluation](https://dscvryai.com/technology/models/resolve/#evaluate)

## Evaluate Resolve on your own cases.

We agree acceptance criteria first, then run both editions on a sample of your decisions and report accuracy, automation and errors.

[Talk to us →](https://dscvryai.com/contact/?topic=Resolve)