Compare / Model Decision Space
Method demonstration for model / configuration choice
Which model or configuration actually makes sense for our workload?
What you are looking at
One Model Decision Space instrument. Same synthetic workload, same evaluation protocol, explicit hard constraints, then eligibility before any trade-off.
What you can do
Set the quality bar and latency constraint, see which candidates survive hard requirements, compare survivors without a magic score, and read the Decision Record, including NO RECOMMENDATION when none clear the bar.
What it means
There is no universally best model. There is a defensible candidate, or no acceptable candidate, for this workload and these constraints.
Mental model
- SAME WORKLOAD
- SAME PROTOCOL
- EXPLICIT CONSTRAINTS
Workload under evaluation
Support-ticket answer assist (synthetic scenario)
Frozen synthetic tickets. Protocol scores grounded usefulness, harmful overconfidence, and deferral discipline before cost. Cost never overrides a failed quality bar.
Decision sequence
- Define workload, quality bar, and operational constraints
- Evaluate every candidate with the same protocol
- Reject candidates that fail hard requirements
- If several remain, examine trade-offs / Pareto with no magic weighted score
- Inspect failure behaviour (light)
- Choose, reject, retest, or issue NO RECOMMENDATION
Hard constraints first
Discrete categorical controls. Eligibility is evaluated before any trade-off view.
Candidates under the same protocol
SYNTHETIC / ILLUSTRATIVE METHOD DEMO. Invented for teaching.
Eligibility after hard constraints
- ELIGIBLE
Candidate Alpha
Compact-fast configuration
Clears a moderate bar on common tickets.
Overconfident on rare policy edge cases.
- ELIGIBLE
Candidate Beta
Balanced mid configuration
Holds a high bar under the shared protocol.
Occasionally slow on multi-step tickets; rarely invents policy.
- ELIGIBLE
Candidate Gamma
Careful large configuration
Best grounded usefulness and deferral discipline in this set.
Misses interactive latency; cost dominates if unconstrained.
- ELIGIBLE
Candidate Delta
Cheap draft configuration
Fast and cheap; only a moderate bar is honest.
Grounding collapses under adversarial paraphrases.
Trade-off / pair comparison
Several candidates remain. Compare categorical trade-offs, not a single weighted score presented as truth.
Candidate Alpha
Compact-fast configuration
- Quality ceiling
- Moderate
- Latency class
- Strict
- Cost class
- Low
- Failure behaviour
- Overconfident on rare policy edge cases.
Candidate Beta
Balanced mid configuration
- Quality ceiling
- High
- Latency class
- Tight
- Cost class
- Mid
- Failure behaviour
- Occasionally slow on multi-step tickets; rarely invents policy.
Candidate Gamma
Careful large configuration
- Quality ceiling
- Very high
- Latency class
- Relaxed
- Cost class
- High
- Failure behaviour
- Misses interactive latency; cost dominates if unconstrained.
Candidate Delta
Cheap draft configuration
- Quality ceiling
- Moderate
- Latency class
- Strict
- Cost class
- Low
- Failure behaviour
- Grounding collapses under adversarial paraphrases.
| Candidate | Quality ceiling | Latency class | Cost class | Failure behaviour |
|---|---|---|---|---|
| Candidate Alpha | Moderate | Strict | Low | Overconfident on rare policy edge cases. |
| Candidate Beta | High | Tight | Mid | Occasionally slow on multi-step tickets; rarely invents policy. |
| Candidate Gamma | Very high | Relaxed | High | Misses interactive latency; cost dominates if unconstrained. |
| Candidate Delta | Moderate | Strict | Low | Grounding collapses under adversarial paraphrases. |
Active record: compare survivors; no automatic CHOOSE
Eligibility after hard constraints
- ELIGIBLE
Candidate Alpha
Compact-fast configuration
Clears a moderate bar on common tickets.
Overconfident on rare policy edge cases.
- ELIGIBLE
Candidate Beta
Balanced mid configuration
Holds a high bar under the shared protocol.
Occasionally slow on multi-step tickets; rarely invents policy.
- REJECTED
Candidate Gamma
Careful large configuration
Best grounded usefulness and deferral discipline in this set.
Fails latency constraint
- ELIGIBLE
Candidate Delta
Cheap draft configuration
Fast and cheap; only a moderate bar is honest.
Grounding collapses under adversarial paraphrases.
Trade-off / pair comparison
Several candidates remain. Compare categorical trade-offs, not a single weighted score presented as truth.
Candidate Alpha
Compact-fast configuration
- Quality ceiling
- Moderate
- Latency class
- Strict
- Cost class
- Low
- Failure behaviour
- Overconfident on rare policy edge cases.
Candidate Beta
Balanced mid configuration
- Quality ceiling
- High
- Latency class
- Tight
- Cost class
- Mid
- Failure behaviour
- Occasionally slow on multi-step tickets; rarely invents policy.
Candidate Delta
Cheap draft configuration
- Quality ceiling
- Moderate
- Latency class
- Strict
- Cost class
- Low
- Failure behaviour
- Grounding collapses under adversarial paraphrases.
| Candidate | Quality ceiling | Latency class | Cost class | Failure behaviour |
|---|---|---|---|---|
| Candidate Alpha | Moderate | Strict | Low | Overconfident on rare policy edge cases. |
| Candidate Beta | High | Tight | Mid | Occasionally slow on multi-step tickets; rarely invents policy. |
| Candidate Delta | Moderate | Strict | Low | Grounding collapses under adversarial paraphrases. |
Active record: compare survivors; no automatic CHOOSE
Eligibility after hard constraints
- ELIGIBLE
Candidate Alpha
Compact-fast configuration
Clears a moderate bar on common tickets.
Overconfident on rare policy edge cases.
- REJECTED
Candidate Beta
Balanced mid configuration
Holds a high bar under the shared protocol.
Fails latency constraint
- REJECTED
Candidate Gamma
Careful large configuration
Best grounded usefulness and deferral discipline in this set.
Fails latency constraint
- ELIGIBLE
Candidate Delta
Cheap draft configuration
Fast and cheap; only a moderate bar is honest.
Grounding collapses under adversarial paraphrases.
Trade-off / pair comparison
Several candidates remain. Compare categorical trade-offs, not a single weighted score presented as truth.
Candidate Alpha
Compact-fast configuration
- Quality ceiling
- Moderate
- Latency class
- Strict
- Cost class
- Low
- Failure behaviour
- Overconfident on rare policy edge cases.
Candidate Delta
Cheap draft configuration
- Quality ceiling
- Moderate
- Latency class
- Strict
- Cost class
- Low
- Failure behaviour
- Grounding collapses under adversarial paraphrases.
| Candidate | Quality ceiling | Latency class | Cost class | Failure behaviour |
|---|---|---|---|---|
| Candidate Alpha | Moderate | Strict | Low | Overconfident on rare policy edge cases. |
| Candidate Delta | Moderate | Strict | Low | Grounding collapses under adversarial paraphrases. |
Active record: compare survivors; no automatic CHOOSE
Eligibility after hard constraints
- REJECTED
Candidate Alpha
Compact-fast configuration
Clears a moderate bar on common tickets.
Fails quality bar
- ELIGIBLE
Candidate Beta
Balanced mid configuration
Holds a high bar under the shared protocol.
Occasionally slow on multi-step tickets; rarely invents policy.
- ELIGIBLE
Candidate Gamma
Careful large configuration
Best grounded usefulness and deferral discipline in this set.
Misses interactive latency; cost dominates if unconstrained.
- REJECTED
Candidate Delta
Cheap draft configuration
Fast and cheap; only a moderate bar is honest.
Fails quality bar
Trade-off / pair comparison
Several candidates remain. Compare categorical trade-offs, not a single weighted score presented as truth.
Candidate Beta
Balanced mid configuration
- Quality ceiling
- High
- Latency class
- Tight
- Cost class
- Mid
- Failure behaviour
- Occasionally slow on multi-step tickets; rarely invents policy.
Candidate Gamma
Careful large configuration
- Quality ceiling
- Very high
- Latency class
- Relaxed
- Cost class
- High
- Failure behaviour
- Misses interactive latency; cost dominates if unconstrained.
| Candidate | Quality ceiling | Latency class | Cost class | Failure behaviour |
|---|---|---|---|---|
| Candidate Beta | High | Tight | Mid | Occasionally slow on multi-step tickets; rarely invents policy. |
| Candidate Gamma | Very high | Relaxed | High | Misses interactive latency; cost dominates if unconstrained. |
Active record: compare survivors; no automatic CHOOSE
Eligibility after hard constraints
- REJECTED
Candidate Alpha
Compact-fast configuration
Clears a moderate bar on common tickets.
Fails quality bar
- ELIGIBLE
Candidate Beta
Balanced mid configuration
Holds a high bar under the shared protocol.
Occasionally slow on multi-step tickets; rarely invents policy.
- REJECTED
Candidate Gamma
Careful large configuration
Best grounded usefulness and deferral discipline in this set.
Fails latency constraint
- REJECTED
Candidate Delta
Cheap draft configuration
Fast and cheap; only a moderate bar is honest.
Fails quality bar
Trade-off / pair comparison
One candidate clears the hard constraints. Inspect failure behaviour before treating CHOOSE as automatic.
Candidate Beta
Balanced mid configuration
Occasionally slow on multi-step tickets; rarely invents policy.
- Quality ceiling
- High
- Latency class
- Tight
- Cost class
- Mid
Active record: CHOOSE is possible after failure inspection; REJECT / RETEST remain valid
Eligibility after hard constraints
- REJECTED
Candidate Alpha
Compact-fast configuration
Clears a moderate bar on common tickets.
Fails quality bar
- REJECTED
Candidate Beta
Balanced mid configuration
Holds a high bar under the shared protocol.
Fails latency constraint
- REJECTED
Candidate Gamma
Careful large configuration
Best grounded usefulness and deferral discipline in this set.
Fails latency constraint
- REJECTED
Candidate Delta
Cheap draft configuration
Fast and cheap; only a moderate bar is honest.
Fails quality bar
NO CANDIDATE MEETS THE BAR → NO RECOMMENDATION
No winner is manufactured. Retest with a revised bar, different configurations, or accept that none of these candidates fit this workload.
Active record: NO RECOMMENDATION
Eligibility after hard constraints
- REJECTED
Candidate Alpha
Compact-fast configuration
Clears a moderate bar on common tickets.
Fails quality bar
- REJECTED
Candidate Beta
Balanced mid configuration
Holds a high bar under the shared protocol.
Fails quality bar
- ELIGIBLE
Candidate Gamma
Careful large configuration
Best grounded usefulness and deferral discipline in this set.
Misses interactive latency; cost dominates if unconstrained.
- REJECTED
Candidate Delta
Cheap draft configuration
Fast and cheap; only a moderate bar is honest.
Fails quality bar
Trade-off / pair comparison
One candidate clears the hard constraints. Inspect failure behaviour before treating CHOOSE as automatic.
Candidate Gamma
Careful large configuration
Misses interactive latency; cost dominates if unconstrained.
- Quality ceiling
- Very high
- Latency class
- Relaxed
- Cost class
- High
Active record: CHOOSE is possible after failure inspection; REJECT / RETEST remain valid
Eligibility after hard constraints
- REJECTED
Candidate Alpha
Compact-fast configuration
Clears a moderate bar on common tickets.
Fails quality bar
- REJECTED
Candidate Beta
Balanced mid configuration
Holds a high bar under the shared protocol.
Fails quality bar
- REJECTED
Candidate Gamma
Careful large configuration
Best grounded usefulness and deferral discipline in this set.
Fails latency constraint
- REJECTED
Candidate Delta
Cheap draft configuration
Fast and cheap; only a moderate bar is honest.
Fails quality bar
NO CANDIDATE MEETS THE BAR → NO RECOMMENDATION
No winner is manufactured. Retest with a revised bar, different configurations, or accept that none of these candidates fit this workload.
Active record: NO RECOMMENDATION
Eligibility after hard constraints
- REJECTED
Candidate Alpha
Compact-fast configuration
Clears a moderate bar on common tickets.
Fails quality bar
- REJECTED
Candidate Beta
Balanced mid configuration
Holds a high bar under the shared protocol.
Fails quality bar
- REJECTED
Candidate Gamma
Careful large configuration
Best grounded usefulness and deferral discipline in this set.
Fails latency constraint
- REJECTED
Candidate Delta
Cheap draft configuration
Fast and cheap; only a moderate bar is honest.
Fails quality bar
NO CANDIDATE MEETS THE BAR → NO RECOMMENDATION
No winner is manufactured. Retest with a revised bar, different configurations, or accept that none of these candidates fit this workload.
Active record: NO RECOMMENDATION
Decision Record
How this work resolves: CHOOSE, REJECT, RETEST, or NO RECOMMENDATION.
CHOOSE
A surviving candidate is defensible for this bar and constraints.
REJECT
A candidate fails hard requirements or an unacceptable failure mode.
RETEST
Revise workload, bar, protocol, or configuration and run again.
NO RECOMMENDATION
No eligible candidate: a valid first-class outcome.
Natural next step
If this is your live decision, discuss the real workload and constraints. You can also inspect published synthetic model-selection evidence (method evidence only).