# Models may judge. Code must guarantee. The boundary I use when an AI product must be helpful without being allowed to break a promise. - **Canonical URL:** https://vidhata.me/notes/models-may-judge-code-must-guarantee - **Author:** Vidhatanand V. (Vid) - **Role:** Fractional CTO and AI Systems Architect **Published:** 01 Sep 2026 | **Reading time:** 9 minute read ## Why I hold this position I have become less interested in asking whether a model is intelligent enough. The question I ask first is whether the surrounding system knows what the model is allowed to decide. That change sounds small, but it changes the architecture of an AI product. A model is useful precisely because it can work with ambiguity. It can read an imperfect sentence, infer intent, recognise an object, compare weak signals, or draft a plausible response. The same quality makes it a poor place to enforce a promise. Consent, money, permissions, hard safety rules, irreversible actions, and critical state changes cannot depend on what seems likely in one model response. ## The problem appears when helpfulness becomes authority. Language models are trained to continue, answer, and help. In many products that instinct is valuable. In a sensitive conversation it can become the risk. A model may correctly detect distress and still offer an intervention at the wrong time. It may repeat the offer after a person has declined. It may sound confident while moving beyond the boundary the product has promised to respect. I saw the same shape of problem in very different systems. A planning product can use AI to understand preferences, but eligibility and safety rules need a versioned deterministic layer. A computer-vision model can estimate whether a component is present, but the conditions that stop a line, preserve evidence, and return control to an operator need explicit states. The model judges. The system decides what that judgement is permitted to change. ## I separate interpretation, policy, and action. The cleanest design I know has three boundaries. The interpretation layer turns messy input into a narrow structured judgement. The policy layer applies rules the organisation is willing to defend. The action layer changes the world through a tool, a workflow, a message, or a machine. Each boundary has a contract, and each transition can be inspected. This does not mean surrounding every model call with hundreds of brittle if statements. It means being precise about the promise. If the promise is that a declined intervention will not be repeated, store the decision and enforce it in code. If a recommendation must never include an ineligible option, apply the eligibility rule before generation. If an industrial action needs two signals and an operator acknowledgement, represent that sequence as a state machine rather than hoping a prompt remembers it. ## The boundary also makes the product easier to improve. When everything lives inside a prompt, a failure becomes an argument about wording. When judgement and guarantees are separated, the team can ask a better question. Was the interpretation wrong, was the policy incomplete, or did the action layer violate its contract? Evaluation becomes more useful because each layer can fail for a different reason. The design also keeps providers replaceable. A stronger model may improve interpretation next month, but it should not silently rewrite the rules around permissions, budgets, escalation, or consent. Product behaviour remains stable while intelligence can change behind a typed capability boundary. ## Human control is a designed state, not an apology. Teams sometimes treat human review as temporary scaffolding that will disappear when the model gets better. I treat it as a product decision. Some judgements have a human consequence that cannot be reduced to a confidence score. The system should know when to pause, what evidence to present, who can act, and how the eventual decision is recorded. The practical lesson is simple. Use AI where ambiguity is the work. Use deterministic software where the product makes a promise. Keep a person in control where neither layer can honestly carry the consequence alone. That is how I make an AI product more useful without pretending uncertainty has disappeared. ## What I take into the next build - Write down the promises the product must never improvise. - Give model outputs narrow, typed meanings before policy or tools consume them. - Test interpretation, policy, and action as separate failure surfaces. - Make human review an explicit state with evidence and ownership. ## Continue reading [All Fractional CTO field notes](https://vidhata.me/notes) ## Work with Vid Start with the actual technical pressure: [bring the problem](https://vidhata.me/hire).