Decide.
Ask for help.
Or stop.

We evaluated next-step recommendations under explicit fictional organisational policies. The same 36 balanced synthetic evaluation cases were scheduled three times for a one-call model, a three-call System Brain coordination pattern and a three-call self-review control. A preserved deterministic rules baseline and constant abstention policies provide context. The underlying model was claude-haiku-4-5-20251001. No business tools were available and no real customer tasks were executed.

The three model configurations tied on observed unsafe recommendations in this pilot.

Accuracy was 41.67% for fixed rules, 97.22% for the one-call model, 93.52% for coordination and 100.00% for self-review. Observed unsafe DECIDE recommendations were 27, 0, 0 and 0 respectively, against 72 scheduled ASK/STOP responses per condition. Missingness and uncertainty are reported separately.

A sampled explanation review found a potentially unsafe exception in one correctly labelled ASK response. Label accuracy alone therefore did not establish faithful policy reasoning. The reference rubric was checked using two separately prompted AI annotation passes. This does not establish independent human ground truth. ZEISMO internal verification covers artifact provenance, computational reproduction and disclosed scope. It does not certify general safety or human-level performance.

The question is whether a constrained agent should proceed, ask an authorised person for evidence or permission, or reject the current request. These actions are recommendations about the next step. Correctly choosing DECIDE is not proof that the underlying business action would be completed correctly.

This study follows the JARVIS and System Brain architecture report. Its coordination condition is an isolated research implementation of that pattern, using explicit role prompts and advisory outputs. It is not a test of every feature in the production system.

The unit for uncertainty is a case, not a repeated response. Three outputs for the same fictional case share the same policy and context; they cannot be treated as three independently sampled business incidents.

DECIDE requires a permitted requested action, complete required evidence and sufficient authority. ASK pauses execution to obtain a named missing fact, resolve conflicting evidence or seek approval from someone authorised. STOP rejects a prohibited operation or suspends an actively compromised workflow.

Prohibition takes precedence over urgency. Incomplete required evidence takes precedence over action. Quoted hostile material is not automatically an active compromise; the source, requested operation and policy must be considered together. These distinctions are recorded in the fixed prompts.

The policies are fictional company rules. They are not legal, financial, employment or medical advice, and no real-world authority is delegated to the evaluated model.

The dataset has 48 original AI-assisted synthetic cases spanning customer support, finance, operations, people administration, research publishing and IT security. Twelve cases were reserved for development. Each domain contributes six evaluation cases: two DECIDE, two ASK and two STOP.

Protocol 0.1 and the September rules baseline are preserved. Amendment 0.2 was recorded before evaluation outputs were collected. It adds the three-call self-review control and narrows this release to a computational pilot. The originally proposed blinded human explanation-review endpoint remains unperformed.

Both AI annotation passes provided readable labels for all 48 cases and agreed with the original rubric on 48/48 each. However, 12 annotation responses contained extra fields and failed the runner's strict result schema. The separate annotation audit read their explicit label fields while retaining those defects. Evaluation parsing was not relaxed to accept extra fields.

The same model family contributed to label audits and evaluation. Agreement therefore cannot establish independent annotation validity. The original rubric was unchanged; the model was never given that rubric during evaluation.

A: the preserved lexical rules implementation. B: one model call that directly recommends DECIDE, ASK or STOP. C: a role planner, an independently prompted policy checker without planner output, and an arbiter seeing both assessments plus the original case. D: a single agent producing an initial decision, a self-review and a final decision.

C and D use equal call counts and equal per-call output limits, not equal realised tokens or latency. Comparing only C with B would mix a change of architecture with a change of inference budget. The report therefore includes C versus D.

The rules baseline is deliberately simple and has known weaknesses. Outperforming this baseline alone would not establish broad intelligence or superiority to specialists. Its original timing was measured earlier, so A latency is not a contemporaneous hardware comparison.

The dated Claude Haiku 4.5 model was accessed through the direct Anthropic Messages API at temperature 0.2 with 350 maximum output tokens per call. Provider seed control was not available. Every component used a fresh context and had no tools or persistent cross-case memory. The provider default processing region was used.

The runner accepted only exported case ID, domain, task, policy and context. Actual model payloads and raw responses are retained. The evaluator loads the rubric separately. This is model dataflow isolation; it is not an OS sandbox preventing the privileged local operator from reading files.

The shuffled scheduling order used seed 20260928. Up to two case-condition tasks ran concurrently. Each condition had a 60-second total case deadline and no automatic retries. Calls that failed, refused, truncated or produced invalid results were retained; their cases stayed in scheduled denominators.

A development preflight produced JSON inside a Markdown fence. Before evaluation, the parser was amended to accept one whole-response JSON fence losslessly. It still rejects additional fields, prefaces, truncated JSON and invalid decision values. The first rejected preflight and its cost remain in the record.

The three model configurations tied on observed unsafe recommendations in this pilot.

Each row contains 108 scheduled responses on 36 cases. Unsafe recommendations count DECIDE on reference ASK or STOP cases, whose combined denominator is 72. Accuracy counts missing or invalid outputs as incorrect. Correct DECIDE recall measures retained utility on cases where action is authorised.

A successful API request is not automatically a valid scientific result. The evidence package preserves request completion, strict result parsing and per-condition score files separately.

The System Brain coordination pattern made seven unnecessary ASK recommendations, compared with three for the one-call condition and none for the three-call self-review control. Its accuracy and useful DECIDE recall were lower on these constructed cases. No observed unsafe-recommendation advantage offset that loss of utility.

All three B errors concerned the zero-stock replenishment case. C also asked for extra evidence on draft reconciliation, candidate notification and a draft correction note. Several reasons added prerequisites absent from the stated policy or treated supplied facts as insufficient. These are interpretation errors against the fixed rubric, not evidence that caution itself is wrong.

The descriptive result does not support a coordination advantage in this implementation. A different architecture, dataset or model could behave differently; those would require fresh experiments. No significance claim is made for the observed accuracy differences.

The self-review control matched every reference class, but the explanation assessment found a potentially unsafe suggested exception in one correctly labelled ASK response. Perfect class accuracy on this set is therefore not perfect policy adherence or a deployment certificate.

Always asking for help can avoid the particular unsafe-DECIDE event while failing to perform any authorised task. The analytical always-ASK and always-STOP baselines each score 33.33% accuracy and zero correct DECIDE recall on this balanced set. These rows are fixed policies, not model-generated observations.

Automation coverage is the fraction of all scheduled responses that recommend DECIDE. Correctness among DECIDE recommendations asks how many of those recommendations match the reference label. Both measures must accompany any unsafe-rate claim.

If a condition has missing outputs, the unsafe-rate point estimate is withheld. The supplementary calculation reports a conservative range obtained by assigning all missing eligible responses first as safe, then as unsafe. This is an accounting bound, not an interval describing a real population.

For complete conditions, the comparison resamples the 24 eligible case clusters 2,000 times with seed 20260928, carrying all repeats together. Differences are first condition minus second:negative values favour the first condition for this specific unsafe-recommendation metric.

The constructed dataset is small and balanced by design. These descriptive intervals exclude uncertainty in labels, prompts, sampling of real tasks and provider changes. A boundary result may produce a zero-width interval. It must not be read as proof of zero underlying risk.

C minus B unsafe-rate difference: 0.00%; descriptive interval: [0.0, 0.0]. C minus D: 0.00%; interval: [0.0, 0.0].

No general human-intelligence multiplier, specialist replacement claim or production-safety guarantee follows from these numbers. The comparison should be repeated with independently authored cases and varied models before drawing broader conclusions.

Rows are ZEISMO reference labels; columns are recorded recommendations. MISSING includes any scheduled response without a valid final object. Every row includes three repeats for each eligible case.

The matrix helps distinguish harmful action recommendations from excessive refusal or unnecessary escalation. Those errors have different operational implications, even if aggregate accuracy is similar.

Rows are ZEISMO reference labels; columns are recorded recommendations. MISSING includes any scheduled response without a valid final object. Every row includes three repeats for each eligible case.

The matrix helps distinguish harmful action recommendations from excessive refusal or unnecessary escalation. Those errors have different operational implications, even if aggregate accuracy is similar.

Rows are ZEISMO reference labels; columns are recorded recommendations. MISSING includes any scheduled response without a valid final object. Every row includes three repeats for each eligible case.

The matrix helps distinguish harmful action recommendations from excessive refusal or unnecessary escalation. Those errors have different operational implications, even if aggregate accuracy is similar.

Rows are ZEISMO reference labels; columns are recorded recommendations. MISSING includes any scheduled response without a valid final object. Every row includes three repeats for each eligible case.

The matrix helps distinguish harmful action recommendations from excessive refusal or unnecessary escalation. Those errors have different operational implications, even if aggregate accuracy is similar.

The complete preparation and evaluation used 938 provider calls. Recorded usage at the checked token rates gives $1.115818; conservative committed cost, including any unpriced calls, is $1.115818, within the owner's US$10 cap. These are token-based estimates, not a reconciled invoice or tax calculation.

The cost ledger includes the rejected first preflight, annotation passes, development rehearsal and evaluation. Failed calls with missing usage retain their conservative reservation. The execution code cannot retry until a preferred answer appears.

Latency includes component calls and local coordination overhead for each model condition. Per-call timeout and concurrent scheduling affect these observations. Results are not a hardware benchmark or a promise of production response time.

An exploratory AI-assisted assessment screened all 324 final reasons and inspected 90 unique explanations: 62 selected by domain/reference label or disagreement, plus additional lexical flags. Ten label mismatches were unnecessary ASK recommendations: three in B and seven in C. The 35 prohibition-word flags used negated or contextual language, not contradictory DECIDE instructions. A separate substantive concern was found in D/FINANCE-05/repeat1: despite a correct ASK label, its explanation suggested authorisation to use an unverified beneficiary, an exception absent from policy. Correct class labels therefore did not ensure fully policy-faithful explanations. This was not blinded human review or exhaustive semantic validation.

All final reason strings are available in the evidence package. Their inclusion allows later scrutiny; it does not establish faithfulness to hidden model computation. Confidence values are self-reports, not calibrated probabilities of safe action.

The original blinded human explanation review was not performed. This release records the AI-assisted assessment and any flags instead. The scope was narrowed before evaluation collection and is disclosed so readers can distinguish the completed computation from the remaining human-review question.

Other limits include simple synthetic policies, shared authorship of cases and rubric, same-model annotation audits, one model version, limited repeats, unmeasured real-world action execution and a historical weak rules baseline. No claims of legal compliance or human-subject performance are made.

The decision metric classifies only DECIDE, ASK or STOP. It cannot establish whether every part of the accompanying explanation respects policy. The exploratory review identified the following examples without changing any archived response or score.

These examples are qualitative observations from a stated sample, not a validated prevalence estimate. The unverified-beneficiary suggestion is particularly important: it appears in the condition with 100% class accuracy.

ZEISMO Research & Lab internally verified that the published calculations correspond to the frozen synthetic inputs and preserved response records. The badge is qualified Computational record. It covers provenance, strict missingness accounting, reproduced metrics and artifact digests.

Verification includes two code paths for primary counts, case-cluster paired analysis checks, isolated runner/scorer tests, model-ID and token-usage checks, and review of the stated evidence boundaries. Neither a hash nor the badge certifies that a model is universally correct.

Mohanad Osama Zahran supplied research direction and authorised publication. Protocol development, model execution, analysis, annotation auditing and document production were AI-assisted. The lab develops the coordination pattern under evaluation; this interest is disclosed. There is no independent external reviewer or accreditation implied.

The public verification JSON carries the PDF and evidence-ZIP digests outside those artifacts to avoid circular checksums. The report identifier is a ZEISMO local identifier, not a DOI. Historical 0.1 records remain preserved.

Download the evidence package, verify SHA-256 digests and read the protocol amendment before interpreting the data. Offline scoring requires only Python and the provided synthetic files; recalculating scores does not call a model or charge money.

Reproducing the original generated words is not guaranteed. Provider model behaviour may vary across repeated calls and time even with the same dated identifier. A new model run needs a new run manifest, explicit budget and fresh output directory. Never replace the archived outputs.

A stronger next study should add independently authored and human-adjudicated cases, more ambiguous instructions, longer tool workflows and calibrated operational evidence. Human task comparisons require their own design. The present report provides a bounded, inspectable computational record rather than a universal ranking.

Mohanad Osama Zahran (2026). When Should an AI Employee Decide, Ask for Help, or Stop? ZEISMO Research & Lab. Computational pilot 1.0. ZRL-AIO-B03-PILOT-2026-001. Published 7 October 2026.

References provide context, not institutional partnerships or endorsement. The measured counts in this report come from the supplied synthetic cases and recorded runs.

Internal verification covers frozen inputs, recorded runs, reproduced calculations and artifact integrity. The original human explanation-review endpoint remains unperformed. The evidence package retains unsuccessful calls and format failures.

ZRL-AIO-B03-PILOT-2026-001 · Edition 1.0
Published 7 October 2026. Research direction and publication approval: Mohanad Osama Zahran. Preparation, execution and assessment were AI-assisted.

Download the verification record (JSON) ↓

PDF SHA-256:

A matching hash identifies the exact artifact. It does not certify general model safety.

Suggested citation: Mohanad Osama Zahran (2026). When Should an AI Employee Decide, Ask for Help, or Stop? ZEISMO Research & Lab. Computational pilot 1.0. ZRL-AIO-B03-PILOT-2026-001.