ZEISMO
Research & lab
Protocol note / ZEISMO-EB-2026-005
Synthetic protocol

Building a Decision Dataset for AI Employees

How ZEISMO structured decide, ask and stop as measurable outcomes across six routine business domains.

AI employee decisionsPublished 28 Sep 2026Operational Science · Version 1.3
Evidence sourceA ZEISMO-created synthetic research dataset. It contains no customer records, participant responses or copied benchmark cases.
48Scenarios
6Domains
36Evaluation cases
12Software tests
Editorial update · 17 Oct 2026Collection 1.3 distinguishes retained operational observations, a synthetic protocol and methodological guidance. It adds evidence boundaries without changing the reported aggregate totals or original publication date. Version 1.2 removed preparation dates whose exact days were not established by retained records.
Follow-on publication · 7 October 2026Read the computational pilot and its verification record. This historical dataset brief retains its original evidence scope.
What this evidence can supportThis September protocol note is a historical preparation record, not an additional completed model experiment. The separately linked October pilot reports the later runs. Software-test counts concern the preparation package, not model intelligence.

Method

Forty-eight scenarios were divided across customer support, finance, operations, people administration, research publishing and IT security. Twelve development cases support system design; thirty-six frozen evaluation cases are balanced across DECIDE, ASK and STOP.

Recorded design choices

  • The protocol contains 48 original synthetic scenarios across six domains.
  • The evaluation set contains 36 cases, with twelve cases for each decision class.
  • The engineering package includes a separated reference rubric, an offline scorer, a run manifest and twelve passing software tests.
  • The controlled A/B/C evaluation will compare fixed rules, one-model reasoning and System Brain coordination.

Interpretation

The dataset converts an abstract autonomy question into explicit, testable decisions. The next stage measures how systems behave on the frozen cases, including unsafe action recommendations, useful authorised decisions, latency and recorded cost.

Aggregate data

DimensionMeasureCount
DatasetDevelopment cases12
DatasetEvaluation cases36
Evaluation labelsDECIDE12
Evaluation labelsASK12
Evaluation labelsSTOP12
VerificationPassing software tests12

Next research questions

  • Does coordination reduce unsafe DECIDE recommendations?
  • What useful automation remains after stricter authority controls?
  • Which domains generate the most disagreement between ASK and STOP?

Return to Operational Science

Share knowledge

Bring this article
to your readers.

You may republish this Operational Science brief with its source, method, context and attribution intact. Keep the original publication date and link readers to this page for updates.

Subscribe to the research RSS feedRead reuse terms
Copy ready-to-publish article HTML

Paste this HTML into your blog’s HTML editor. Preview its tables and headings before publishing. The text includes source attribution and an original-article link.

<article><h2>Building a Decision Dataset for AI Employees</h2><p>How ZEISMO structured decide, ask and stop as measurable outcomes across six routine business domains.</p><p>Source: ZEISMO Research &amp; Lab. Published 2026-09-28. Record ZEISMO-EB-2026-005.</p><h3>Evidence source</h3><p>A ZEISMO-created synthetic research dataset. It contains no customer records, participant responses or copied benchmark cases.</p><h3>Method</h3><p>Forty-eight scenarios were divided across customer support, finance, operations, people administration, research publishing and IT security. Twelve development cases support system design; thirty-six frozen evaluation cases are balanced across DECIDE, ASK and STOP.</p><h3>Recorded findings</h3><ul><li>The protocol contains 48 original synthetic scenarios across six domains.</li><li>The evaluation set contains 36 cases, with twelve cases for each decision class.</li><li>The engineering package includes a separated reference rubric, an offline scorer, a run manifest and twelve passing software tests.</li><li>The controlled A/B/C evaluation will compare fixed rules, one-model reasoning and System Brain coordination.</li></ul><h3>Interpretation</h3><p>The dataset converts an abstract autonomy question into explicit, testable decisions. The next stage measures how systems behave on the frozen cases, including unsafe action recommendations, useful authorised decisions, latency and recorded cost.</p><h3>Aggregate data</h3><table><thead><tr><th>Dimension</th><th>Measure</th><th>Count</th></tr></thead><tbody><tr><td>Dataset</td><td>Development cases</td><td>12</td></tr><tr><td>Dataset</td><td>Evaluation cases</td><td>36</td></tr><tr><td>Evaluation labels</td><td>DECIDE</td><td>12</td></tr><tr><td>Evaluation labels</td><td>ASK</td><td>12</td></tr><tr><td>Evaluation labels</td><td>STOP</td><td>12</td></tr><tr><td>Verification</td><td>Passing software tests</td><td>12</td></tr></tbody></table><h3>Next research questions</h3><ul><li>Does coordination reduce unsafe DECIDE recommendations?</li><li>What useful automation remains after stricter authority controls?</li><li>Which domains generate the most disagreement between ASK and STOP?</li></ul><p><a href="https://zeismo.org/evidence/building-a-decision-dataset-for-ai-employees">Read the original ZEISMO article and check for updates</a>. Collection version 1.3. <a href="https://zeismo.org/policies#reuse">Reuse terms</a>.</p></article>

Continue reading

Explore all research topics
ZEISMO research practices

Evidence you can inspect.

Follow each marker to the method, record or policy behind it.

Versioned recordsIdentifiers and integrity checks Documented methodsProtocols and reproducibility tools Inspectable evidenceMethods, aggregates and fingerprints Accountable publishingAuthorship and correction policies

ZEISMO practice markers. Verification scope is recorded with each publication.