Workflows and controlAI RADAR · STUDY ·

01

The key finding

How often did executions of discovered tasks receive full credit?

50%

Full credit

Finding a task does not mean completing it

Requests for clarification and partly correct results also fell short of full credit.

Limitation: The analysis pools different models and repeated executions, not independent workplace assignments.

02

The study

How often did executions of discovered tasks receive full credit?

Full credit50%
Less than full credit50%

4,943 task executions pooled across models and runs. Full credit means meeting every criterion for the required system change and delivered content.

What the study shows

  • Of 4,943 executions of previously discovered tasks, 49.8 percent received full credit.
  • The remaining 50.2 percent included clarification requests, incomplete content, wrong targets and other deviations.
  • Requests for clarification and partly correct results also fell short of full credit.

How solid is this?

Method

One simulated organisation with 44 employees over six months, 94 actor-days and 120 tasks. Seventeen models ran in the same agent three times each.

Limitations

  • One synthetic organisation, not a field test.
  • The 4,943 executions repeat tasks across models and runs.
  • Not a general error rate; clarification can be sensible in real work.
03

What we take from it

Test separately which tasks the agent finds and what it actually changes. Check recipients, records and content before granting it more permissions.

Your 10-minute check

Tick the boxes and go through them with your team. Fewer than three ticks? Then it is worth a closer look.

Our interpretation: Recognising tasks and executing them correctly require separate tests. A list of discovered work does not establish a completed result.

What does this mean for your company? Book a conversation ↗