Five questions decide whether a recurring workflow should become an AI agent. Two of them produce a no on their own. Run all five on the loop your client-service team keeps rebuilding by hand.
These are the questions we ask on a diagnostic call at SideKind. Read the two that produce a no first; a workflow that fails either one fails it whether or not anybody says so.
You can run them yourself. Nothing here requires talking to us.
The five questions
The five questions test variance, volume, provability, reach, and consequence; ask them in that order, because the cheap ones disqualify the most candidates.
Complexity is a poor sorting variable. Each of the five has a test you can run today, on one loop, without buying anything.
| # | The question | What a bad answer means |
|---|---|---|
| 1 | Can the person who does this today write down the rule? | You want automation or a script, not an agent |
| 2 | How many times a year does this actually run? | No. A custom build will not pay back |
| 3 | Can you write the sentence that says it's working? | No. A project with no test is one we don't take |
| 4 | Could a new hire do this with the logins they'd get on day one? | Not yet. Fix the reach first |
| 5 | What does a wrong answer cost, and how would you find out? | Yes. This is what decides the rollout |
1. Can the person who does this today write down the rule?
If the rule fits on one page and holds every time, you want automation. Not an agent. Automation costs less and breaks in ways you can predict.
Judgment is the only ingredient you pay extra for. If the steps are identical every run, somebody on your team already did the deciding, and encoding that decision as a rule is a solved problem.
Anthropic publishes engineering guidance to find the simplest solution possible, and only increase complexity when needed. A model vendor wrote that. The same guide assigns well-defined tasks to workflows, and saves agents for flexibility at scale.
The common false positive is a loop called judgment because nobody ever wrote the rule down. "Triage" is usually the word.
Ask whoever does it to walk you through twenty recent cases. Fifteen of them usually collapse into a two-column lookup, and the interesting part is the five exceptions. That is a routing table with an escalation path, not an agent.
Our own process names the simpler answers: automation, a hire, or a process fix, and two of them have their own field notes. Start with the line between an agent, an automation, and a script. Then read when the honest answer is a person.
2. How many times a year does this actually run?
Count your real runs, not the felt ones. Eleven runs a year will not pay back a custom build, however much each one hurts.
Pain and frequency are different variables, while the loops that hurt most are often the rarest. The annual planning rebuild is awful and happens once; a quarterly board pack happens four times.
Both deserve fixing. Neither is worth a custom agent: a build is a fixed cost, and the return arrives one run at a time.
Do the arithmetic on a napkin: hours you save per run, times runs per year. Two hundred runs that each save four minutes is thirteen hours a year; forty runs that each save two hours is eighty. Annoyance is a poor proxy for either number.
Then run the other half: what a build costs, and what it costs to keep running. That half has its own arithmetic. Both clear, or the idea does not deserve a scope.
Gartner predicts that more than 40% of agentic AI projects will be canceled by the end of 2027. Escalating cost is the first cause it names.
This question disqualifies more candidates than any other, including ones we would have enjoyed building. If nothing on your list clears the bar, look at the list, because volume is exactly why some loops get picked first. Here are the ones client-service teams reach for.
3. Can you write the sentence that says it's working?
Write one sentence a stranger could check without asking anyone how they feel. If you cannot, the answer is no. We publish that disqualifier in advance: if we can't define the test, we don't take the project.
You can spot the sentences that fail. "The team spends less time on it." "The briefs are better." "It feels faster." Each one gets settled by polling people who already have an opinion.
METR ran a randomized trial with 16 experienced developers on 246 real tasks. In METR's measurement they finished 19% slower with AI tools, then estimated afterward that AI had made them 20% faster. Self-report got the direction wrong.
A testable version names a number, a population, and a place to measure it. Here is one.
"First response to an inbound form fill goes out inside five minutes, measured on every lead, with exceptions counted rather than excused."
The number does not have to be ambitious, but it has to exist before your build starts. A bar agreed afterward is not a bar.
This prevents a specific failure: a good agent nobody can defend six months later. Without a number agreed up front, every argument about whether it works becomes an argument about whether people like it. People like things less in month four than in week one, and that is how a working project gets quietly switched off.
Most of the work we turn down fails here, on nothing technical. It has its own field note.
4. Could a new hire do this with the logins they'd get on day one?
If a competent new hire could not do this work on their first morning with the logins you would give them, an agent cannot either. The verdict is "not yet." Fix the access and ask again in a month.
Agents read from systems. When the deciding information lives in someone's head, in an unstructured thread, or in a spreadsheet on one person's laptop, the agent does not stop and ask. It produces fluent, confident output from incomplete inputs; output that looks finished does not get checked.
OpenAI and Georgia Tech researchers argue that models are trained to guess when uncertain, because evaluation scoring rewards guessing over admitting it. A missing field does not produce silence; it produces a plausible answer.
Reach failures disguise themselves: "not connected to that tool" and "connected, asking the wrong question" arrive as the same message. We spent roughly six rounds of fixes on one connector's reconnect prompt before anybody read the actual HTTP response. The request had been malformed the whole time.
So test the reach during scoping. Somebody pulls the exact field the work depends on and it comes back; that is the answer you need. "Yes, it has an API" answers a different question.
That makes reach a scheduling problem: fix the access, or accept that one slice of your workflow stays manual. The same loop clears a month later.
5. What does a wrong answer cost, and how would you find out?
Two variables live here: what a wrong answer costs, and whether anyone would notice. Teams forget the second.
Cheap and visible is the ideal shape for your first build, because mistakes surface the same day and cost an apology. Expensive and visible works if a human gates that path. Expensive and invisible is where you should not go.
Cheap and invisible is the dangerous one. Errors accumulate and nothing surfaces them.
The EBU and the BBC graded over 3,000 AI answers to news questions; almost half carried a significant issue, and a third had serious sourcing problems. Somebody had to read every one of those answers to find that out.
We learned that the expensive way, on our own system.
We logged a status on every action. It said "ok." That only meant no error had been thrown. The real verdict lived in a separate audit record, so an action could fail cleanly and still log success.
The dashboard was green for work that was not happening.
So the first question about any agent action is no longer "did it run." It is "which record proves it did the thing."
This question rarely kills a project. It decides the rollout. A build goes to shadow mode first, then supervised, then autonomous. That is how an agent earns the right to act.
It is also the best argument for making your first one dull. A first build is an experiment in whether your team trusts the thing. So pick a loop where being wrong is cheap and obvious.
What a "no" sounds like
A no is a redirect, and it lands in one of four places.
- Automation or a script. The rule is writable, so write it; a script costs less to run and fails loudly instead of creatively.
- A person. The volume is real and the judgment is the actual job, so sometimes the answer is a hire. Our diagnostic call produces that outcome too.
- A process fix. The loop exists because of an upstream decision nobody revisited. A form asking for the wrong six fields, or an approval step added after one incident in 2023.
- Leave it alone. It runs four times a year, it takes an afternoon, and the afternoon is fine.
Two of the five can end the conversation on their own. Better that they end it early. A workflow that runs eleven times a year loses money for whoever pays for the build. One with no test sentence can never be shown to have worked, so it gets argued about until somebody turns it off.
If it's a yes
A yes means your loop clears all five: judgment, volume, a test, reach, and a survivable failure. What follows is a scope. Not a proof of concept.
Design takes five to ten business days. You get one paragraph describing the agent's job. A list of every system it touches. A list of every action it may take. And the test sentence, signed.
Fixed price for the build, transparent retainer to run it, three to four weeks from signed scope to live. Cancel any month.
Common questions
How much does an AI agent cost to build?
No honest number exists without knowing which systems the agent touches and how often your loop runs. A build is a fixed cost; running it is a recurring one. Both have to clear against the hours the loop consumes today. SideKind quotes a fixed price for the build and a transparent retainer to run it, and the diagnostic call is free. The arithmetic gets its own field note.
Can I use an off-the-shelf AI agent instead of a custom build?
An off-the-shelf agent is often the right call, and the same five questions decide it. That agent still has to reach your systems, and it still needs a sentence that says it is working. Question four kills more off-the-shelf agents than custom builds, because they reach whatever their vendor integrated and nothing else. If one you can buy passes all five, buy it.
Do I need a developer on staff to run an AI agent?
Somebody on your side has to own the test sentence and read the audit trail, and that person does not have to be an engineer. SideKind runs the operate phase under the retainer; day one hands you a workspace, a scoped tool list, a management dashboard, and an audit trail. Cancel any month, or take the agent in-house whenever you want it.
How long does shadow mode run before an agent goes autonomous?
Shadow mode runs until the test sentence has enough real runs behind it to settle the question; volume sets that clock. A loop that fires daily accumulates the evidence in weeks, while one that fires forty times a year takes most of a year. Promotion follows the evidence: shadow first, then supervised, then autonomous.
What the call actually is
The call runs thirty minutes, free, with no pitch at the end. You leave with a yes, a no, or a not-yet, and a no comes with the cheaper thing to do instead. The rest of the process is written out here. Book the call here. Bring the loop you cannot stop thinking about.