What Turns an AI Pilot Into a Working Business System?

Introduction — The Demo Is Not the Operating System

A company can put AI into the hands of most of its employees and still be unable to answer a simple question: Is it creating value?

That is the uncomfortable position Denis O'Shea described when we spoke. His technology services company had deployed Microsoft Copilot to roughly two-thirds of its people. Leadership wanted staff to explore the tools, and there were competitions and incentives to encourage adoption.

But exploration did not create a system for deciding where AI belonged, who actually needed it, or how work would change. According to Denis, an internal scan then identified around 33,000 sensitive files with excessive access or sharing. The team had to classify information, tighten permissions and rethink how agents would interact with company systems. When the board asked whether the investment was working and whether everyone should receive access, he did not have an objective answer.

This is his reported experience, not an independently audited case study. Even with that distinction, it exposes a problem every ambitious leader should recognize: technological access, employee activity, safe operation and commercial value are four different things.

The temptation is to treat AI as a product that can simply be rolled out. Give people accounts, run a pilot, collect a few impressive examples, and expand the deployment. Yet the real work of a company is far less tidy than the demo. Information belongs to different departments. Decisions involve exceptions. People have different skills and incentives. Customers do not care how clever a workflow looks; they care whether the company delivers.

In my conversations with founders, operators, investors and technology specialists, I found different explanations for why this gap appears. Some point to data. Some point to people. Others focus on software testing, leadership or the economics of what happens after a launch. They are not describing the same problem, and they do not all agree on the answer.

That is precisely why these perspectives belong together.

This guide asks a practical question: What must be true before a promising AI pilot becomes a business system that deserves to scale?

We will start earlier than the technology, with the job the business needs to accomplish. Then we will examine why a successful experiment can still be a weak operating model. Later chapters will address data, permissions, workflow design, human capability, verification and long-term ownership. The conclusion will bring those decisions into a practical production-readiness check.

There is no automatic success formula here. The aim is to help entrepreneurs see what they need to prove before putting more money, work and trust behind an AI initiative.

01 — The Job Before the Tool

One of the most useful ways to question a technology project comes from someone whose work began long before today's AI tools.

Tony Ulwick recalled working on IBM's PCjr in the 1980s. The company invested in preparing the product, only to discover at launch that the market judged it a failure. His response was not simply to ask how IBM might have built a better computer. He became interested in a prior question: What would customers have needed the product to accomplish, and how could success have been measured before building it?

That question shaped his work on what he calls Outcome-Driven Innovation. In his interview, Tony explained the idea of a job map: identify the job people are trying to do, understand the steps involved, then identify the outcomes by which they judge each step. Those outcomes can involve time, error, effort, reliability and other characteristics that matter to the person doing the job.

This is a helpful starting point for an AI discussion because executives often begin at the other end. They hear that a tool can summarize calls, answer questions, generate proposals or automate tasks. Then they look for a place to install it.

But being able to automate an activity does not establish that the activity was important, that it was the real source of the delay, or that speeding it up will improve the customer's experience.

Imagine a company with a slow process for responding to customer complaints. It could use AI to draft a reply in seconds. That may be useful. But what if the expensive part of the process is identifying who has the authority to approve a refund? What if the complaint requires information from three systems, or the customer has already contacted the company twice? A faster draft may make a visible activity more efficient without resolving the job that matters.

That illustration is an editorial example, not a result from one of these interviews. Its purpose is to separate three questions: What can be automated? What should improve? And what evidence would tell us it actually improved?

Luke Girgis arrives at a similar starting point from the operator's side, but his method is different. He described entering a struggling food e-commerce company and breaking work down into small tasks. He wanted to understand what it took for a customer to place an order, for the food to be produced, and for delivery to happen. He also examined repeated administrative tasks and the way jobs were organized around them.

Luke's sequence is straightforward: audit the work, architect a better process, activate the technology, then accelerate what proves useful.

Tony's focus is the outcome people want. Luke's focus is the actual work needed to produce it. These are related ideas, not interchangeable methods. Together they give us a stronger test than asking whether a company has found an interesting AI use case.

First define the result. Then observe the work. Only then decide what a tool should do.

Luke's experience also carries a complication we must not ignore. He said one turnaround involved a very large reduction in staff while the company moved toward break-even. Elsewhere in the interview he argues that automation should free people from wasted time. Those two experiences are not automatically consistent. The human and organizational consequences of redesigned work deserve serious attention later in this guide. We should not use an operational success story to suggest that every stakeholder experiences the outcome as a success.

02 — The Pilot's False Sense of Success

A pilot can create convincing evidence that people are using AI while producing very little evidence that the company is better at its work.

Garrett Fritz described seeing what he called AI theater: executives approve AI spending and expect productivity to improve, but leave individual employees and middle managers to work out the implementation on their own. Different people select different tools. Some become faster. Others see little benefit. The company has activity everywhere but no consistent way to compare outcomes.

One of his more revealing observations concerns incentives. He said some employees were reluctant to share what they had learned because they feared that proving their jobs had become easier might put those jobs at risk.

This is not an independently measured claim about how often employees behave that way. It is an important management observation: a company can ask people to improve work while creating a reason for them to hide the improvement.

Denis O'Shea's account presents a related problem from a different angle. His company did not lack enthusiasm or access to tools. It lacked the business analysis and measurement system needed to understand which uses mattered. Only after problems emerged did his team invest more deeply in training, data classification and technical foundations.

These are not stories of AI technology simply failing to function. They are stories of organizational readiness failing to keep pace with adoption.

Saurabh Gupta, who leads a company specializing in data infrastructure, draws attention to the information beneath the model. He argues that AI projects become difficult to expand when data is scattered, lacks dependable context, or must be connected through a growing collection of separate tools. In his interview, he described a manufacturing customer's attempt to turn telemetry information into operational value. According to his account, the work advanced when the team connected the supporting data foundation to a specific use case rather than treating the platform as the entire objective.

We should distinguish his underlying point from his company's commercial solution. His argument about data context is relevant, but a particular vendor approach is not automatically the answer for every business.

Likewise, Saurabh cited a striking failure-to-scale percentage for AI pilots. We should not repeat that number as a general industry fact without locating the research, understanding its sample and seeing how it defines both pilot and scale. The argument here does not depend on an alarming statistic.

The three interviews suggest a practical distinction.

A technical pilot asks whether the system can carry out a task. An operational pilot asks whether that task can be carried out reliably by the people, data and processes of a real company. A business evaluation asks whether the resulting work is better enough to justify its cost and risk.

These distinctions are my editorial synthesis, not a framework the three guests jointly developed.

Consider the difference between an AI assistant that answers ten carefully selected questions and one that must answer hundreds of real customer questions. In the second case, the assistant may encounter incomplete records, conflicting instructions, private information, new kinds of exceptions and questions that should be sent to a human. Success on the first set does not prove readiness for the second.

A stronger pilot therefore needs more than attractive screenshots or estimates of minutes saved. It needs a baseline, a clearly defined owner, examples from real work, an account of failure cases, and agreement on what an unacceptable result looks like.

03 — Data You Can Trust—and Access You Can Defend

The previous chapter ended with a question that is easy to overlook when an AI demonstration goes well: Does the information behind the system deserve the level of trust and access we are about to give it?

There are two different problems hidden inside that question. One is whether the information is useful enough to support the task. The other is whether the system should be allowed to see or use it in the first place.

A company can solve the first problem and make the second dangerously worse.

Denis O'Shea encountered that distinction when his company expanded the use of AI across its workforce. In his account, the initial effort focused on putting tools into people's hands. Only afterward did the team discover how much existing company information was accessible beyond its intended audience. Denis reported that an internal scan found around 33,000 sensitive files that were overexposed or overshared.

What makes his story significant is not the size of the number by itself. It is the order of events. The business introduced a new way of finding and working with information before it had sufficiently examined who could already access that information.

Denis used a straightforward American example in our conversation: an employee might search for a W-2 payroll document, a draft employment agreement, or a performance improvement plan. Those documents may have existed in shared folders for years. A new AI interface does not necessarily invent the underlying access problem. But by making information easier to discover and retrieve, it may make an old problem much harder to ignore.

His team responded by classifying data, reviewing permissions and applying sensitivity controls. He also raised the problem of AI agents that connect to business applications, databases and APIs. Each agent may have an identity, a set of privileges and an owner. If that owner leaves, the company must still know what the agent does, which information it can reach and who has the authority to change or stop it.

There is an important distinction here: putting information into a central location or applying a label is not, by itself, a guarantee of safe access. A company's controls must actually work across the systems and situations in which the AI is used.

Saurabh Gupta approaches the data problem from another direction. In his conversation, he argued that companies frequently concentrate on models and computing power while underestimating the difficulty of giving those models reliable business context.

Consider the manufacturing customer he described. The company had accumulated telemetry from consumer products and wanted to understand product performance and warranty issues. The information existed, but converting it into useful answers required more than attaching a model to a database. According to Saurabh, the team had been considering a lengthy data-foundation effort; his company instead worked with the customer on a first use case and its supporting data platform.

Saurabh's preferred approach, offered through his company's product, packages data with information about its meaning, quality, origin, governance and transformation. He calls these reusable units data products. The important idea for an entrepreneur is not the particular vendor architecture. It is the practical difference between finding a number and understanding what that number means.

Imagine that an AI system reports an increase in product failures. Is it using warranty claims, returned items, repair requests or all three? Does a repeat claim count twice? Are the records from the same period and product version? Without those definitions, a confident answer can still lead to a poor decision.

This is an illustrative example, not a new claim about Saurabh's customer. It shows why business context matters even when information is technically available.

The tension between Denis and Saurabh is productive. Denis warns us about access to information that should not be exposed. Saurabh warns us about access to information that cannot be interpreted reliably. Those are separate questions. Solving one does not resolve the other.

A further perspective comes from Alex Rawitz, whose interview concerned connected devices rather than workplace AI. He described a model in which vehicle owners could choose to share selected data with services such as battery warranties or insurance. Consent and permission were part of the proposed design, not something added after data had already been distributed.

His example is a useful analogy, not a tested enterprise AI security model. It does, however, force a question that belongs in every data conversation: Who has the right to make this information available, and for what purpose?

A business leader does not need to design the entire technical architecture personally. But someone must be able to explain why an AI system can use a particular record, how the information will be interpreted, and what happens when access should be denied.

04 — Rebuild the Work, Not Just the Software

Technology projects have a habit of treating today's workflow as a permanent feature of the business.

A task exists, so someone builds a tool to perform it faster. A form requires approval, so the tool routes it to the same manager. Information moves between spreadsheets, so the new system copies it automatically.

The process may become faster without becoming better.

Luke Girgis described a more fundamental approach when he stepped into an e-commerce food company that, according to his account, was losing about $400,000 per month. He was not an industry specialist when he arrived. Rather than begin with an organizational chart or a set of software products, he traced the actual work: how a customer ordered, how food was produced and how it reached the customer's door.

Then he broke that work into smaller tasks and reconsidered how those tasks should fit together.

This matters because an organizational chart tells us who has a job. It does not necessarily tell us how value moves through the business—or where it stops.

Luke described a four-part sequence: audit, architect, activate, accelerate. First, investigate where the business is struggling. Next, redesign the highest-value workflow before choosing or building the technology. Then introduce the change in the selected process, and expand the parts that show useful results.

The order is the point. Automation follows the diagnosis; it does not replace it.

His turnaround also reveals why we must evaluate the human cost of operational change. Luke said the restructuring involved letting go of 70% of the staff, while the company subsequently moved toward break-even. That is his account of a particular crisis, not a recommended staffing formula. A business surviving may be a meaningful result for those who retain their jobs, but it does not erase the consequences for the people who lost theirs. And without independent verification, the case cannot prove that AI, rather than other restructuring decisions, caused the financial improvement.

There is another, less dramatic part of Luke's experience that illuminates the same principle. He described an artist-management business where managers spent time moving information between tools and updating administrative records. The redesign aimed to remove much of that repetitive work so managers could concentrate on activities that helped artists build their careers.

Here the relevant question is not how many automated steps the system contains. It is whether removing those steps gives people more time for the work that creates value.

This is where the conversation with Chris Majer becomes useful. Chris argues that modern organizations generate much of their value through coordination between people. Different specialists must make promises, transfer information, resolve exceptions and act in ways that produce a useful result for an internal or external customer.

An AI workflow is therefore not merely a chain of automated tasks. It also alters the agreements between people. Who notices when a handoff fails? Who has permission to override an automated recommendation? Who speaks to the customer when the process encounters something outside its design?

Those questions cannot always be answered by adding another automation.

An instructive comparison comes from Nick Foy, whose work focuses on enterprise resource planning systems. Nick described taking actual sales orders, purchase orders and manufacturing transactions from a client's company and running those examples through the proposed new ERP environment. His team could then show what daily business would look like after implementation, rather than rely only on meetings about how the process was supposed to work.

An ERP implementation is not an AI pilot. The technologies and risks differ. Yet the underlying discipline is worth borrowing: test the future process with real work before asking the entire organization to depend on it.

Nick also warned against customizing ordinary processes simply because a company has historically handled them in a particular way. He encourages clients to spend attention and money on what genuinely makes their business different.

That introduces a useful decision for any AI project. Some activities should be standardized. Some should be improved. Some may be automated. Others are distinctive because of the judgment, relationships or personal service involved. Treating them all as interchangeable tasks would be a design failure.

05 — Adoption Requires Practice, Not Just Licenses

There is a difference between giving someone a capability and helping them develop the competence to use it.

Chris Majer made that distinction sharply in our conversation. He described working with athletes and other people who must perform under pressure, and brought the lesson into leadership development: understanding a principle is not the same as being able to put it into practice.

You can read an explanation of delegation without becoming good at delegating. You can attend a workshop on conflict without becoming more effective in a difficult conversation. Knowing what a tool can do is only the beginning of learning how to use it when work becomes complicated.

AI adoption is vulnerable to the same confusion.

A company may provide accounts, run an introductory training session and count the number of people who have opened a tool. Those figures are easy to produce. But they say little about what employees can do with the system when their information is incomplete, the answer looks suspicious, or a customer needs something the automated process was not designed to handle.

Chris's perspective suggests that useful training should include the situations people will actually face. Rather than only teaching features, ask people to perform a task, inspect the result, recover from a mistake and decide when a person needs to take over.

Garrett Fritz introduces another variable: people's willingness to participate.

In his work assessing organizational AI maturity, he encountered employees who had discovered valuable uses for AI but were reluctant to share them. According to Garrett, some feared that management would interpret their productivity gains as proof that their jobs were no longer needed.

The issue was not a shortage of clever prompts. It was a conflict between the message from leadership—become more efficient—and the perceived consequence of success.

An organization cannot reliably build shared capability if the people learning the most believe that sharing their knowledge puts them at risk. That does not mean every company must promise that automation will never change staffing levels. Such a promise might not be credible. It means leaders must be clear about what they are trying to improve, what changes are being considered, and how employees can participate without being misled.

Luke Girgis's restructuring example makes this especially important. We have already seen that an operating turnaround can involve substantial layoffs. It would be misleading to use only his stories of removing administrative burden and conclude that AI adoption necessarily creates more rewarding work for everybody. The outcomes can differ by company, role and individual.

A useful counterpoint comes from Felienne Hermans, a computer science educator who spoke about teaching children to program. She was concerned that the availability of instant AI-generated answers could discourage the practice through which students develop understanding and creativity.

Felienne compared learning to activities that require repetition. In her view, having a system produce an answer is not always the same thing as learning to produce, evaluate or question one yourself.

Her example comes from education, not a controlled study of workplace AI adoption. But it raises an important question for managers: Which forms of effort should the new system remove, and which forms of effort are part of developing the judgment the business still needs?

Consider an employee who uses AI to prepare a customer proposal. If the system collects accurate background information and produces a first structure, the employee may have more time to understand the customer's specific situation. That could be a useful division of work.

But if the employee simply sends the generated proposal without understanding the recommendations, the company may be reducing the very expertise it needs to recognize a weak answer. That is a possible failure mode, not an assertion that all AI-assisted proposal writing has this effect.

Steve Cadigan contributes a broader workforce perspective. He argues that many people increasingly value opportunities to learn and develop, rather than assuming a job should offer the same skills and responsibilities for decades. His position is a practitioner observation about changing career expectations, not a universal statistic. It nevertheless gives companies another reason to treat AI implementation as a learning program rather than simply a technology purchase.

These interviews lead to a practical distinction between access, ability and trust.

Access means the employee has the tool. Ability means the employee can use it appropriately in real work, including recognizing errors and limits. Trust means the employee can participate openly, raise concerns and share useful practices without being deceived about the purpose of the change.

A business needs all three if it wants adoption that survives the first wave of enthusiasm.

06 — Verification Before Trust

The previous chapter asked whether the people using an AI system have the competence to recognize a bad result. Now we need to ask a more uncomfortable question.

What if the result looks good enough that nobody thinks to question it?

AI makes a particular kind of mistake attractive. A response can be clear, complete, well written and wrong. A new software feature can appear to work in a demonstration while quietly breaking an existing customer journey. The speed of producing an answer or a change can exceed the time available to inspect it.

This creates two different problems: proving that software behaves as intended, and proving that the information behind a decision deserves trust.

Gal Vered approaches the first problem through software testing. He works on tools designed to test applications as developers increasingly use AI to write code. In our conversation, Gal argued that the central bottleneck was shifting. Writing code was becoming easier; understanding whether it worked correctly was becoming more difficult.

His response is not simply to ask one AI system to judge another and accept the answer.

Gal described a process in which an AI agent examines a software change, identifies behavior that should be tested, and writes a test script. The resulting script can then run through the application and produce a repeatable result. In his example, the agent can generate tests using Playwright, a framework that automates browser interactions.

The distinction matters. An AI agent may help design a check, but the check itself can execute in a more predictable way. Rather than repeatedly asking a model, “Does this feature seem correct?”, the team can ask whether a specified action produces a specified observable outcome.

For example, if a customer updates a delivery address, does the system save the change? Does the new address appear on the order? Does the old address remain in an audit record where required? Does a later software change break that behavior?

A repeatable test can reveal a regression that might otherwise escape attention.

But we should not turn Gal's testing approach into a promise that all AI behavior can be made deterministic. His examples concern software applications and testable user interactions. A passing test proves only that the conditions covered by that test passed at that time. It does not guarantee that every customer scenario, data condition or future AI response will be correct. Someone must still decide what constitutes acceptable behavior, which cases deserve tests, and what risks the tests do not cover.

That brings us to the second problem.

Mike Ryan works in investment analysis, where a well-formed but poorly sourced answer can distort an important decision. He told me that a central part of his team's work is matching each AI task to the most relevant and trustworthy source material. A financial statement, research report and spreadsheet may all contribute useful evidence, but not every source deserves the same weight for every question.

Mike's approach is informed by a simple limitation: an AI system can produce an answer even when the available information is incomplete or poorly selected. Better instructions alone do not repair missing, outdated or irrelevant evidence.

He described his operating philosophy as AI plus one: powerful AI workflows supporting a capable person who directs the work and remains responsible for interpreting what the information means.

That is a useful philosophy from his particular field, not proof that every AI workflow can or should be supervised by exactly one person. A system processing thousands of low-risk transactions will need a different oversight design from a tool that helps an investment professional assess a complex company.

What connects Gal and Mike is more important than the difference between software engineering and investment analysis.

A trustworthy result needs a way to be challenged.

For Gal, the challenge may be an automated test that exposes incorrect application behavior. For Mike, it may be checking the evidence, assumptions and judgment behind an investment conclusion. For the business owner, the lesson is to ask which kind of verification a particular task requires.

Consider a customer-support AI system. A software test might confirm that it can open a customer record, retrieve an order status and send a response. Those are useful checks. But they do not establish that the response is appropriate when the order contains disputed information, a customer has received conflicting promises, or an agent has retrieved a private record it should never see.

The business may need several layers of review: technical behavior, data access, factual correctness, quality of service and escalation when the answer is uncertain. No single check substitutes for the others.

Denis O'Shea's experience with overshared documents, introduced earlier, adds a crucial warning. A system could retrieve information correctly and still violate the company's access boundaries. Technical success is not always an acceptable business result.

07 — Who Owns the System After the Launch?

A pilot often has a champion. That person understands why it exists, knows which prompts are effective, has the relevant files and can usually find a way around unexpected problems.

A working business system cannot depend on that person being available forever.

Denis O'Shea raised a specific version of this problem when he talked about AI agents. Suppose an employee creates an agent that accesses a customer relationship management system and a finance application. The agent performs useful work and becomes part of a routine. Then the employee leaves.

Who now knows which systems the agent can access? Who can explain its behavior? Who can revoke a permission, repair a connection or turn it off? What happens to tasks that were scheduled to run without further human attention?

Denis described the danger of an orphaned agent: automation that continues to exist after its effective owner, design rationale or operational knowledge has disappeared.

His concern is not that all agents inevitably become dangerous. It is that a company can create a new class of operational dependency without first deciding how those dependencies will be managed.

The issue extends beyond agents. It applies to any AI-supported process built around one employee's account, an undocumented integration, a private collection of prompts or an outside developer whose engagement ends after deployment.

Matt Strippelhoff offers a complementary way to think about that responsibility. He works with companies commissioning custom digital solutions. In our conversation, he explained why treating software development as a project with a beginning, middle and end can be especially troublesome for mid-sized businesses that do not maintain a large internal engineering team.

His alternative is to consider the result an asset that the company will need to support and maintain.

The initial build is only one part of the investment. An operational system may require updates, monitoring, error resolution, changes to integrations, security work, support and adjustments as the business evolves. Matt encourages buyers to consider the total cost of ownership over time and to understand how the responsibility for that work will be covered.

These are his observations from custom software services, not an independently established cost formula for every AI implementation. But the underlying question is directly relevant: What are we really buying when we commission something we intend to depend on?

If a pilot produces an impressive result but nobody has budgeted for maintenance, testing, support or a transition to another provider, the company's investment is incomplete.

There is a commercial dimension here, too. A tool purchased on a subscription can create continuing obligations even when the initial excitement fades. A custom agent can depend on multiple paid services. A workflow can require a model, a data connection and a human review process. Sometimes buying a service will be more practical than building one. Sometimes a particular integration will become a liability. The right answer depends on the business, but the decision must include the future, not only the launch demonstration.

A third perspective comes from Lior Weinstein, who works with nontechnical business leaders and technology teams. He described founders who hire a CTO or a development company and then assume the provider must be brilliant because the technical explanation is beyond their understanding.

Lior's objection was simple: a leader cannot collaborate effectively with a professional who cannot explain what is being proposed, how it works and how the two sides will work together.

He also uses a personal distinction between being in charge and being in control. The leader must know the destination and retain responsibility for strategic choices, without personally steering every technical action.

Applied to AI, these ideas suggest that executive ownership is not the same as writing code. A founder does not need to know every detail of a system's implementation. But the founder, or another accountable executive, should be able to ask for and understand the essential agreements: the intended outcome, accepted risks, operating costs, decision rights, escalation rules and conditions for stopping.

This is where all three perspectives come together.

Denis asks whether an automation has an owner. Matt asks whether the business is prepared to maintain the asset. Lior asks whether leadership can govern technical work without needing to perform it personally.

They are not describing one shared framework. Their combination does, however, expose a useful weakness in the way companies think about technology projects.

A project can be successfully delivered and still be operationally ownerless.

08 — The AI Production Readiness Gate

Imagine being asked to approve the next stage of an AI project.

The team shows a persuasive demonstration. Some employees are excited. A vendor says the technology is ready. A department leader believes it will reduce costs. The proposed budget looks reasonable compared with hiring another team.

What would you need to see before saying yes?

After comparing the different experiences in these interviews, I would not ask for one grand measure of AI maturity. I would ask for evidence across eight separate operating questions.

I call this editorial decision tool the 21CE AI Production Readiness Gate.

It is an original synthesis for this FIELD GUIDE, informed by the interviews but not a framework any individual guest has claimed to develop, a scientifically validated score or a guarantee of project success.

Its purpose is narrower and more practical: help a business leader identify what still needs to be demonstrated before a successful experiment becomes a lasting operating commitment.

Eight questions before expansion

These gates are connected, but they should not be collapsed into one number.

A system can have excellent data and poor adoption. It can reduce costs and make customers worse off. It can pass technical tests while lacking anyone with authority to handle an unusual case.

A simple average would hide precisely the weaknesses the guide is trying to expose.

Three decisions—not one reassuring score

For each gate, record one of three decisions.

Proceed means the team has enough relevant evidence for the proposed bounded next stage. It does not mean every risk has disappeared, or that the system is ready for unrestricted use.

Redesign means the business case may remain useful, but part of the workflow, technical arrangement or operating responsibility needs to change before expansion.

Hold means there is a consequential unanswered issue that makes the proposed next stage inappropriate at present—such as uncontrolled sensitive-data access, no way to stop an unsafe process, or no accountable owner for a high-impact decision.

These are management decisions, not mathematical thresholds. A “proceed” in one gate cannot cancel a serious “hold” in another.

The more consequential the task, the stronger the evidence should be. Internal assistance with low-risk administrative drafts is not the same as making financial commitments, changing customer entitlements, handling employee records or supporting decisions that affect health and safety.

A worked example: the customer-support pilot

Consider a fictional mid-sized company that tests an AI assistant for its customer-support team.

The initial pilot looks promising. The assistant retrieves order statuses, suggests replies and saves time in selected cases. Staff members like it, and the vendor proposes expanding access to the full team.

The business leader runs the eight-gate review.

Outcome: The company knows that customers want quicker, correct answers about orders, and it has a baseline for response times. That is useful evidence.

Workflow: The assistant has been tested against ordinary inquiries, but not against disputed refunds or orders changed after shipping. The company chooses redesign: expand the test cases and clarify escalation.

Information: The assistant retrieves order data, but no one has yet established whether every support employee should see every customer note. That may justify hold for wider access until permission boundaries are checked.

Adoption: The pilot team can use the tool, but new staff have not practiced recognizing unsupported answers. The company plans supervised training rather than assuming the success of early adopters applies to everyone.

Reliability: The system has answered prepared questions well. The company has not yet measured how often it fails on contradictory or incomplete records. It needs a more representative evaluation.

Ownership: A pilot manager can troubleshoot the system, but there is no agreed substitute during leave or after departure. The company names a business owner, a technical owner and an escalation contact.

Economics: The pilot estimates time saved, but does not yet establish whether customers receive more accurate answers, whether recontacts fall, or how much ongoing review and maintenance cost. The financial evaluation remains incomplete.

Continuity and human effects: The company needs to decide what support employees should continue to learn, how their responsibilities change, and who maintains the underlying information sources.

This is not a case from any guest's business. It is a worked editorial illustration.

The sensible result is not automatically to abandon the project. Nor is it to scale merely because most boxes look positive. The company can keep learning within a controlled pilot while holding back wider deployment until critical access, reliability and ownership questions are resolved.

That is a much more useful decision than declaring the pilot either a triumph or a failure.

How to use the gate in a real meeting

Give the owner of the proposed rollout a single page with eight rows. For each row, require four entries:

1. Evidence: What actual observation or document supports the claim that this area is ready?
2. Owner: Who is accountable for that evidence and the decision it informs?
3. Decision: Proceed, redesign or hold—for this specific next stage.
4. Next action: What must be done, by whom, before the decision is reviewed again?

A meeting should end with a named next step, not just a vague request for further confidence.

When evidence is missing, say it is missing. Do not manufacture numerical scores to make uncertainty look scientific. When a decision is held, define what would change it. When a bounded expansion is approved, define the conditions that would trigger a pause.

The gate also creates an important conversation between the technical team and the people who will live with the result.

Tony Ulwick asks us to define the outcome that matters. Luke Girgis pushes us to inspect the real work. Saurabh Gupta brings attention to data and context. Denis O'Shea exposes permissions and ownerless agents. Chris Majer and Garrett Fritz show why adoption is a matter of human practice and incentives. Gal Vered emphasizes observable system behavior. Mike Ryan requires credible evidence and responsible judgment. Matt Strippelhoff extends the question to maintenance and total cost. Felienne Hermans reminds us not to remove the effort through which people build their own capabilities.

Those are not ten endorsements of a common tool. They are ten useful reasons to resist a premature yes.

The question is no longer whether AI can produce an impressive result in a controlled setting.

It is whether the business understands enough about the work, the risks and the people to take responsibility for using that result every day.

Conclusion — Scale a Capability, Not a Demo

Imagine handing the customer-support pilot from the previous chapter to the colleague who opens the queue tomorrow morning. The first customer asks a question the pilot handled well last week. This time, the account record and the policy document disagree. What can that colleague see? Who can resolve the discrepancy? What happens to the customer while they wait?

That handover is a useful way to end a pilot review. Ask someone who did not build the system to take a real case through it. Let them explain where the answer came from, where they used their own judgment and what they would do if the same case arrived on a busy afternoon. Listen especially closely when they say, “I usually ask someone.” Find out who that person is and what they know. Their answer may reveal work that never appeared in the pilot report.

Then go back to the outcome you chose at the start. In the support example, a faster first reply has limited value if the customer has to call again to get the issue resolved. Look at the whole case, including the time a colleague spent checking or correcting the answer. That gives the team a fair basis for deciding where the system helped and where it moved work onto someone else.

Before the meeting ends, ask the owner to finish this sentence in writing. **“We are ready to use this for , because we observed . If ___ happens, ___ will pause the process. We will review the result on ___.”** A gap in that sentence gives the team a specific question to take back into the next test.

Sometimes the right next step is one more queue, one more team or one more kind of case. Sometimes it is fixing a permission before another customer enters the pilot. What makes either decision worthwhile is that someone can explain it, act on it and come back with evidence. That is when the business begins to own the capability it has built.

Key Takeaways: From AI Pilot to Working Business System

• Start with the job, not the tool. Define what should improve for the customer or the business before selecting AI.
• Treat a pilot as a learning exercise. Technical success, operational readiness and business value require different evidence.
• Verify both the meaning of the data and the right to access it. A useful answer can still rely on misleading or improperly accessible information.
• Redesign actual work before automating it. Map handoffs, exceptions and responsibilities, not only ideal workflows.
• Build competence and honest participation. Licenses and training attendance do not demonstrate ability, judgment or employee trust.
• Challenge the system before trusting it. Test difficult cases, trace consequential answers and make failures observable.
• Name owners who will remain accountable after launch. Cover access, changes, incidents, costs, maintenance and handover.
• Expand only when evidence supports the next bounded step. Use the eight readiness gates to choose proceed, redesign or hold.

Practical companion

Bring the eight gates to a team meeting. Record the evidence, the owner and the next decision.

Sources & editorial method

This FIELD GUIDE is an original editorial synthesis by Martin Piskoric based on interviews from 21st Century Entrepreneurship. Guest examples and viewpoints are attributed to their speakers. The connections, field tests and AI Production Readiness Gate are the author’s analysis. Individual experiences are not universal benchmarks, and the decision aid is not a validated assessment.