Implementing AI in Your Business: From Problem to Pilot

Bringing AI into a business doesn't end with buying a few tool subscriptions. A better starting point is choosing one specific process, naming an owner for it and measuring results at a small scale. If it isn't clear which problem is being solved, even impressive output can add cost and extra work.

This article offers a suggested pattern for designing a first pilot. The timings and metrics are examples and should be adjusted to your organization; no specific percentage of cost reduction or sales growth is guaranteed.

Define the problem before choosing a tool

Pick a repetitive process whose inputs and outputs can be described. For example, categorizing support requests, preparing draft replies or summarizing internal reports you're allowed to share could be candidates. Sensitive decisions about people or financial commitments are not good choices for a first, unsupervised pilot.

Write down on one page: who does the work, where the input comes from, who receives the output and what happens if there's an error. If those answers aren't clear, document the process first.

Measure the current state

Before rolling out a solution, record a few representative samples of how the work is done today. Don't measure speed alone. Time spent on human review, how much needs correcting, serious errors and the satisfaction of internal users are all worth measuring.

For example, in a pilot for drafting support replies you could record how many drafts are usable after review, how many contain wrong information and the total time from preparation to approval. A fast but wrong answer isn't a success. Set a stop condition from the start too, such as any data leak or a reply sent without approval.

Limit data and access

• Use made-up samples, or data you're clearly permitted to use.
• Decide what must never go into an external service, including passwords, confidential contracts and customer identifiers.
• Don't let employees' personal accounts replace managed, organizational access.
• Grant only the permissions the pilot needs, and name someone responsible for revoking them.

Data quality matters too. Old documents, contradictory answers and messy naming can't be fixed just by adopting a new tool. Identify the authoritative version of each source and who keeps it up to date.

Run a limited pilot with human review

For example, you could limit a short pilot to one small group and one category of requests. At this stage the output should be a draft, reviewed by a named person before it's sent or recorded. Going back to the old way of working should be simple and always available.

NIST's voluntary AI Risk Management Framework sets out four functions: Govern, Map, Measure and Manage. It can help organize questions about responsibility, use, evaluation and responding to risk. The approach in this article is not an official adaptation of the framework or a certification of compliance with it. NIST framework

Calculate the real cost

Cost isn't just the tool subscription. Also record the time spent preparing data, connecting to existing systems, training staff, reviewing output, maintenance and handling errors. Compare the result with the previous method on similar work.

If the tool has shifted review work onto someone else, don't leave that time out. Also decide how the work continues if the service goes down or the quality of its answers changes.

Decide whether to continue, adjust or stop

At the end of the pilot, summarize the results in a short report: what improved, which errors remain, what it cost in total and which parts still need supervision. Expanding use only makes sense when the criteria you set in advance have been met.

• If the output isn't reliable, narrow the scope of the problem or fix the data source.
• If the cost of review outweighs the benefit, stop the pilot or change its design.
• If the result is good, define the next stage with an owner, a budget and fresh evaluation criteria.

Common questions from managers

Do we need a custom model from the start?

Not necessarily. Whether you need a custom solution depends on the type of data, confidentiality, the quality you expect and integration with existing systems. Clarify the problem and constraints first.

Which department should we start with?

One with a well-defined problem, an available owner and a way to measure results. Choosing a department just because a tool is popular is not a good criterion.

When should we expand the project?

When the pilot's evidence shows acceptable quality and a defensible cost, and there's a clear process for handling errors. To practice these decisions, explore the Applied AI course. To talk through a specific process, you can also contact Fekrano.