The real cost of “just add AI”

AI projects rarely fail because the model is weak. They fail because nobody defined the goal, the owner or the measure of success before the work started.

This post covers what "just add AI" ends up costing, the usual mistakes, and the process we use to avoid them.

What bolting AI on costs

Adding AI to an existing process without redesigning the process tends to create four problems.

1. No strategy. There is no clear goal, owner or metric, so effort goes into demos instead of anything the business needs.
2. Bad data. Inputs are messy and nothing feeds corrections back. The model learns the mess.
3. No path to production. The pilot runs on one laptop, with no plan for uptime, monitoring or fallback.
4. A worse experience. Users get extra steps and answers they cannot check, so they stop trusting it and stop using it.

A better model does not fix any of these. It only makes the failure more expensive.

The usual mistakes

• Starting with a model instead of a job. "Let's use an LLM" rather than "let's help finance close the books faster".
• Measuring model accuracy instead of business impact.
• Building retrieval, fine-tuning and a vector database before anyone has shown they are needed.
• Proofs of concept with nobody responsible for running them afterwards.
• Leaving privacy and personal data until the end.

The process we use

1. Start with a job, not a model

Write one sentence: "Help [role] achieve [outcome] by automating or assisting [task]."

For example:

• Help support agents resolve tickets faster by drafting first responses.
• Help finance cut manual entry by extracting structured data from invoices.

2. Pick a narrow use case

Look for work that happens often, has an effect you can measure, and has a clear definition of done. If the task carries legal, financial or safety risk, put a person in the loop from the first day.

3. Use the simplest thing that works

1. See whether a rule or a template covers most cases. Start there if it does.
2. If not, try basic machine learning or extraction.
3. Only then use an LLM: prompting first, then retrieval, then fine-tuning.

Ship something small and add complexity when the results ask for it.

4. Decide the metric before writing code

| Function | What to measure | | -------- | -------------------------------------------------------- | | Support | First-contact resolution, handle time, satisfaction | | Finance | Minutes saved, errors avoided, cycle time | | Sales | Qualified replies, pipeline created, time to first touch |

Make the number visible to everyone involved from the start.

5. Design the workflow around the model

A production AI feature is a workflow with a model inside it. Plan for:

• people reviewing output where judgment matters,
• a fallback when the model is not confident,
• a way to learn from the edits and approvals people make.

6. Run the pilot as if it were live

Log prompts, inputs, outputs, errors and latency. Validate output and protect private data. Write down what happens when the model provider has an outage or changes behavior.

7. Scale only after the pilot pays off

Expand once the pilot has hit its business target and passed the operational checks.

Where to start

These use cases usually have a clear metric and a short path to production:

• ticket triage and drafted replies,
• extracting data from documents,
• drafting sales outreach,
• search over internal knowledge.

Pick one and run it as a four-week pilot.

Checklist before you build

1. The problem, the owner and the metric are written down.
2. Input data exists, along with examples of correct output.
3. Privacy and retention rules are set.
4. The points where a person reviews output are mapped.
5. The fallback path has been tested.
6. Logging and monitoring are in place.
7. The rollout from pilot to full use is planned.

Technical defaults

• Prefer small tools you can combine over large platforms.
• Start with prompting and add retrieval only when you need it.
• Keep prompts focused and context small.
• Use models you can swap for another provider later.
• For sensitive data, consider models that run on the device or on your own servers.

To plan an AI pilot with us, get in touch.