A model is a tool that guesses well. That is its strength and it is the whole problem. The guess is invaluable when the task has no rules, and it is an operational headache when the task has them.
A great many of the tasks smaller companies want rid of have rules. Which is why the answer to them is not AI.
Two kinds of task
Almost every workflow sorts into two piles with a single question: can you write the rule down?
If an invoice from a particular supplier always goes to a particular account, if an order above a certain amount always has to be approved by a particular person, if an email containing the word complaint always goes to a particular queue, then you have a rule. Rules should be coded, not guessed.
A coded rule does the same thing every time, costs almost nothing to run, can be tested before release and can be explained afterwards. Four properties a model does not have.
What the model actually costs
The price per call is rarely the problem. What costs is three other things.
The uncertainty
A rule-driven workflow that works today works the same way in six months. A workflow built on a model can start answering differently because the supplier updated something you do not control. That means somebody has to keep an eye on it, and that time is a running cost.
The control
A rule-driven workflow can be tested exhaustively. You feed in twenty cases and see twenty correct outcomes. A model-driven workflow can be tested by sampling, never in full, and that means you have to build a path for whatever comes out wrong. That path is often more work than the workflow itself.
The explanation
The day somebody asks why the workflow did what it did, you want to be able to point at a line. With a rule the line is there. With a model there is a line of reasoning that sounded plausible at the time.
The four cases where a model is the right answer
There is work that cannot be coded as a rule, and there a model is not merely acceptable but clearly better.
- Unstructured text coming in. Invoices and delivery notes that look different from every supplier, emails written by people, scanned documents. Pulling the fields out of the mess is exactly what a model is good at, and it is hard to write rules for.
- Sorting with fuzzy boundaries. Deciding whether an incoming case is a complaint, an order or a question. No exact rule exists, but the boundary is obvious to a person reading it.
- Summarising for a person. Pulling fifteen emails in a thread together into a paragraph somebody will read and then decide on. The fact that a person reads it afterwards makes the error risk manageable.
- Text out that will be reviewed anyway. A draft reply, a product description, a first version of minutes. The model takes you from a blank page to something to correct.
The pattern is worth noticing. In all four cases the job is to interpret or produce language, and in three of them a person looks at the result before anything happens.
The most common right answer is a mix
In practice a good workflow often looks like this: a model reads the messy incoming text and pulls out the fields. Then ordinary code takes over and does everything else, meaning the check against the rule, the lookup against the register, the entry in the system and the log.
The model gets what it is good at, which is interpretation, and it does it in one place where an error can be caught. The code gets what it is good at, which is doing exactly the same thing every time.
Put the model where the text is messy. Put the code where the consequence is expensive.
Why we say this out loud
It is not in our short term interest. An engagement with AI in it can be charged at a higher rate, and it sounds better when it has to be described.
But we sell workflows that have to keep running after we leave the room, with no hosting and no support contract. A workflow we built more complicated than it needed to be comes back as a problem, and then we have earned money once and lost a reference.
So the question we ask at every mapping session is the simplest one there is: can the rule be written down? If it can, we write it down.