Skip to content

Insights

Cloud AI, a local model, or neither: how I decide

By Matt Parker5 min read

In short: Whether a workflow should use a cloud AI service, a model running on hardware you control, or no AI at all comes down to three checks. Map the data that actually flows through the task. Read the provider's business terms on training and retention and write down the date. Then ask whether rules would do the job better. Each option has a cost, and the honest answer for many tasks is a plain script.

Nearly every first call includes a version of the same question, usually phrased as “is it safe to put our stuff in there?” It deserves a better answer than a shrug or a sales pitch. Here is the process I use to decide, in the order I use it.

First, what data is actually in the workflow

Before any tool is named, I map the fields the task touches. Customer names. Addresses. Health information. Card numbers. Contract terms. Internal notes someone wrote in a hurry. Two things happen almost every time. The workflow turns out to carry less sensitive data than the owner feared, and one or two fields turn out to be more sensitive than anyone had noticed. That map is the first page of the written plan, and it is worth having even if no AI is ever involved.

Then, what the provider's terms say

A consumer chat app and a business tier of the same product are different contracts. The questions I read for: is customer input used to train models, how long is it retained, who can see it, and where is it stored. I write the answers down with the date and a link to the page, because these terms change and a decision made on last year's terms is not a decision.

If the business tier's terms fit the data map, a cloud service is usually the fastest and most capable option, and I say so. The configuration still matters: the least access that works, retention set deliberately, and a named person who owns the account.

When a local model earns its keep

Some data cannot leave your environment, by regulation or by contract or by your own judgment. Some workflows run at a volume where paying per request stops making sense. In those cases a model running on hardware you control is the right call, and I'll build it that way.

It is a decision with a bill attached. Local means hardware to buy or rent, someone to keep it patched, and in many cases less capability than the frontier cloud models. I don't recommend it as a reflex, and I'm wary of anyone who does. It is one option, chosen when the data map calls for it.

When neither is the answer

A surprising share of “AI” requests are rules wearing a costume. Match this invoice to that purchase order. Send the report every Monday. Flag the form with a missing field. A lookup table, a scheduled script, or a form with validation does those jobs more cheaply, more predictably, and in a way an auditor can read. When that is the case, that is what I build, and the word AI does not appear on the invoice.

What you get in writing

The data map. The provider decision and the terms it rested on, dated. The list of steps where a person reviews the output before it goes anywhere that matters. It all goes in the same folder as the runbook, so the next person to touch the workflow can see why it was built the way it was.

The data and security page has the standing version of these commitments.

If you want a straight read on which of these fits your business, book a free discovery call. If AI isn't the right tool for your problem, I'll tell you.

← All insights