Cloud vs Local — Decision Tree for Regulated AI
Decision map · cloud vs local for regulated AI

Where should this workflow run?

Ask it per workflow (or per step) — never once for the whole company. Follow the branches: NO goes left, YES goes right.

Start · per workflow Does this step touch sensitive / regulated data (PHI, PII) that must not leave your perimeter?
If NO — data can leave Does the task need top-tier reasoning (hard, high-accuracy)?
If YES — data is sensitive Can the data be reliably de-identified before the smart step?
CLOUD / LOCAL Small, cheap model. Simple task, no secrets — run it wherever is cheapest, cloud or local.
CLOUD Frontier cloud model in your private tenant. Nothing sensitive here — use the best brains available.
De-ID not possible? Is a local open-weight model accurate enough for this task?
HYBRID Strip the PHI locally, then send only de-identified data to a frontier cloud model. Best of both.
CLOUD · LOCKED Private cloud, fully locked down. Azure OpenAI in-tenant + BAA, region-locked, no-training — when you truly need the big model on raw sensitive data.
LOCAL Local open-weight model, on-prem. Llama / Mistral via ONNX inside your own network — nothing ever leaves the building.
NO ◀
▶ YES
NO ◀
▶ YES
NO ◀
▶ YES
NO ◀
▶ YES
CLOUD — runs on the provider's servers (private tenant) HYBRID — split by step: sensitive local, reasoning in cloud LOCAL — runs on your own hardware, nothing leaves
The catch

Where the model runs is only half the answer. Whatever branch you land on, the real safety comes from what you wrap around it — logging, validation, human-in-the-loop on high cost-of-error steps, and an audit trail. A local model with no validation is still a bomb.

Digital Gateway · Denys Matrosov · dm@igateway.digital