On-premise vs cloud AI: what each is, and when you need on-premise.

In short: on-premise AI runs on hardware inside your own building, so documents and questions are processed where they already live. Cloud AI sends them to a provider’s servers to be processed. On-premise AI infrastructure is needed when your obligations depend on where client information travels and who can access it, as they do for legal privilege, health records and professional confidentiality. Comparing hosted options? See private cloud vs on-premise AI.

If your business handles material you are obliged to protect, the useful question about AI is not "which model is best". It is "which deployment posture do our obligations permit". This page defines the four postures on the market, compares them honestly, and explains where each one makes sense. It is written to be useful even if you end up choosing a different column to ours.

The four deployment postures

1. Public cloud AI. Consumer tools such as free ChatGPT accounts. Prompts are processed on the provider's servers, and on consumer tiers they can be used to train future models. Fast, capable and free, which is exactly why staff use them quietly. For client work in a regulated practice, this posture is generally indefensible.

2. Enterprise cloud AI. Paid business tiers: ChatGPT Enterprise, Claude for enterprise use, Microsoft Copilot. These are serious products. The reputable providers do not train on your business data by default, hold strong security certifications, and offer admin controls. What does not change is the architecture: every prompt and every document still travels to the provider's infrastructure, and for the major providers that infrastructure and the company behind it sit under foreign jurisdiction.

3. Private cloud and sovereign hosting. Dedicated capacity in an Australian data centre. Data stays onshore, which resolves the offshore transmission question, and you still depend on a shared facility, network transit and a hosting relationship. Built yourself, it is a genuine infrastructure project: provisioning, scaling, patching and paying for GPUs whether they are busy or idle. Managed, it is what Ermos Edge provides: a private instance for your business, hosted and answered in Australia. Our private cloud vs on-premise comparison covers this posture in detail.

4. On-premise. The AI runs on hardware inside your building with no path to the internet. Nothing is transmitted because there is nothing to transmit on. Historically this meant building it yourself; the point of a managed unit like Dominion is getting this posture without the project.

The honest comparison

Public cloudEnterprise cloudPrivate cloud / sovereign hostingOn-premise
Model capabilityFrontierFrontierYour choice of open modelsStrong open models, tuned to your business over time
Where your data goesProvider servers, may train modelsProvider servers; training off by defaultAustralian data centreNever leaves your building
JurisdictionForeignForeign for the major providersAustralianYour premises, Australian law
Cost modelFree or low per userPer seat, per monthInfrastructure + engineeringPer unit, per month
AuditabilityProvider policy documentsProvider policy documents and certificationsContract + facility auditsPoint at the unit; your security team can inspect the boundary
Failure modesData absorbed into models; staff misusePolicy change, subpoena or breach at the provider; offshore processing findingsFacility or network incident; project overrunsHardware failure (managed under support); no external failure surface

Being fair to the cloud column

Enterprise cloud AI is the right answer for many businesses. The frontier models are more capable at open-ended general reasoning than anything you can run locally, the per-seat pricing is easy to start small with, and providers such as OpenAI, Anthropic and Microsoft invest heavily in security. If your obligations permit client material to be processed on offshore infrastructure under contract, enterprise cloud may be all you need.

The reason this page exists: for many Australian professional practices, that "if" fails. Legal professional privilege, health information rules and professional confidentiality obligations turn on where information travels and who can access it; whether the provider trains on it is a separate question. "Not trained on" and "never transmitted" are different promises, and only one of them can be verified by standing in your own server room.

What "on-premise" means for your data, precisely

An Ermos unit has no route to the internet for your data. Your team connects over your local network; documents are indexed and queried on the unit; the model runs on the unit. Support and updates for Dominion happen through sessions you initiate and supervise, and health monitoring watches the hardware, never your files. The security claim is structural rather than contractual, which is why it is the one posture a compliance officer can verify without reading a trust portal.

Where Dominion sits

Dominion is the fourth column as a managed service: an AI unit installed in your office, running models tuned to your business by ErmosIQ, with monitoring, updates and support included. You get the posture without hiring an infrastructure team.

Dominion is priced per unit, per month on a 12-month term, sized to your practice: unlimited users, no per-user licences, no usage charges. Exact pricing is confirmed in the scoping consultation.

For the full cost picture including worked per-seat comparisons, see what on-premise AI actually costs.

Common questions

How does on-premises AI differ from cloud-based AI?
The difference is where the work happens. With cloud-based AI, each question and the documents behind it travel to a provider's servers, the answer is generated there, and the provider's terms decide what is kept. With on-premises AI, the model and your documents sit on hardware in your building and the answer is generated there. Cloud is quicker to start and leads on open-ended general reasoning. On-premises keeps working without the internet and keeps client material inside the building.
What is the difference between legal AI that processes data on-premises and legal AI in the cloud?
Cloud legal AI sends matter documents and prompts to the vendor's infrastructure, so the practice has to be satisfied about confidentiality, privilege, retention and who at the vendor can reach the material. On-premises legal AI processes the same documents on a unit inside the practice, so no third party receives them. The supervision and verification duties are the same either way.
On-premises vs cloud infrastructure for AI: how do they compare?
Cloud infrastructure is rented, scales instantly, and processes your documents on the provider's servers, for the major providers overseas. On-premises infrastructure is a unit in your building that processes everything locally, works without the internet, and is priced per unit rather than per seat. Capability for general reasoning favours the cloud; capability for work grounded in your own documents, and every obligation that turns on where information travels, favours on-premises. Our provider checklist sets out the questions to ask either way.
What should we look for in a provider offering on-prem or private cloud options?
Where the model runs, who runs it, what leaves your environment per question, what is kept, whether your content can train a model, who can read it, which law governs the operator, and what happens when you leave. Our guide to choosing an on-premise or private cloud AI provider works through twelve questions.
What is on-premises AI infrastructure, and when is it needed?
It is the hardware and software that let an AI model run inside your own building: a server with enough processing power, the model itself, and the index of your documents. It is needed when your obligations or your clients' terms depend on information staying on your premises, or when you want AI that keeps working with no internet connection.
Is enterprise cloud AI safe for client data?
The reputable enterprise tiers are secure, certified products that do not train on your business data by default. The open question for a regulated business is not security practice but architecture and jurisdiction: your material is still transmitted to and processed on the provider's infrastructure, which for the major providers sits under foreign law. Whether that is acceptable depends on your specific obligations.
Where does my data go on an on-premise AI unit?
The AI hardware has no route to the internet for your data. Documents are stored, indexed and queried entirely on a unit inside your building, and staff reach it over your local network. Nothing is transmitted to any provider, so there is no transmission to analyse for compliance purposes.
Is on-premise AI less capable than ChatGPT or Claude?
For open-ended general knowledge, frontier cloud models lead. For work grounded in your own documents, matters and precedents, a local model tuned to your business answers from material those cloud models will never see. Most day-to-day professional work is the second kind.
What about private cloud or sovereign hosting in Australia?
It resolves the offshore question and suits businesses whose documents already live in the cloud. Built in-house it is a real engineering commitment; a managed service such as Ermos Edge removes that project. The remaining difference from on-premise is that processing happens in a data centre instead of your building, and whether that matters depends on your obligations.

See Dominion answering from your own documents.

A demonstration takes 30 minutes, uses no client data, and comes with no obligation.

Book a demo