Self-Hosted AI for Enterprise: What Actually Has to Stay Inside Your Environment?

Self-hosted AI means inference runs on a model endpoint inside an environment the organisation controls, rather than requiring an external model API. What should stay inside depends on the workload. The prompt, retrieved context, model and weights, output, and record of the exchange each have a different sensitivity and operational role.

Most organisations answer at company level. A blanket ban treats every summarisation or classification task as equally sensitive. A blanket exception assumes a single approved service is suitable for every data source and every decision. Neither approach examines what the workload actually touches.

Start with the data entering the AI step, the authority given to its output and the record required afterwards. A workload using public material to draft non-sensitive text has a different boundary from one retrieving case records, interpreting operational telemetry or preparing a consequential system action. Self-hosting is one architectural response, not a universal requirement for enterprise AI.

The workload boundary also prevents policy from becoming a debate about labels. Teams can describe a specific flow: the source systems involved, the fields retrieved, the endpoint called, the people allowed to review the result and the action that follows. Security, data and operations owners can then make one decision against observable facts instead of trying to approve or reject AI as a single category.

The phrase "the model processes our data" compresses five distinct things into one. Separate them before deciding where inference should run.

1. Prompt

The prompt contains the instruction given to the model. A generic instruction may carry little sensitivity, but prompts can also encode internal policy, decision criteria or operating logic. Self-hosting keeps the assembled instruction inside the controlled runtime and gives the organisation visibility into the prompt a workflow actually used.

Review the final assembled prompt, not only the reusable template. A template may look harmless while variables insert account details, incident notes or excerpts from an internal procedure at run time. Prompt construction is therefore part of the data flow and should be versioned with the workflow that creates it.

2. Retrieved context

Retrieved context is the material added so the model can answer about a specific case. This is usually where regulated, personal or commercially sensitive content enters. The key control is retrieval scope: which sources can be searched, which passages can be returned for this identity, and whether those passages leave the estate. Internal inference lets retrieval and processing remain inside the same boundary.

Hosting the model internally does not correct an over-broad retrieval design. The workflow still needs to pass the caller's identity and permissions into the retrieval step, limit the result to what the task requires and preserve enough evidence to explain which material informed the output.

3. Model and weights

The model and its weights are the versioned artefact that performs inference. Confidentiality is only one concern. Availability, update timing and change control matter too. With an internal endpoint, the organisation imports, registers and changes the model deliberately instead of inheriting an external service's default version. The sovereign AI architecture describes that operating model.

Version discipline matters because a workflow is more than its visible logic. The same prompt and context can produce a different result after a model change. Recording the model version makes that dependency explicit and lets an organisation test a proposed change against representative cases before assigning it to operational runs.

4. Output

Output inherits the sensitivity of the context that produced it and can introduce new risk if it is treated as authority. Self-hosting keeps the response inside the boundary. Governance still has to determine what can consume it, what deterministic checks follow, and whether a person must approve the next action.

That distinction separates interpretation from authority. A model can turn an unstructured document into proposed fields, but a deterministic check should still reject missing identifiers or values outside policy. A model can summarise a case, but the person accountable for the case can remain responsible for approving the resulting action.

5. Record of the exchange

The record connects input, retrieved context, model version, output, policy checks and reviewer decision to one run. It is often more sensitive than any single prompt because it reconstructs the whole case. Keeping that evidence under the organisation's retention policy makes later review possible without relying on a vendor console. See governance and audit for the wider execution record.

Retention should be intentional. Some teams need the complete exchange for investigation and audit. Others may retain references, hashes or selected fields because the source material has a shorter retention period. Self-hosting provides control over that choice. It does not decide the policy on the organisation's behalf.

Internal inference removes the external model API from the execution path. Prompts, retrieved context and outputs can stay inside the customer-controlled estate, and a disconnected workflow can continue to call a local endpoint. Other workflow integrations may still connect outside that estate when the organisation deliberately configures them.

Control also creates operating responsibilities. The customer owns model capacity, endpoint availability and model updates. A model version becomes a governed decision tied to workflow change control, rather than a default that can change outside the organisation's process.

Capacity is a workload property too. A background classification queue can tolerate delay in a way that an interactive approval process cannot. Endpoint health, concurrency and failure handling need to match the process. A governed workflow should expose a failed or unavailable inference step clearly rather than silently switching models or sending the payload across another boundary.

This is not an argument against cloud deployment. On premises, private cloud, sovereign cloud and air-gapped environments can all sit inside a customer-controlled boundary. The meaningful question is who controls the runtime and whether inference depends on an external vendor service. The operational definition of sovereignty applies the same test to execution, identity, secrets and evidence.

In a governed workflow, the model interprets. It can classify material, extract structure or prepare a recommendation. It does not receive open-ended authority to decide what happens next.

Deterministic checks in Rules + Code constrain the path around the AI step. They can validate required fields, enforce thresholds, reject an out-of-policy output or choose which approval path applies. Where accountability is consequential, human approval pauses and resumes execution with the relevant context attached.

The AI step then lands in the same execution record as connector actions and code nodes. Model version, inputs, output, checks, approval and system action remain part of one trace rather than separate records that have to be reconciled later. This is the practical basis of governed AI automation.

This structure also makes exceptions visible. If a reviewer rejects the interpretation, the record can show whether the run stopped, returned for correction or followed an approved alternate path. The important property is continuity: the exception remains attached to the work that caused it instead of becoming an email or ticket with no reliable connection to the original run.

Consider a case review triggered by a new record. The workflow retrieves only the approved context, applies deterministic preparation, calls an internal model endpoint for interpretation, and checks the result before presenting it to a named reviewer. Only an approved result reaches the target system. The audit step retains the complete path.

That sequence is deliberate: Trigger → Integration → Rules + Code → Sovereign AI → Human Approval → System Action → Audit. The sovereign AI solution view shows where this pattern applies across operational workflows.

Not every run needs every node. A low-consequence classification may proceed after deterministic validation, while a case affecting access, money or a regulated record may require named approval. The architecture remains consistent because interpretation, policy and authority are separate steps that can be composed according to the workload.

Use workload traits rather than a company-wide label. Several traits can apply at once, and the strictest requirement may determine the design. External model use can remain reasonable where the payload, service boundary and consequence have been explicitly accepted.

Apply the table to one real workflow with the people who own its data, infrastructure and outcome. Write down the boundary and the accepted external dependencies. A concrete decision can be reviewed when the workflow changes. A general policy statement usually cannot tell an engineer whether a particular payload is allowed to cross a particular endpoint.

The table is a design aid, not a compliance conclusion. Record the reasoning, name the boundary and test the resulting architecture. For externally exposed AI applications, Nexus Cerberus provides assessment and reporting against the application surface without sitting inline in the request path.