The Paradigm That Has Not Run Its Course
Cloud-native infrastructure — the workload-centric, containerised, orchestrated infrastructure model that Kubernetes represents — has not been superseded. It remains the right infrastructure model for the overwhelming majority of enterprise application workloads in 2026, and the maturity of the tooling, the community, and the operational practices around Kubernetes makes it more capable and easier to operate than it was five years ago.
The claim that a new infrastructure paradigm is emerging is therefore not a claim that cloud-native is being replaced. It is a claim that AI workloads are sufficiently different from conventional application workloads in their infrastructure requirements that they are forcing adaptations to the cloud-native model that, at sufficient scale, constitute a new set of infrastructure design principles running alongside the existing ones.
Understanding which principles from the cloud-native model extend naturally to AI workloads, which require adaptation, and which genuinely do not apply is the architectural judgment that enterprise infrastructure leaders need to develop now, because the organisations that develop it correctly will design infrastructure that serves both conventional and AI workloads efficiently, and the organisations that get it wrong will build either AI-specific infrastructure that does not integrate with their conventional infrastructure or conventional infrastructure that does not support their AI workloads at the required performance.
What Transfers from Cloud-Native to AI Infrastructure
The cloud-native principles that extend naturally to AI infrastructure are the ones grounded in operational and governance fundamentals rather than in the specific characteristics of stateless application workloads.
Infrastructure as code is as applicable to AI infrastructure as to conventional application infrastructure, and more important in some respects: the GPU cluster configuration, the model serving deployment, and the AI data pipeline are all more complex than typical application infrastructure, and the reproducibility that infrastructure as code provides is more valuable for infrastructure that is harder to recreate from memory.
Declarative configuration and reconciliation loops work for AI infrastructure as they do for application infrastructure. The desire to declare the target state of the AI serving infrastructure and have the orchestration system converge toward that state is as valid for model serving as for application serving.
Observability as a platform capability applies to AI infrastructure, with the additions noted elsewhere: the AI workload’s observability requirements include model-specific metrics that conventional observability does not provide, but the infrastructure for collecting, storing, and alerting on metrics is shared between conventional and AI workloads.
Policy and governance automation through admission control and policy-as-code applies directly. The same OPA policies that enforce security standards for application deployments can enforce security standards for AI workload deployments, with AI-specific policies added for the AI-specific requirements.
What Requires Adaptation
The cloud-native principles that require adaptation for AI workloads are those that assume the stateless, fungible compute model that conventional application workloads conform to.
The scheduling model requires the most significant adaptation. Cloud-native scheduling assumes that compute is fungible: any node that satisfies the CPU and memory request is an acceptable placement for the workload. AI inference workloads violate this assumption: the GPU type, the GPU memory, and the GPU count are non-fungible requirements that must be satisfied by the specific node the workload is scheduled to. The adaptation is GPU-aware scheduling that accounts for these requirements, which Kubernetes supports through device plugins and extended resources but which requires configuration and management that goes beyond the default scheduler.
The resource model requires adaptation for the GPU memory characteristics described earlier: non-swappable, fixed per GPU, with allocation failure modes that differ from CPU and RAM allocation failures. The adaptation involves the GPU resource management approaches (time-slicing, MIG partitioning, full allocation) and the scheduling policies that account for them.
The networking model requires adaptation for the high-bandwidth, low-latency networking that distributed AI training workloads require. The standard CNI networking that serves application workloads is adequate for inference serving but may be insufficient for distributed training that requires GPU-to-GPU communication at network bandwidth levels that exceed what standard Ethernet CNI provides. InfiniBand or RDMA networking for training clusters is an infrastructure adaptation that is well outside the conventional cloud-native networking model.
The storage model requires adaptation for the large-file, high-sequential-read access patterns that model loading generates. The object storage model that serves application artefact distribution is not optimised for the repeated parallel loading of large model files to multiple GPU nodes. The adaptation involves storage architecture choices — high-performance object storage, parallel file systems, model caching layers — that are specific to AI workload storage patterns.
What Genuinely Does Not Transfer
The cloud-native assumption that does not transfer to AI inference workloads is horizontal scaling as the primary resilience and performance scaling mechanism.
Conventional application workloads scale horizontally: add more replicas when demand increases, remove replicas when demand decreases, and the stateless replicas are interchangeable in their ability to serve requests. This model works because the resource unit (a container with its CPU and memory allocation) can be made as small as necessary to match the demand increment, and the per-request resource consumption is consistent across requests.
AI inference workloads break these assumptions. The resource unit is not a container but a GPU, which is expensive, scarce, and comes in discrete sizes. The per-request resource consumption is highly variable: a short prompt to a large language model consumes GPU memory and compute very differently from a long document. The cold start time for AI inference, which involves loading large model files from storage into GPU memory, is measured in minutes rather than the seconds of container startup time.
The implication is that the scaling model for AI inference is fundamentally different from the scaling model for application workloads: AI inference scales through intelligent request routing and batching that maximises GPU utilisation, not through rapid horizontal scaling of lightweight containers. The infrastructure that serves AI inference needs to be designed for this scaling model, not adapted from the container auto-scaling model.
The Design Question That Determines the Outcome
The enterprise infrastructure architect who is planning for AI workloads alongside conventional application workloads has one design question that determines most of what follows: shared infrastructure or separate infrastructure?
Shared infrastructure — extending the existing Kubernetes cluster fleet with GPU node pools to serve AI workloads alongside application workloads — is the model that most enterprises with mature platform engineering are finding more operationally efficient. The governance, the monitoring, the deployment pipeline, and the operational processes are shared between application and AI workloads, reducing the operational burden and enforcing consistent standards.
Separate infrastructure — dedicated AI clusters with their own management, monitoring, and governance — is the model that produces the best performance for high-intensity AI workloads but at the cost of operational fragmentation and the platform engineering immaturity that comes from building a new infrastructure model from scratch.
The answer for most enterprises is shared infrastructure for inference and development workloads, with dedicated infrastructure only for the distributed training workloads that have bandwidth requirements that shared infrastructure cannot meet. This is the architectural conclusion that practitioner experience at the organisations running AI at scale in 2026 is consistently supporting.
The architectural work is not starting from scratch. It is extending what works with what AI requires.
