Why GPU infrastructure needs its own security layer

For most of the last decade, security architecture for compute infrastructure followed a familiar pattern: firewalls and inspection sat at the network edge, host-based agents handled endpoint protection, and GPU servers were treated as just another category of compute to be wrapped in the same controls as everything else. That pattern is breaking down for AI infrastructure, and the reason is structural rather than incidental.

AI factories concentrate enormous value into a small number of physical locations. A single cluster can represent hundreds of millions of pounds of hardware, host proprietary model weights worth far more than the infrastructure itself, and process datasets that are frequently sensitive by definition, healthcare records, financial data, proprietary R&D. At the same time, these environments are typically multi-tenant, with different customers or teams sharing the same physical fabric, and they run at throughput levels where traditional security appliances become the bottleneck rather than the protection.

The industry’s response, visible across announcements from NVIDIA, Cisco, Check Point and Palo Alto Networks over the past few months, has converged on a common architectural answer: push security enforcement down to the data processing unit (DPU) layer, directly inside the GPU server, rather than relying on centralised appliances at the network perimeter.

The logic is straightforward once you consider the traffic pattern. In an AI factory, the front-end network, where inference and training requests arrive, where storage systems exchange datasets and checkpoints, and where multi-tenant workloads share servers, is where almost all traffic needs inspection, not a sampled subset. Centralised firewalls cannot operate at the line rate AI fabrics run at without becoming the slowest link in the system. Host-based agents consume CPU and GPU resources that are, by design, the most expensive and contended resources in the building. Enforcing security policy on the DPU itself, the same component already handling networking and storage offload for the GPU, allows inspection and segmentation to happen inline, at full throughput, without taxing the compute the infrastructure exists to provide.

What this looks like in practice is now shipping as reference architecture rather than concept. Cisco’s Secure AI Factory extends its firewall platform to NVIDIA’s DPU hardware, enabling stateful segmentation and air-gapped enforcement directly on AI servers, with the explicit goal of zero performance trade-off. Check Point has taken a similar approach with its own AI factory security blueprint, embedding firewall and threat prevention into the same DPU layer via NVIDIA’s software platform, while extending protection up the stack to cover inference APIs and model endpoints against prompt injection, data exfiltration and adversarial queries, attack categories that traditional web application firewalls were never designed to handle. NVIDIA has also extended this model into operational technology, partnering with several established security vendors to bring the same DPU-based, agentless zero-trust approach to the industrial and energy systems that power AI facilities themselves.

The common thread across all of these is security built in from the start rather than layered on afterwards, aligning with broader guidance from national cybersecurity bodies that AI infrastructure security needs to be embedded into the fabric, the hardware and the orchestration layer from inception, not retrofitted once systems are already in production.

For organisations building or operating AI infrastructure, particularly multi-tenant environments serving external customers, the practical implication is that security architecture decisions need to be made at the same time as compute and networking decisions, not after. A DPU-based security layer is now a mainstream, vendor-supported option rather than a niche configuration, and retrofitting it into a live multi-tenant environment is considerably harder than designing for it from the start. The infrastructure layer that used to sit comfortably below the security boundary is now, in AI factories, the security boundary itself.