VCF 9.1 is now generally available, and its Private AI Services update is not a cosmetic refresh. Broadcom has made a serious push to turn VMware Cloud Foundation into a credible platform for private, production-grade AI. The release adds support for newer GPU hardware, high-speed GPU networking, better model and GPU visibility, more data-source integrations, and deployment options for disconnected environments.
Do not misunderstand the message, though. VCF 9.1 does not magically turn every vSphere cluster into an AI platform. AI infrastructure still requires deliberate architecture: GPU capacity, fast networking, data governance, model lifecycle management, and a realistic operating model. What VCF 9.1 delivers is a far stronger foundation for organizations that want to run these workloads in their own environment without discarding the operational discipline they already use for virtual infrastructure.
The hardware story: NVIDIA Blackwell and AMD Instinct
The most obvious change is expanded accelerator support. VCF 9.1 adds support for NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs and NVIDIA HGX B200 systems. It also supports the NVIDIA HGX platform with NVLink Switch, which is relevant when a workload genuinely needs multiple GPUs to behave as one high-bandwidth resource. This matters for large language model training and high-throughput inference, where moving data between GPUs can become the bottleneck long before the raw compute capacity is exhausted.
For multi-node AI workloads, VCF 9.1 supports NVIDIA ConnectX-7 NICs and BlueField-3 DPUs with Enhanced DirectPath I/O. That enables GPUDirect RDMA and GPUDirect Storage, reducing the overhead involved in moving training data and model traffic across hosts. The useful part is not just lower latency. The useful part is that VCF aims to preserve familiar operational capabilities such as HA, DRS, vMotion, and live patching around the workload infrastructure while the AI stack consumes high-performance hardware.
VCF 9.1 also strengthens the AMD story. The AMD Instinct MI350 series brings substantial HBM3E capacity and high FP4/FP6 performance for organizations that want an alternative accelerator path. The practical benefit is not simply vendor choice. More GPU memory can reduce the number of accelerators required for a model, which can materially change the cost and physical footprint of an AI deployment. Broadcom and AMD position the MI350 platform for models with up to 520 billion parameters on a single GPU, subject to the obvious reality that model precision, context length, KV cache, and batch size determine whether a real workload fits.
There is an important caveat here. Do not read “virtualization benefits” as “a passthrough GPU can move around as freely as an ordinary virtual machine.” VCF gives you platform-level capabilities such as high availability, ESX Live Patch, Storage vMotion, snapshots, and virtual hardware hot-add or removal. However, GPU virtualization and vMotion capabilities for the AMD MI350 are identified as future support in the source presentation. Plan GPU placement, maintenance, and failure handling accordingly rather than assuming magic mobility.
DirectPath I/O becomes more practical
A traditional GPU passthrough design gives a virtual machine excellent performance but can make placement and lifecycle management awkward. VCF 9.1 improves that situation with DirectPath I/O support for GPUs.
Fixed DirectPath I/O remains the straightforward option when a single virtual machine or Kubernetes node needs exclusive GPU access. Dynamic DirectPath I/O is more interesting for a cloud platform. It maps a workload to a GPU device type instead of a hard-coded physical PCI address. That abstraction helps solve the initial placement problem and lets VCF apply more of its normal resource-management logic to GPU-enabled workloads.
This is the architectural point that matters: AI infrastructure cannot remain a collection of manually pinned, fragile servers if it is going to be operated as a private cloud service. Dynamic device assignment is not a replacement for careful capacity planning, but it is a necessary step away from treating every GPU host as a snowflake.
Private AI Services becomes a real service layer
The infrastructure is only useful if developers and data scientists can consume it without opening a ticket for every experiment. VCF Private AI Services is designed to provide that service layer.
Private AI Services is enabled per Supervisor Namespace. Each namespace can have its own API, user interface, indexing components, database access, model endpoints, and resource boundaries. Model endpoints run in a VKS cluster and can use CPU or GPU resources according to the selected inference engine. The presentation identifies vLLM, llama.cpp, and Infinity as engine choices.
This multi-tenant design matters because it gives platform teams a way to separate teams, applications, data sources, and resource consumption. Without that boundary, a so-called private AI platform quickly becomes a shared GPU playground with little accountability and even less governance.
CPU inference is not a compromise; it is a placement decision
VCF 9.1 adds CPU-based inference through llama.cpp. This is a sensible addition, not a headline-grabbing substitute for GPUs. Smaller models, development workloads, proof-of-concept environments, and applications with modest latency requirements do not always need an expensive GPU.
Running appropriate models on CPUs can reduce cost and keep scarce GPU capacity available for workloads that genuinely need acceleration. The wrong approach is to put every model on GPUs because “AI equals GPU.” The right approach is to profile the workload, set an acceptable latency target, measure throughput, and use the cheapest architecture that meets the requirement.
Better observability for GPU and model operations
AI projects fail operationally when teams cannot see where capacity is going or why a model endpoint is slow. VCF 9.1 addresses this with GPU observability in VCF Operations and AI Metrics Observability for model endpoints.
Operators can drill into GPU utilization, memory usage, temperature, and other granular statistics. On the model side, the new observability capabilities expose request and token-consumption information so MLOps teams can identify heavily used models, bottlenecks, and cost drivers. The public dashboards are designed for Grafana, so do not treat monitoring as an automatic checkbox. You still need to deploy the observability stack, define the metrics that matter, and act on the results.
For an AI platform, the important measures are not limited to GPU percentage. Track token throughput, time to first token, end-to-end response latency, cache utilization, endpoint availability, and cost per useful outcome. A GPU that shows high utilization is not automatically productive. It may simply be serving a badly designed application.
MCP, Google Workspace, and governance
VCF 9.1 adds support for the Model Context Protocol, or MCP. MCP provides a standard way for an AI assistant or agent to interact with internal content repositories and external tools such as Oracle, Microsoft SQL Server, ServiceNow, GitHub, Slack, and PostgreSQL. The advantage is obvious: you should not have to create and maintain a custom connector for every tool an agent needs to use.
But MCP is not a security product. It is a protocol. The security value comes from the governance around it: authentication, role-based access control, explicit tool authorization, and clear data boundaries. Administrators must decide which tools an agent can access and what the agent is permitted to do. If you expose an overly privileged MCP server to an agent, you have not built secure automation. You have built a fast route to a security incident.
The Data Indexing and Retrieval Service now also supports Google Workspace, including Google Docs, Sheets, and Slides. This lets organizations build knowledge bases for retrieval-augmented generation directly from internal Google Workspace content instead of forcing users to export documents to PDF first. The data is fetched, crawled, indexed, broken into text chunks, and stored with metadata and vector embeddings for retrieval.
Air-gapped deployment is now part of the story
For regulated industries, defense environments, and organizations with strict isolation requirements, an internet-connected AI platform is often unacceptable. VCF 9.1 supports air-gapped deployment of Private AI Services through the Artifact Mirroring Tool. The tool mirrors the required artifacts into a private OCI registry and generates the configuration files needed for installation in the disconnected environment.
This does not make air-gapped AI easy. It makes it achievable without inventing an unsupported deployment process. You still need to manage artifact provenance, image updates, vulnerability remediation, storage capacity, and operational procedures. But the necessary deployment path is now built into the platform rather than left as an afterthought.
The bottom line
The value of VCF 9.1 Private AI Services is not a single GPU model, an acronym, or a dashboard. Its value is the combination of modern accelerators, fast networking, private data integration, multi-tenant consumption, observability, and enterprise operations.
If your organization wants to run AI workloads in its own data center, VCF 9.1 is a substantial step forward. It gives platform teams a more credible way to deliver AI as an internal service while keeping control of the infrastructure, data, and operational model.
The next step is not to buy the largest GPU server you can find. The next step is to define your use cases, measure the model requirements, design the data and security boundaries, and validate the operational failure scenarios. Then decide whether the VCF 9.1 capabilities solve the real problem. That is how you avoid turning an AI initiative into an expensive collection of idle accelerators.
