With VCF 9.1, Broadcom introduced Zero Touch Provisioning (ZTP), a capability that takes a bare-metal server at a remote edge site from power-on to a fully registered host in a vSphere cluster, without an engineer on site. No manual configuration, no PXE infrastructure, no USB sticks. Just a network cable and a power button. While testing this in my lab, I ran into a problem that I suspect others will hit as well: my ESX installation consistently stalled at "Reading installation script" whenever the management network lived on a tagged VLAN. Here is what happened, why it happens, and the one-line fix that solved it.

How ZTP network boot works

Starting with VCF 9.1, the management of UEFI HTTPS Boot moves from Auto Deploy to Zero Touch Provisioning. Instead of the classic PXE chain with TFTP and iPXE, ZTP uses a UEFI HTTPS boot URL served straight from vCenter. Three components make this work. The Boot Routing Service (BRS) exposes a single vCenter-level boot URL (https://<vcenter>/vmw/brs/vbmx64.efi) that applies to every physical machine in your environment. The new first-stage bootloader, vSphere Boot Manager (vBM), replaces iPXE and submits hardware identifiers such as UUID, MAC address, and SMBIOS data to BRS. BRS evaluates those identifiers against the Auto Deploy rule engine and redirects the host to a cluster-specific boot URL served by the Network Boot Service (NBS), which hosts the ESX boot images on the vCenter filesystem under /vmw/nbs/clusters/<cluster-id>/.

You can watch this two-stage pipeline on the console of the booting host. After the BRS redirect, the host pulls its boot configuration from the NBS cluster URL:

Loading https://10.1.1.11/vmw/nbs/clusters/domain-c1358/529f948b-99e8-626c-91f5-54819ccaa5b9/image-0/boot.cfg

So far, so good. The installer boots, the modules load, and the automated installation kicks off. In my setup the host is connected to a trunk port, the management network is tagged VLAN 100, and IP addresses on that network are handed out by a DHCP server.

The problem

The installation ran fine right up to the point where ESX was installed and the installer needed to fetch and process the installation script. There it hung, indefinitely, on:

Reading installation script

The reason turned out to be simple once I understood what the installer was doing. When the ESX installer brings up its management interface, it creates vmk0 without a VLAN ID. That is perfectly fine when the management network is untagged or carried as the native VLAN on the switch port. But on a trunk port with a tagged management VLAN, an untagged vmk0 is talking into the void: its DHCP requests never reach the DHCP server on VLAN 100, the host never gets an IP address, and the installer can never retrieve the installation script it needs to continue. Hence the eternal "Reading installation script".

The fix: add vlanid to boot.cfg

The ESX installer supports a boot option called vlanid, which tells it to tag the installer network interface (and consequently vmk0) with a specific VLAN ID. The trick is getting that option into the ZTP boot flow, and the boot screen itself points you to the right place: the boot.cfg file that NBS serves to the host.

The cluster-specific Network Boot Folders live on the vCenter Server Appliance filesystem, matching the URL you see on the console. Take the path from the boot screen, in my case:

/vmw/nbs/clusters/domain-c1358/529f948b-99e8-626c-91f5-54819ccaa5b9/image-0/boot.cfg

SSH into the vCenter appliance, locate that boot.cfg, and open it. Inside you will find the kernelopt line that carries the boot options for the installer. All you need to do is append the vlanid option directly after weasel (the ESX installer):

kernelopt=runweasel vlanid=100

Save the file and boot the edge host again. This time the installer tags its network interface with VLAN 100, vmk0 comes up on the correct management VLAN, the DHCP server on the management network hands out an IP address, and the installation script is fetched and executed without any drama. The host completed the rest of the ZTP workflow and registered itself in the vSphere cluster exactly as advertised: zero on-site interaction.

Wrapping up

Zero Touch Provisioning in VCF 9.1 is an impressive piece of automation: a single UEFI HTTPS boot URL, no PXE or TFTP infrastructure, and hardware-aware routing to the right cluster image via BRS and NBS. But the current workflow assumes an untagged management network during the installation phase. If your edge sites use a tagged management VLAN with DHCP, as many production networks do, the installer will stall at "Reading installation script" because vmk0 comes up without a VLAN ID and never obtains an address.

The workaround is a single boot option: edit the boot.cfg that NBS serves for your cluster image and add vlanid=<your management VLAN> to the kernelopt line, right after weasel. It would be great to see this exposed as a first-class setting in the ZTP configuration in a future release, so the installer inherits the management VLAN from the cluster network profile automatically. Until then, this one-liner gets your tagged edge sites provisioned hands-free.

A first-time journey with the ESP32-C6 DevKitM-1 — from detecting the board over USB to a smooth rainbow LED cycling through the full spectrum, entirely from the command line.

Why the ESP32-C6?

The ESP32-C6 is Espressif's newest mainstream chip: RISC-V core, Wi-Fi 6, Bluetooth 5, and IEEE 802.15.4 (Thread/Zigbee) — all for € 12,50  What caught my eye was the DevKitM-1 form factor: compact, USB-C, and a WS2812B RGB LED already soldered on. A perfect first target for a no-IDE, command-line-only workflow on macOS.

Instead of reaching for the Arduino IDE or VS Code with PlatformIO, I wanted to understand each step: find the port, install the toolchain, compile, flash. This is what actually happened — including the parts that didn't go smoothly.

Step 1 — Find the Board

Plug in the ESP32-C6 over USB-C and macOS creates a serial device automatically. The factory firmware is already running — a rainbow demo that cycles through colors, confirming the hardware is alive.

ls /dev/cu.*
/dev/cu.Bluetooth-Incoming-Port
/dev/cu.usbserial-110          # ← there it is

One port. No guessing. /dev/cu.usbserial-110 is what we'll target for both flashing and monitoring.

Step 2 — Install arduino-cli

The Arduino IDE is a fine tool, but it carries a lot of GUI overhead that gets in the way of scripting and automation. arduino-cli gives you the full Arduino ecosystem — board packages, libraries, compilation, upload — as a clean command-line program.

brew install arduino-cli
arduino-cli version
arduino-cli  Version: 1.5.1

Step 3 — Add the ESP32 Board Package

Out of the box, arduino-cli knows nothing about Espressif boards. We point it at the official board index, then install the ESP32 core.

arduino-cli config add board_manager.additional_urls \
    https://espressif.github.io/arduino-esp32/package_esp32_index.json
arduino-cli core update-index
arduino-cli core install esp32:esp32@3.0.7

Heads up: During installation the Espressif download server returned a 504 Gateway Timeout on the xtensa-esp-elf-gdb debugger package. Workaround: download the ARM64 macOS build manually from the espressif/binutils-gdb releases page and extract it into ~/Library/Arduino15/packages/esp32/tools/xtensa-esp-elf-gdb/12.1_20231023/. After that, the installation completes normally.

Step 4 — The Right LED Library

The onboard LED is a WS2812B — a "smart" RGB LED with its own protocol. You can't just toggle a GPIO pin; you need a library that generates the precise timing the chip expects.

FastLED is the obvious choice, but version 3.10.5 has a compilation error against ESP32 core 3.0.7: it references dma_burst_size and MALLOC_CAP_CACHE_ALIGNED that were added in a later IDF version. Adafruit NeoPixel compiles cleanly:

arduino-cli lib install "Adafruit NeoPixel"
Installed Adafruit NeoPixel@1.15.5

Step 5 — The Code

Version 1: basic color cycling

The first version switches between red, green, blue, and off — one color every 500 ms. Useful for verifying the library works and that you have the right pin and color order (NEO_GRB for this board).

esp32c6-blink.ino C++ / Arduino

#include <Adafruit_NeoPixel.h>

#define LED_PIN   8
#define NUM_LEDS  1

Adafruit_NeoPixel strip(NUM_LEDS, LED_PIN, NEO_GRB + NEO_KHZ800);

void setup() {
    strip.begin();
    strip.setBrightness(50);
    strip.show();
}

void loop() {
    strip.setPixelColor(0, strip.Color(255, 0,   0));  strip.show(); delay(500);
    strip.setPixelColor(0, strip.Color(0,   255, 0));  strip.show(); delay(500);
    strip.setPixelColor(0, strip.Color(0,   0,   255));  strip.show(); delay(500);
    strip.setPixelColor(0, strip.Color(0,   0,   0));    strip.show(); delay(500);
}

Version 2: smooth rainbow with HSV

Mixing RGB values to get a smooth color transition is tedious — you'd need to interpolate three channels separately. The HSV color model makes this trivial: keep Saturation and Value fixed, and simply increment the Hue from 0 to 65535. The library maps that arc directly onto the LED, and you get the full visible spectrum with no math.

esp32c6-blink.ino C++ / Arduino

#include <Adafruit_NeoPixel.h>

#define LED_PIN   8
#define NUM_LEDS  1

Adafruit_NeoPixel strip(NUM_LEDS, LED_PIN, NEO_GRB + NEO_KHZ800);

void setup() {
    strip.begin();
    strip.show();
}

void loop() {
    for (long hue = 0; hue < 65536; hue += 32) {
        strip.setPixelColor(0, strip.ColorHSV(hue, 255, 150));
        strip.show();
        delay(8); // 2048 steps × 8 ms ≈ 16 s per full cycle
    }
}

The three arguments to ColorHSV: hue (0–65535), saturation (255 = fully saturated), value/brightness (150 out of 255 — bright but not eye-searing). Changing hue += 32 to a larger step speeds up the cycle; changing delay(8) fine-tunes the smoothness.

Step 6 — Compile and Flash

Two commands. The --fqbn flag is the "fully qualified board name" that tells the compiler exactly which chip it's targeting.

arduino-cli compile \
    --fqbn esp32:esp32:esp32c6 \
    ./esp32c6-blink \
    --output-dir /tmp/esp32c6-build
Sketch uses 221778 bytes (16%) of program storage space.

arduino-cli upload \
    --fqbn esp32:esp32:esp32c6 \
    --port /dev/cu.usbserial-110 \
    --input-dir /tmp/esp32c6-build \
    ./esp32c6-blink
Hash of data verified.
Hard resetting via RTS pin...

The board resets automatically after flashing and starts running the new program immediately. No button presses, no manual reboot.

What I Learned

What's Next

The ESP32-C6 has Wi-Fi 6, Bluetooth 5, and Thread built in — a single LED barely scratches the surface. Logical next steps: read a DHT22 temperature sensor and push the data to an MQTT broker over Wi-Fi, or use the LED as a status indicator for something external (a build pipeline, a home automation event). The command-line workflow scales to all of it.

VCF 9.1 is now generally available, and its Private AI Services update is not a cosmetic refresh. Broadcom has made a serious push to turn VMware Cloud Foundation into a credible platform for private, production-grade AI. The release adds support for newer GPU hardware, high-speed GPU networking, better model and GPU visibility, more data-source integrations, and deployment options for disconnected environments.

Do not misunderstand the message, though. VCF 9.1 does not magically turn every vSphere cluster into an AI platform. AI infrastructure still requires deliberate architecture: GPU capacity, fast networking, data governance, model lifecycle management, and a realistic operating model. What VCF 9.1 delivers is a far stronger foundation for organizations that want to run these workloads in their own environment without discarding the operational discipline they already use for virtual infrastructure.

The hardware story: NVIDIA Blackwell and AMD Instinct

The most obvious change is expanded accelerator support. VCF 9.1 adds support for NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs and NVIDIA HGX B200 systems. It also supports the NVIDIA HGX platform with NVLink Switch, which is relevant when a workload genuinely needs multiple GPUs to behave as one high-bandwidth resource. This matters for large language model training and high-throughput inference, where moving data between GPUs can become the bottleneck long before the raw compute capacity is exhausted.

For multi-node AI workloads, VCF 9.1 supports NVIDIA ConnectX-7 NICs and BlueField-3 DPUs with Enhanced DirectPath I/O. That enables GPUDirect RDMA and GPUDirect Storage, reducing the overhead involved in moving training data and model traffic across hosts. The useful part is not just lower latency. The useful part is that VCF aims to preserve familiar operational capabilities such as HA, DRS, vMotion, and live patching around the workload infrastructure while the AI stack consumes high-performance hardware.

VCF 9.1 also strengthens the AMD story. The AMD Instinct MI350 series brings substantial HBM3E capacity and high FP4/FP6 performance for organizations that want an alternative accelerator path. The practical benefit is not simply vendor choice. More GPU memory can reduce the number of accelerators required for a model, which can materially change the cost and physical footprint of an AI deployment. Broadcom and AMD position the MI350 platform for models with up to 520 billion parameters on a single GPU, subject to the obvious reality that model precision, context length, KV cache, and batch size determine whether a real workload fits.

There is an important caveat here. Do not read “virtualization benefits” as “a passthrough GPU can move around as freely as an ordinary virtual machine.” VCF gives you platform-level capabilities such as high availability, ESX Live Patch, Storage vMotion, snapshots, and virtual hardware hot-add or removal. However, GPU virtualization and vMotion capabilities for the AMD MI350 are identified as future support in the source presentation. Plan GPU placement, maintenance, and failure handling accordingly rather than assuming magic mobility.

DirectPath I/O becomes more practical

A traditional GPU passthrough design gives a virtual machine excellent performance but can make placement and lifecycle management awkward. VCF 9.1 improves that situation with DirectPath I/O support for GPUs.

Fixed DirectPath I/O remains the straightforward option when a single virtual machine or Kubernetes node needs exclusive GPU access. Dynamic DirectPath I/O is more interesting for a cloud platform. It maps a workload to a GPU device type instead of a hard-coded physical PCI address. That abstraction helps solve the initial placement problem and lets VCF apply more of its normal resource-management logic to GPU-enabled workloads.

This is the architectural point that matters: AI infrastructure cannot remain a collection of manually pinned, fragile servers if it is going to be operated as a private cloud service. Dynamic device assignment is not a replacement for careful capacity planning, but it is a necessary step away from treating every GPU host as a snowflake.

Private AI Services becomes a real service layer

The infrastructure is only useful if developers and data scientists can consume it without opening a ticket for every experiment. VCF Private AI Services is designed to provide that service layer.

Private AI Services is enabled per Supervisor Namespace. Each namespace can have its own API, user interface, indexing components, database access, model endpoints, and resource boundaries. Model endpoints run in a VKS cluster and can use CPU or GPU resources according to the selected inference engine. The presentation identifies vLLM, llama.cpp, and Infinity as engine choices.

This multi-tenant design matters because it gives platform teams a way to separate teams, applications, data sources, and resource consumption. Without that boundary, a so-called private AI platform quickly becomes a shared GPU playground with little accountability and even less governance.

CPU inference is not a compromise; it is a placement decision

VCF 9.1 adds CPU-based inference through llama.cpp. This is a sensible addition, not a headline-grabbing substitute for GPUs. Smaller models, development workloads, proof-of-concept environments, and applications with modest latency requirements do not always need an expensive GPU.

Running appropriate models on CPUs can reduce cost and keep scarce GPU capacity available for workloads that genuinely need acceleration. The wrong approach is to put every model on GPUs because “AI equals GPU.” The right approach is to profile the workload, set an acceptable latency target, measure throughput, and use the cheapest architecture that meets the requirement.

Better observability for GPU and model operations

AI projects fail operationally when teams cannot see where capacity is going or why a model endpoint is slow. VCF 9.1 addresses this with GPU observability in VCF Operations and AI Metrics Observability for model endpoints.

Operators can drill into GPU utilization, memory usage, temperature, and other granular statistics. On the model side, the new observability capabilities expose request and token-consumption information so MLOps teams can identify heavily used models, bottlenecks, and cost drivers. The public dashboards are designed for Grafana, so do not treat monitoring as an automatic checkbox. You still need to deploy the observability stack, define the metrics that matter, and act on the results.

For an AI platform, the important measures are not limited to GPU percentage. Track token throughput, time to first token, end-to-end response latency, cache utilization, endpoint availability, and cost per useful outcome. A GPU that shows high utilization is not automatically productive. It may simply be serving a badly designed application.

MCP, Google Workspace, and governance

VCF 9.1 adds support for the Model Context Protocol, or MCP. MCP provides a standard way for an AI assistant or agent to interact with internal content repositories and external tools such as Oracle, Microsoft SQL Server, ServiceNow, GitHub, Slack, and PostgreSQL. The advantage is obvious: you should not have to create and maintain a custom connector for every tool an agent needs to use.

But MCP is not a security product. It is a protocol. The security value comes from the governance around it: authentication, role-based access control, explicit tool authorization, and clear data boundaries. Administrators must decide which tools an agent can access and what the agent is permitted to do. If you expose an overly privileged MCP server to an agent, you have not built secure automation. You have built a fast route to a security incident.

The Data Indexing and Retrieval Service now also supports Google Workspace, including Google Docs, Sheets, and Slides. This lets organizations build knowledge bases for retrieval-augmented generation directly from internal Google Workspace content instead of forcing users to export documents to PDF first. The data is fetched, crawled, indexed, broken into text chunks, and stored with metadata and vector embeddings for retrieval.

Air-gapped deployment is now part of the story

For regulated industries, defense environments, and organizations with strict isolation requirements, an internet-connected AI platform is often unacceptable. VCF 9.1 supports air-gapped deployment of Private AI Services through the Artifact Mirroring Tool. The tool mirrors the required artifacts into a private OCI registry and generates the configuration files needed for installation in the disconnected environment.

This does not make air-gapped AI easy. It makes it achievable without inventing an unsupported deployment process. You still need to manage artifact provenance, image updates, vulnerability remediation, storage capacity, and operational procedures. But the necessary deployment path is now built into the platform rather than left as an afterthought.

The bottom line

The value of VCF 9.1 Private AI Services is not a single GPU model, an acronym, or a dashboard. Its value is the combination of modern accelerators, fast networking, private data integration, multi-tenant consumption, observability, and enterprise operations.

If your organization wants to run AI workloads in its own data center, VCF 9.1 is a substantial step forward. It gives platform teams a more credible way to deliver AI as an internal service while keeping control of the infrastructure, data, and operational model.

The next step is not to buy the largest GPU server you can find. The next step is to define your use cases, measure the model requirements, design the data and security boundaries, and validate the operational failure scenarios. Then decide whether the VCF 9.1 capabilities solve the real problem. That is how you avoid turning an AI initiative into an expensive collection of idle accelerators.
 

Application modernization and the shift toward containerization are forcing IT infrastructures to evolve. Where organizations once maintained a strict separation between virtual machines and container environments, VMware Cloud Foundation (VCF) 9 fundamentally changes this dynamic. With VCF 9, containers are promoted to first-class citizens, running directly alongside traditional VMs on the same platform. This article takes a deep dive into the VCF 9 architecture for native containers, the role of vSphere Pods and the new Container Service (introduced in VCF 9.1), and explains how administrators and DevOps engineers can rapidly deploy workloads without requiring deep Kubernetes expertise.

The Architecture: vSphere Supervisor and vSphere Pods

To understand native containers on vSphere, we need to examine the underlying architecture of VCF 9. The heart of this integration is the vSphere Supervisor. Once enabled on a VCF Workload Domain, the vSphere cluster is transformed into a platform capable of orchestrating not only virtual machines, but also Kubernetes workloads.

Within the Supervisor, workloads are organized using vSphere Namespaces — logical boundaries (resource pools) to which compute, memory, and storage resources are assigned through Storage Policy Based Management. Within these namespaces, DevOps engineers can deploy containers directly.

What Is a vSphere Pod?

The fundamental unit for running a native container in VCF is the vSphere Pod. Unlike a standard Kubernetes pod, which shares the Linux kernel of the underlying worker node with other pods, a vSphere Pod provides strict hardware-level isolation.

A vSphere Pod is essentially an ultra-lightweight virtual machine, powered by a purpose-built Photon OS kernel. Inside this PodVM, the container runtime (such as containerd) executes the OCI-compliant (Open Container Initiative) container image. This architecture delivers three critical advantages.

Strong Isolation. Because there is no shared kernel, the risk of container escapes or noisy neighbor interference is virtually eliminated. This makes vSphere Pods ideal for multi-tenant or security-sensitive environments.

Performance and Speed. Despite being technically a virtual machine, the footprint is so minimal that startup times are comparable to those of traditional containers.

Integrated Management. vSphere DRS (Distributed Resource Scheduler) manages the placement of vSphere Pods across ESXi hosts in exactly the same way it handles virtual machines, ensuring optimal resource distribution at all times.

The VCF Container Service: Containers Without Kubernetes Complexity

While running containers through Kubernetes is powerful, setting up and managing Kubernetes clusters — for example through vSphere Kubernetes Service (VKS) — requires considerable expertise. VCF 9.1 addresses this with the introduction of the Container Service.

The Container Service is a simplified, managed interface available through VCF Automation or the Local Consumption Interface (LCI). It acts as a high-level wrapper around vSphere Pods, enabling users to simply provide an OCI image URL and let the platform handle the rest. The platform intelligently determines whether a workload should run as a Kubernetes Deployment or StatefulSet based on the workload specification.

Use Cases for the Container Service

The Container Service is the right tool in the following scenarios:

• You need to run a container quickly without provisioning a full Kubernetes cluster.
• CI/CD pipelines require short-lived ephemeral workloads such as build jobs or test runners.
• You are building a hybrid application where a database runs in a traditional VM and the frontend runs in a container, both managed from the same vSphere interface.
• Your team lacks dedicated Kubernetes expertise but wants to benefit from containerized workloads.

Hands-On: Deploying a Native Container

There are two primary methods for deploying a native container (vSphere Pod) in VCF 9: through the graphical Container Service interface (VCF 9.1 and later) or declaratively via the command line using kubectl and YAML manifests.

Method 1: The Graphical Container Service (VCF 9.1)

With the Container Service, deploying an application — for example an NGINX web server — is reduced to a few steps.

Step 1 — Access. Log in to VCF Automation or the Local Consumption Interface and navigate to the Container Service.

Step 2 — Basic Configuration. Provide a name for the container and enter the image URL (e.g., ghcr.io/nginxinc/nginx-unprivileged). Set the CPU and memory requests; the platform provides sensible defaults.

Step 3 — Advanced Options (Optional). For more specific requirements, the Container Service exposes the full range of capabilities you would expect from a Kubernetes workload: storage (PersistentVolumes backed by vSAN), load balancing (NSX-provisioned LoadBalancer services), ConfigMaps, Secrets, and sidecar containers.

As you build the configuration in the UI, VCF automatically generates the corresponding Kubernetes YAML in real time. This manifest can be downloaded and used directly in any automation pipeline or GitOps workflow.

Method 2: Declarative Deployment via kubectl (YAML)

For platform teams working with Infrastructure as Code, the traditional Kubernetes-native approach remains fully supported. Once a vSphere Administrator has created a vSphere Namespace and assigned the appropriate permissions, a DevOps engineer can connect to the Supervisor Control Plane endpoint and apply manifests directly.

The following is an example manifest that deploys a vSphere Pod running NGINX:

apiVersion: v1

kind: Pod

metadata:

  name: nginx-native-pod

  namespace: my-vsphere-namespace

spec:

  containers:

  - name: nginx

    image: nginx:latest

    ports:

    - containerPort: 80

    resources:

      requests:

        memory: "512Mi"

        cpu: "500m"

      limits:

        memory: "1Gi"

        cpu: "1000m"

Apply the manifest using the standard kubectl command:

kubectl --kubeconfig vsphere-kubeconfig apply -f nginx-pod.yaml

The Supervisor translates this Kubernetes object directly into a vSphere Pod, DRS places it on an appropriate ESXi host, and NSX handles network isolation — all transparently, without any additional configuration required from the engineer.

Network and Storage Integration

Networking. Every vSphere Pod receives a unique IP address assigned by NSX. Network isolation is enforced at the virtual NIC level, not merely through software-defined rules within a shared operating system. This means container traffic can be routed, inspected, and filtered through NSX Distributed Firewall policies in exactly the same way as VM traffic — a significant advantage for security and compliance teams.

Storage. vSphere Pods consume storage through Container Native Storage (CNS). Persistent volumes are attached directly as VMDKs to the PodVM, governed by the Storage Policy Based Management (SPBM) rules defined in vCenter. This guarantees not only data persistence across Pod restarts, but also predictable I/O performance, since resource guarantees are enforced at the VM level rather than relying on shared OS-level scheduling.

Conclusion

VMware Cloud Foundation 9 definitively bridges the gap between traditional infrastructure and modern application development. By running containers as vSphere Pods directly on the hypervisor, organizations gain the speed and portability of containers combined with the strict security and isolation of virtual machines. With the addition of the Container Service in VCF 9.1, the barrier to getting started with containers is lower than ever. It is one platform, with one unified operational experience, for all workloads.

What if you could run your own AI model locally — no subscription, no data leaving your computer, no internet required? With LM Studio, that's exactly what you get. In this post I'll walk you through installing it on a Mac, loading a model, and making your first API call.

What is LM Studio?

LM Studio is a desktop application for macOS, Windows, and Linux that lets you download and run large language models (LLMs) locally. It comes with a built-in chat interface and — importantly for developers — a local API server that is fully compatible with the OpenAI API. That means any app or script already talking to ChatGPT can be pointed at your own machine instead.

Installation

On a Mac with Apple Silicon, installation is straightforward. Download the latest .dmg from the LM Studio website, open it, and drag the app to your Applications folder. The whole process takes about a minute.

Your First Model: Gemma 4

When you open LM Studio for the first time, it asks you to download a model. The default suggestion is Google's Gemma-4-e4b — a solid choice for a first model. It's the effective 4B parameter version of Gemma 4 and supports text, image input, reasoning, and tool calling. The download is about 6.86 GB.

Starting the API Server

Once your model is loaded, head to the Developer tab (the </> icon in the left sidebar) and click Start Server. LM Studio will spin up a local API server at http://localhost:1234.

You can verify it's running by opening a browser and visiting:

http://localhost:1234/v1/models

Or from the terminal:

curl -s http://localhost:1234/v1/models | python3 -m json.tool

You'll get back a JSON list of the models currently loaded — in our case Gemma-4-e4b and a bundled embedding model.

Sending Your First Message via the API

The API is OpenAI-compatible, so the request format will look familiar:

curl -s http://localhost:1234/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"google/gemma-4-e4b","messages":[{"role":"user","content":"Say hello and introduce yourself briefly."}]}'

The model responds almost instantly and — a nice touch — the response includes a reasoning_content field showing the internal thinking process before the final answer was produced. Gemma 4 reasons before it responds, similar to OpenAI's o1 series.

What Can You Build With This?

Because the local server speaks the OpenAI protocol, you can:

• Point Python or Node.js scripts at http://localhost:1234/v1 using the standard openai SDK
• Connect tools like AnythingLLM to build a private RAG setup over your own documents
• Use editor plugins like Continue.dev for local AI-assisted coding
• Experiment with different open-source models (Llama, Mistral, Phi, Qwen, and more) without any per-token costs

from openai import OpenAI

client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")

MODEL = "google/gemma-4-e4b"
history = []

print("Chat gestart. Typ 'exit' of 'quit' om te stoppen.\n")

while True:
    user_input = input("Jij: ").strip()
    if not user_input:
        continue
    if user_input.lower() in ("exit", "quit"):
        print("Tot ziens!")
        break

    history.append({"role": "user", "content": user_input})

    print("Model: ", end="", flush=True)
    response_text = ""

    stream = client.chat.completions.create(
        model=MODEL,
        messages=history,
        stream=True,
    )

    for chunk in stream:
        delta = chunk.choices[0].delta.content
        if delta:
            print(delta, end="", flush=True)
            response_text += delta

    print()
    history.append({"role": "assistant", "content": response_text})

Why This Matters

Running AI locally gives you full control: no data leaves your machine, there are no API costs, and you're not dependent on external services. For development, testing, or privacy-sensitive workflows, a local model is a powerful addition to your toolkit.

LM Studio makes the barrier to entry remarkably low. If you have an Apple Silicon Mac with enough RAM, you can go from zero to a running local AI in under 15 minutes.