Introduction to Microsoft Foundry Local — How to Distribute a Local AI Environment Across Your Company with a Single Installer

Many companies face the dilemma of wanting to use cloud AI services without allowing sensitive data to leave the organization. Cloud AI presents challenges that are hard to overcome, such as data privacy concerns, bandwidth costs, and latency in on-site environments with unstable network connectivity.

In April 2026, Microsoft directly addressed this challenge by making Foundry Local generally available (GA). An end-to-end solution for running AI models on local devices, it requires no Azure subscription, has zero token charges, and works offline. This article provides an overview of Foundry Local, covering installation, basic operations, and key enterprise adoption takeaways.

What is Foundry Local?

Foundry Local is a local AI inference runtime provided by Microsoft. It allows you to run a subset of the same model catalog found in cloud-based Azure AI Foundry directly on your local PC or workstation.

Key features include:

• Lightweight runtime — An ONNX Runtime-based inference engine that can be embedded into applications at approximately 20 MB
• Curated model catalog — Provides models quantized and compressed for device use, including Phi-4, Qwen, DeepSeek, Mistral, GPT OSS, and Whisper
• Automatic hardware acceleration — Automatically detects GPUs/NPUs and selects the optimal execution provider, falling back to CPU if no GPU is present
• OpenAI-compatible API — Use the OpenAI SDK directly; existing code runs simply by replacing the endpoint
• Cross-platform — Supports Windows, macOS (Apple Silicon), and Linux
• Data never leaves the device — Both prompts and outputs are processed entirely locally

Because it uses the same SDKs (C#, JavaScript, Python, Rust) as the cloud version of Azure AI Foundry, you can smoothly implement a hybrid workflow: "first develop and test locally, then scale to the cloud in production."

Installation Steps

Windows

On Windows, you can install it with a single winget command.

winget install Microsoft.FoundryLocal

macOS（Apple Silicon）

Use Homebrew.

brew tap microsoft/foundrylocal
brew install foundrylocal

Linux

Download the installer from the GitHub Releases page, or use packages tailored to each distribution.

# GitHub リリースページからダウンロード
# https://aka.ms/foundry-local-installer

Verifying the Installation

After installation, verify operation by running the following in your terminal.

foundry --version

If a service connection error occurs, you can resolve it with foundry service restart.

Supported Models List and Selection Guide

Foundry Local features a lineup of models optimized for local execution. You can check the full list using the foundry model list command.

Chat Completion (Text Generation) Models

Speech Transcription Models

Multimodal Models

How to Choose a Model

• PCs with 8 GB of RAM or less → Lightweight models such as Phi-4-mini or Qwen 2.5-0.5B
• PCs equipped with a GPU (8 GB VRAM or more) → Mid-sized models such as Phi-4 or Qwen 2.5-7B
• High-performance workstations (16 GB VRAM or more) → GPT OSS 20B, Qwen 2.5-14B
• Audio transcription needed → Whisper
• Image recognition needed → Phi-4-multimodal

Models are automatically downloaded upon first use and cached locally.

Basic Usage

Testing Models via CLI

The simplest way to use it is running models interactively from the CLI.

# モデル一覧を確認
foundry model list

# GPU モデルだけを絞り込み
foundry model list --filter device=GPU

# モデルを対話モードで実行（初回はダウンロードが走る）
foundry model run phi-4-mini

Running foundry model run proceeds in order: model download → loading → interactive session. Enter a prompt, and you will receive a response right on the spot.

Using as an OpenAI-Compatible API

Foundry Local launches an OpenAI-compatible REST API server locally. Code written with the existing OpenAI SDK runs almost without modification.

# Python での例
from openai import OpenAI

# Foundry Local のエンドポイントに接続
client = OpenAI(
    base_url="http://localhost:PORT/v1",  # PORT は foundry service status で確認
    api_key="not-needed"  # ローカルなので認証不要
)

response = client.chat.completions.create(
    model="phi-4-mini",
    messages=[
        {"role": "user", "content": "社内の情報セキュリティポリシーを要約してください"}
    ]
)

print(response.choices[0].message.content)

You can check the port number with foundry service status.

Embedding into Apps via SDK

For more full-fledged development, use the SDKs available for each language.

# JavaScript
npm install foundry-local-sdk openai

# Python
pip install foundry-local-sdk openai

# C#
dotnet add package Microsoft.AI.Foundry.Local

# Rust
cargo add foundry-local-sdk

Using the SDK lets you programmatically control model downloading, loading, and inference, and even allows running in-process inference without going through a server.

Service Management Commands

It is also helpful to remember the service management commands used on a daily basis.

# サービスの状態確認（エンドポイント URL も表示）
foundry service status

# サービスの再起動
foundry service restart

# ロード中のモデル一覧
foundry service ps

# キャッシュの確認と管理
foundry cache list
foundry cache location
foundry cache remove <モデル名>

Use case

Here are scenarios where Foundry Local proves especially powerful.

Internal Chatbots

Ideal when you want to automate internal inquiries with AI, but do not want to send FAQ data or company policies to the cloud. Combining Foundry Local + RAG (retrieval-augmented generation) allows you to complete all responses based on internal knowledge entirely locally.

Summarizing and Analyzing Confidential Documents

When processing confidential documents such as contracts, financial reports, or performance reviews with AI, you want zero risk of data leakage. With Foundry Local, you can summarize and categorize without routing any data over the network.

Inference in Offline Environments

AI can be utilized in environments where internet connectivity is restricted, such as factories, medical facilities, and construction sites. By pre-downloading models, it operates completely offline.

Development and Testing Environments

You can experiment locally as many times as needed without worrying about cloud API usage costs. A hybrid setup switching to Azure AI Foundry in production is ideal.

Comparison with Ollama and LM Studio

Foundry Local is not the only local AI execution tool available. Here is an overview of how it differs from major alternatives.

Which One Should You Choose?

• When Foundry Local is suitable: Integration with Microsoft products is crucial, you want to embed AI directly into applications, you prefer automatic hardware optimization, or you are planning hybrid operations with Azure AI Foundry
• When Ollama is suitable: You want to experiment with a wide variety of models, prefer API-first development, or prioritize performance (inference speed) above all else
• When LM Studio is suitable: You want to try models easily through a GUI, require access for non-engineers, or are a beginner getting started with local AI

Enterprise Adoption Considerations

Here is a summary of points for IT administrators and business leaders to consider prior to adoption.

Distribution Methods

• Windows: In addition to winget, an MSIX package is provided, allowing integration with distribution tools such as Intune and SCCM
• macOS: Utilizes a Homebrew Tap. Can be used in conjunction with MDM (mobile device management) tools
• Pre-distributing models: Workflows can include downloading models with foundry model download <モデル名> and distributing the cache folder (checked via foundry cache location) over a shared drive

GPU Requirements

While it can run solely on a CPU without a GPU, inference speed will drop significantly. When distributing internally, it is critical to select model sizes matched to the specifications of the target PCs.

License

• Foundry Local itself is provided under the MIT License
• Each AI model has an individual license (verifiable with foundry model info <モデル名> --license)
• Execution providers such as NVIDIA CUDA and Intel OpenVINO also have individual licenses
• Because commercial use permissions vary by model, be sure to verify them prior to deployment

Reassuring Security Factors

• Prompts and outputs are processed entirely locally and are not sent to Microsoft
• Network communication occurs only during initial model download and when updating execution providers
• Because no Azure subscription is required, no dependency on cloud services is created

Conclusion

Microsoft Foundry Local is a tool that significantly lowers the barrier to running local AI.

• Installation finishes with a single command
• Provides security peace of mind because data never leaves the device
• Existing code and knowledge remain directly applicable thanks to its OpenAI-compatible API
• Cross-platform support across Windows, Mac, and Linux
• Enables hybrid local and cloud operations through integration with Azure AI Foundry

For enterprises that want to leverage AI but cannot send data externally, Foundry Local will be a compelling option. Start by installing it with winget install Microsoft.FoundryLocal (for Windows) and experience interacting with AI using foundry model run phi-4-mini.

---

If you need help adopting or building your AI environment, feel free to contact GleamHub.