Serves local LLM, vision, embedding and reranker models on Apple Silicon Macs through OpenAI- and Anthropic-compatible APIs, with continuous batching and an SSD-backed KV cache.
Use it now
Keep it with up to 9 other tools you're setting up, then share them all as one link.
Install the CLI
Install it as its own instructions say.
Its page: github.com/jundot/omlx
Call the API
Auth An API key from Jun Kim
Its page: github.com/jundot/omlx
Compare
Press Compare to add it to the tray at the bottom of the window, then pick up to 2 others anywhere on the site.
- Access
- CLIAPI
- Auth
- API Key
- Skills
- 4
Skills
Tiered KV Cache
Keeps hot KV blocks in RAM and offloads cold ones to SSD, restoring matching prefixes from disk instead of recomputing, even after restarts.
OpenAI and Anthropic APIs
Exposes streaming chat and text completions, Anthropic Messages, embeddings, rerank and model listing endpoints under /v1 for existing clients.
Multi-Model Serving
Loads LLMs, VLMs, embedding models and rerankers in one server, with LRU eviction, model pinning, per-model TTL and a memory limit.
Agent Integrations
Sets up OpenClaw, OpenCode, Codex, Hermes Agent, Copilot, Pi and DeepSeek Harness against the local server from the admin dashboard.