Download OrionPod
Available for macOS, Windows and Linux
v0.6.0-beta
Bulma LatestMulti-instance with CLI
macOS
Universal (Apple Silicon + Intel) · DMG · ~59 MB
Linux
x64 · AppImage · ~109 MB
Linux
x64 · deb · ~40 MB
Linux
x64 · rpm · ~40 MB
Windows
x64 · Installer · ~22 MB
- › Models now run in llama.cpp's own `llama-server`, bundled with OrionPod and kept close to upstream (release pinned in `scripts/llama-server.version`). Each model's own chat template is used, so new model families work as soon as llama.cpp supports them
- › Tool calling (`tools`, `tool_choice`), structured output (`response_format` with a JSON schema), image input for vision models (the `mmproj` projector next to a model is picked up automatically) and reranking (`/v1/rerank` with a reranker model) work over the API
- › Instances serve several requests at once (continuous batching) and reuse the cached prompt across turns, so agents and multiple clients no longer queue behind each other
- › Anthropic Messages API (`/v1/messages`, `/v1/messages/count_tokens`) and OpenAI Responses API (`/v1/responses`): Claude Code, Codex and OpenCode run against a local model. API keys are accepted as `Authorization: Bearer` or `x-api-key`
- › `orionpod launch claude|codex|opencode [--model X]` starts the agent wired to a local instance (starting one sized for agents if needed). Instances → "Use with…" shows the same setup as copyable commands. Claude Code is told the model's real context window and to send no telemetry while it runs here
- › Context is sized automatically by default: as much as the model was trained for and free memory allows, up to 32,768 tokens (was a fixed 2,048). Setting a context length still means exactly that length. Requests that don't fit get a clear `context_length_exceeded` error naming the window, and the chat shows when older messages were left out
- › Fixed the chat budgeting every model against a 4,096-token window whatever its real context
- › Per-instance and per-model engine settings: KV cache precision (`q8_0` roughly halves context memory), flash attention, parallel requests, speculative decoding (MTP for models with an MTP head, n-gram, or a draft model), MoE expert offload to the CPU, GPU layers, chat template override and extra `llama-server` arguments. Set them when starting an instance, per model (Models → Settings) or as defaults (Settings → Engine). The instance card shows what actually applied, and why when something couldn't
- › Fit advisor (Models → Settings, `orionpod fit <model>`): reads the model's architecture and llama.cpp's view of your GPUs, then explains whether it fits and which context, KV cache, GPU layers or MoE offload make it fit
- › `orionpod bench` and Instances → Inspect → "Check & benchmark" measure prefill and generation speed, first-token time and speculative acceptance on this machine, and keep a history to compare settings
- › `orionpod check <model>` and the same panel run an agent compatibility suite (chat, streaming, tool call, tool-result round trip, JSON schema, Anthropic Messages with tools, Responses, long-context recall). Results show as a badge on the model, and `orionpod check --matrix` prints the matrix for every model checked
- › Request inspector: Instances → Inspect lists every request an instance served with route, status, duration, first-token time, tokens, speed, cache hits, speculative acceptance, finish reason and the request and response bodies (kept in memory only; `orionpod requests`)
- › Prometheus metrics at `/metrics` on every instance (requests by route and status, tokens, latency and first-token histograms, context-limit hits, speculative tokens, memory) followed by llama.cpp's own metrics. Optional OpenTelemetry trace export per request (`[server].otel_endpoint` or `--otel-endpoint`)
- › Gateway: `orionpod gateway` (or Instances → "Start gateway", optionally with the app) serves every model on one port (8400), routes by the request's `model` (with aliases), starts an instance on demand and stops gateway-started ones after 15 idle minutes or when more than two run
- › Use models other apps downloaded without copying them: LM Studio, Ollama, the Hugging Face cache, llama.cpp's cache, Jan and GPT4All (Models → "From other apps", `orionpod import`). Removing a linked model never deletes its file
- › `orionpod pull` downloads every part of split GGUFs and the vision projector of multimodal repos, and pulls MLX repos as MLX models
- › MLX engine on Apple Silicon (optional, needs `pip install mlx-lm`): MLX models from Hugging Face run in `mlx_lm.server` behind the same API, metrics and inspector
- › Find instances on your network: network-bound instances announce themselves over mDNS (without their key), and Instances → "On your network" lists them so the chat can use one with its key. No account involved. HTTPS with `--tls-cert` / `--tls-key`. On macOS the permission prompt says why it is needed
- › Secure by default: an instance bound beyond localhost refuses to start without an API key (`--allow-unauthenticated` to override on a trusted network), and each engine runs on a private loopback port behind a random key
- › Local only, and checkable: a network ledger records every outbound connection OrionPod makes (Settings → Privacy & network, `orionpod network`). The update check can be turned off, and the Observability page no longer contacts a third-party host to test connectivity
- › Settings → Privacy & network shows the open-source notices (llama.cpp and every bundled dependency). The About dialog credits llama.cpp and shows a release's SHA-256 when an update is available
- › MCP servers: add Model Context Protocol servers (stdio or HTTP, Claude Desktop / Cursor `mcpServers` layout) in Settings → Tools or `~/.orionpod/mcp.json`. Their tools join the agent tools in the chat and `orionpod run`, and each MCP tool call asks for approval first (`orionpod mcp` lists them)
- › Settings → Tools lists every tool the chat's agent can call, built-in and from each MCP server, with its description and the server's connection state. Each tool has its own switch, and the chat and `orionpod run` only get the tools that are on; the Agent tools switch (and the chat's Tools chip) still turns them all on or off. Add an MCP server with a form (name, then a command and arguments or a URL and headers), edit, disable or remove it there, or use "Edit JSON" for the raw `mcp.json`. `orionpod mcp` marks switched-off tools
- › Plugins from the AgentHub marketplace: Settings → Tools → Browse plugins (or `orionpod plugins search|info|install|update|rm|list`) searches its catalog and installs the plugins' MCP servers and skills. Before anything is installed, a review screen shows the source, the pinned commit, the exact command each MCP server runs, the skills, missing programs (like `npx`) and the Claude Code-only parts left out. Plugins live in `~/.agents/plugins`, a folder other agents can read too; removing one also removes its MCP servers
- › Skills: the chat and `orionpod run` offer the model every skill installed from a plugin or put in `~/.agents/skills` (a `SKILL.md` folder). The model sees only each skill's name and description and loads one with the `load_skill` tool when a task needs it; each skill has its own switch in Settings → Tools
- › Run a skill yourself with `/`: typing `/` in the chat lists your skills (arrow keys and Enter or Tab to pick), and `/uv add requests to my project` sends the request with the `uv` skill's instructions included, so the model follows them without having to call a tool. Works whether or not agent tools are on; the message shows a skill tag. `orionpod run` takes the same `/<skill> …` commands and lists them with `/skills`
- › MCP servers and git now start from an app opened from Finder even when `node`, `npx`, `uvx` or `git` live in Homebrew or similar: the app also searches the login shell's `PATH`
- › Reasoning models: their thinking shows as a collapsible "Reasoning" section above the answer
- › `orionpod doctor` checks the engine, GPUs, MLX, memory, running instances and the network ledger. `orionpod ps` shows each instance's context and speed
- › An engine that crashes is restarted automatically (up to three times in ten minutes), and engines left behind by a killed instance are cleaned up
- › New look: a translucent liquid-glass interface over a slowly drifting aurora background, gradient primary buttons, glowing gradient sliders and a larger, book-like type scale set in JetBrains Mono by default
- › Settings → Appearance: pick the interface typeface from built-in fonts (JetBrains Mono, Literata, Inter, Geist, Geist Mono), any font installed on your computer, or a font file you add (copied to `~/.orionpod/fonts/`), and scale all text from 85% to 140% with a live preview
- › An MCP server that doesn't answer no longer freezes the chat: each server gets 15 seconds to connect, a failed one is retried after 30 seconds (or right away from Settings → Tools → Test connections), other messages don't wait on it, and Stop works while servers are still connecting
- › The empty chat screen is a still, pixelated lake under frosted glass: the OrionPod logo drops into it every few seconds and the pointer stirs it, with ripples running off the edges instead of bouncing back (off when the system asks for reduced motion)
- › Disabled sliders look disabled: a grey, hatched track with a flat thumb and no glow. Sidebar opacity is disabled on Windows and Linux, where the sidebar is always solid
- › Context length is a slider that snaps from Auto through 2K to 256K (capped at what the model was trained for), in Settings → Engine and per model
- › Bento layouts across the app: Settings → Model and Advanced, the Models page (library size, the loaded model and a grid of model cards with size, parameters and context), each model's settings (fit, benchmark and checks, engine settings) and the Instances page
- › Observability is a bento dashboard with new glass charts: smooth glowing area charts with hover readouts (generation speed, latency), stacked prompt/completion bars per request, donuts for the token mix and memory (OrionPod, other apps, free), arc gauges for the context window and CPU, and live CPU and network throughput graphs
- › The network status dot breathes while connected, and instance health dots pulse while an instance is serving
- › Running instances stand out on the Instances page: a tinted glass card with a glowing border, blue-violet for model instances (strongest for the chat's), teal for the gateway and red when an instance has a problem
- › Starting an instance or the gateway is a single play button
- › Instance logs are color-coded by level and source, with search, level and source filters and newest/oldest sorting; llama-server output is parsed the same way
- › Instance metrics show as cards: each Prometheus metric with its type, origin (OrionPod or llama.cpp), help text, labels and a readable value (histograms show count, average, p50 and p95). Search, filter by origin and type, sort by name, value or type, or switch to the raw text
- › The request inspector can search, filter by outcome (succeeded, failed, in flight) and route, and sort by time, duration, tokens or speed
- › Chat and the welcome screens use more of a wide window
- › New "Midnight" look: a deep navy palette taken from the OrionPod brand, one restrained ice-blue accent, and consistent buttons, fields, sliders and badges throughout
- › Redesigned layout: the sidebar now holds navigation, your recent chats (rename, export and delete on hover) and a model status card with system usage, downloads and the version. The separate chat list column, the top header and the bottom status bar are gone
- › Redesigned chat: replies read as plain text in a centered column instead of boxed bubbles, tool calls show as a compact chip you can expand, and copy, pin and speed stats sit under each reply
- › The message box floats at the bottom with a round send button, a tools on/off toggle and a stop button while generating. A new chat opens on a short greeting with the message box in the middle and suggestions that start a prompt
- › With no model loaded, the chat lists your downloaded models with a memory-fit badge so you can load one right there, or links to the model browser
- › The toolbar above the chat shows the conversation title, a model picker (also ⌘K / Ctrl K) and how full the context window is
- › A chat keeps streaming while you look at another page
- › Fixed spacing that the old CSS reset was silently removing, which ran reply paragraphs together and left inline code without padding. Also fixed an empty bubble that appeared when a reply only called a tool
- › On macOS the sidebar shows the translucent vibrancy. Settings › Appearance › "Sidebar opacity" (was "Darkness") controls how much shows through
- › Models, Instances, Observability, Settings, the ⌘K model switcher, the welcome wizard, the About and quit dialogs and the notifications all use the Midnight look: quiet cards, sentence-case labels, numbers in Geist Mono, and one main button per view
- › Models: "Upload Model" is now "Add local model", the loaded model no longer shows a disabled Delete, and the fit badges read "Fits well", "Tight on memory" and "Too large"
- › Instances: each instance's status dot shows generating, healthy, loading, error or unreachable
- › Settings: the Appearance / Model / Advanced tabs are a segmented control, and on/off settings are switches
- › Fixed the ⌘K model switcher not closing on Escape or a click outside it. Escape now also closes the About, download and instance log dialogs, and an open dialog takes Escape first instead of also stopping a reply
- › Larger, more readable text across the app: body text is now 18px (was 14px), secondary text 14 to 16px (was 12px), and badges and labels no smaller than 13px (were 10 to 11px), with headings and code blocks scaled up to match
- › New `orionpod` command line tool, bundled with the app (inside the `.app` on macOS, next to the app on Windows and Linux):
- › New **Instances** page lists every running instance, whether the app or a terminal started it, with health, model, port, uptime, requests, tokens per second, memory and CPU. From there you can start an instance (any installed model, optional port), stop one, read its log, copy its base URL, or switch the chat to it
- › Chat now runs through an instance. Loading a model starts one, or reuses an instance that already serves that model, so the model loaded in the app can also serve other apps
- › When you quit with instances the app started still running (closing the window, the macOS menu or ⌘Q), OrionPod asks whether to stop them or keep them serving, and can remember the choice (also set on the Instances page)
- › "Install command line tool" on the Instances page links `orionpod` into `/usr/local/bin` (asking for admin rights on macOS when needed) or `~/.local/bin`
- › Fixed the quantization shown for local `Q4_K_M` / `Q5_K_M` / … files, which showed as `Q4_K` / `Q5_K`
- › Windows (x64) and Linux (x64) builds: an NSIS installer for Windows, and AppImage, `.deb` and `.rpm` packages for Linux
- › GPU acceleration on Windows and Linux through Vulkan, so one build covers NVIDIA, AMD and Intel GPUs
- › Windows and Linux use the native title bar on an opaque window (no macOS vibrancy there), and the About dialog and Settings show the right platform and data folder
- › Double-clicking the header now zooms the window like a native title bar. On macOS it follows the "Double-click a window's title bar to" setting (zoom, fill, minimize or nothing)
- › Update notifications link to the download for your OS. Builds other than macOS fall back to the releases page when the update server lists no download for them
- › Hugging Face downloads, update checks and HTTP MCP servers no longer trust the eight compromised eMudhra intermediate certificates (October 2026). HTTPS is checked against the OS trust store, which macOS and Windows have already updated; on Linux, keep the `ca-certificates` package current
- › `crates/orionpod-runtime` no longer links llama.cpp (`llama-cpp-2` is removed). `engine/` supervises engine processes (`process.rs`: launch, readiness, log forwarding, shutdown; `llama.rs`: binary lookup and argument building; `mlx.rs`), and `server/` is a proxy in front of them (`models.rs` lifecycle and crash watchdog, `proxy.rs` forwarding with usage capture and error shaping, `telemetry.rs` metrics and inspector, `otel.rs` OTLP/HTTP JSON export)
- › New runtime modules: `gguf` (GGUF header reader, split files, projector lookup), `settings` (`EngineSettings` layering and auto context), `gateway`, `agents` (Claude Code / Codex / OpenCode setup), `bench`, `compat`, `mcp` (client and orion-core tool adapter), `lan` (mDNS), `netlog` (outbound ledger), `models::fit` and `models::import`
- › The desktop chat and `orionpod run` use `InstanceChatBackend` (an orion-core `ChatBackend` over `/v1/chat/completions`) instead of templating prompts themselves; `InstanceBackend` is removed. `orionpod run` without a running instance starts a private in-process instance (`server::serve_ephemeral`)
- › `config.toml`: `context_length = 0` means auto (configs still on the old 2,048 default are migrated once, tracked by `config_version`), plus `[engine]` defaults, `[gateway]` (`port`, `idle_ttl_secs`, `max_instances`, `aliases`, `autostart`), `[privacy]` (`update_check`, `network_log`) and new `[server]` keys (`allow_unauthenticated`, `advertise`, `tls_cert`, `tls_key`, `otel_endpoint`, `inspector`). Models carry `source`, `format` and `settings` in the registry
- › Instance records gain `kind` (model or gateway), `engine_pids` and `tls`; `InstanceStatus` gains `engine` (kind, version, role, context, slots, modalities, applied settings, notes) and cache, speculation, context-limit and first-token figures. New endpoints: `/metrics`, `/orionpod/requests[/{id}]`, `/orionpod/engine/log`; `LoadRequest` takes `settings`
- › `scripts/build-llama-server.sh` builds a static `llama-server` (Metal / Vulkan) from the pinned release into `target/debug/` (`just engine`) or as a sidecar (`just build`); `tauri.bundle.conf.json` ships it as a second `externalBin`. CI caches the build per version. On macOS it builds without BLAS (Accelerate's new BLAS interface needs macOS 13.3; Metal does the work) for macOS 11 (arm64) / 10.15 (x86_64), like the old in-process build
- › `scripts/build-sidecar.sh` and `build-llama-server.sh` also stage per-architecture sidecars for universal macOS builds, which `cargo tauri build --target universal-apple-darwin` requires
- › `scripts/generate-notices.cjs` (`just notices`, run by `just build` after `llama-server` so the full llama.cpp license is picked up on a clean build) writes `THIRD_PARTY_NOTICES.md` and a CycloneDX SBOM (`sbom.cdx.json`), both bundled with the app. `just compat-matrix` writes `docs/compatibility.md`
- › `just publish` uploads `SHA256SUMS` with each release and adds per-asset and per-OS `sha256` to the website's release data; the update check reads `sha256` from `latest.json` (the website needs to pass that field through)
- › New dependencies: `mdns-sd`, `axum-server` (rustls), `getrandom`; `async-trait` in the app
- › On macOS the app sets its own menu bar (`app_menu` in `src-tauri/src/lib.rs`) with a custom Quit item routed through `quit_allowed`; `src-tauri/Info.plist` adds `NSLocalNetworkUsageDescription` and `NSBonjourServices` (`_orionpod._tcp`)
- › orion-core 0.9 with the `plugins` feature: catalog parsing, the git-backed `PluginStore` (`~/.agents/plugins`) and `PluginContents` live there; `orionpod-runtime::plugins` adds the network ledger (`plugin-catalog`, `plugin-download`), `mcp.json` wiring (servers marked with `plugin`, renamed `<plugin>-<name>` on a clash, user switches and env kept on update) and skills (`agent_skills`, `with_skills`). Requires orion-core 0.9.0
- › `config.toml` gains `[plugins]` (`marketplace`, `catalog_url`, `disabled_skills`); `agent_tools` takes the `SkillSet` and adds `load_skill`, which runs without approval like the built-ins. New `which` module (`on_path`, `search_path`) replaces the private PATH lookup in `engine/mlx.rs`
- › `send_prompt` and `orionpod run` expand a leading `/skill` with orion-core's `SkillSet::expand_command` (`plugins::expand_skill_command`, skills from `command_skills`); saved conversations keep the expanded message and the chat shows it via `lib/skills.ts` (`parseExpanded`, mirroring `skills::parse_expanded`), which also keeps conversation titles as typed
- › New commands: `search_plugins`, `prepare_plugin_install`, `confirm_plugin_install`, `cancel_plugin_install`, `list_installed_plugins`, `remove_plugin`, `list_skills`; new `PluginBrowserDialog` and `PluginInstallDialog`
- › `config.toml` gains `[appearance]` (`font_family`, `font_source`, `custom_font`, `font_scale`); new commands `list_system_fonts` (via `fontdb`), `list_custom_fonts`, `install_font`, `read_font` in `src-tauri/src/commands/appearance.rs`
- › Midnight tokens are translucent (`surface`, `raised`, `well`) with a base-layer backdrop blur, plus `violet` and `teal` accents; text sizes scale with `--font-scale` and the font follows `--app-font` (`src/lib/appearance.ts`). Fonts ship via `@fontsource-variable` (JetBrains Mono, Inter, Literata)
- › New `src/lib/logs.ts` and `src/lib/prometheus.ts` parsers, `ContextSlider` and `AppearanceSettings` components, and shared filter UI in `src/components/Instances/Filters.tsx`
- › Per-tool switches: `disabled_tools` in `config.toml` (built-ins) and `disabledTools` per server in `mcp.json`. `tools::agent_tools` gathers the agent's tools for `send_prompt` and `orionpod run` (master switch, built-ins, enabled MCP tools); `McpHub::tools` filters by `disabledTools` and keeps live connections when only tool switches change. `McpStatus.tools` is now `ToolInfo` (name, label, description, enabled)
- › New commands `list_tools`, `set_mcp_tool_enabled`, `save_mcp_server`, `remove_mcp_server` and `set_mcp_server_enabled`; `McpHub::connect` and `McpHub::drop_stale`. `McpSettings.tsx` is replaced by `ToolsSettings.tsx`, `McpServerDialog.tsx` and `McpJsonDialog.tsx`
- › `McpHub` connects outside its locks (in parallel, with `CONNECT_TIMEOUT` and `RETRY_AFTER` backoff); `McpHub::refresh` retries failed servers for the settings commands, and `send_prompt` stops waiting on MCP connects when Stop sets the abort flag
- › New UI primitives `ui/Tile.tsx` (`Tile`, `TileIcon`, `Bento`) and `ui/LiveDot.tsx`, a `useElementWidth` hook, and `Observability/charts.tsx` (`AreaChart`, `StackedBars`, `Donut`, `Gauge`, `Legend`) replacing `BarChart`, `GaugeRing`, `Sparkline` and `MetricCard`
- › Design tokens live in `src/styles/globals.css` under `@theme` (`ground`, `sidebar`, `surface`, `raised`, `well`, `line`, `fg`…`fg-4`, `accent`, `ready`, `caution`, `danger`, plus `rounded-chip/control/panel/float`), so they work as Tailwind classes (`bg-surface`, `text-fg-3`, `rounded-control`). The old `--color-bg-*` / `--color-text-*` / `--blur-*` variables are removed, and components no longer use inline `style` for static layout. Base styles sit in `@layer base` so utilities can override them
- › Removed the unlayered `* { margin: 0; padding: 0 }` rule, which overrode every Tailwind spacing utility (Tailwind's preflight already resets margins and padding)
- › New primitives in `src/components/ui/`: `Button` and `IconButton`, `Input` and `Select`, `Slider` (accent fill via `--fill`) and `Switch`. `lib/cx.ts` joins class names
- › `hooks/useConversations` owns the saved-conversation list and the open conversation, shared by the sidebar and the chat. `Layout/PageToolbar` is the content area's drag region. `Header.tsx`, `Footer.tsx` and `Chat/InputBar.tsx` are removed; `Chat/Composer.tsx`, `Chat/ModelChip.tsx`, `Chat/EmptyStates.tsx`, `Layout/StatusDot.tsx` and `Layout/Logo.tsx` are new
- › The macOS overlay title bar and traffic-light position (18, 24) are set in `tauri.conf.json` instead of at runtime
- › New dependencies: `@fontsource-variable/geist` and `@fontsource-variable/geist-mono`
- › The Rust code is split into three crates:
- › Instances register in `~/.orionpod/instances/<pid>.json` and log to `~/.orionpod/logs/instances/serve-<pid>.log`. Besides the OpenAI routes they serve `/health`, `/orionpod/status`, `/orionpod/load`, `/orionpod/unload`, `/orionpod/tokenize` and `/orionpod/shutdown`. POSTs must be JSON, and a loopback bind rejects foreign `Host` headers (DNS-rebinding protection)
- › `[server]` section in `config.toml`: `host`, `base_port`, `api_key` and `on_quit` (`ask` / `stop` / `keep`)
- › `scripts/build-sidecar.sh` builds the CLI and stages it at `src-tauri/binaries/orionpod-cli-<triple>`. `just build` passes `tauri.bundle.conf.json` to add it as an `externalBin`, and builds it universal on macOS. `just dev` builds the CLI first, `just cli <args>` runs it from source, and `just install-cli` links a release build into `~/.local/bin`
- › New dependencies: `axum`, `tokio-stream` (runtime) and `clap`, `rustyline`, `indicatif` (CLI). Tests cover the OpenAI request and response shapes, stop sequences, model resolution and pull references
- › `window-vibrancy` is now a macOS-only dependency
- › Windows and Linux builds need the Vulkan SDK (`VULKAN_SDK`) to build `llama-server`
- › `.cargo/config.toml` links the C runtime statically on Windows (`+crt-static` and `LLAMA_STATIC_CRT=1`), so the installer doesn't need the VC++ Redistributable
- › Platform window overrides in `tauri.windows.conf.json` and `tauri.linux.conf.json`. The `.deb` / `.rpm` depend on the Vulkan loader
- › `just build` is now per-OS (`[macos]` / `[windows]` / `[linux]` recipes)
- › New `.github/workflows/build.yml` builds Windows and Linux on `v*` tags (and on demand). `just publish` waits for that run, uploads its bundles alongside the DMG, and `update-web-release.cjs` adds them to `releases.js` with a `platform` / `format` per asset (`SKIP_CI=1` skips this)
- › `update-web-release.cjs` also writes a per-OS `downloads` map (`{ macos, windows, linux }`, AppImage for Linux) to each `releases.js` entry, and `UpdateInfo` reads the same map from `latest.json`
- › The Windows GPU probe no longer flashes a console window (`CREATE_NO_WINDOW`)
- › New `title_bar_double_click` command reads `AppleActionOnDoubleClick` via `NSUserDefaults` (new macOS-only `objc2-foundation` dependency). `AppError` gains a `Window` variant for `tauri::Error`
- › Dependency refresh for the eMudhra CA compromise: `rustls-webpki` 0.103.16 and `webpki-root-certs` 1.0.10 (the patched releases), a full `cargo update`, llama.cpp b11784, `eslint` 10.13, `typescript-eslint` 8.72 and `lucide-react` 1.60. TypeScript stays on 6.0 until `typescript-eslint` supports 7. The bundled `llama-server` is built without OpenSSL and makes no outbound HTTPS
v0.5.1
Chihiro Stableorion-core 0.8, a llama.cpp update, and cleaner streaming of emoji and multi-byte text.
macOS
Universal (Apple Silicon + Intel) · ~32 MB
- › Fixed garbled characters (`�`) in streamed replies when an emoji or other multi-byte character was split across tokens. The end-of-turn check ran each token through the streaming UTF-8 decoder before the output did, so the decoder saw those bytes twice
- › `orion-core` bumped **0.5 → 0.8**. Built-in tools move to the new `Tool::execute(args, ctx: &ToolContext)` signature and build their output with `ToolOutput::new(..).with_details(..)`. `ToolResultData` and the `AgentEvent` TS union now mirror the new `pending` flag and the `tool_denied` / `tool_resolved` / `paused` / `nested` events
- › `llama-cpp-2` 0.1.138 → 0.1.160: tokenization, EOS lookup, and token-to-piece decoding move from `LlamaModel` to `LlamaVocab` (`model.vocab()`), since `str_to_token` / `token_eos` / `token_to_piece` and `AddBos` were removed upstream
- › Rust deps updated to latest: `reqwest` 0.13 (now with the `query` feature, needed for `RequestBuilder::query`; rustls is the default TLS backend), `sysinfo` 0.39, `dirs` 7, `window-vibrancy` 0.8, Tauri 2.12 and plugins, plus a full `cargo update`
- › Frontend deps updated to latest: Vite 8, `@vitejs/plugin-react` 6, ESLint 10 / `@eslint/js` 10, `lucide-react` 1.x, React 19.3, Tauri JS API 2.12 and plugins, Tailwind 4.3, Prettier 3.9, `globals` 17, `@types/node` 26
- › TypeScript pinned to **6.0** rather than 7.0. `typescript-eslint` only supports `<6.1`, because TS 7 ships no programmatic API yet
- › `eslint-plugin-react-hooks` is now registered in `eslint.config.js` (`rules-of-hooks`, `exhaustive-deps`). It was already installed, but the plugin was never loaded, so the existing `eslint-disable` comments for it failed under ESLint 10
- › Fixed new clippy lints from Rust 1.97 (`unnecessary_sort_by`, `manual_checked_ops`) and empty `catch` blocks flagged by ESLint 10
v0.5.0-beta
FutabaConversations that persist and export, beta agent tools, and correct prompt formatting beyond ChatML.
macOS
Universal (Apple Silicon + Intel) · ~26 MB
- › Chat template coverage — models in the Llama 3, Mistral/Mixtral, Alpaca, and Vicuna prompt formats are now templated correctly instead of being wrapped in ChatML. Previously every non-ChatML model was fed a ChatML-formatted prompt, degrading output quality
- › The Runtime Controls "Chat Template" override now takes effect — choosing Llama 3 / Mistral / Alpaca / Vicuna (or leaving it on Auto-detect) actually drives prompt formatting; unimplemented choices fall back to auto-detect
- › Per-message inference stats — each assistant message now shows measured `tokens/sec · time-to-first-token · total time` once the turn completes (previously the live speed readout disappeared when generation ended)
- › Accurate GPU name in Observability — the system card now reports the real graphics adapter on every platform instead of the CPU name (macOS) or a hardcoded "Unknown" (Linux/Windows)
- › Context-aware compatibility — a model's "will it run" estimate now accounts for KV-cache growth at the configured context length, so a model can shift from comfortable to risky as the window grows; estimates are labelled as such in the UI
- › Conversation persistence — chats are saved automatically and survive restarts. A conversation list down the left of the Chat view lets you switch between past chats, with the most recent restored on launch
- › Rename, delete, and export conversations — rename inline, delete with a confirmation, and export any chat to Markdown or JSON via a native save dialog (defaulting to `~/Downloads`)
- › Each conversation remembers the model it was created with; resuming one under a different (or no) model loaded shows a warning
- › Agent tools (beta) — enable "Agent Tools" in Settings → Advanced and the model can call built-in tools (`get_current_time`, `calculator`). Tool calls run automatically and each invocation appears in the chat as an expandable card showing its arguments and result. Off by default, since injecting tool instructions can affect chat quality on smaller models
- › Pin important messages — a pin button on each chat message keeps it in the model's context window even as the conversation grows past the limit; pins persist across restarts
- › Context pruning choice — in Settings → Model, pick how the context window is trimmed when it fills: *Sliding window* (drop the oldest turns, default) or *Summarize older messages* (fold them into a pinned summary so their gist is kept). Summarize spends one extra model call when it kicks in
- › `orion-core` extracted into its own MIT-licensed, open-source repository and published to crates.io as **v0.5.0**; `src-tauri` now depends on it via `orion-core = "0.5"` from crates.io instead of the in-tree workspace crate. The agent harness (Agent loop, `LlmBackend`, context pipeline, templates, tools) is unchanged — only its home and distribution moved. The local `orion-core/` crate and its workspace membership were removed
- › `orion-core` (bumped to 0.2.0): new `Llama3Template`, `MistralTemplate`, `AlpacaTemplate`, and `VicunaTemplate` implementations of the `ChatTemplate` trait, exported from the crate root
- › `Llama3Template` — `<|begin_of_text|>` + `<|start_header_id|>{role}<|end_header_id|>` / `<|eot_id|>` blocks; tool results use the `ipython` role
- › `MistralTemplate` — `<s>[INST] … [/INST]` turns with `</s>` after each assistant reply; the system prompt is merged into the first user instruction via a custom `format()` (Mistral has no system role), while `format_system`/`format_message` return token-representative fragments for context-budget accounting
- › `AlpacaTemplate` — `### Instruction:` / `### Response:` blocks with the standard preamble when no system prompt is set
- › `VicunaTemplate` — `USER:` / `ASSISTANT:` turns (FastChat v1.1 spacing) with the standard preamble fallback
- › Shared `render_tools()` helper renders the tool-instruction block (description list + `tool_call` JSON convention) uniformly across all templates; `ChatMLTemplate` refactored to use it
- › `detect_template()` now matches GGUF metadata template strings to the right family (Llama 3 headers → ChatML → `[INST]` → Alpaca → Vicuna), falling back to ChatML when no marker matches
- › `template_from_name()` resolves the manual-override dropdown values (with aliases like `mixtral`, `llama-3.1`) to a template, returning `None` for unimplemented names
- › `commands/inference.rs::resolve_template()` applies the active template per prompt — a non-empty override wins, otherwise the loaded model's GGUF template string drives auto-detection; only swapped when it changes (the KV cache reconciles a format change via prefix matching)
- › 14 new tests in `orion-core/tests/template_tests.rs` (per-family format correctness, tool rendering, GGUF detection, name resolution)
- › Measured request metrics — `MetricsCollector` records a request only when `AgentEvent::GenerationStats` arrives, using its measured TTFT and generation time; the `1000 / tps` TTFT fallback in `commands/inference.rs` is removed, and aborted/failed turns are skipped rather than logged with fabricated timing that skewed session averages
- › Frontend consumes the `generation_stats` event — measured stats attach to the assistant message and `MessageBubble` renders `t/s · TTFT · total`
- › `observability::gpu_name()` detects the real adapter per platform (macOS `system_profiler` Chipset Model; Linux `nvidia-smi` → `lspci`; Windows `Win32_VideoController` via PowerShell), memoized in a `OnceLock` since the observability page polls it frequently
- › Context-scaled compatibility estimate — runtime memory modelled as `weights + 10% compute overhead + context-scaled KV cache` instead of a fixed `1.2×` multiplier; the configured `context_length` is threaded through the backend and the mirrored frontend `assessCompatibility`
- › Update endpoint verified against the live `https://orionpod.com/api/latest.json`; new `deserializes_live_endpoint_payload` test pins the payload shape and the release-vs-beta version ranking so endpoint/struct drift fails at test time rather than silently at runtime
- › Removed dead code surfaced in a codebase audit: the unused `useInference` stub hook and the half-wired vibrancy toggle (`set_vibrancy` command, its `invoke_handler` registration, and the commented-out `setVibrancy` wrapper). Window vibrancy is unaffected — it stays always-on via the unconditional apply at startup in `lib.rs`
- › Headless DMG assembly — `just build` now bundles only the `.app` (`cargo tauri build --bundles app`) and creates the DMG via `scripts/bundle-dmg.sh` with `hdiutil` directly, applying the pre-captured window layout from `src-tauri/dmg/DS_Store` instead of scripting Finder with AppleScript. No window pops up during the build and no GUI session is required, so releases can run unattended on remote machines. The DMG itself is unchanged: same styled layout, volume icon, and `Applications` drop link
- › Conversation store (`src-tauri/src/conversations.rs`) — one `<id>.json` per conversation under `~/.orionpod/conversations/` holding metadata + the canonical `orion_core::Message` history; a borrowed `ConversationRecord` pins the persisted `message_count` so the `ConversationSummary` listing reads headers without parsing message bodies, and `conversation_path()` guards against path traversal
- › `AgentState` tracks an `active_conversation_id`; `send_prompt` auto-saves the agent's message history after each completed turn (the agent is the source of truth), assigning an id lazily on the first turn and preserving the original creation time, (possibly renamed) title, and model association on later saves. `clear_conversation` detaches the active conversation
- › New IPC commands in `commands/conversations.rs`: `list_conversations`, `load_conversation` (replaces agent messages + resets the KV cache), `new_conversation`, `delete_conversation`, `rename_conversation`, and `export_conversation` (renders Markdown/JSON and writes from Rust, since the frontend `fs` scope doesn't cover arbitrary user-chosen paths)
- › Frontend: `ChatPage` wrapper owns conversation state (list, active id, restore-on-mount, model-mismatch warning) and composes the new `ConversationList` panel with `ChatWindow`; `ChatWindow` converted from a `forwardRef` imperative handle to a parent-driven view (`viewNonce` / `viewMessages` reset, `onTurnComplete`, `onNewChat`). New typed IPC wrappers and `Conversation` / `ConversationSummary` types
- › `orion-core` tool execution loop — `Agent::prompt` now parses tool calls from the model's reply, runs the matching registered `Tool`, appends a `ToolResult` message, and loops back to the LLM until it returns a tool-free answer (bounded by `AgentConfig::max_tool_iterations`, default 8), emitting `ToolExecStart` / `ToolExecUpdate` / `ToolExecEnd`. New `parse_tool_calls()` + `ParsedToolCall` implement the lenient `tool_call` JSON convention (fenced `tool_call`/`json` blocks, or a whole-message bare JSON object); `ToolUpdateCallback` re-exported; 4 new tests in `orion-core/tests/tool_tests.rs`
- › Built-in tools live in `src-tauri/src/tools/` — `get_current_time` (chrono) and `calculator` (a small dependency-free recursive-descent evaluator in `tools/eval.rs`, no external eval crate). Registered with the agent per prompt from the new `tools_enabled` config flag (off by default); the existing `agent-event` emitter already forwards `ToolExec*` events to the frontend
- › Frontend: `ChatWindow` now opens assistant bubbles on `turn_start` (so multi-turn tool continuations render correctly) and shows tool execution via the new `ToolCallCard`; `ChatPage` reload reconstructs tool cards from persisted `tool_calls` + `tool_result` messages; Agent Tools toggle added to Settings → Advanced. `ChatMessage` gains a `"tool"` role + `ToolActivity`
- › orion-core prune strategies — `Message` gains a `pinned` flag (`Message::pinned()` builder, `Agent::set_pinned(id, bool)`); pinned messages always survive context pruning, turn-aware so a pin never orphans its pair. New `PruneStrategy::Summarize`: when the window overflows the agent folds the oldest dropped turns (and any prior summary) into a single pinned summary via one extra backend call (best-effort, falls back to sliding window). Turn-selection extracted into `plan_prune()` → `PrunePlan` (`prepare_context` is now a thin formatter over it); `Agent::replace_messages` bumps the id counter past restored ids
- › App wiring — `AppConfig.prune_strategy` synced to the agent per prompt; new `set_message_pinned` IPC persists the pin to the conversation store; `ChatWindow` reconciles the agent's message id onto live bubbles (from `message_end`) so they can be pinned, `MessageBubble` gains a pin button + indicator and renders summary messages as a muted note; "Context Pruning" selector added to Settings → Model
v0.4.0-beta
YukiRe-aligning with focus on UX
macOS
Universal (Apple Silicon + Intel) · ~25 MB
- › Faster multi-turn chat — the model's context (KV cache) is now reused across turns instead of rebuilt on every message, lowering per-message latency and removing the redundant Metal warm-up after the first response
- › Observability now reports the real time-to-first-token per request (previously an estimate derived from tokens/sec)
- › Context Window card on Observability page — gauge ring showing token usage ratio, used/max/available counts, messages in context, and pruned message count
- › Keyboard shortcuts: `⌘N` new chat, `⌘K` quick model switcher, `⌘,` open settings, `Esc` stop generation
- › Quick model switcher — command palette (`⌘K`) with search, keyboard navigation, load/unload
- › Dynamic window title — shows "OrionPod — ModelName" when a model is loaded
- › Window size and position remembered across launches
- › Skeleton loader for model list (replaces plain text spinner)
- › Smooth animations: message appearance, sidebar active indicator, modal transitions
- › Persistent inference context — `InferenceEngine` keeps one `llama_context` per loaded model (created lazily, reused across turns; recreated only when context size or thread count changes). `generate()` is now `&mut self`
- › Incremental KV cache — each new prompt is reconciled against the tokens resident in the cache via longest-common-prefix; only the diverging suffix is decoded (`clear_kv_cache_seq`), and generated tokens are appended for reuse on the next turn
- › KV cache reconciles with orion-core pruning automatically (truncates at the divergence point, system-prompt prefix retained) and resets on `clear_conversation`, model switch/unload, and context overflow via `InferenceEngine::reset_context()`
- › Metal compute pipelines now compile once per loaded model instead of once per turn
- › `AgentEvent::GenerationStats` — new event carrying real tokens generated, tokens/sec, time-to-first-token, and generation time; `MetricsCollector` records actual TTFT instead of approximating it as `1000/tps`
- › Frontend `AgentEvent` union extended with the `generation_stats` variant
- › `tauri-plugin-window-state` added for persistent window geometry
- › `useKeyboardShortcuts` hook for global shortcut handling
- › `ChatWindow` converted to `forwardRef` to expose `clearChat()` for programmatic reset
- › Context budget metrics now recorded to `MetricsCollector` — wired `ContextBudget` agent event data through to session metrics
- › `SessionMetrics` TypeScript type extended with `context_used_tokens`, `context_max_tokens`, `context_messages_in_context`, `context_messages_pruned`
- › `ChatTemplate` trait in orion-core — pluggable prompt formatting with `format()`, `format_system()`, `format_message()`, `assistant_prefix()`
- › `ChatMLTemplate` default implementation
- › `detect_template()` — auto-selects template from GGUF metadata string, falls back to ChatML
- › Pair-wise context pruning — user+assistant turns pruned as units, never orphans a question or answer
- › Template-aware token accounting — budget counts template overhead (`<|im_start|>`, `<|im_end|>`, etc.) per message, not just raw content
- › System prompt + tool schema tokens deducted from context budget before conversation pruning
- › `CoreError::Context` on overflow — clear error when system prompt or latest message exceeds context budget
- › `prepare_context()` replaces separate `prune_messages()` + `format_chatml()` — single function for prune, format, and budget accounting
- › `Agent::with_template()` constructor and `set_template()` for runtime template switching
- › 17 new tests in `orion-core/tests/context_tests.rs` (pair-wise pruning, overflow errors, template overhead, tool budget, ChatML formatting, detect_template)
v0.3.0-alpha
DaruHello World orion-core, an inbuilt harness
macOS
Universal (Apple Silicon + Intel) · ~24 MB
- › Structured agent event system — chat now receives rich lifecycle events (start, delta, end, error, warning) instead of raw token strings
- › Token budget bar in chat toolbar — shows context window usage (e.g. "1,200 / 4,096 tokens") with visual warning when >80% full
- › Discord community link added to About modal
- › System prompt support — backend accepts custom system prompts via `set_system_prompt` command
- › Inference parameter tuning — `set_inference_params` command for runtime temperature, context size, and thread count updates
- › `orion-core` integration: `LlamaCppBackend` implements `LlmBackend` trait, wrapping `InferenceEngine` for backend-agnostic agent loop
- › `AgentState` replaces `InferenceState` — all inference commands route through `orion_core::Agent` for conversation state, context pruning, and ChatML prompt formatting
- › `agent-event` Tauri event channel replaces `token-stream` — emits all 13 `AgentEvent` variants (`agent_start`, `message_delta`, `message_end`, `context_budget`, `error`, etc.)
- › `set_system_prompt` and `set_inference_params` IPC commands
- › `InferenceParams` extended with `n_threads` field (defaults to `available_parallelism - 2`)
- › Removed `inference/streaming.rs` (`TokenEvent`, `ChatMessage`, `format_chatml` superseded by orion-core types)
- › Removed dead `format_prompt` and `truncate_to_fit` from `InferenceEngine` (context pipeline now in orion-core)
- › Frontend: `AgentEvent` discriminated union type (13 variants), `AgentMessage`, `ToolCall`, `ToolResultData` types added to `lib/types.ts`
- › Frontend: `setSystemPrompt()` and `setInferenceParams()` IPC wrappers in `lib/tauri.ts`
- › Frontend: `ChatWindow.tsx` migrated from `token-stream` listener to `agent-event` with full event handling
- › Zero compiler warnings (Rust), zero TypeScript errors
v0.2.3-beta
FayeSuggested Models and Onboarding
macOS
Universal (Apple Silicon + Intel) · ~24 MB
- › "Surprise Me" model discovery — one-click random suggestions from a curated list of small, high-quality models
- › Curated model list: TinyLlama 1.1B, Qwen 2.5 (0.5B/1.5B/3B), Phi 3.5 Mini, Gemma 2 2B, StableLM 2 1.6B, SmolLM2 1.7B, Llama 3.2 (1B/3B)
- › "Try Another" re-roll button — skips already-seen and already-downloaded models
- › Inline download with progress tracking directly from the suggestion card
- › HuggingFace search result caching (5-minute TTL) — fewer API calls, faster repeat searches
- › Quantization variant badges on search result cards — see available quants at a glance
- › Sort search results by downloads, likes, or recent activity
- › Download pause/resume — pause active downloads, resume later (supports HTTP Range)
- › Cancel download button with proper cleanup
- › Disk space check before downloading — warns if insufficient space
- › Download complete toast with "Load now?" action button
- › Active download indicator in footer bar
- › First-run welcome wizard — guided setup: download a starter model, auto-load, start chatting
- › Partial download recovery — detects incomplete downloads after app crash and offers resume
- › `src/lib/curatedModels.ts` — maintainable curated model list as a typed constant array
- › `SurpriseCard` component with full download lifecycle (progress, pause/resume/cancel)
- › `SearchCache` with TTL-based expiry in `HuggingFaceClient`
- › `DownloadManagerState` with `DownloadHandle` for cancel/pause control
- › `DownloadSidecar` metadata JSON written alongside downloaded GGUFs
- › `available_disk_space()` using `sysinfo::Disks` for volume-aware space check
- › `cancel_download`, `pause_download`, `resume_download`, `list_partial_downloads`, `get_available_disk_space` IPC commands
- › Download resume via HTTP `Range` header with `.gguf.part` file detection
- › `WelcomeWizard` component with 4-step flow (welcome → download → loading → ready)
- › `useDownloads` hook now tracks download completion transitions for toast notifications
v0.2.2-alpha
EdYour models, your rules
macOS
Universal (Apple Silicon + Intel) · ~24 MB
- › GGUF metadata extraction — model cards now show parameter count, context length, and architecture
- › Runtime controls — functional thread count, temperature, and context length sliders in Settings
- › Chat template auto-detection from GGUF metadata with manual override dropdown (ChatML, Llama 3, Mistral, Gemma, Phi-3, DeepSeek, etc.)
- › Context overflow handling — oldest messages automatically pruned when conversation exceeds context window
- › Model status events — real-time loading/ready/error/unloaded status via Tauri events
- › Update notification toast with download button when a new version is available
- › Actionable toast notifications (toasts can now have clickable action buttons)
- › `GgufModelInfo` struct with full GGUF header metadata (params, layers, heads, embedding dim, architecture, chat template)
- › `InferenceEngine::format_prompt()` uses `apply_chat_template()` from llama.cpp with ChatML fallback
- › `InferenceEngine::truncate_to_fit()` for pair-wise context pruning
- › `model-status` Tauri event channel with `ModelStatusEvent` payload
- › `AppConfig` extended with `chat_template` option for manual override
- › `generate()` accepts configurable `n_threads` and `context_length` from config
- › `ModelMetadata` enriched with `context_length`, `architecture`, `chat_template` fields (backward-compatible via `#[serde(default)]`)
- › Auto-update check via `https://orionpod.com/api/latest.json` on app launch
- › `check_for_updates` Rust IPC command with semver-aware version comparison
- › `useUpdateCheck` hook (5s delayed, silent fail, non-blocking)
- › `update-web-release.cjs` script for automated release metadata updates
- › Changelog auto-extraction from `CHANGELOG.md` into `releases.js`
v0.2.1-rc1
Kusanagi DeprecatedFirst public release. Metal GPU acceleration, HuggingFace model browser, real-time observability.
macOS
Universal (Apple Silicon + Intel) · ~30 MB
- › Chat interface with streaming responses and markdown rendering
- › HuggingFace model browser with hardware compatibility filtering
- › GGUF model support — download from HuggingFace or upload local files
- › Real-time observability dashboard (tokens/s, memory, latency, GPU usage)
- › Metal GPU acceleration on Apple Silicon
- › Model parameter controls (temperature, context length, top-p, top-k)
- › Toast notifications and user-friendly error messages
- › Glassmorphism UI with macOS vibrancy
- › Tauri v2 + React + TypeScript + Rust + llama.cpp
- › orion-core agent harness crate (backend-agnostic)
- › Universal macOS binary (Apple Silicon + Intel), ~30 MB
- › Starts in under 2 seconds, <50 MB RAM idle
- › Zero telemetry, zero analytics, zero cloud dependencies
System Requirements
- ✓ macOS 10.15 (Catalina) or later
- ✓ Apple Silicon recommended, for Metal GPU acceleration
- ✓ Intel Macs work too
- ✓ Windows 10 or 11 (x64)
- ✓ A Vulkan-capable GPU driver (NVIDIA, AMD or Intel) for GPU acceleration
- ✓ Without one, models run on the CPU
- ✓ x64 with glibc 2.35 or newer (Ubuntu 22.04, Debian 12)
- ✓ WebKitGTK 4.1
- ✓ A Vulkan-capable GPU driver for GPU acceleration
8 GB of RAM is a comfortable minimum for 7B models. The fit advisor tells you how any model fits before you load it.
Open the DMG → drag OrionPod to Applications → launch. If macOS says it can't be opened, here's what to do.