Download OrionPod
Currently available for macOS
v0.5.0-beta
Futaba LatestConversations that persist and export, beta agent tools, and correct prompt formatting beyond ChatML.
macOS
Universal (Apple Silicon + Intel) · ~26 MB
- › Chat template coverage — models in the Llama 3, Mistral/Mixtral, Alpaca, and Vicuna prompt formats are now templated correctly instead of being wrapped in ChatML. Previously every non-ChatML model was fed a ChatML-formatted prompt, degrading output quality
- › The Runtime Controls "Chat Template" override now takes effect — choosing Llama 3 / Mistral / Alpaca / Vicuna (or leaving it on Auto-detect) actually drives prompt formatting; unimplemented choices fall back to auto-detect
- › Per-message inference stats — each assistant message now shows measured `tokens/sec · time-to-first-token · total time` once the turn completes (previously the live speed readout disappeared when generation ended)
- › Accurate GPU name in Observability — the system card now reports the real graphics adapter on every platform instead of the CPU name (macOS) or a hardcoded "Unknown" (Linux/Windows)
- › Context-aware compatibility — a model's "will it run" estimate now accounts for KV-cache growth at the configured context length, so a model can shift from comfortable to risky as the window grows; estimates are labelled as such in the UI
- › Conversation persistence — chats are saved automatically and survive restarts. A conversation list down the left of the Chat view lets you switch between past chats, with the most recent restored on launch
- › Rename, delete, and export conversations — rename inline, delete with a confirmation, and export any chat to Markdown or JSON via a native save dialog (defaulting to `~/Downloads`)
- › Each conversation remembers the model it was created with; resuming one under a different (or no) model loaded shows a warning
- › Agent tools (beta) — enable "Agent Tools" in Settings → Advanced and the model can call built-in tools (`get_current_time`, `calculator`). Tool calls run automatically and each invocation appears in the chat as an expandable card showing its arguments and result. Off by default, since injecting tool instructions can affect chat quality on smaller models
- › Pin important messages — a pin button on each chat message keeps it in the model's context window even as the conversation grows past the limit; pins persist across restarts
- › Context pruning choice — in Settings → Model, pick how the context window is trimmed when it fills: *Sliding window* (drop the oldest turns, default) or *Summarize older messages* (fold them into a pinned summary so their gist is kept). Summarize spends one extra model call when it kicks in
- › `orion-core` extracted into its own MIT-licensed, open-source repository and published to crates.io as **v0.5.0**; `src-tauri` now depends on it via `orion-core = "0.5"` from crates.io instead of the in-tree workspace crate. The agent harness (Agent loop, `LlmBackend`, context pipeline, templates, tools) is unchanged — only its home and distribution moved. The local `orion-core/` crate and its workspace membership were removed
- › `orion-core` (bumped to 0.2.0): new `Llama3Template`, `MistralTemplate`, `AlpacaTemplate`, and `VicunaTemplate` implementations of the `ChatTemplate` trait, exported from the crate root
- › `Llama3Template` — `<|begin_of_text|>` + `<|start_header_id|>{role}<|end_header_id|>` / `<|eot_id|>` blocks; tool results use the `ipython` role
- › `MistralTemplate` — `<s>[INST] … [/INST]` turns with `</s>` after each assistant reply; the system prompt is merged into the first user instruction via a custom `format()` (Mistral has no system role), while `format_system`/`format_message` return token-representative fragments for context-budget accounting
- › `AlpacaTemplate` — `### Instruction:` / `### Response:` blocks with the standard preamble when no system prompt is set
- › `VicunaTemplate` — `USER:` / `ASSISTANT:` turns (FastChat v1.1 spacing) with the standard preamble fallback
- › Shared `render_tools()` helper renders the tool-instruction block (description list + `tool_call` JSON convention) uniformly across all templates; `ChatMLTemplate` refactored to use it
- › `detect_template()` now matches GGUF metadata template strings to the right family (Llama 3 headers → ChatML → `[INST]` → Alpaca → Vicuna), falling back to ChatML when no marker matches
- › `template_from_name()` resolves the manual-override dropdown values (with aliases like `mixtral`, `llama-3.1`) to a template, returning `None` for unimplemented names
- › `commands/inference.rs::resolve_template()` applies the active template per prompt — a non-empty override wins, otherwise the loaded model's GGUF template string drives auto-detection; only swapped when it changes (the KV cache reconciles a format change via prefix matching)
- › 14 new tests in `orion-core/tests/template_tests.rs` (per-family format correctness, tool rendering, GGUF detection, name resolution)
- › Measured request metrics — `MetricsCollector` records a request only when `AgentEvent::GenerationStats` arrives, using its measured TTFT and generation time; the `1000 / tps` TTFT fallback in `commands/inference.rs` is removed, and aborted/failed turns are skipped rather than logged with fabricated timing that skewed session averages
- › Frontend consumes the `generation_stats` event — measured stats attach to the assistant message and `MessageBubble` renders `t/s · TTFT · total`
- › `observability::gpu_name()` detects the real adapter per platform (macOS `system_profiler` Chipset Model; Linux `nvidia-smi` → `lspci`; Windows `Win32_VideoController` via PowerShell), memoized in a `OnceLock` since the observability page polls it frequently
- › Context-scaled compatibility estimate — runtime memory modelled as `weights + 10% compute overhead + context-scaled KV cache` instead of a fixed `1.2×` multiplier; the configured `context_length` is threaded through the backend and the mirrored frontend `assessCompatibility`
- › Update endpoint verified against the live `https://orionpod.com/api/latest.json`; new `deserializes_live_endpoint_payload` test pins the payload shape and the release-vs-beta version ranking so endpoint/struct drift fails at test time rather than silently at runtime
- › Removed dead code surfaced in a codebase audit: the unused `useInference` stub hook and the half-wired vibrancy toggle (`set_vibrancy` command, its `invoke_handler` registration, and the commented-out `setVibrancy` wrapper). Window vibrancy is unaffected — it stays always-on via the unconditional apply at startup in `lib.rs`
- › Headless DMG assembly — `just build` now bundles only the `.app` (`cargo tauri build --bundles app`) and creates the DMG via `scripts/bundle-dmg.sh` with `hdiutil` directly, applying the pre-captured window layout from `src-tauri/dmg/DS_Store` instead of scripting Finder with AppleScript. No window pops up during the build and no GUI session is required, so releases can run unattended on remote machines. The DMG itself is unchanged: same styled layout, volume icon, and `Applications` drop link
- › Conversation store (`src-tauri/src/conversations.rs`) — one `<id>.json` per conversation under `~/.orionpod/conversations/` holding metadata + the canonical `orion_core::Message` history; a borrowed `ConversationRecord` pins the persisted `message_count` so the `ConversationSummary` listing reads headers without parsing message bodies, and `conversation_path()` guards against path traversal
- › `AgentState` tracks an `active_conversation_id`; `send_prompt` auto-saves the agent's message history after each completed turn (the agent is the source of truth), assigning an id lazily on the first turn and preserving the original creation time, (possibly renamed) title, and model association on later saves. `clear_conversation` detaches the active conversation
- › New IPC commands in `commands/conversations.rs`: `list_conversations`, `load_conversation` (replaces agent messages + resets the KV cache), `new_conversation`, `delete_conversation`, `rename_conversation`, and `export_conversation` (renders Markdown/JSON and writes from Rust, since the frontend `fs` scope doesn't cover arbitrary user-chosen paths)
- › Frontend: `ChatPage` wrapper owns conversation state (list, active id, restore-on-mount, model-mismatch warning) and composes the new `ConversationList` panel with `ChatWindow`; `ChatWindow` converted from a `forwardRef` imperative handle to a parent-driven view (`viewNonce` / `viewMessages` reset, `onTurnComplete`, `onNewChat`). New typed IPC wrappers and `Conversation` / `ConversationSummary` types
- › `orion-core` tool execution loop — `Agent::prompt` now parses tool calls from the model's reply, runs the matching registered `Tool`, appends a `ToolResult` message, and loops back to the LLM until it returns a tool-free answer (bounded by `AgentConfig::max_tool_iterations`, default 8), emitting `ToolExecStart` / `ToolExecUpdate` / `ToolExecEnd`. New `parse_tool_calls()` + `ParsedToolCall` implement the lenient `tool_call` JSON convention (fenced `tool_call`/`json` blocks, or a whole-message bare JSON object); `ToolUpdateCallback` re-exported; 4 new tests in `orion-core/tests/tool_tests.rs`
- › Built-in tools live in `src-tauri/src/tools/` — `get_current_time` (chrono) and `calculator` (a small dependency-free recursive-descent evaluator in `tools/eval.rs`, no external eval crate). Registered with the agent per prompt from the new `tools_enabled` config flag (off by default); the existing `agent-event` emitter already forwards `ToolExec*` events to the frontend
- › Frontend: `ChatWindow` now opens assistant bubbles on `turn_start` (so multi-turn tool continuations render correctly) and shows tool execution via the new `ToolCallCard`; `ChatPage` reload reconstructs tool cards from persisted `tool_calls` + `tool_result` messages; Agent Tools toggle added to Settings → Advanced. `ChatMessage` gains a `"tool"` role + `ToolActivity`
- › orion-core prune strategies — `Message` gains a `pinned` flag (`Message::pinned()` builder, `Agent::set_pinned(id, bool)`); pinned messages always survive context pruning, turn-aware so a pin never orphans its pair. New `PruneStrategy::Summarize`: when the window overflows the agent folds the oldest dropped turns (and any prior summary) into a single pinned summary via one extra backend call (best-effort, falls back to sliding window). Turn-selection extracted into `plan_prune()` → `PrunePlan` (`prepare_context` is now a thin formatter over it); `Agent::replace_messages` bumps the id counter past restored ids
- › App wiring — `AppConfig.prune_strategy` synced to the agent per prompt; new `set_message_pinned` IPC persists the pin to the conversation store; `ChatWindow` reconciles the agent's message id onto live bubbles (from `message_end`) so they can be pinned, `MessageBubble` gains a pin button + indicator and renders summary messages as a muted note; "Context Pruning" selector added to Settings → Model
v0.4.0-beta
YukiRe-aligning with focus on UX
macOS
Universal (Apple Silicon + Intel) · ~25 MB
- › Faster multi-turn chat — the model's context (KV cache) is now reused across turns instead of rebuilt on every message, lowering per-message latency and removing the redundant Metal warm-up after the first response
- › Observability now reports the real time-to-first-token per request (previously an estimate derived from tokens/sec)
- › Context Window card on Observability page — gauge ring showing token usage ratio, used/max/available counts, messages in context, and pruned message count
- › Keyboard shortcuts: `⌘N` new chat, `⌘K` quick model switcher, `⌘,` open settings, `Esc` stop generation
- › Quick model switcher — command palette (`⌘K`) with search, keyboard navigation, load/unload
- › Dynamic window title — shows "OrionPod — ModelName" when a model is loaded
- › Window size and position remembered across launches
- › Skeleton loader for model list (replaces plain text spinner)
- › Smooth animations: message appearance, sidebar active indicator, modal transitions
- › Persistent inference context — `InferenceEngine` keeps one `llama_context` per loaded model (created lazily, reused across turns; recreated only when context size or thread count changes). `generate()` is now `&mut self`
- › Incremental KV cache — each new prompt is reconciled against the tokens resident in the cache via longest-common-prefix; only the diverging suffix is decoded (`clear_kv_cache_seq`), and generated tokens are appended for reuse on the next turn
- › KV cache reconciles with orion-core pruning automatically (truncates at the divergence point, system-prompt prefix retained) and resets on `clear_conversation`, model switch/unload, and context overflow via `InferenceEngine::reset_context()`
- › Metal compute pipelines now compile once per loaded model instead of once per turn
- › `AgentEvent::GenerationStats` — new event carrying real tokens generated, tokens/sec, time-to-first-token, and generation time; `MetricsCollector` records actual TTFT instead of approximating it as `1000/tps`
- › Frontend `AgentEvent` union extended with the `generation_stats` variant
- › `tauri-plugin-window-state` added for persistent window geometry
- › `useKeyboardShortcuts` hook for global shortcut handling
- › `ChatWindow` converted to `forwardRef` to expose `clearChat()` for programmatic reset
- › Context budget metrics now recorded to `MetricsCollector` — wired `ContextBudget` agent event data through to session metrics
- › `SessionMetrics` TypeScript type extended with `context_used_tokens`, `context_max_tokens`, `context_messages_in_context`, `context_messages_pruned`
- › `ChatTemplate` trait in orion-core — pluggable prompt formatting with `format()`, `format_system()`, `format_message()`, `assistant_prefix()`
- › `ChatMLTemplate` default implementation
- › `detect_template()` — auto-selects template from GGUF metadata string, falls back to ChatML
- › Pair-wise context pruning — user+assistant turns pruned as units, never orphans a question or answer
- › Template-aware token accounting — budget counts template overhead (`<|im_start|>`, `<|im_end|>`, etc.) per message, not just raw content
- › System prompt + tool schema tokens deducted from context budget before conversation pruning
- › `CoreError::Context` on overflow — clear error when system prompt or latest message exceeds context budget
- › `prepare_context()` replaces separate `prune_messages()` + `format_chatml()` — single function for prune, format, and budget accounting
- › `Agent::with_template()` constructor and `set_template()` for runtime template switching
- › 17 new tests in `orion-core/tests/context_tests.rs` (pair-wise pruning, overflow errors, template overhead, tool budget, ChatML formatting, detect_template)
v0.3.0-alpha
DaruHello World orion-core, an inbuilt harness
macOS
Universal (Apple Silicon + Intel) · ~24 MB
- › Structured agent event system — chat now receives rich lifecycle events (start, delta, end, error, warning) instead of raw token strings
- › Token budget bar in chat toolbar — shows context window usage (e.g. "1,200 / 4,096 tokens") with visual warning when >80% full
- › Discord community link added to About modal
- › System prompt support — backend accepts custom system prompts via `set_system_prompt` command
- › Inference parameter tuning — `set_inference_params` command for runtime temperature, context size, and thread count updates
- › `orion-core` integration: `LlamaCppBackend` implements `LlmBackend` trait, wrapping `InferenceEngine` for backend-agnostic agent loop
- › `AgentState` replaces `InferenceState` — all inference commands route through `orion_core::Agent` for conversation state, context pruning, and ChatML prompt formatting
- › `agent-event` Tauri event channel replaces `token-stream` — emits all 13 `AgentEvent` variants (`agent_start`, `message_delta`, `message_end`, `context_budget`, `error`, etc.)
- › `set_system_prompt` and `set_inference_params` IPC commands
- › `InferenceParams` extended with `n_threads` field (defaults to `available_parallelism - 2`)
- › Removed `inference/streaming.rs` (`TokenEvent`, `ChatMessage`, `format_chatml` superseded by orion-core types)
- › Removed dead `format_prompt` and `truncate_to_fit` from `InferenceEngine` (context pipeline now in orion-core)
- › Frontend: `AgentEvent` discriminated union type (13 variants), `AgentMessage`, `ToolCall`, `ToolResultData` types added to `lib/types.ts`
- › Frontend: `setSystemPrompt()` and `setInferenceParams()` IPC wrappers in `lib/tauri.ts`
- › Frontend: `ChatWindow.tsx` migrated from `token-stream` listener to `agent-event` with full event handling
- › Zero compiler warnings (Rust), zero TypeScript errors
v0.2.3-beta
FayeSuggested Models and Onboarding
macOS
Universal (Apple Silicon + Intel) · ~24 MB
- › "Surprise Me" model discovery — one-click random suggestions from a curated list of small, high-quality models
- › Curated model list: TinyLlama 1.1B, Qwen 2.5 (0.5B/1.5B/3B), Phi 3.5 Mini, Gemma 2 2B, StableLM 2 1.6B, SmolLM2 1.7B, Llama 3.2 (1B/3B)
- › "Try Another" re-roll button — skips already-seen and already-downloaded models
- › Inline download with progress tracking directly from the suggestion card
- › HuggingFace search result caching (5-minute TTL) — fewer API calls, faster repeat searches
- › Quantization variant badges on search result cards — see available quants at a glance
- › Sort search results by downloads, likes, or recent activity
- › Download pause/resume — pause active downloads, resume later (supports HTTP Range)
- › Cancel download button with proper cleanup
- › Disk space check before downloading — warns if insufficient space
- › Download complete toast with "Load now?" action button
- › Active download indicator in footer bar
- › First-run welcome wizard — guided setup: download a starter model, auto-load, start chatting
- › Partial download recovery — detects incomplete downloads after app crash and offers resume
- › `src/lib/curatedModels.ts` — maintainable curated model list as a typed constant array
- › `SurpriseCard` component with full download lifecycle (progress, pause/resume/cancel)
- › `SearchCache` with TTL-based expiry in `HuggingFaceClient`
- › `DownloadManagerState` with `DownloadHandle` for cancel/pause control
- › `DownloadSidecar` metadata JSON written alongside downloaded GGUFs
- › `available_disk_space()` using `sysinfo::Disks` for volume-aware space check
- › `cancel_download`, `pause_download`, `resume_download`, `list_partial_downloads`, `get_available_disk_space` IPC commands
- › Download resume via HTTP `Range` header with `.gguf.part` file detection
- › `WelcomeWizard` component with 4-step flow (welcome → download → loading → ready)
- › `useDownloads` hook now tracks download completion transitions for toast notifications
v0.2.2-alpha
EdYour models, your rules
macOS
Universal (Apple Silicon + Intel) · ~24 MB
- › GGUF metadata extraction — model cards now show parameter count, context length, and architecture
- › Runtime controls — functional thread count, temperature, and context length sliders in Settings
- › Chat template auto-detection from GGUF metadata with manual override dropdown (ChatML, Llama 3, Mistral, Gemma, Phi-3, DeepSeek, etc.)
- › Context overflow handling — oldest messages automatically pruned when conversation exceeds context window
- › Model status events — real-time loading/ready/error/unloaded status via Tauri events
- › Update notification toast with download button when a new version is available
- › Actionable toast notifications (toasts can now have clickable action buttons)
- › `GgufModelInfo` struct with full GGUF header metadata (params, layers, heads, embedding dim, architecture, chat template)
- › `InferenceEngine::format_prompt()` uses `apply_chat_template()` from llama.cpp with ChatML fallback
- › `InferenceEngine::truncate_to_fit()` for pair-wise context pruning
- › `model-status` Tauri event channel with `ModelStatusEvent` payload
- › `AppConfig` extended with `chat_template` option for manual override
- › `generate()` accepts configurable `n_threads` and `context_length` from config
- › `ModelMetadata` enriched with `context_length`, `architecture`, `chat_template` fields (backward-compatible via `#[serde(default)]`)
- › Auto-update check via `https://orionpod.com/api/latest.json` on app launch
- › `check_for_updates` Rust IPC command with semver-aware version comparison
- › `useUpdateCheck` hook (5s delayed, silent fail, non-blocking)
- › `update-web-release.cjs` script for automated release metadata updates
- › Changelog auto-extraction from `CHANGELOG.md` into `releases.js`
v0.2.1-rc1
Kusanagi DeprecatedFirst public release. Metal GPU acceleration, HuggingFace model browser, real-time observability.
macOS
Universal (Apple Silicon + Intel) · ~30 MB
- › Chat interface with streaming responses and markdown rendering
- › HuggingFace model browser with hardware compatibility filtering
- › GGUF model support — download from HuggingFace or upload local files
- › Real-time observability dashboard (tokens/s, memory, latency, GPU usage)
- › Metal GPU acceleration on Apple Silicon
- › Model parameter controls (temperature, context length, top-p, top-k)
- › Toast notifications and user-friendly error messages
- › Glassmorphism UI with macOS vibrancy
- › Tauri v2 + React + TypeScript + Rust + llama.cpp
- › orion-core agent harness crate (backend-agnostic)
- › Universal macOS binary (Apple Silicon + Intel), ~30 MB
- › Starts in under 2 seconds, <50 MB RAM idle
- › Zero telemetry, zero analytics, zero cloud dependencies
System Requirements
- ✓ macOS 10.15 (Catalina) or later
- ✓ Apple Silicon (M-series) recommended for Metal GPU acceleration
- ✓ 8 GB RAM minimum for 7B models
Open the DMG → drag OrionPod to Applications → launch. That's it.