Ai Keeper/Guides/Models, engines, and runtimes
← All workflows
Collection 02 · Local inference

Models, engines, and runtimes

Acquire model files, install the software that serves them, create instances, choose roles, tune safely, and clean up without breaking active work.

21 workflowsBeginner-first

Find one workflow

Filter this collection by name, outcome, provider, engine, channel, mode, or command.

21 shown
Workflow 01

Install or repair required engines

Let Ai Keeper install the supported command-line pieces used to run local models.

Beginner5–15 minResult · Required engine rows are healthy✓ Source-audited workflow
Engines hub reporting all required dependencies installed, with Re-check All and Update All.
Each row shows the resolved path, install source, version policy, and when it was last checked. Read this before blaming a model for an engine failure.
Before you start
  • Internet access.
  • Enough disk space for managed tools.

  1. Open Engines.

  2. Click Fix Required. Keep the app open while each required row installs or verifies.

  3. If one row fails, expand Output Log, read the last specific error, then use that row's Repair.

  4. Click Re-check after fixing an external requirement such as Xcode command-line tools.

You are done when
No required row shows Missing, Broken, or Fix Required.
If something looks wrong
The installer asks for system softwareFollow the macOS prompt, return to Engines, and Re-check. Ai Keeper cannot silently grant system permissions.
A preview release brokeChoose the Stable release channel for that engine, then Repair.
Workflow 02

Install an optional specialist engine

Add mlx-serve, mlx-optiq, ds4, MTPLX, vmlx, TurboQuant, llama-diffusion, or another specialist only when a model or feature needs it.

Intermediate5–20 minResult · One optional engine is available✓ Source-audited workflow
Before you start
  • Know why Automatic is insufficient for this model.
  • Read the engine description and machine requirements first.

  1. Open Engines and click Install Optional….

  2. Select only the engine you need. For example, ds4 is a high-memory DeepSeek V4 specialist; mlx-vlm is for multimodal MLX; llama.cpp engines require GGUF.

  3. Choose Stable unless you are intentionally testing a pre-release, then install.

  4. Open its row after completion and run Re-check.

You are done when
The engine row shows an installed version and a healthy resolved executable or environment.
If something looks wrong
Unsure which engine to chooseDo not install one. Keep the instance on Automatic; Ai Keeper routes compatible models.
The model format does not matchGGUF belongs with llama.cpp-family engines; MLX repositories belong with MLX engines.
Workflow 03

Download a recommended Cookbook model

Use hardware-aware recommendations and create a ready-to-start instance in one pass.

Beginner10–60 minResult · A model and instance are created✓ Source-audited workflow
Download > Cookbook showing the detected Mac, the Fits my machine filter, and Download & Set Up on each recommendation.
The header reads your chip, unified memory, memory tier, and GPU budget — that is what Fits my machine filters against.
Before you start
  • Internet access.
  • A models directory with enough space.

  1. Open Download > Cookbook.

  2. Enable Fits my machine. Choose the job you need: chat, coding, vision, embedding, reranking, speech, or another catalog role.

  3. Open a card and read its memory fit, model size, format, and intended runtime role.

  4. Click Download & Set Up. Keep the app open until both download and setup complete.

  5. Open Runtime > Instances and review the created instance before starting it.

You are done when
The model appears under Runtime > Models and a matching instance appears under Runtime > Instances.
If something looks wrong
Download is slowModel files are often several gigabytes. Pause other large transfers and keep free disk space above the model size.
The card says it barely fitsChoose a smaller quantization for a better first experience.
Workflow 04

Download a model from Hugging Face

Search the wider catalog when the Cookbook does not contain the exact model you want.

Intermediate10–60 minResult · A chosen repository is stored locally✓ Source-audited workflow
Current Download hub with Cookbook and HuggingFace available in the top navigation.
Choose HuggingFace when you already know the repository; return to Cookbook when you want a machine-fit recommendation.
Before you start
  • Know whether you need MLX or GGUF.
  • For gated models, have the required Hugging Face access.

  1. Open Download > HuggingFace and search by organization, model family, or repository name.

  2. Filter for MLX when you want Apple-Silicon-native repositories, or GGUF when you want llama.cpp. Sort and inspect rather than downloading the first name match.

  3. Open the model card. Check parameter count, quantization, disk size, license, and whether the model is chat, vision, embedding, reranking, STT, or TTS.

  4. Choose a destination and start the download. If a GGUF repository offers many files, select one quantization file rather than every variant.

  5. After completion, open Runtime > Models and confirm the detected role and format.

You are done when
The model has a real local path, non-zero size, detected format, and no incomplete-download warning.
If something looks wrong
A GGUF repo is enormousIt probably contains multiple quantizations. Download one suitable .gguf file plus any required projector, not the whole repository.
The model needs permissionAccept its license on Hugging Face and configure authentication; do not paste tokens into prompts.
Workflow 05

Create and start a runtime instance

Turn an installed model file into a listening local server that Chat and the proxy can use.

Beginner3–10 minResult · The instance reaches Ready✓ Source-audited workflow
Current Runtime Models detail showing the Create New Instance action for a selected local model.
Select a model that fits the machine, review its size and capabilities, then use Create New Instance to begin runtime configuration.
Before you start
  • A complete model under Runtime > Models.
  • Required engine dependencies are healthy.

  1. Open Runtime > Instances and choose New Instance.

  2. Select the model and confirm its runtime role. Use Chat/LLM for ordinary conversation; choose Embedding, Reranker, Vision, STT, or TTS only for matching models.

  3. Leave Backend Selection on Automatic. Keep Host on localhost for a private first run, and accept the suggested free port.

  4. Run Optimize if offered, save the instance, then click Start.

  5. Wait for Ready. If it fails, use Diagnose before editing random fields.

You are done when
The instance shows Ready, a listening local endpoint, and the correct role; it also appears in the relevant model picker.
If something looks wrong
Port already in useChoose another suggested port; only one listener can own a port.
Process exits during loadThe model may not fit. Stop other heavy processes in Monitor or use a smaller model.
Wrong roleA reranker or embedding model will not behave like a chat model even if the server starts.
Workflow 06

Optimize and tune an instance safely

Let the app choose a baseline, then change one measurable setting at a time.

Intermediate10–20 minResult · The model is stable at your chosen tradeoff✓ Source-audited workflow
Current Runtime Models detail showing the selected model and its Create New Instance action.
Start from one known model and create its instance before tuning; keep the recommended engine until Diagnostics gives a concrete reason to override it.
Before you start
  • A stopped or editable instance.
  • A repeatable prompt or benchmark for comparison.

  1. Open the instance detail and click Optimize. Review the chosen engine and memory/context recommendations.

  2. Choose a Speed/Quality profile before touching individual sampling controls.

  3. Change only one area at a time: context window, output limit, cache/KV choice, or sampling. Save and restart when the UI says a restart is required.

  4. Open Lab > Benchmark or repeat the same Chat prompt. Record TTFT, tokens/second, memory, and whether output quality stayed acceptable.

  5. Use Reset if several changes made the result worse.

You are done when
The instance starts repeatedly, stays within comfortable memory pressure, and improves the metric you actually care about.
If something looks wrong
Bigger context seems freeSome engines preallocate the entire context and use much more RAM at startup.
Many knobs changed at onceReset, then test one change. Otherwise you cannot tell which setting helped.
Workflow 07

Set up embedding, reranker, vision, STT, or TTS roles

Create specialist instances so routing sends each job to a model designed for it.

Intermediate10–30 minResult · A specialist endpoint passes its task✓ Source-audited workflow
Current Runtime Models tab with the MTP / Assistants category and model-format details visible.
Specialist roles begin with a compatible model. Confirm the asset advertises the required capability before creating its instance and assigning a role.
Before you start
  • A model built for the chosen specialist role.
  • A compatible engine installed.

  1. In Runtime > Models, inspect the model's detected capabilities and read its model card.

  2. Create a new instance and choose the matching role: Embedding for vectors, Reranker for ordering results, Vision for image input, STT for transcription, or TTS for speech output.

  3. Leave the engine Automatic unless the model requires a specialist path. Save and start.

  4. Test the matching feature: index a tiny document, run a rerank request in Playground, attach an image in Chat, transcribe a short clip, or synthesize one sentence.

You are done when
The role-specific action returns the expected type of output and the instance is not offered as an ordinary chat model unless it truly supports chat.
If something looks wrong
BGE model classified unexpectedlyUse the explicit Reranker role for a reranking checkpoint and test /v1/rerank.
Speech model starts but Chat ignores itSpeech models are used by Voice/Media routes, not as conversational LLMs.
Workflow 08

Stop, reveal, or delete models safely

Free memory or disk space without removing files still used by running instances.

Beginner3–10 minResult · Resources are reclaimed without broken references✓ Source-audited workflow
Current Runtime Models tab with folders, model sizes, fit badges, and per-model actions.
Storage actions — reveal, delete, duplicate detection, favorites, folders, and hiding — begin under Runtime > Models.
Before you start
  • Know which conversations or automations depend on the model.

  1. To free memory only, open Runtime > Instances, select the running instance, and click Stop.

  2. To inspect disk use, open Runtime > Models, select the model, and use Show in Finder.

  3. Before deletion, review the model's dependent instances. Remove or repoint those instances first.

  4. Use the model's Delete action and confirm the exact path and size. Clear download cache separately under System > Settings > Storage if needed.

You are done when
The stopped process disappears from Monitor, or the deleted model disappears from Runtime > Models and no remaining instance references it.
If something looks wrong
Need temporary spaceStop the instance or clear cache first; deleting model weights forces a future re-download.
Finder deletion left a stale entryUse the app's refresh/re-check so it can quarantine or reconcile missing assets.
Workflow 09

Run a model with oMLX

The default local engine for most Apple Silicon chat and tool-calling models. This walkthrough covers the primary MLX route.

Beginner10–30 minResult · oMLX reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with MLX.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find oMLX, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with MLX. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select oMLX as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows oMLX as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important oMLX constraintUse Automatic first; install the optional grammar components when structured output is required.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select oMLX manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 10

Run a model with mlx-serve

A native Zig server with OpenAI and Anthropic APIs, prompt lookup, and chunked prefill. This walkthrough covers the specialist MLX route.

Advanced10–30 minResult · mlx-serve reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with MLX.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find mlx-serve, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with MLX. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select mlx-serve as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows mlx-serve as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important mlx-serve constraintInstall it from Engines before selecting it manually.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select mlx-serve manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 11

Run a model with mlx-optiq

OptiQ quantized models, mixed-precision KV cache, MTP, adapters, and Gemma 4 vision sidecars. This walkthrough covers the specialist MLX / OptiQ route.

Advanced10–30 minResult · mlx-optiq reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with MLX / OptiQ.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find mlx-optiq, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with MLX / OptiQ. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select mlx-optiq as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows mlx-optiq as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important mlx-optiq constraintUse an OptiQ-compatible model and verify adapter/base-model compatibility.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select mlx-optiq manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 12

Run a model with ds4 (DwarfStar)

DeepSeek V4 Flash/PRO GGUF serving with DSML tools and compressed disk-backed cache. This walkthrough covers the specialist DeepSeek V4 GGUF route.

Advanced10–30 minResult · ds4 (DwarfStar) reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with DeepSeek V4 GGUF.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find ds4 (DwarfStar), and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with DeepSeek V4 GGUF. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select ds4 (DwarfStar) as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows ds4 (DwarfStar) as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important ds4 (DwarfStar) constraintRequires roughly 96 GB or more unified memory; never force its unsupported CPU path.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select ds4 (DwarfStar) manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 13

Run a model with vllm-mlx

Continuous batching, tool calling, adapters, and mature OpenAI-compatible serving. This walkthrough covers the compatibility fallback MLX route.

Intermediate10–30 minResult · vllm-mlx reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with MLX.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find vllm-mlx, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with MLX. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select vllm-mlx as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows vllm-mlx as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important vllm-mlx constraintPrefer Automatic unless a model or client specifically needs this compatibility path.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select vllm-mlx manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 14

Run a model with vMLX

JANG-family support, continuous batching, multimodal controls, adapters, and native Anthropic messages. This walkthrough covers the specialist MLX / JANG route.

Advanced10–30 minResult · vMLX reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with MLX / JANG.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find vMLX, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with MLX / JANG. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select vMLX as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows vMLX as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important vMLX constraintConfirm the model family and installed vMLX version before manual selection.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select vMLX manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 15

Run a model with mlx-vlm

Direct multimodal MLX serving for vision-language models. This walkthrough covers the manual escape hatch Multimodal MLX route.

Intermediate10–30 minResult · mlx-vlm reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with Multimodal MLX.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find mlx-vlm, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with Multimodal MLX. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select mlx-vlm as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows mlx-vlm as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important mlx-vlm constraintUse only for a model identified as multimodal; a text-only model gains nothing from it.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select mlx-vlm manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 16

Run a model with mlx-flash

Low-memory weight streaming and optional disk-backed KV cache. This walkthrough covers the manual escape hatch MLX route.

Intermediate10–30 minResult · mlx-flash reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with MLX.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find mlx-flash, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with MLX. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select mlx-flash as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows mlx-flash as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important mlx-flash constraintDisk streaming trades speed and SSD activity for lower memory pressure.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select mlx-flash manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 17

Run a model with llama.cpp

The general GGUF engine with Metal offload, grammar, adapters, and broad model compatibility. This walkthrough covers the primary GGUF route.

Beginner10–30 minResult · llama.cpp reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with GGUF.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find llama.cpp, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with GGUF. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select llama.cpp as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows llama.cpp as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important llama.cpp constraintDownload one suitable GGUF quantization, not every file in a repository.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select llama.cpp manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 18

Run a model with llama.cpp TurboQuant

TurboQuant KV/weight compression and compatible MTP draft models. This walkthrough covers the specialist GGUF route.

Advanced10–30 minResult · llama.cpp TurboQuant reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with GGUF.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find llama.cpp TurboQuant, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with GGUF. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select llama.cpp TurboQuant as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows llama.cpp TurboQuant as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important llama.cpp TurboQuant constraintStart from an ordinary llama.cpp configuration, then enable only model-supported TurboQuant features.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select llama.cpp TurboQuant manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 19

Run a model with MTPLX

Native multi-token prediction for compatible Qwen3-Next-style MTP models. This walkthrough covers the specialist MTP-marker MLX route.

Advanced10–30 minResult · MTPLX reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with MTP-marker MLX.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find MTPLX, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with MTP-marker MLX. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select MTPLX as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows MTPLX as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important MTPLX constraintThe model must include the expected MTP markers; structured constraints may require an optional component.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select MTPLX manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 20

Run a model with Rapid-MLX

Fast serving with parser auto-detection, PFlash prompt compression, bounded caches, DSpark, multimodal routes, and request constraints. This walkthrough covers the manual escape hatch MLX route.

Intermediate10–30 minResult · Rapid-MLX reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with MLX.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find Rapid-MLX, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with MLX. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select Rapid-MLX as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows Rapid-MLX as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important Rapid-MLX constraintReview parser detection, public model ID, and any PFlash warning after installation or update; current releases no longer support the old cloud-routing flags.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select Rapid-MLX manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.
Workflow 21

Run a model with llama-diffusion

A managed adapter for masked or diffusion language models in GGUF format. This walkthrough covers the specialist Diffusion-LM GGUF route.

Advanced10–30 minResult · llama-diffusion reaches Ready and answers one test prompt✓ Source catalog + live Runtime navigation verified
Before you start
  • A model compatible with Diffusion-LM GGUF.
  • Enough free storage and unified memory for the selected model.
  • Ai Keeper is running in Server mode.

  1. Open Engines, find llama-diffusion, and run Install or Repair until its health check passes.

    The model file and the serving engine are separate requirements.

  2. Open Download and acquire one model explicitly compatible with Diffusion-LM GGUF. Read the model card before downloading a quantization.

  3. Open Runtime > Instances, create an instance for that model, then select llama-diffusion as the engine. Use a manual choice only because this walkthrough is testing this exact backend.

  4. Run Optimize, review the proposed settings, and keep unsupported or model-specific switches at their defaults.

  5. Click Start. Wait for Ready, then open Chat and ask: Reply with the engine name and one short sentence.

You are done when
Runtime shows llama-diffusion as Ready, Chat returns a streamed answer, and the instance diagnostics show no unsupported launch flag.
If something looks wrong
Important llama-diffusion constraintThis is for compatible diffusion language models, not image diffusion checkpoints.
Automatic chooses another engineThat is expected when Automatic finds a safer match. Select llama-diffusion manually only for this deliberate compatibility test.
The process exits during launchOpen instance Diagnostics and Logs; confirm model format, memory fit, and engine installation before changing tuning flags.

Keep going

← Every Work ModePrevious collectionChat and Assistant →Next collection