Documentation · llamay 0.4.3
llamay
The sovereign inference engine: one program that runs language models on
your own machine, serves the OpenAI, Anthropic and Ollama APIs on
localhost:11435, and sends nothing anywhere unless you tell it
to. These pages cover what it does in version 0.4.3, how to use each part,
and where each part stops.
Start with what you want to do
I just want to chat
Install the Mac, Windows or Linux app, pick a model, and ask. No terminal, no account.
The apps →I'm a developer
Point curl, the OpenAI SDK, the Anthropic SDK or the Ollama client at llamay. Five minutes, real output.
API in five minutes →I'm deploying it for a team
Keys, directory sign-in, TLS and smart cards, the enterprise profile, the audit record and metrics.
Deploying for a team →I'm reviewing it for security
What leaves the machine, every control and how to see it work, how a release is verified, and what is not done.
Security review →Find a task
| I want to… | Read |
|---|---|
| Install llamay, or upgrade or remove it | Install · Quickstart |
| Download a model, or use the ones Ollama already has | Getting models |
| Get JSON that always matches my schema, or have the model call my functions | Structured output and tools |
| Read a picture, a scanned PDF or a video | Images, scans and video |
| Turn speech into text | Speech to text |
| Answer questions from my documents, with citations | Documents and citations · Embeddings and reranking |
| Remove personal data before text goes anywhere | Removing personal data |
| Check a model translates well enough | Evaluating translation |
| Pay for a long prompt once, fork it, save it, move it | Contexts · The state engine |
| Serve it to other people, safely | Deploying for a team · Serving · Security and identity |
| Watch it in production | Observability |
| Move an existing app over from OpenAI, Anthropic, Ollama or llama.cpp | Migrating · Integrating |
| Check a download is genuine | Verifying a release |
| Download or delete my hosted account | Your hosted account |
| Look up a command, a flag, a route or an error | CLI · HTTP API · Errors · Troubleshooting |
What llamay is
- One program. A single static binary per platform, the same one inside every app. It reads GGUF model files and runs them on the CPU, or on a GPU through Metal, CUDA or Vulkan.
- Local by default. It listens on loopback. It has no telemetry. The only call it makes unasked is a daily update check, and one environment variable turns that off.
- Compatible. OpenAI's, Anthropic's and Ollama's APIs on one address, plus MCP, so existing clients need a new base URL and nothing else.
- A context is an object. A conversation's computed state can be forked, saved to a file, restored, and moved to another server — the part of llamay that is not a reimplementation of anything else.
- Controlled when shared. Named callers, TLS, a hash-chained audit record, classification markings, encrypted capture and sealed state, for when it serves more than one person.
Three claims, and where each is checked
| Claim | What it means | Where it is checked |
|---|---|---|
| A context is an object | Fork it, snapshot it, restore it, move it. A fork of a 32-page context eight ways allocates zero pages. | State engine — a reference implementation run in lockstep under randomised operations |
| It is deterministic | The same input gives the same output regardless of thread count or batch composition, and a fork continues with a logit difference of zero. | llamay verify, which you can run on your own model file |
| It computes what llama.cpp computes | Identical token counts on eight real models, and within 1.3% of its perplexity on eleven of thirteen measured. | The matrix — greedy text and perplexity on pinned files |
What it does not do
Named here rather than discovered later. Each is expanded on the page it belongs to.
- No Mamba or state-space models. They are refused at load. DeepSeek-V2 and V3 are refused by name.
- Images on the CPU only. Vision models read pictures on the CPU path, at a minute or two per image on a laptop. Images
- WAV only, no timestamps for speech to text. Speech
- No training. LoRA adapters are served, not trained. The i-quants are read, not written.
- A few JSON Schema keywords are refused rather than silently ignored —
not,if/then,patternProperties— andoneOfis treated asanyOf. The matrix - No independent certification. The security controls are features; llamay has not been assessed or accredited. What is not done
Conventions in these pages
- Examples were run against llamay 0.3.0, mostly with
qwen2.5:0.5bon an Apple M4 laptop CPU. Output shown is what came back. A small model's answers are not a big one's; the shapes are the same. - The recorded demos each name the build and machine they were recorded on.
llamayis lowercase everywhere.- In diagrams, an amber box is the thing the diagram is about, a teal one is proved by a test, and a dashed one is optional.