Start here

Install llamay

Every platform, every package, and what to do after it lands. Pick your row, follow it to the end, and you will have a model answering on your own machine with nothing configured and no account.

Which one do you want?

macOS

The app (easiest)

1. Download llamay_<version>_macos.dmg from the download page.
2. Open it and drag llamay to Applications.
3. Launch it. It is signed and notarized, so it opens without a warning.

The app bundles the engine and manages the model store under ~/.llamay. Install Command Line… in its menu-bar menu links llamay into /usr/local/bin (it asks for an administrator password), so the command line and the app share one set of downloaded models rather than each keeping its own. The app needs macOS 13 or later on Apple silicon; an Intel Mac takes the command line below. What the app does once it is open is on the apps page.

The command line

curl -fsSL https://llamay.com/install.sh | sh

The script detects your architecture, downloads the matching build and the release's SHA256SUMS, and checks the sum before installing — that check is not optional, and the script refuses rather than skipping it on a machine with no sha256 tool. The binary goes to /usr/local/bin, which it will ask for sudo to write.

To avoid the password prompt entirely, install somewhere you already own:

LLAMAY_INSTALL_DIR="$HOME/.local/bin" curl -fsSL https://llamay.com/install.sh | sh

Apple Silicon and the GPU

On an M-series Mac, take the Metal build. It is a separate artifact because it needs cgo and the plain one does not:

Check which one you are running at any time — the backend field says device when the GPU is in use and cpu when it is not. This is the plain darwin_arm64 build, so it says cpu:

curl -s localhost:11435/healthz

{"backend":"cpu","id":"590bc10b059c7189","instance":"caacd23f45f3d8ee","model":"qwen2.5:0.5b","status":"ok","version":"0.3.0"}

From the tarball, by hand

VER=$(curl -fsSLI -o /dev/null -w '%{url_effective}' https://github.com/AzmxAI/llamay-releases/releases/latest | sed 's|.*/tag/v||')
curl -fsSLO https://github.com/AzmxAI/llamay-releases/releases/download/v$VER/llamay_${VER}_darwin_arm64_metal.tar.gz
curl -fsSLO https://github.com/AzmxAI/llamay-releases/releases/download/v$VER/SHA256SUMS

# Verify before you extract, not after.
shasum -a 256 -c SHA256SUMS --ignore-missing

tar xzf llamay_${VER}_darwin_arm64_metal.tar.gz
sudo install -m 0755 llamay /usr/local/bin/llamay
llamay --version

xattr -d com.apple.quarantine llamay

Windows

The installer

1. Download llamay_<version>_windows_amd64_setup.exe — or the arm64 one on a Snapdragon or Surface Pro X.
2. Run it. It is signed with Azure Trusted Signing, so SmartScreen lets it through.
3. It installs to %LOCALAPPDATA%\Programs\llamay, adds that to your PATH, and offers to start llamay at logon.

Per-user by default, which is why there is no elevation prompt. For a machine-wide install into Program Files, run it elevated.

The logon task runs llamayw.exe, a launcher with no console window, so nothing opens a terminal at logon and there is no window whose closing stops the server. Its output goes to %LOCALAPPDATA%\llamay\logs. The Start Menu has llamay (the chat window), llamay (native window), and llamay server, which runs the server in a console on purpose, for watching it. What the windows do is on the apps page.

Unattended installs — for scripts, and for Intune or Group Policy wrapping the installer as a Win32 app:

llamay_<version>_windows_amd64_setup.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART `
    /CURRENTUSER `
    /TASKS="addtopath,startserver" `
    /LOG="C:\Temp\llamay-setup.log"

/ALLUSERS instead of /CURRENTUSER installs for the whole machine and needs elevation; /DIR="…" picks the folder. /TASKS replaces the default selection rather than adding to it — name every task you want. Uninstalling removes the PATH entry and the logon task, and leaves your models where they are.

PowerShell, one line

irm https://llamay.com/install.ps1 | iex

Same shape as the shell script: detect, download, verify the checksum, install, add to PATH. Add -AllUsers to install into Program Files instead, which needs an elevated shell.

& ([scriptblock]::Create((irm https://llamay.com/install.ps1))) -AllUsers

Open a new terminal after installing. A PATH change does not reach a shell that was already running.

The portable zip

No installer, no PATH change, nothing written outside the folder — useful on a locked-down machine or a USB stick.

$ver = ((Invoke-WebRequest "https://github.com/AzmxAI/llamay-releases/releases/latest" -UseBasicParsing).BaseResponse.ResponseUri.AbsoluteUri -replace '.*/tag/v','')
Invoke-WebRequest "https://github.com/AzmxAI/llamay-releases/releases/download/v$ver/llamay_${ver}_windows_amd64.zip" -OutFile llamay.zip -UseBasicParsing
Expand-Archive llamay.zip -DestinationPath .\llamay
.\llamay\llamay.exe --version

Running it at logon

A scheduled task rather than a Windows service, and deliberately: llamay wants a user's model store and a desktop session, which a service running as SYSTEM has neither of.

schtasks /Query  /TN "llamay server"      # the installer creates this
schtasks /End    /TN "llamay server"      # stop it now
schtasks /Delete /TN "llamay server" /F   # stop it starting at logon

Linux

Debian, Ubuntu, Mint, Pop!_OS

VER=$(curl -fsSLI -o /dev/null -w '%{url_effective}' https://github.com/AzmxAI/llamay-releases/releases/latest | sed 's|.*/tag/v||')
curl -fsSLO https://github.com/AzmxAI/llamay-releases/releases/download/v$VER/llamay_${VER}_amd64.deb
sudo apt install ./llamay_${VER}_amd64.deb

Use arm64.deb on a Raspberry Pi 5, an Ampere box or Graviton.

Fedora, RHEL, Rocky, Alma, openSUSE

VER=$(curl -fsSLI -o /dev/null -w '%{url_effective}' https://github.com/AzmxAI/llamay-releases/releases/latest | sed 's|.*/tag/v||')
sudo dnf install https://github.com/AzmxAI/llamay-releases/releases/download/v$VER/llamay-${VER}-1.x86_64.rpm

Use aarch64.rpm on ARM. On openSUSE, zypper install takes the same URL.

The service the packages install

Both packages ship a systemd unit that runs as its own llamay user with a store at /var/lib/llamay/models. The package enables and starts it when it installs, and restarts it on an upgrade. It downloads nothing on its own: with no model it waits, and answers /healthz with "models":0.

systemctl status llamay
journalctl -u llamay -f
sudo systemctl disable --now llamay      # if you would rather start it yourself

One store for the service and for you. The store is shared: it is group-writable by llamay, and /etc/profile.d/llamay.sh points a login shell's LLAMAY_MODELS at it — so a model you llamay pull in a terminal is one the service, and Studio, can serve. Writing to it needs the group, once:

sudo usermod -aG llamay "$USER"     # then log out and back in

Pulls made from Studio go through the service and need no group. Set LLAMAY_MODELS yourself to keep a store of your own instead.

It listens on loopback. To change the address, the model or anything else, override rather than editing the unit — the unit belongs to the package, and an upgrade replaces it and would take your edit with it:

sudo systemctl edit llamay

[Service]
ExecStart=
ExecStart=/usr/bin/llamay serve -m /var/lib/llamay/models/qwen.gguf -concurrency 4

The empty ExecStart= is required. Without it systemd appends to the existing one and the service tries to start twice.

The desktop window

llamay-desktop is a separate package that puts a llamay window in the applications menu. It needs the engine package from the same release and WebKitGTK, and is built for Ubuntu 22.04 or later, Debian 12 or later, and current Fedora:

sudo apt install ./llamay_${VER}_amd64.deb ./llamay_${VER}_linux_amd64_desktop.deb
# Fedora: sudo dnf install ./llamay-${VER}-1.x86_64.rpm ./llamay_${VER}_linux_amd64_desktop.rpm

Any distribution, from the tarball

curl -fsSL https://llamay.com/install.sh | sh

Works on anything with a shell and curl: Arch, Alpine, NixOS, a container, a CI runner. No package manager involved, and no root needed if you set LLAMAY_INSTALL_DIR to somewhere you own.

With root, it installs the same system service as the packages, with the same shared store, and adds you to the llamay group. Installed into a directory you own, it installs a systemctl --user service instead, on your own store in ~/.llamay — no root, and nothing outside your home.

NVIDIA and AMD

Take linux_amd64_gpu.tar.gz. It carries both the CUDA and Vulkan backends in one binary, and the loader is dlopened at first use — so there is no CUDA toolkit to install, no libvulkan-dev, and the same file runs on a machine with no GPU at all.

llamay serve -m model.gguf
# then, in another terminal
curl -s localhost:11435/healthz | grep -o '"backend":"[a-z]*"'

"device" means a GPU is doing the work. "cpu" means the driver was not found or declined, and the server says which in its first log line rather than quietly running slowly.

Android

llamay_<version>_android_arm64.tar.gz is the command line for arm64 Android devices: no installer, no app, CPU only. Unpack it in a terminal on the device and run ./llamay. It is built and published with every release; these pages have not tested it on a device. See the apps page.

Containers

There is no public container image. A container is one file and a volume, so build one from the Linux tarball:

# Dockerfile
FROM debian:12-slim
ARG VER=0.4.3
ADD https://github.com/AzmxAI/llamay-releases/releases/download/v${VER}/llamay_${VER}_linux_amd64.tar.gz /tmp/llamay.tgz
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates \
 && rm -rf /var/lib/apt/lists/* \
 && tar xzf /tmp/llamay.tgz -C /usr/local/bin ./llamay && rm /tmp/llamay.tgz \
 && useradd --system --home /models llamay && mkdir -p /models && chown llamay /models
USER llamay
ENV LLAMAY_MODELS=/models LLAMAY_NO_UPDATE_CHECK=1
EXPOSE 11435
ENTRYPOINT ["/usr/local/bin/llamay"]
CMD ["serve", "-addr", ":11435"]

docker build -t llamay .
docker run --rm -v llamay-models:/models llamay pull qwen2.5:0.5b
docker run --rm -p 11435:11435 -v llamay-models:/models \
  -e LLAMAY_API_KEY="$(openssl rand -hex 32)" llamay

• -addr :11435, every interface, is needed inside a container, because loopback there is reachable by nothing. Set a key either way; -p 11435:11435 publishes the port on every interface of the host, so if nothing else must reach it, put the host's loopback address in front of the port.
• The volume holds the models, so they survive the container.
• Check the tarball against SHA256SUMS first (verifying a release), or pin its digest in the build.
• For NVIDIA, use the linux_amd64_gpu tarball and run with --gpus all; the image then needs the NVIDIA container runtime on the host.

This Dockerfile is a starting point written for these docs; it was not run as part of preparing them.

What the one-line install actually does, pointed at a scratch directory so nothing on this machine moved:

~/llamay $ # what the install line actually does, pointed at a scratch dir
~/llamay $ LLAMAY_INSTALL_DIR=build/demo/work/bin LLAMAY_NO_SERVICE=1 \
>   sh packaging/install.sh
llamay 0.1.58 — darwin/arm64 (gpu)
downloading llamay_0.1.58_darwin_arm64_metal.tar.gz
checksum ok
installed build/demo/work/bin/llamay

run:  llamay serve
~/llamay $ # it verified a signature-grade checksum before it wrote anything
~/llamay $ build/demo/work/bin/llamay version
llamay         0.1.58
build          go1.26.8 darwin/arm64, cgo on, tags metal,accelerate
revision       8e08433a6e86
kernels        neon+i8mm
gpu            metal built in; metal:Apple M4
formats        23: F32, F16, Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, Q8_1, Q2_K, Q3_K, Q4_K, Q5_K, Q6_
K, Q8_K, IQ2_XXS, IQ2_XS, IQ3_XXS, IQ4_NL, IQ3_S, IQ2_S, IQ4_XS, BF16, MXFP4
architectures  17 entries, 39 names: azmx (azmx-one, llama, mistral), qwen2 (qwen3), gemma,
gemma2, gemma3 (gemma3_text), azmx-code (azmx_code, azmxcode), gpt-oss (gptoss, gpt_oss, ope
nai-moe), phi3 (phi3.5), stablelm (stablelm2), gpt2, falcon (rw), gptneox (gpt_neox, gpt-neo
x, neox), phi2, qwen3moe (qwen3_moe), qwen2moe (qwen2_moe), azmx-ocr (ocr, ctc), azmx-embed
(bert, embedding)

Verifying what you downloaded

Every release publishes SHA256SUMS over the artifacts as they were uploaded. The installers check it for you; if you download by hand, check it yourself.

--ignore-missing matters: the file covers every artifact in the release and you downloaded one of them. Without it the check lists every file you did not download as missing, which reads like a failure.

SHA256SUMS is itself signed with Sigstore. To check that signature with cosign, and the macOS and Windows code signatures, see verifying a release.

Pulling a model, with a transfer that resumes:

~/llamay $ # a model this machine does not have yet
~/llamay $ llamay pull gemma3:270m
pulling manifest
pulling 735af2139dc6
  [========================] 100.0%  278.0 MiB / 278.0 MiB  (5.0 MiB/s)
verifying sha256 digest
writing manifest
stored gemma3:270m (278.0 MiB)
  sha256:735af2139dc652bf01112746474883d79a52fa1c19038265d363e3d42556f7a2

llamay run -m gemma3:270m -p "hello"
~/llamay $ llamay models
name                                     size  architecture           store
all-minilm:latest                    43.8 MiB  bert (embedder)        llamay
gemma2:2b                             1.6 GiB  gemma2 (decoder)       llamay
gemma3:270m                         278.0 MiB  gemma3 (decoder)       llamay
gpt2:q8_0                           169.4 MiB  gpt2 (decoder)         llamay
qwen2.5:0.5b                        379.4 MiB  qwen2 (decoder)        llamay
smollm2:135m                        258.3 MiB  llama (decoder)        llamay
azmx-code-test:latest                 1.1 GiB  gemma3 (decoder)       ollama

6 in ~/.llamay/models, 1 in ~/.ollama/models

Your first five minutes

Identical on every platform once llamay is on your PATH.

# 1. Get a model. A 7B is a good first choice on 16 GB; the 0.5B runs anywhere.
llamay pull hf:Qwen/Qwen2.5-7B-Instruct-GGUF/qwen2.5-7b-instruct-q4_k_m.gguf -as qwen2.5:7b
llamay pull qwen2.5:0.5b

# 2. Ask it something, with no server involved.
llamay run -m qwen2.5:0.5b -chat -p "why is the sky blue?"

# 3. Start the server.
llamay serve

# 4. Use it from anything that speaks OpenAI.
curl -s localhost:11435/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"","messages":[{"role":"user","content":"hello"}]}'

Already have Ollama? llamay reads its store, so every model you have pulled is available with no re-download: llamay run -m llama3.2:3b.

Which model fits your machine

The weights are the number people plan for and the KV cache is the one that surprises them: it grows with context length times concurrency. -kv vq8 roughly halves it, at a cost the verifier measures rather than asserts.

Upgrading

llamay update fetches the same build you have: the Metal build stays Metal, the Linux GPU build stays CUDA and Vulkan, and a CPU build stays CPU. If a release has no file for your build, the update stops and names the missing file rather than installing a different build. On Windows it runs the installer only for a copy the installer put there; a copy from install.ps1 or an unpacked .zip has its llamay.exe and llamayw.exe replaced where they are.

Your models are not touched by an upgrade — they live in the store, not beside the binary. Restart the server afterwards; a running process keeps the old binary mapped.

Uninstalling

# macOS, installed by the script: the LaunchAgent, then the binary
launchctl bootout gui/$(id -u)/com.llamay.server
rm ~/Library/LaunchAgents/com.llamay.server.plist
sudo rm /usr/local/bin/llamay /usr/local/bin/llamay-server

# Linux, installed by the script: the service, then the binary
sudo systemctl disable --now llamay
sudo rm /etc/systemd/system/llamay.service /etc/profile.d/llamay.sh
sudo rm /usr/local/bin/llamay /usr/local/bin/llamay-server
# (with LLAMAY_INSTALL_DIR in your home it is a user service instead:
#  systemctl --user disable --now llamay; rm ~/.config/systemd/user/llamay.service)

# a tarball you unpacked yourself
rm /path/to/llamay

# Debian / Ubuntu
sudo apt remove llamay

# Fedora / RHEL
sudo dnf remove llamay

# the models, which no uninstall removes for you
rm -rf ~/.llamay          # or /var/lib/llamay for a packaged service

On Windows, use Add or Remove Programs, or delete the folder if you took the zip. The model store is left alone on purpose: it is often tens of gigabytes somebody spent an evening downloading, and an uninstaller that deletes it without asking is one people only forgive once.

When it does not work

Where to go next

Quickstart is the same first session in fewer words. The live playground runs against the server you just started. Recipes is what to build with it, and Serving is what to do when it is not only your machine calling.