Download llamay

macOS 13 Ventura or later

Apple Silicon and Intel

The one-liner takes the Metal build on Apple Silicon and the portable one on Intel. It installs llamay to /usr/local/bin and a LaunchAgent that runs as you, so the server is there after a reboot.

Then the server is on http://127.0.0.1:11435. Read the script before you run it — it is served from the repository it lives in.

Or an app, and the binaries

llamay.app drag to Applications · signed and notarised, so it opens with no security warning

CLI · Apple Silicon Metal and Accelerate

CLI · Apple Silicon, portable no cgo — the reference kernels, for reproducing a number

CLI · Intel portable, AVX2

glibc · x86-64 and arm64

Linux

Installs a systemd service running as its own llamay user, with the binary at /usr/bin/llamay. Set LLAMAY_NO_SERVICE=1 beforehand to install only the CLI.

The script refuses rather than continuing if the machine has no sha256sum or shasum: an installer that skips the check is one that runs whatever it was handed.

Or a package

Debian · Ubuntu · x86-64.deb

Debian · Ubuntu · arm64.deb

Fedora · RHEL · SUSE · x86-64.rpm

Fedora · RHEL · SUSE · arm64.rpm

CLI with CUDA and Vulkanneeds a driver, no toolkit

CLI · x86-64portable

CLI · arm64portable

Removing the package keeps your models: they can be tens of gigabytes, and reinstalling does not bring them back.

Windows 10 or later

Windows

Per user, so it needs no administrator prompt. Adds llamay to PATH and registers a logon task, so the server is there after a reboot.

Paste it in PowerShell. Read the script. The installers are Authenticode signed through Azure Trusted Signing.

Or an installer

Installer · x86-64signed

Installer · arm64signed

CLI · x86-64portable

CLI · arm64portable

Go 1.26

From source

The default build has no cgo and nothing outside the standard library, which is what makes make cross six targets from one machine rather than six machines.

from a checkout

$ make build          # ./bin/llamay, portable, no cgo
$ make build-metal    # + Metal
$ make build-fast     # + Metal and Accelerate
$ make build-cuda     # + CUDA — no toolkit, no card needed to build
$ make build-vulkan   # + Vulkan — no SDK, no loader needed to build
$ make check          # fmt, vet, tests, and the invariants

Before you install

Five things worth knowing first.

It is slower than llama.cpp at decoding

On an M4, llamay decodes at about 68% of llama.cpp's rate and 64% of Ollama's. Its x86 prefill is 1.4× to 2× llama.cpp's on three of four models. Both columns, every round, are on the throughput page.

Two models need the exact KV cache

gpt-oss and phi3-mini exceed the quality bound under vq8. Run them with -kv f32. llamay verify tells you per file, rather than leaving you to notice.

gpt2 is 4.67% off llama.cpp's perplexity

Where every other architecture tested is inside 1.3%. It is a known defect in that block, not a rounding difference, and it is written up rather than omitted.

The Windows and Linux builds have no GUI

The chat window is macOS only today. Elsewhere llamay is a CLI and a server — which is the whole surface for anyone who wanted that anyway.

Nothing is sent anywhere

The server listens on loopback, and no provider is configured until you add one. If you put it on 0.0.0.0, set LLAMAY_API_KEY in the same breath.

The scripts are served from the repository

llamay.com/install.sh is deployed from packaging/install.sh on every push, which is the only way to keep the reviewed script and the executed one from drifting.