The Path › Chapter 2 of 9

Your first local brain

an open model on your own computer, offline

About 20 minutes · for strangers · bricks: awdk

Teach

What a model is

A language model is a very large file of numbers — billions of them — and a program that uses those numbers to turn text in into text out. That is the whole thing. When people say "the AI", they mean the numbers plus the program that runs them. Running it is called serving: the program loads the file into memory and waits on a port (a numbered door on your machine) for questions.

Think of a vinyl record and a turntable. The record is the model; the turntable is the server; the music only exists while both are together. The record is useless on its own, and the turntable plays whatever you put on it.

What "8B" means

"8B" is eight billion numbers. Each number normally takes 16 bits, so an 8B model is about 16 GB — too big for most laptops' memory. Quantization stores each number in fewer bits. At 4 bits the same model is ~4.5 GB and almost as good. At 1 bit — which is what Bonsai does — a 27-billion-number model fits in 3.8 GB and runs on a plain CPU with 8 GB of RAM.

Fewer bits is a trade: smaller and faster, very slightly less precise. The kit picks the trade that fits your machine, so you do not have to.

The open model stack

Two families of brains are Aitherium's own, and both are open:

  • The orchestrator model — the open 8-billion-parameter brain the platform itself runs

on. This is the one adk quickstart-local downloads for you (an openly published GGUF file, quantized to fit your RAM).

  • Bonsai — the 1-bit family from PrismML. Bonsai-27B is the "big brain in a tiny

box" option, and because 1-bit files need a special version of the server, the kit ships it as a container (adk bonsai-local, optional below).

Open means: the file is public, you can download it, and nothing about it phones home. Unplug your network after this chapter — it still answers.

What quickstart-local does, step by step

It prints five steps as it goes:

  1. Detects your hardware — CPU or GPU, how much memory.
  2. Picks a backend — the program that will serve the model: plain llama.cpp

on a CPU (no dependencies), Ollama if you already have it, vLLM on an NVIDIA card with Docker.

  1. Installs it and downloads a model sized for you. This is the slow step.
  2. Verifies it by asking the model a question and checking an answer comes back.

A failed check fails the command — "quickstart" means proven working.

  1. Registers the brain with the kit, so every later chapter can find it.

Do

1

This picks the fastest backend your machine has (llama.cpp on a plain CPU, Ollama if you already have it, vLLM on NVIDIA + Docker), downloads an open GGUF model sized for your RAM, checks it answers, and saves the endpoint. The first run downloads several GB; later runs are instant.

bash
adk quickstart-local

You should see: five numbered steps, [1/5] Detecting hardware... through [5/5], then Config saved: and an endpoint such as http://localhost:8200/v1

If not: Each step prints what it is doing. If step 3 fails to download, check your disk has 6 GB free and run the same command again - the download resumes where it stopped.

2optional

Bonsai-27B is a 27-billion-parameter model squeezed to about one bit per weight - 3.8 GB, runs on a CPU with 8 GB RAM. The 1-bit format needs PrismML's own llama.cpp build, which is why it ships as a container.

bash
adk bonsai-local

You should see: Starting Bonsai-27B ... and a healthy endpoint on port 8090. If the container image is not on your machine yet, it BUILDS it first from PrismML's public sources - a 3.8 GB download plus a compile, up to ~40 minutes once, seconds ever after.

If not: This one needs Docker Desktop and the Bonsai container image. Without them, skip - the brain from the previous step is already running. Bonsai is explained on the left; you can come back for it.

Check you are done

bash
adk backend status

You should see: Backend: llamacpp (or ollama / vllm) and Endpoint: http://localhost:... - the brain the kit will use from now on

What you learned

One estate, 8 doors

These projects are organs of the same system — the same identity plane, the same agent runtime, the same operational discipline. Follow any of them.

the osaitherium.comThe Living OS these four repos are cut from. Open, no login.pythonAither ADKBuild AI agent fleets — 3 lines, any backend, local or cloud.markdownawskillsPortable agent skills — self-contained procedures an agent loads on demand.markdownawknowledgeHow to run a coding agent so the result survives — the laws, with evidence.cudaaitherkvcacheNear-optimal KV cache quantization for LLM inference — sub-byte compression.powershellAitherZeroPowerShell 7+ automation framework — numbered, self-describing scripts.writingThe BlogHow all of it is actually built, measured and operated.pythonawrunA priority-aware queue and dispatcher for agentic runs and ad-hoc CI builds. It also judges whether the runner pool is big enough for the queue it is draining, and can ask a host to grow it -- reserving capacity is zero-sum, so a saturated pool needs more of it, not a different share of it.

The ecosystem this belongs to

Every Aitherium repository publishes an aither-manifest.json beside its page, and each surface reads all of them — so the network is browsable from any node in it.