Quick start

Qwen3.8-27B on one Blackwell GPU, served over an OpenAI-compatible endpoint. No Python, no PyTorch, no CUDA toolkit: the binary loads the driver and cuBLAS at first use, and everything specific to the model is in the manifest it verifies before running.

What you need

• Linux x86_64 or aarch64, an NVIDIA driver for CUDA 13 (r580 or newer), libcublas.so.13 on the loader path (a CUDA 13 toolkit or runtime image)
• A GB300 (sm_103a): what the published Qwen3.8 kernels were built for

Install

curl -fsSL https://kern-baa.pages.dev/install.sh | sh      # kern: run, test, bench, verify

kern-serve is built from source for now: cd crates/kern-serve && cargo build --release (details in Install).

Get the model

hf download Pegainfer/kern-qwen38-sm103 --local-dir kern-qwen38     # kernels + manifests
hf download Qwen/Qwen3.8-27B                                        # prints the checkpoint path, if you don't have one yet

The checkpoint is used as published: the manifest says which tensor fills which buffer, and tokenizer.json, the chat template and generation_config.json beside the shards supply the tokenizer and the stop tokens.

Serve

Three flags name the artifact: the manifest, the directory of kernels it pins by hash, and the checkpoint. Start the server on one GPU:

kern-serve --manifest kern-qwen38/manifests/qwen3.8-27b.json --kernels kern-qwen38/cubins \
           --weights <your_qwen38_checkpoint_path> --served-model-name qwen3.8-27b

Then, from another shell, a completion:

curl -s http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3.8-27b", "prompt": "The capital of France is", "max_tokens": 32}'

A chat turn, answered directly (Qwen3.8 thinks first unless told not to):

curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3.8-27b", "messages": [{"role": "user", "content": "Write a haiku about Rust."}],
       "max_tokens": 64, "chat_template_kwargs": {"enable_thinking": false}}'

The same, streamed:

curl -N http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3.8-27b", "messages": [{"role": "user", "content": "Write a haiku about Rust."}],
       "max_tokens": 64, "stream": true}'

With thinking on, the reasoning arrives as reasoning deltas and the answer as content. /v1/models lists what is served, /metrics is Prometheus.

--served-model-name is the id the API answers to (default: the manifest's model). --port defaults to 8000. --gpus 0,1,2,3 drives several GPUs in lockstep when the manifest declares a topology; this one is single-GPU.

Generate without a server

kern run --manifest kern-qwen38/manifests/qwen3.8-27b.json --kernels kern-qwen38/cubins \
         --weights <your_qwen38_checkpoint_path> --prompt "The capital of France is" --steps 64

The same three flags, one greedy sequence. Diagnostics go to stderr (the manifest verified, every kernel resolved by its pinned hash, the weights assembled from the shards, the decode step captured as one CUDA graph); the text goes to stdout.

Speculative decoding

Qwen3.8-27B has a DFlash2 draft. One more download, the draft's manifest, and the draft's checkpoint as a second --weights:

hf download incoai/Qwen3.8-27B-DFlash2

kern-serve --manifest kern-qwen38/manifests/qwen3.8-27b-dflash2.json --kernels kern-qwen38/cubins \
           --weights <your_qwen38_checkpoint_path> --weights <your_dflash2_checkpoint_path> \
           --served-model-name qwen3.8-27b-dflash2
kern run   --manifest kern-qwen38/manifests/qwen3.8-27b-dflash2.json --kernels kern-qwen38/cubins \
           --weights <your_qwen38_checkpoint_path> --weights <your_dflash2_checkpoint_path> \
           --prompt "The capital of France is" --steps 64

Every step drafts 7 tokens and verifies them in one pass; the manifest carries the draft, verify and accept programs, the runtime is unchanged. Greedy output is the same text as plain decode. On one GB300:

	plain	DFlash2
single sequence	105 tok/s	261 tok/s, 4.0 tokens accepted per step

What you just ran

The binary contains no model. Which kernel runs where, how the draft is verified, which checkpoint tensor fills which buffer: all of it is in the manifest, and the runtime verified it before executing. Read The model artifact for what that contract is, and Test a kernel change for how a swapped kernel earns its way into a manifest.