profile · v3 · rocm
16k context on a 35B MoE
q8 KV cache trades 3 % decode for context that actually holds a repo map. 22.4 GB resident at 16k.
flags
llama-server-ngl 99 -c 16384 -fa 1 -ctk q8_0 -ctv q8_0 --parallel 1 -b 1024To use these, create a custom profile from one you already have and paste the flags in:
hal0 profile create my-kv-q8-longctx --from <a-profile-you-have> --flags "<copied flags>"
hal0 does not install community .toml files directly. The dashboard's Profiles page does the same thing.
history
- v32026-06-19-c 16384 at q8 cache — 22.4 GB resident, 3 % decode given up
- v22026-05-31-b 1024
- v12026-05-10initial