Written by a human:

I see alot of talk on tech twitter about getting your Strix halo’s working with this new model.

I also see the snags happening in laguna fork llama.cpp build that I wanted to iron out for the community.

This is NOT captured in the llama.cpp command that Laguna would have you run.

Community is seeing approx 20ish token per second generation speeds and , I do not thing they are focused on “Long Horizon Agentic Coding” which would take advantage of the d-flash via code generation.

My proof: My baseline t/s without spec decoding is around 22 t/s after benching.

Im currently hitting 34 t/s during long running code generation in OpenCode (the models use case) over the run you will see token speeds move from 26 up to the 34 t/s mostly hovering in the 26-33 t/s range.

Im running with this command:

~/AI/llama.cpp/laguna-llama-vulkan-build/llama.cpp/build/bin/llama-server --host 0.0.0.0 --port 8080 --ctx-size 250000 -ngl 999 -m laguna-s-2.1-Q4_K_M.gguf -md laguna-s-2.1-DFlash-B
F16.gguf --spec-type draft-dflash  --spec-draft-n-max 4 --spec-draft-p-min 0.55  --jinja  --device Vulkan0 -fa on -b 16384 -ub 2048  -ctk q8_0 -ctv q8_0

Heads Up: Questionable Quants Where Floating Around Launch Day

Since I see people making claims their model is bunk and others saying its not. Its a common thing for the community to get their hands on bad models. At least I was able to confirm this in the past and only reminded with the wildly varied results from users in the wild with this model. That was even before the NVP8 issue, I’ll explain.

Poolside slid out a new NVP8 quant themselves in response to peoples reported issues, so great example of a how bad quants can happen. Always try multiple models, do testing if these are going into prod.

I am using the official Poolside released quantization of laguna-s-2.1-Q4_K_M.gguf. See here : https://huggingface.co/poolside/Laguna-S-2.1-GGUF

One Gotcha: Snag To Build Laguna llama.cpp Fork On Many Platforms

See Poolside Hugging Face model card on getting the forked repo, and branch you need to build from. Command looks like this as of this writing:

git clone --branch laguna https://github.com/poolsideai/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build -j

You might have seen the error in the build process with speculative.ccp. While it is still not merged (at the time of writing) its a simple fix on line 15 of common/speculative.ccp by importing cmath.

Check if fix below is merged first and skip this step , ask your AI coding assistant to fix with issue in hand , or add the missing import line to the same speculative.cpp file in your repo that is missing.

issue: https://github.com/poolsideai/llama.cpp/pull/3

fyi, my Vulkan version (1.4.341.0+dfsg1-1) the newest version and then I build with:

  rm -rf build;
cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_VULKAN=ON

cmake --build build --config Release -j$(nproc)

Benched / Real Code Generation Tuned Using D-Flash

To evaluate a model like Laguna, traditional single-turn coding prompts (e.g., “Write a bubble sort algorithm”) are irrelevant. The model is built to survive multi-hour, hundreds-of-step loops inside real sandboxes. That is how I will test it.

OpenCode is given all permissions and a full spec for making ReAct AI agent. I also test by generating 20 python class examples in all different themes not related to each other.

D-Flash models it pairs with are great …. for this purpose, your tokens per second will drop though especially if you use with a deep research agent, creative writing, etc. Its not really made for that. You also need to really focus in on the tune here are you’ll create problems

Anyway team, I think I am done playing with the tune on this model on Strix Halo. Now going to build on my AI agents and hone in my AI coding mutating tests and input fuzzying pipeline among other things in response to this tweet from the creator of the book “Clean Code”:

Later Gator.