Krafton's Raon-1 mini on a Korean Document-QA Benchmark: 300 Questions with Known Answers, Run on an RTX 3090 and Two GTX 1080 Tis

Every number here comes from runs on my own machines. The benchmark, retrieval and judges are the same as in Local RAG Scored on 300 Questions. This is the preview release, measured within two days of launch, so the final release may score differently. (Korean version: 크래프톤 Raon-1 mini 채점.)

TL;DR

• Accuracy (main judge / second judge): thinking on 74.2% / 76.9%, model-card sampling 74.9% / 76.6%, thinking off 72.6% / 74.9%.
• With thinking off, I found no meaningful difference between Raon (~4B active) and the dense qwen3.8 27B (thinking off, 71.2% / 72.2%). It clearly beat LG's Korean EXAONE 3.5 7.8B (63.2%) and Alibaba's qwen3.5 9B (65.6%).
• With thinking on, qwen3.8 27B pulls ahead (77.3% / 80.6%). Both judges had qwen3.8 winning more of the discordant questions, but the difference is significant only under the second judge. The thinking gain was also small for Raon: +1.7 points vs +6.0 for qwen3.8 (main judge; Raon's gain is not significant).
• With thinking on at temperature 0, 16 of 320 requests spent the whole 8,192-token budget reasoning, 14 of them returning an empty answer. One run with the model-card sampling had 11.
• It doesn't run in Ollama yet. Built from Krafton's llama.cpp branch, it takes 2.0 s per question on one RTX 3090 (thinking off) and 16 s on two GTX 1080 Tis from 2017.
• On 20 questions whose answers are not in the documents, every configuration answered "not found" 20 out of 20 times.

What Raon-1 mini is

An open-weight model Krafton released on October 21, 2026 (Apache 2.0).

• MoE with about 30B total parameters (28.8B per the model card), about 4B (4.3B) active per token
• Hybrid attention: 18 layers of Krafton's linear attention (Raon Delta Attention) plus 7 full attention layers
• Up to 262K tokens of context; positioned for Korean and English chat, reasoning and agents. Krafton's own numbers include IFBench 84.2, AIME 2026 93.3 and SWE-Verified 70.8 (model card)

Krafton's numbers cover math, coding and instruction following. I wanted to know something narrower: give it Korean documents and a question; does it give the right answer?

Setup

Same as the previous post.

Benchmark	Allganize Korean RAG evaluation set, 300 questions (finance, public sector, medical, legal, commerce); 299 scorable (one has no gold page) + 20 unanswerable questions I wrote
Retrieval	Hybrid (kiwi-morpheme BM25 + bge-m3) top 5 pages within each domain. Every model gets the same 5 pages
Instruction	"Answer only from the excerpts; if it's not there, say you can't find it"
Raon	Q4_K_M (19.7 GB), Krafton's llama.cpp branch (commit fb89566), context 16,384
Three conditions	① thinking on, temperature 0 ② thinking on, model-card sampling (temp 1.0, top_p 0.95, top_k 20, presence penalty 1.5) ③ thinking off, temperature 0
Output cap	8,192 tokens with thinking, 1,024 without
Judges	Main: qwen3.6 27B (88.5% agreement with the O/X labels published with the dataset). Second: Claude Opus 5.5

Where it lands

Model	Main judge	Second judge	Gold page retrieved	Paragraph	Text	Table	Image
qwen3.8 27B, thinking	77.3%	80.6%	90.1%	87%	84%	80%	44%
Claude Opus 5.5	76.6%	80.3%	89.7%	85%	84%	80%	46%
Raon-1 mini, model-card sampling	74.9%	76.6%	87.7%	89%	80%	74%	35%
Raon-1 mini, thinking	74.2%	76.9%	86.0%	87%	76%	76%	39%
Claude Sonnet 5.5	73.9%	79.3%	86.0%	83%	78%	82%	40%
Raon-1 mini, thinking off	72.6%	74.9%	84.8%	88%	71%	68%	37%
Claude Haiku 5.5	72.2%	78.3%	85.6%	83%	73%	76%	40%
qwen3.8 27B, thinking off	71.2%	72.2%	84.0%	85%	76%	68%	35%
gemma4 31B	70.2%	72.9%	81.5%	84%	78%	66%	33%
gpt-oss 20B	69.6%	71.6%	79.4%	82%	80%	70%	30%
gemma4 12B	66.6%	68.2%	77.4%	84%	62%	58%	32%
qwen3.5 9B	65.6%	66.9%	76.5%	84%	64%	52%	32%
gemma4 26B MoE	63.5%	68.9%	74.5%	78%	67%	56%	30%
EXAONE 3.5 7.8B	63.2%	61.5%	72.8%	81%	60%	50%	32%

The three Raon rows are from the RTX 3090. Sorted by the main judge; per-type columns use the main judge. Question counts: paragraph 147, text 45, table 50, image 57; "gold page retrieved" covers the 243 questions where the gold page made the top 5. All other rows are unchanged from the previous post.

Paired comparisons (exact McNemar, 299 questions; "only A / only B" = questions only that side got right)

Comparison	Main judge (p)	Second judge (p)
Raon thinking vs qwen3.8 27B thinking	7 / 16 (0.09)	3 / 14 (0.01)
Raon thinking vs Claude Opus 5.5	8 / 15 (0.21)	5 / 15 (0.04)
Raon thinking off vs qwen3.8 27B thinking off	14 / 10 (0.54)	19 / 11 (0.20)
Raon thinking off vs qwen3.5 9B	28 / 7 (below 0.001)	34 / 10 (below 0.001)
Raon thinking off vs EXAONE 3.5 7.8B	38 / 10 (below 0.001)	49 / 9 (below 0.001)
Raon thinking on vs off	14 / 9 (0.41)	15 / 9 (0.31)
Raon model-card sampling vs thinking at temp 0	7 / 5 (0.77)	7 / 8 (1.0)

How to read it:

• Thinking off: no difference detected between Raon and qwen3.8 27B (which is not proof that they are equal). That's a good result for a model with ~4B active parameters.
• Thinking on: qwen3.8 27B is ahead. Both judges agree on the direction, but under the main judge it's within chance (p = 0.09), so I won't call Raon clearly worse.
• Clearly ahead of EXAONE 3.5 7.8B. Size, architecture and release date all differ, so this experiment can't say why.
• Thinking barely helped Raon (on vs off 14 / 9, not significant). On this benchmark, running it with thinking off at about 2.3x the speed is a reasonable choice.

Questions where it only thought

With thinking on, the model writes a long reasoning trace before answering. With an 8,192-token output cap, 16 of 320 requests at temperature 0 used the whole budget reasoning, and 14 of those came back with an empty answer.

Temperature 0 always picks the most likely next token, which makes repetition loops more likely. So I reran with the sampling the model card recommends (temperature 1.0, top_p 0.95, top_k 20, presence penalty 1.5).

Condition	Hit 8,192 tokens	Empty answers	Accuracy (main / second)
Thinking on, temperature 0	16	14	74.2% / 76.9%
Thinking on, model-card sampling	11	11	74.9% / 76.6%

Each setting ran once, and only 6 of the looping questions overlap between the two runs, so I can't claim the sampling reduced loops. It did not remove them, and accuracy was the same. If you run it with thinking, use the model-card sampling and handle responses that hit the output cap. qwen3.8 27B hit no cap on the same prompts.

It doesn't run in Ollama yet: building it

As of October 22, Raon-1 mini does not run in Ollama or in mainline llama.cpp. Upstream support for the new attention is "pending", so you build Krafton's branch.

git clone --depth 1 -b raon-1-mini-preview https://github.com/krafton-ai/llama.cpp raon-llama.cpp
cd raon-llama.cpp
# 86 for RTX 3090 (Ampere), 61 for GTX 1080 Ti (Pascal)
CC=gcc-12 CXX=g++-12 cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 \
  -DCMAKE_CUDA_HOST_COMPILER=g++-12 -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-server llama-cli

• CUDA 12.0 with gcc 13 (Ubuntu 24.04's default): CUDA 12.0 officially supports host compilers up to gcc 12, so I built with gcc 12 as above.
• The weights are at KRAFTON/Raon-1-mini-preview-gguf on Hugging Face. The repo was gated on launch day and opened the next day (October 22).

./build/bin/llama-server -m raon-1-mini-preview-Q4_K_M.gguf -ngl 99 -c 16384 --jinja --port 8090

Thinking is toggled per request. Add "chat_template_kwargs": {"enable_thinking": true} to an OpenAI-style request and the reasoning comes back in reasoning_content, the answer in content. It reasons in English even when asked in Korean. The model card does not recommend turning thinking off.

Speed and memory: one RTX 3090 vs two GTX 1080 Tis

	One RTX 3090	Two GTX 1080 Tis
VRAM	about 19.9 GB (all on GPU)	10.9 GB + 10.3 GB (all on GPU)
Prompt processing	about 2,800 tok/s	about 330 tok/s
Generation	about 96 tok/s	about 10 tok/s
Per question, thinking off	2.0 s	16 s
Per question, thinking on	4.8 s	39 s (28 questions only)

Medians over the 320-question runs (prompts were about 2,500 tokens). I didn't run all 320 with thinking on the 1080 Tis because one looping question takes about 13 minutes at 10 tok/s to fill 8,192 tokens (measured 784–812 s).

• It runs on Pascal (2017). Krafton's new attention worked without errors on the old cards, and thinking-off accuracy over 320 questions was 74.2% / 75.9%, close to the 3090's 72.6% / 74.9%. Same model and settings, different GPU and build: 9 verdicts flipped (7 only right on the 1080 Ti side, 2 only on the 3090). qwen3.8 flipped 11 in the previous post.
• It is slow there. On the same two 1080 Tis, a similar-sized MoE, qwen3.6 35B-A3B, generated about 20 tok/s. My guess is that the new attention kernels aren't tuned for old cards yet; I did not verify that.

Pitfalls

1. No Ollama: ollama pull won't get it. The model card lists Transformers, vLLM and MLX as well, but for a lightweight GGUF setup the branch build above is the route today.
2. Thinking loops at temperature 0: see above. Use the model-card sampling.
3. Serving to the LAN from WSL: with mirrored networking, the Windows Hyper-V firewall blocks inbound connections. You need an admin to add a rule for the port; I used a port that was already open.
4. Windows holding GPU memory: on the same 3090, Windows-side apps held 9.5 GB one day, so the 19.7 GB Q4_K_M didn't fit; the next day it was 0.7 GB and everything fit. Check nvidia-smi first.

Limitations

• Preview release. Krafton says a full release and technical report will follow.
• One quantization (Q4_K_M). BF16 or higher-bit quants may differ.
• One run per condition. In the previous post the same model flipped about 11 verdicts across hardware and versions, so don't rank 1–2 point gaps.
• LLM judges. The second judge (Opus) has been more lenient toward Claude answers, so compare local models on both judges.
• One benchmark (64 Korean public documents).

Reproduce it

Per-question verdicts and scripts are in the repro bundle (README in Korean, scripts commented in English). It now includes the Raon verdicts and gen_openai.py, which generates through llama-server's OpenAI-compatible API (SAMPLING=card for the model-card settings).

Related

• Local RAG Scored on 300 Questions: the benchmark and all the other models
• 35B MoE on two 1080 Tis: old-GPU speed for a similar-sized MoE