Can a Local LLM Redact the Same Document Five Times? 100 Korean Documents, Graded
I planted fake personal data in 100 Korean documents, asked each model to redact every document five times, and graded against the answer key. qwen3.8 27B on one RTX 3090 passed 96 documents in all five tries, close to Claude Sonnet 5.5 (99); with a regex pass first it reached 99. Small models that fit a 1080 Ti passed half or fewer, and qwen3.5 9B masked the form fields while missing names inside sentences.