Frontier quality on one card
Practitioners are compressing open models onto a single consumer graphics card and reporting scores close to what the paid frontier models get.
-
A 3-bit local Qwen3.8-27B scored within 0.3 points of Claude Opus 4.6 on BrainBench. Single source, self-judged. Report
-
An Intel Arc B70 32GB ran Qwen3.8-27B Q6_K at 28 to 32 tokens per second on 200,000-token context. Report
-
Nvidia's Jetson Orin Nano 2: 78 TOPS in 8GB, but no price and nothing shipping until 2027. Nvidia
-
Daniel Han claims GLM-5.3-Flash quantizes to 4-bit at 93% retained accuracy, "essentially a local Opus 4.7." Single source. Han
-
SHIPUnsloth's biggest release yet trims package size 44% and defaults AMD to Vulkan
Version 0.1.807-beta, out September 8, ships over 200 fixes: Vulkan replaces ROCm as AMD's default for a 20% prefill speedup, Windows binaries are now code-signed, and MLX quantized KV caches use up to 74% less memory. Unsloth v0.1.807-beta
-
Offline, fully local coding agents are starting to do real production work, not just pass benchmarks.
Thomson Reuters engineer Rittika Jindal's Oct 27 Latent Space post fixed a real Decimal-rounding bug with Wi-Fi off, using OpenCode and Ollama to run a local Qwen3.5 9B model end to end - a demonstrated result, not a benchmark score. Rittika Jindal, Coding From the Himalayas With an Offline AI Agent