Offline CPU Voice Loop for Ollama and LM Studio Agents
r/LocalLLaMA top day47 days agoNew ToolA r/LocalLLaMA post introduces an offline voice loop for talking to local models through Ollama, LM Studio, or vLLM. The stack uses Silero VAD, Parakeet TDT 0.6B v3 STT, and Supertonic TTS 3, all running on CPU so GPU memory stays available for the LLM. The author reports measured CPU-only benchmarks, agent integrations, cross-platform installers, and an MIT-licensed GitHub release.
AMD Highlights Unified Memory Architecture for Future AI Systems
r/LocalLLaMA top day47 days agoHardwareA Reddit post in r/LocalLLaMA links to coverage of AMD discussing unified memory architecture and its role in future product roadmaps. The post says AMD believes UMA could help shape next-generation architectures and notes Ryzen AI MAX 400 series systems, also referred to by the community as Gorgon Halo. It frames the topic as part of an ongoing LocalLLaMA discussion about whether unified-memory x86 systems could matter for local AI workloads.
qwen3.6-27b Users Report Repeated Tool Call Loops
r/LocalLLaMA top day47 days agoIncidentA Reddit user on r/LocalLLaMA says qwen3.6-27b can fall into repeated tool-call loops during use. They report spending two days adjusting parameters such as temperature and top-k without resolving the issue. The post is a troubleshooting question rather than a confirmed bug report, asking whether other local model users have seen similar behavior.
Benchmarking Google Eloquent Exposes Major On-Device Dictation Reliability Issues
r/LocalLLaMA top day47 days agoBenchmarkA LocalLLaMA user tried to benchmark Google’s new fully local dictation app, Eloquent, against open ASR models such as Qwen3-ASR and NVIDIA Parakeet V3. The tester reported that roughly half of dictations returned only fragments, even during manual use. When Eloquent produced complete transcripts, its word error rate was competitive, but the missing-output behavior made the app unreliable for evaluation and practical use.
DiffusionGemma: Google Launches High-Speed Open-Weight Gemma Diffusion Model★ 76
Simon Willison's Weblog47 days agoReleaseSimon Willison highlights Google’s new DiffusionGemma, an Apache 2 licensed open-weight Gemma model. He connects it to last year’s brief Gemini Diffusion preview, which he measured at 857 tokens per second. NVIDIA is currently hosting the model for free on its NIM cloud API, where Willison generated 2,409 tokens in 4.4 seconds, implying at least 500 tokens per second.
Google DeepMind Releases DiffusionGemma: Open Source Model with 4x Local AI Execution Speed Improvement
Ars Technica AI47 days agoReleaseGoogle DeepMind has released DiffusionGemma, an open-source model that brings diffusion-based generation to text tasks. Unlike autoregressive LLMs that generate one token at a time, diffusion models can produce outputs in parallel, dramatically cutting latency. The result is reportedly a 4x speed improvement for local AI inference, making on-device deployment significantly more practical.
LocalLLaMA User Weighs QAT Gemma 31B GGUF Quants for RTX 3060
r/LocalLLaMA top day47 days agoCommentaryA Reddit user with an RTX 3060 12GB and 32GB DDR3 RAM is evaluating new QAT-based Gemma 31B GGUF quantizations. They currently run an older Unsloth Gemma 31B IQ3_XXS build at long context, with some tensor and mmproj offloading to CPU. The post asks which Q2-Q3 quant to choose, whether QAT changes quality expectations, and whether MTP would help or hurt under tight VRAM limits.
Robotaxi Safety Must Be Built In, Not Added Later
NVIDIA Blog47 days agoCommentaryNVIDIA argues that robotaxi safety requires more than perception and driving decisions. The post presents Halos OS as a production safety foundation covering a certifiable OS, standardized interfaces, AI guardrails and large-scale validation. It also highlights global robotaxi collaborations using DRIVE Hyperion and the broader Halos stack across training, simulation and in-vehicle inference.
πfs: the data-free filesystem that “stores” data in π
Hacker News (AI keywords)47 days agoNew Toolπfs is an open-source FUSE-style filesystem built around a deliberately absurd idea: data does not need to be stored if it can be located in pi. It records metadata such as file names and positions in pi, then reconstructs content from those locations. The project is more technical humor and conceptual demonstration than practical storage or AI tooling.
Seeking the Best Open-Source Coding AI for an RTX 5070 PC
r/LocalLLaMA top day47 days agoOpinionA Reddit user on r/LocalLLaMA is looking for the most powerful open-source AI coding model that can run on their Windows 11 desktop. Their system includes an AMD Ryzen 7 7700 CPU, RTX 5070 GPU, and 32GB of DDR5 RAM. The intended use cases are writing, coding, and debugging, but the post itself does not include benchmark results, candidate models, or community recommendations.
llama.cpp Merges MTP Optimization Removing Padding and Extra D2D Copies
r/LocalLLaMA top day47 days agoReleasellama.cpp merged PR #24086, which changes ggml_gated_delta_net so MTP passes snapshot count K as an operation parameter instead of deriving it from tensor shape. The change removes a padding workaround and copies emitted snapshots into the recurrent cache with a single strided ggml_cpy. Benchmarks on DGX Spark with Qwen3.6-35B-A3B-UD-Q4_K_M.gguf showed about a 4% throughput gain, with wall time falling from 21.71s to 20.91s.
DiffusionGemma: 4x faster text generation★ 74
Google DeepMind Blog47 days agoReleaseGoogle’s DiffusionGemma is an Apache 2.0 experimental open model using text diffusion instead of standard autoregressive decoding. The 26B MoE model activates 3.8B parameters during inference and is designed for low-latency local workflows. Google claims up to 4x faster generation on dedicated GPUs, while noting that output quality is below standard Gemma 4 and production-quality use cases should still prefer Gemma 4.
Lemonade v10.7 Adds Omni Models, Benchmarks, and Cross-Vendor GPU Support
r/LocalLLaMA top day47 days agoReleaseLemonade v10.7 marks a project-level shift toward working-group-driven development, with 19 contributors involved in the release. The update improves LMX-Omni virtual models for Open WebUI and OpenAI-compatible multimedia clients, introduces the `lemonade bench` CLI, and expands backend support. CUDA, Vulkan, llama.cpp, stable-diffusion.cpp, FastFlowLM, and vLLM are part of the broader push toward cross-vendor local AI performance.
NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
NVIDIA Blog47 days agoReleaseGoogle DeepMind released DiffusionGemma, an experimental open model built for fast text generation. NVIDIA says it optimized the model for GeForce RTX GPUs, RTX PRO platforms, and DGX Spark systems. Instead of generating text one word at a time, DiffusionGemma produces multiple words in parallel to reduce latency for single-user workloads.
DiffusionGemma: The Developer Guide — Google Developers Blog
r/LocalLLaMA top day47 days agoTutorialGoogle has released a comprehensive developer guide for DiffusionGemma, a text-generation model that uses masked diffusion rather than autoregressive next-token prediction. Unlike standard Gemma models, DiffusionGemma iteratively denoises a fully masked sequence to produce output, enabling a fundamentally different generation paradigm. The guide targets developers looking to integrate or experiment with diffusion-based LLMs using Google's tooling.
DiffusionGemma: 4x Faster Text Generation★ 76
Hacker News (AI keywords)47 days agoReleaseGoogle released DiffusionGemma, a 26B MoE experimental open model using text diffusion instead of token-by-token autoregressive decoding. It can generate blocks of text in parallel, reaching up to 4x faster output on dedicated GPUs. The model targets local, speed-sensitive workflows, but Google says its output quality is below standard Gemma 4 and recommends Gemma 4 for quality-critical production use.
SenseNova U1 Adds an Infographic-Specific Fine-Tune
r/LocalLLaMA top day47 days agoReleaseA Reddit post highlights a new infographic-specific fine-tune for SenseNova U1-8B-MoT, trained with an extended multi-task phase for structured visual output. The reported benchmarks show large gains in IGenBench infographic accuracy and chart understanding, with smaller improvement in text rendering. Aesthetic score appears roughly unchanged, suggesting the update mainly improves information structure and visual reasoning rather than overall visual polish.
Apache Burr: Open-Source State Machine Framework for Building Reliable AI Agents
Hacker News (AI keywords)48 days agoNew ToolApache Burr provides a state-machine-based architecture for building reliable AI agents, making complex multi-step LLM workflows predictable and testable. It includes built-in tracing, observability, and a local visualization UI, allowing developers to replay and debug agent execution step by step. Model-agnostic and integrable with LangChain, LlamaIndex, and major LLM providers, it also supports state persistence and human-in-the-loop workflows for production use.
Bonsai LM 1-bit and 1.58-bit Benchmarks on Jetson Orin Nano Super
r/LocalLLaMA top day48 days agoBenchmarkA LocalLLaMA post benchmarks five Bonsai LM models, from 1.7B to about 8B parameters, on a $250 Jetson Orin Nano Super 8GB using llama.cpp CUDA. The tests compare 7W, 15W, 25W, and MAXN modes across latency, throughput, energy per token, and thermals. The main takeaway is that 25W is usually the best efficiency/performance point for models up to 4B, while Bonsai-8B may favor 15W for lower power.
MooreThreads Releases MusaCoder-27B Code LLM on Hugging Face
r/LocalLLaMA top day48 days agoReleaseMooreThreads, a Chinese GPU semiconductor company best known for its MUSA compute platform, has released MusaCoder-27B on Hugging Face alongside a technical paper on arXiv. The 27B-parameter model is positioned as a code-generation LLM, extending MooreThreads' ambitions beyond hardware into the AI model layer. Its public availability on Hugging Face signals an open-weights approach, making it accessible to local-inference practitioners and researchers evaluating alternatives to Western-origin coding models.
OpenLumara Creator Challenges Reddit to Hack Its Public Agent Instance
r/LocalLLaMA top day48 days agoIncidentThe creator of OpenLumara posted a public challenge asking r/LocalLLaMA users to try breaking into a Discord-hosted instance of the local-model agent. They claimed common prompt-engineering attacks would not work because modules and sandboxes were heavily locked down. The post later listed several successful findings, including missing path traversal protection, an authorization-check bypass, and another undisclosed exploit pending a fix.
Reddit Debate: Apple and Microsoft Push Local-First AI
r/LocalLLaMA top day48 days agoOpinionA Reddit user claims Apple and Microsoft have both made strong moves toward local-first AI, pointing to Apple Core AI materials and Microsoft Surface Laptop Ultra announcements. The post argues that Apple’s emphasis on local, private, no-cost AI and Microsoft’s Surface/Nvidia direction could reshape expectations for consumer hardware. However, it is an opinion-driven market prediction, not a confirmed financial or technical analysis.
Qwen3.6-MTP-27B on Tesla V100: llama.cpp Throughput Tuning Question
r/LocalLLaMA top day48 days agoBenchmarkA Reddit user is running Qwen3.6-MTP-27B-MTP in Q4_K_M GGUF format with llama.cpp server on a 32GB Tesla V100. They report one peak of 55 tokens per second, but typical throughput is closer to 44-48 TPS. The post asks whether flags such as parallelism, speculative MTP draft settings, KV cache quantization, flash attention, and a 262K context window are limiting performance without improving output quality.
How Useful Is qwopus Compared With Qwen3.6 27B for Coding?
r/LocalLLaMA top day48 days agoOpinionA Reddit user on r/LocalLLaMA asks for practical comparisons between qwopus and Qwen3.6 27B, specifically for coding work. They note conflicting community opinions, with some users calling qwopus worse and others saying it is much better. In their own simple tests, they did not notice clear differences and want feedback from people using these models for agentic coding.
Charting Local LLM Releases: 2025 Was the Peak, Not 2026
r/LocalLLaMA top day48 days agoCommentaryA r/LocalLLaMA community member shared visualizations tracking the volume of local LLM releases over time. Contrary to the perception that 2026 has been an unusually prolific year, the data indicates the actual release peak occurred in 2025. The poster attributes the misperception to the outsized quality improvements in 2026 making it feel more eventful than it quantitatively was.
Gemma 4 12B Unified Audio Loses Speech Attention with Large System Prompts
r/LocalLLaMA top day48 days agoCommentaryA developer building a single-pass voice assistant with Gemma 4 12B unified (encoder-free audio/vision/text model) finds that audio attention collapses once the system prompt grows to ~21k tokens. The model then ignores or hallucinates instead of responding to the spoken input. The issue reproduces identically on vLLM, llama.cpp, and LiteRT-LM, pointing to an architectural attention-saturation limit rather than a stack-specific bug.
Without Open Source LLMs, US AI Companies Could Have Monopolized the Technology
r/LocalLLaMA top day48 days agoOpinionThis r/LocalLLaMA post argues that open-source LLMs are an ethical duty because AI has broad social impact. The author worries that without open models, US AI companies could have monopolized access and potentially limited availability to US firms. They also frame China’s release of powerful open-source LLMs as a contribution to humanity, despite political disagreements.
Anthropic Is Accused of Nerfing Fable for Other LLM Development
r/LocalLLaMA top day48 days agoCommentaryA r/LocalLLaMA post claims Anthropic may be intentionally limiting Fable when users ask it to help build other LLMs. The source is a short Reddit post with screenshot context, not a formal benchmark or verified disclosure. Discussion centers on trust in hosted closed models, unclear safety boundaries, and why local or open-weight LLMs may be necessary for serious AI development work.
Unsloth releases GGUF version of Cohere North-Mini-Code 1.0 (30B A3B MoE) on Hugging Face
r/LocalLLaMA top day48 days agoReleaseUnsloth uploaded a GGUF version of Cohere's North-Mini-Code 1.0 to Hugging Face, making local inference possible for this 30B A3B MoE coding-focused model. The poster links the release to llama.cpp PR #24260, suggesting new architecture support may be required. No benchmarks or test results have been shared yet; this is an early community resource post.
Anthropic Claude Fable 5: Mythos-Class Power with Controversial Terms★ 84
Latent Space48 days agoReleaseAnthropic released Claude Fable 5 as its first broadly available Mythos-class model, alongside restricted Mythos 5 access. Benchmarks and ecosystem reports show strong gains in coding, long-horizon agentic tasks, research, and vision. The controversy centers on 30-day retention for Mythos-class traffic and silent interventions that may reduce effectiveness on frontier LLM development tasks, raising trust, reproducibility, and open AI concerns.