Startup Subquadratic Claims It Broke Through a Core LLM Mathematical Bottleneck
MIT Tech Review AI39 days agoBusinessMiami-based AI startup Subquadratic emerged from stealth last month claiming to have solved a fundamental mathematical bottleneck that has held back large language models for nearly a decade. Initial reactions from the research community were skeptical, with technical details appearing sparse relative to the scale of the claim. The company has since begun releasing more substantive supporting evidence, and MIT Technology Review investigates whether it can back up its assertions.
How LLMs Actually Work
Hacker News (AI keywords)54 days agoTutorialThe article explains how modern LLMs convert text into token IDs, embeddings, and position-aware vectors before passing them through stacked transformer blocks. It covers attention, multi-head attention, KV cache, GQA, feed-forward networks, MoE, residual streams, normalization, and decoding. Its goal is educational: helping readers understand the common architecture behind many current model families and read model cards or papers more confidently.
你也能設計出最先進的 Transformer 位置編碼：從直覺到 RoPE 的數學推導★ 75
Hugging Face Blog610 days agoTutorialThis educational article from Hugging Face aims to guide readers — in the most intuitive, step-by-step way — to "reinvent" RoPE (Rotary Position Embedding)…