Transformers are inherently succinct

An ICLR 2026 paper proves fixed-precision transformers can describe some formal languages with extreme compactness.

This paper studies transformer expressivity through succinctness: how compactly a formalism describes a language. It proves fixed-precision transformers can be exponentially more succinct than LTL and RNNs, and doubly exponentially more succinct than finite automata. The same succinctness makes verification hard, with basic problems such as emptiness and equivalence shown to be EXPSPACE-complete.

摘要由 AI 整理,以原文為準。