Transformers are inherently succinct★ 74
Hacker News (AI keywords)·2d ago·Paper
This paper studies transformer expressivity through succinctness: how compactly a formalism describes a language. It proves fixed-precision transformers can be exponentially more succinct than LTL and RNNs, and doubly exponentially more succinct than finite automata. The same succinctness makes verification hard, with basic problems such as emptiness and equivalence shown to be EXPSPACE-complete.
