Rethinking the Primitives: Next Generation LLM Architecture

Jiankai Sun, Chuanyang Zheng, others

https://web.stanford.edu/ jksun/blog/llm-architecture.html, 2026

Abstract

A layer-by-layer redesign of the Transformer stack. Starting from the outermost layer — how position is encoded — and working inward through attention mechanisms, linear hybrid architectures, sparse expert routing, and finally normalization. Each work finds the hidden mathematical structure of one component and replaces engineering convention with a principled derivation.

BibTeX

@article{sun2026clawsbench,
  title = {Rethinking the Primitives: Next Generation LLM Architecture},
  author = {Sun, Jiankai and Zheng, Chuanyang and others},
  journal = {https://web.stanford.edu/~jksun/blog/llm-architecture.html},
  year = {2026},
  keywords = {machine learning, LLM, reasoning}
}