From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning — part 1/4

Cleaned-text code points [0, 15925). Bibliography omitted here; source.md includes it.

# From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning

###### Abstract

Humans organize knowledge into compact conceptual categories that balance compression with semantic richness. Large Language Models (LLMs) exhibit impressive linguistic abilities, but whether they navigate this same compression-meaning trade-off remains unclear. We apply an Information Bottleneck framework to compare human conceptual structure with embeddings from 40+ LLMs using classic categorization benchmarks ( Rosch, 1973a ; Rosch, 1975 ; McCloskey and Glucksberg, 1978 ) . We find that LLMs broadly agree with human category boundaries, yet fall short on fine-grained semantic distinctions. Unlike humans, who maintain “inefficient” representations that preserve contextual nuance, LLMs aggressively compress, achieving more optimal information-theoretic compression at the cost of semantic richness. Surprisingly, encoder models outperform much larger decoder models in agreement with human categories, suggesting that understanding and generation rely on distinct representational mechanisms. Training-dynamics analysis reveals a two-phase trajectory: rapid initial concept formation followed by architectural reorganization, during which semantic processing migrates from deep to mid-network layers as the model discovers increasingly efficient, sparser encodings. These divergent strategies, where LLMs optimize for compression and humans for adaptive utility, reveal fundamental differences between artificial and natural intelligence. This highlights the need for models that preserve the conceptual “inefficiencies” essential for human-like understanding.

## 1 The Enigma of Meaning in Large Language Models

“The categories defined by constructions in human languages may vary from one language to the next, but they are mapped onto a common conceptual space , which represents a common cognitive heritage, indeed the geography of the human mind.” – Croft (2001) p. 139

Humans excel at organizing knowledge into concepts which are compact categories that achieve remarkable compression while preserving essential meaning ( Murphy, 2004 ) . A single word like “bird” compresses information about thousands of species, yet maintains critical semantic properties (can fly, has feathers, lays eggs). This hierarchical organization (robin → \rightarrow bird → \rightarrow animal; Rosch et al. 1976 ) represents a fundamental cognitive achievement: balancing efficiency with semantic fidelity.

Large Language Models (LLMs) demonstrate striking linguistic capabilities that suggest semantic understanding ( Singh et al., 2024 ; Li et al., 2025 ) . Yet, a critical question remains unanswered: Do LLMs navigate the compression-meaning trade-off similarly to humans, or do they employ fundamentally different representational strategies? This question matters because true understanding, which goes beyond surface-level mimicry, requires representations that balance statistical efficiency with semantic richness ( Tversky, 1977 ; Rosch, 1973c ) .

To address this question, we apply Rate-Distortion Theory ( Shannon, 1948 ) and Information Bottleneck principles ( Tishby et al., 2000 ) to systematically compare LLM and human conceptual structures. We digitize and release seminal cognitive psychology datasets ( Rosch, 1973c ; Rosch, 1975 ; McCloskey and Glucksberg, 1978 ) , which are foundational studies that shaped our understanding of human categorization but were previously unavailable in a machine-readable form. These benchmarks, comprising 1,049 items in 34 categories with membership and typicality ratings, offer unprecedented empirical grounding to evaluate whether LLMs truly understand concepts as humans. It also offers much better quality data than the current crowdsourcing paradigm by relying on experts to design and execute the human experiments.

Analyzing embeddings from 40+ diverse LLMs against these benchmarks, we uncover a fundamental divergence: LLMs and humans employ different strategies when balancing compression with meaning. While LLMs achieve broad categorical agreement with human judgment, they optimize for aggressive statistical compression at the expense of semantic nuance. Humans maintain “inefficient” representations that preserve rich, multidimensional structure essential for flexible reasoning.

This divergence manifests across three dimensions. First, LLMs capture categorical boundaries but miss fine-grained semantic distinctions like item typicality, central to human understanding. Second, our information-theoretic analysis reveals LLMs achieve mathematically “optimal” compression-distortion trade-offs, while human categories appear suboptimal. Third, encoder models surprisingly outperform decoder models in human alignment despite smaller scales, indicating that understanding and generation may require fundamentally different representational strategies.

Through analysis of OLMo-7B across 57 training checkpoints, we further uncover how these strategies emerge during learning: conceptual structure develops via rapid initial formation followed by architectural reorganization, with semantic processing migrating from deep to mid-network layers as models discover increasingly efficient encodings.

These findings challenge the assumption that statistical optimality equals understanding. The apparent “inefficiency” of human concepts may reflect optimization for adaptive flexibility. Our framework and newly-digitized benchmarks provide essential tools for monitoring this critical balance, guiding development toward AI systems that achieve not just compression, but comprehension.

## 2 Research Questions and Scope

Previous work has explored conceptual representations of LLM through multiple lenses: relational knowledge ( Shani et al., 2023 ; Misra et al., 2021 ) , interpretable concept extraction ( Hoang-Xuan et al., 2024 ; Maeda et al., 2024 ) , sparse activation patterns ( Li et al., 2025 ) , similarity preserving architecture Doumbouya et al. (2026) , and embedded geometry, including hierarchical structures ( Park et al., 2025 ) . While insightful, these studies often lack a deep, quantitative comparison of the compression-meaning trade-off using information theory against rich human cognitive benchmarks.

Separately, cognitive science has applied information theory to human concept learning ( Imel and Zaslavsky, 2024 ; Tucker et al., 2025 ; Zaslavsky et al., 2018 ; Sorscher et al., 2022 ) . For example, Zaslavsky et al. (2018) developed an Information Bottleneck framework for color naming efficiency, later extended to animal taxonomies ( Zaslavsky et al., 2020 ) . Yet these cognitive studies typically proceed without connecting to modern LLMs, and tend to focus on a specific domain. One notable example is Wu et al. (2025) , which examined abstraction transfer in humans and LLMs using a behavioral and cognitive modeling level. Our work is different in the sense that it analyzes how information is preserved or distorted inside LLM embedding spaces under controlled clustering transformations.

These two streams, LLM conceptual analysis and cognitive information theory, rarely intersect. We bridge this gap through rigorous comparison of how LLMs and humans navigate the compression-meaning trade-off, grounding our analysis in established cognitive benchmarks. This leads to three research questions:

[RQ1] To what extent do LLM-emergent concepts align with human-defined categories?

[RQ2] Do LLMs exhibit human-like internal structure, particularly item typicality?

[RQ3] How do humans and LLMs differ when balancing compression with semantic fidelity?

Our framework approaches each RQ through a unified lens. [RQ1] examines the categorical alignment, or how information is compressed into discrete groups. [RQ2] probes internal structure, which means how semantic meaning is preserved within categories. [RQ3] employs our ℒ \mathcal{L} objective to evaluate the integrated trade-off. This progression from compression to preservation to their balance mirrors the fundamental challenge both systems face: creating representations that are simultaneously efficient and meaningful.

Figure 1 overviews the data generation and analyses. Human data was collected by asking whether an item i (e.g., chair) is a good example of the category C (furniture). These ratings are aggregated into ranked similarity profiles for each category. Models generate analogous scores using their embeddings. We then compute three metrics: [RQ1] Mutual Information to assess category recoverability, [RQ2] Spearman correlation to measure alignment with human typicality structure, and [RQ3] a rate-distortion objective capturing the trade-off between representation complexity and meaning preservation.

## 3 Benchmarking Against Human Cognition

Investigating LLM-human conceptual alignment requires robust benchmarks and diverse models. This section details both components.

### 3.1 Human Baselines: Empirical Data from Seminal Cognitive Science

We draw on three foundational studies that shaped our understanding of human categorization. Unlike many noisy modern crowdsourced datasets, these classic benchmarks were carefully curated by experts, capturing deep cognitive patterns. We focus on three influential works:

Rosch (1973): This foundational work ( Rosch, 1973a ) explored semantic categories as part of the research program leading to prototype theory ( Rosch, 1973c ) 1 1 1 Prototype theory is only one account of how humans form concepts; exemplar theory offers an alternative based on stored instances. We do not adjudicate between theories, but use this framework because it provides structured data suitable for modeling. Our computational analysis is compatible with alternative accounts. . The theory posits that categories organize around “prototypical” members rather than strict, equally shared features. The dataset includes 48 items in eight common semantic categories (e.g., furniture, bird), with prototypicality rankings (e.g., ‘robin’ as typical bird, ‘bat’ as atypical).

Rosch (1975): Building on prototype theory, Rosch (1975) further detailed how semantic categories are cognitively represented. This work provides typicality ratings for a larger set of 552 items across ten categories (e.g., ‘orange‘ as a prototypical fruit, ‘squash‘ as less so).

McCloskey & Glucksberg (1978): Investigated the ”fuzzy” boundaries of natural categories, showing membership is graded rather than absolute ( McCloskey and Glucksberg, 1978 ) . Covers 449 items in 18 categories with both typicality scores and membership certainty ratings (e.g., ‘dress’ is typical clothing, ‘bandaid’ less so).

While originating from different researchers, these datasets share rigorous experimental designs and provide data on both category assignments and item typicality. We aggregated data from these studies, creating a unified benchmark of 1,049 items across 34 categories. This data, which we have digitized and made publicly available (Appendix C ), offers a high-quality empirical foundation for evaluating the human-likeness of LLMs. 2 2 2 Appendix I shows that polysemy is rare in our data and cannot account for our findings.

### 3.2 Large Language Models Under Study

We analyze 40+ diverse LLMs spanning multiple architectures and scales (300M to 72B parameters) to understand how conceptual representation varies across model design choices. We note that our analysis requires access to LLMs’ embeddings, rather than just output. Thus, we are unable to use any closed-source frontier models such as GPT-5 and Claude.

Model Selection. Our study encompasses three architectural paradigms. Encoder models include the BERT family ( Devlin et al., 2019 ; He et al., 2021 ; Zhuang et al., 2021 ) and CLIP ViT text encoders ( Radford et al., 2021 ) . Decoder models form the majority of our analysis: the Llama family (1B-70B; Touvron et al., 2023a ; Touvron et al., 2023b ; Grattafiori et al., 2024 ), Gemma variants (2B-27B; Team et al., 2024 ; Team et al., 2025 ), Qwen models (0.5B-72B; Bai et al., 2023 ; Yang et al., 2024 ), Phi series ( Javaheripi et al., 2023 ; Abdin et al., 2024 ; Abouelenin et al., 2025 ) , Mistral-7B ( Jiang et al., 2023 ) , GPT-2 ( Radford et al., 2019 ) , and OLMo-7B ( olmo2024 ) . We also include classic static embeddings Word2Vec ( Mikolov et al., 2013a ; Mikolov et al., 2013b ) and GloVe ( Pennington et al., 2014 ) as baselines.

This diverse selection enables us to disentangle effects of architecture (encoder vs. decoder), scale (300M to 72B), and training objectives (understanding vs. generation). Note that encoder-only models are less represented (and are smaller) since recent LLM development has prioritized decoder-only architectures. Complete model specifications appear in Appendix D .

Embedding Extraction. We extract representations at two levels to capture different aspects of conceptual knowledge: (1) static embeddings from input layers (E matrix), capturing context-free lexical knowledge directly comparable to isolated words in human categorization experiments; and (2) contextual embeddings from hidden layers using controlled prompts, revealing how context shapes conceptual structure across network depth.

This dual approach allows us to trace how concepts emerge from basic lexical knowledge to contextualized understanding. Critically, our results prove robust to prompt templates and pooling strategies (Appendix E ). Moreover, despite substantial vocabulary overlap between model families, neither token count nor tokenization patterns correlate with our results (Appendix J ).

## 4 A Framework for Comparing Compression and Meaning

To quantitatively compare how LLMs and humans navigate the fundamental tension between compact representation and semantic richness, we develop a framework that captures both aspects of conceptual organization. Our approach adapts Rate-Distortion Theory ( Shannon, 1948 ) and the Information Bottleneck principle ( Tishby et al., 2000 ) to measure the quality of conceptual systems.

### 4.1 Theoretical Foundations

Human concepts achieve remarkable efficiency: the word “bird” compresses knowledge about thousands of species into a single category, yet preserves critical semantic information (can fly, has feathers, lays eggs). This reflects a fundamental trade-off that any conceptual system must navigate:

• Compression: Grouping diverse items into manageable categories (fewer bits needed)

• Meaning Preservation: Maintaining semantic coherence within groups

Rate-Distortion Theory (RDT; Shannon, 1948 ) formalizes this trade-off for lossy compression. Given data X X and compressed representation X ^ \hat{X} , RDT seeks encodings that minimize: R + λ ​ D = I ⁡ ( X , X ^ ) + λ ​ 𝔼 ​ [ d ⁡ ( X , X ^ ) ] R+\lambda D=I(X;\hat{X})+\lambda\mathbb{E}[d(X,\hat{X})] (1) where R R is the rate (bits required), D D is distortion (information lost), and λ \lambda controls their trade-off.

Information Bottleneck (IB; Tishby et al., 2000 ) extends this by compressing X X into Z Z while preserving information about relevant variable Y Y : min ⁡ I ⁡ ( X , Z ) − β ​ I ​ ( Z , Y ) \min I(X;Z)-\beta I(Z;Y) (2)

Our adaptation: For conceptual representation, we lack an external relevance variable Y Y . Instead, ”relevance” becomes internal semantic coherence, i.e., how well categories preserve within-group similarity. We thus combine RDT’s geometric distortion with IB’s information-theoretic compression, yielding our framework where clustering C C represents items X X by minimizing both the information needed to specify items (compression) and the semantic spread within clusters (distortion).

### 4.2 The ℒ \mathcal{L} Objective: Quantifying the Trade-off

We formalize how clustering C C represents items X X through an objective that combines information-theoretic compression with geometric coherence: