From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning — part 2/4

Cleaned-text code points [15925, 31513). Bibliography omitted here; source.md includes it.

ℒ ( X , C ; β ) = I ⁡ ( X , C ) ⏟ Complexity: bits needed + β ⋅ 1 | X | ​ ∑ c ∈ C ∑ e i ∈ c ‖ e i − e ¯ c ‖ 2 ⏟ Distortion: semantic spread \mathcal{L}(X,C;\beta)=\underbrace{I(X;C)}_{\text{Complexity: bits needed}}+\beta\cdot\underbrace{\frac{1}{|X|}\sum_{c\in C}\sum_{e_{i}\in c}\|e_{i}-\bar{e}_{c}\|^{2}}_{\text{Distortion: semantic spread}} (3)

where β \beta weights the relative importance of compression versus coherence.

#### 4.2.1 The Complexity Term: Measuring Compression

Complexity quantifies how much information the clustering preserves about individual items through mutual information I ⁡ ( X , C ) I(X;C) . Intuitively, if knowing an item’s cluster tells us little about which specific item it is, compression is high (low complexity).

Given | X | |X| items partitioned into clusters of sizes { | C c | } \{|C_{c}|\} : Complexity ​ ( X , C ) = I ⁡ ( X , C ) = log 2 ⁡ | X | − 1 | X | ​ ∑ c ∈ C | C c | ​ log 2 ​ | C c | \text{Complexity}(X,C)=I(X;C)=\log_{2}|X|-\frac{1}{|X|}\sum_{c\in C}|C_{c}|\log_{2}|C_{c}| (4)

This equals the reduction in uncertainty about item identity when told its cluster. Uniform clusters minimize complexity (one | X | |X| cluster; maximum compression), while singleton clusters maximize it (no compression).

#### 4.2.2 The Distortion Term: Measuring Semantic Coherence

Distortion captures how well clusters preserve semantic relationships and meanings by measuring the average squared distance between items and their cluster centroids in embedding space (spread): Distortion ​ ( X , C ) = 1 | X | ​ ∑ c ∈ C | C c | ⋅ σ c 2 \text{Distortion}(X,C)=\frac{1}{|X|}\sum_{c\in C}|C_{c}|\cdot\sigma^{2}_{c} (5)

where σ c 2 = 1 | C c | ​ ∑ e i ∈ c ‖ e i − e ¯ c ‖ 2 \sigma^{2}_{c}=\frac{1}{|C_{c}|}\sum_{e_{i}\in c}\|e_{i}-\bar{e}_{c}\|^{2} is the variance within cluster c c , and e ¯ c \bar{e}_{c} is its centroid.

Low distortion indicates tight, semantically coherent clusters with similar embeddings. This geometric measure captures the intuition of “meaningful” categories: robins and sparrows cluster tightly as similar birds, while bats would increase distortion on downstream tasks if categorized with them.

### 4.3 Connecting Framework to Research Questions

Our framework provides unified metrics for all three research questions:

[RQ1] Categorical Alignment: How do LLMs and humans partition semantic space? The Complexity term I ⁡ ( X , C ) I(X;C) directly measures this by quantifies how many bits are needed to specify individual items given their clusters. Comparing I ⁡ ( X , C Human ) I(X;C_{\text{Human}}) with I ⁡ ( X , C LLM ) I(X;C_{\text{LLM}}) reveals whether both systems create similarly-sized groupings with comparable compression rates. Higher mutual information means finer-grained categories; lower means broader, more compressed groupings.

[RQ2] Internal Semantic Structure: Do LLMs capture human-like typicality signal? The Distortion term measures how well clusters preserve semantic coherence. We tested whether typical items cluster tightly near centroids while atypical items lie farther away. Low distortion with clear center-periphery structure indicates prototype organization that mirrors human cognitive structure.

[RQ3] Compression-Meaning Trade-off: How do different systems balance efficiency against semantic fidelity? The complete ℒ \mathcal{L} objective reveals fundamental optimization strategies. By varying K K (number of clusters) and computing ℒ \mathcal{L} curves, we uncover system priorities: aggressive compressors rapidly achieve low ℒ \mathcal{L} values by sacrificing nuance, while systems preserving semantic richness maintain higher ℒ \mathcal{L} to retain meaningful distinctions. The shape and level of these curves expose whether a system optimizes for statistical efficiency or cognitive utility.

## 5 An Empirical Investigation of Representational Strategies

Building on our information-theoretic framework (Section 4 ) and established benchmarks (Section 3.1 ), we empirically investigate how LLMs and humans navigate the compression-meaning trade-off. For each analysis, we examine both static embeddings and contextual embeddings (across all hidden layers), revealing how and when context shapes conceptual organization.

### 5.1 [RQ1] The Big Picture: Alignment of Conceptual Categories

We first investigate whether LLMs form conceptual categories aligned with humans , which examines how information is compressed into discrete groups (the complexity term in our framework).

Approach: We tested whether LLMs naturally organize our 1,049 items into categories resembling human conceptual structure. Token embeddings were extracted at two levels: (i) static embeddings from input layers (E matrix), representing context-free lexical knowledge; (ii) contextual embeddings from all hidden layers, measured layer-wise to identify peak conceptual alignment. These were clustered using k-means ( K K matching human category counts) and evaluated against human categories using Adjusted Mutual Information (AMI), Normalized Mutual Information (NMI), and Adjusted Rand Index (ARI) metrics. NMI quantifies how much information is shared between the model-derived clusters and the human-labeled categories; AMI refines this measure by correcting for the amount of overlap that would be expected by chance; and ARI assesses the degree of agreement between the two partitions while explicitly accounting for random assignments, providing a complementary view of clustering accuracy.

Broad Categorical Agreement: All 40+ models achieve significant above-chance alignment (Figure 2 (Left)). Even at baseline, static embeddings show substantial alignment (mean A ​ M ​ I ≈ 0.45 AMI\approx 0.45 ), which contextual processing enhances to peak A ​ M ​ I ≈ 0.55 AMI\approx 0.55 . This confirms that LLMs encode human-like categorical boundaries. Full NMI and ARI results in Appendices K - M .

Architecture Matters More Than Scale: Surprisingly, BERT-large-uncased (340M parameters) achieves A ​ M ​ I = 0.60 AMI=0.60 , matching or exceeding models 100× larger. Classic static Word2Vec and GloVe embeddings, despite predating modern architectures by years, reach AMI scores rivaling contemporary LLMs’ peak performance. This suggests that fundamental semantic structure emerges from relatively simple distributional learning, with encoder architectures particularly effective at capturing human-like categories regardless of scale.

### 5.2 [RQ2] Zooming In: Fidelity to Fine-Grained Semantics

Having established broad categorical alignment, we now examine whether LLMs capture the internal semantic structure of categories. Specifically, we check how meaning is preserved within clusters .

Approach: We test whether LLMs encode human-like typicality signals, i.e., whether robins are more “birdy” than penguins. For each item, we compute the cosine similarity with its category name using embeddings (e.g., ‘robin’ → ‘bird’; c ​ o ​ s ​ i ​ n ​ e s ​ i ​ m ​ ( E ⁡ ( r ​ o ​ b ​ i ​ n ) , E ⁡ ( b ​ i ​ r ​ d ) CLOSE cosine_{sim}(E(robin),E(bird) ). We compare these similarities with human typicality ratings using Spearman’s correlation coefficient ρ \rho ( Wissler, 1905 ) .

We employed two analysis approaches: (i) static-layer analysis using embeddings directly from the input layer; (ii) peak AMI layer analysis using contextual embeddings from the layer that maximized AMI in RQ1. For the peak AMI approach, we extracted category embeddings by replacing items with category names in the same prompt template, ensuring consistent contextualization (see Appendix F for template and pooling robustness analysis).

Weak Typicality Alignment: Correlations between LLM internal organization of concepts and human typicality are modest at best (Figure 2 (Right); Tables 3 - 4 in Appendix N ). Static embeddings show weak correlations: BERT achieves ρ = 0.38 \rho=0.38 ( p < 0.05 p<0.05 ), while most decoder models fall below ρ = 0.15 \rho=0.15 . Even when statistically significant, these correlations indicate limited correspondence with human judgments. This shows that the internal concept geometries of models differ from those of humans, with representation-focused models aligning more closely.

Architectural Patterns: Several clear patterns emerge in how different architectures capture typicality. Representation-focused models (Word2Vec, GloVe) and most encoder models (both ViT encoders, BERT-large) demonstrate stronger static-layer performance than decoder-only models (Llama, Gemma, Qwen families). Static correlations range from ρ ≈ 0.25 \rho\approx 0.25 - 0.40 0.40 for representation-focused models versus ρ < 0.15 \rho<0.15 for most decoders.

This divergence likely stems from training objectives: models explicitly trained for representation learning appear more effective at capturing semantic category relationships in their embeddings, while modern decoder-only models, which optimized primarily for next-token prediction, show consistently lower static-layer correlations. The pattern holds across model scales, suggesting architectural design matters more than size for capturing fine-grained semantic similarity.

Layer-wise Analysis Reveals a Trade-off: Comparing static and peak AMI layers exposes an architectural limitation. Peak AMI layers, which are optimal for clustering, show systematically weaker typicality correlations than static layers. This pattern holds across model families: layers that best separate categories (RQ1) poorly preserve within-category structure (RQ2). The implication is clear: current architectures encode different aspects of meaning at different depths, forcing applications to choose between broad categorization and semantic nuance.

Interpretation: The divergence between LLMs and humans reflects fundamentally different organizational principles. Humans judge typicality through rich, multidimensional criteria: robins are typical birds due to size, flight ability, song, etc. This creates graded categories with clear prototypes and cognitive structures that optimize flexible reasoning and generalization.

LLMs, in contrast, appear to encode flatter statistical associations between items and category labels. Although sufficient for categorization and fluent text generation, these representations miss the prototype structure that makes categories cognitively useful. This difference suggests that LLMs optimize for different objectives than human cognition, a hypothesis that we test directly in RQ3 by examining how each system balances compression against semantic preservation.

### 5.3 [RQ3] The Efficiency Angle: The Compression-Meaning Trade-off

Having explored categorical alignment (RQ1) and internal semantic structure (RQ2), we now address our central question: How do LLM and human representational strategies compare when balancing compression against meaning preservation?

Approach: We analyzed human-defined categories and LLM-derived clusters using our ℒ \mathcal{L} objective function (Equation 3 , β = 1 \beta=1 ) and mean cluster entropy ( S α S_{\alpha} ). For LLMs, we performed k-means clustering across various K K values to trace the full compression-meaning frontier.

Results: Our analysis reveals three key patterns (Figure 3 ; full results in Appendix Q ):

LLMs Achieve Superior Statistical Efficiency: Both measures reveal stark differences between LLMs and humans (Figure 3 ; full results in Appendix Q ):

Higher human entropy. Human concepts consistently exhibit higher cluster entropy than LLM clusters at comparable K K values, indicating less statistical compactness but greater internal diversity.

Lower LLM ℒ \mathcal{L} scores. LLM-derived clusters achieve significantly lower ℒ \mathcal{L} values than human categories across all tested K K (Figure 3 b). Since lower ℒ \mathcal{L} signifies more optimal compression-distortion balance, LLMs are demonstrably more “efficient” by this information-theoretic measure.

Architectural differences. The Complexity-Distortion plot (Figure 3 a) reveals that encoder models (BERT, ViT, classic static models) achieve superior trade-offs. Their distortion at any given complexity is lower compared to decoder models, across both static and contextual embeddings.

Statistical Optimality Versus Cognitive Utility: This divergence reveals fundamental differences in optimization pressures. LLMs, trained on massive text corpora, develop maximally efficient statistical representations that minimize redundancy and internal variance. Although human conceptual systems may look suboptimal under information-theoretic measures, prior work indicates that they are structured to support goals such as flexible generalization and causal reasoning rather than maximal compression ( Murphy, 2004 ) . Thus, the differing pressures shaping human and LLM representations help explain this apparent suboptimality.

Our analysis reveals that compression efficiency does not predict functional capability. We find no correlation between ℒ \mathcal{L} scores and downstream performance ( r = − 0.20 r=-0.20 , ρ = 0.51 \rho=0.51 on MMLU; Appendix S ). This suggests that apparent human “inefficiency” reflects optimization for cognitive flexibility rather than statistical compression. While LLMs excel at compact representation, they may sacrifice the semantic richness essential for human-like understanding.

The consistent architectural patterns observed raise fundamental questions: Do understanding and generation require distinct computational strategies? The superiority of representation-focused models suggests that current architectures may be conflating two fundamentally different cognitive tasks.

### 5.4 Emergence During Training: How Divergent Strategies Develop

Having established that LLMs and humans employ divergent representational strategies, we investigate how these strategies emerge. Analysis of OLMo-7B across 57 training checkpoints (1K to 557K steps, approximately 4B to 2.5T tokens) reveals how conceptual structure emerges.

Two-Phase Representational Development. Conceptual organization emerges via two phases (Figure 15 ). First, rapid concept formation (1K-100K steps) establishes basic categorical structure, with AMI rising from near zero to approximately 0.45, achieving 80% of final alignment within just 10% of training. Second, architectural reorganization (100K-500K steps) systematically migrates semantic processing from deeper layers toward mid-network while AMI continues to gradually improve. This migration from layer 29 to layer 23 occurs without sacrificing categorical alignment, suggesting the model discovers more efficient internal representations. Moreover, this double-phase dynamics occurs when testing attention sparsity, effective rank, and ℒ \mathcal{L} values. That is, all of these exhibit the same early rapid shift followed by a slower restructuring phase. This convergence across independent metrics indicates that the model is not merely improving categorical alignment, but reorganizing its internal representations toward increasingly efficient structure. See Appendix H .

Architectural Reorganization as Optimization. The upward migration of semantic processing hints that the model discovers increasingly efficient encodings. Early training relies on deep, memorization-heavy representations, but as training progresses, the model shifts to distributed mid-network encoding. This reorganization may explain apparent “emergent” capabilities: they arise not from learning fundamentally new information but from more efficient internal organization of existing knowledge.