Fine-Tuning Fixes Mode Collapse and Over-Dispersion in LLMs — part 7/7

Cleaned-text code points [93310, 98879). Bibliography omitted here; source.md includes it.

The M n M_{n} generations are i.i.d. draws from q n , T ( ⋅ ∣ x ) q_{n,T}(\cdot\mid x) , so the probability that none lands in U ⁡ ( x ) U(x) is ( 1 − ε n , T ​ ( x ) ) M n \big(1-\varepsilon_{n,T}(x)\big)^{M_{n}} , giving the stated equality. Bernoulli’s inequality ( 1 − ε ) M n ≥ 1 − M n ​ ε (1-\varepsilon)^{M_{n}}\geq 1-M_{n}\varepsilon for ε ∈ [ 0 , 1 ] \varepsilon\in[0,1] yields 1 − ( 1 − ε n , T ​ ( x ) ) M n ≤ M n ​ ε n , T ​ ( x ) 1-(1-\varepsilon_{n,T}(x))^{M_{n}}\leq M_{n}\,\varepsilon_{n,T}(x) . If M n ​ ε n , T ​ ( x ) → 0 M_{n}\,\varepsilon_{n,T}(x)\to 0 this probability vanishes, so with high probability no sample realizes any continuation in U ⁡ ( x ) U(x) . ∎

No validity-aware guarantee. Split an underweighted tail into valid and invalid sets, U valid ​ ( x ) U_{\mathrm{valid}}(x) and U invalid ​ ( x ) U_{\mathrm{invalid}}(x) . Temperature applies the same map u ↦ u 1 / T u\mapsto u^{1/T} to both: q T ​ ( U valid ​ ( x ) ∣ x ) q T ​ ( U invalid ​ ( x ) ∣ x ) = ∑ y ∈ U valid ​ ( x ) q ​ ( y ∣ x ) 1 / T ∑ y ∈ U invalid ​ ( x ) q ​ ( y ∣ x ) 1 / T . \frac{q_{T}(U_{\mathrm{valid}}(x)\mid x)}{q_{T}(U_{\mathrm{invalid}}(x)\mid x)}=\frac{\sum_{y\in U_{\mathrm{valid}}(x)}q(y\mid x)^{1/T}}{\sum_{y\in U_{\mathrm{invalid}}(x)}q(y\mid x)^{1/T}}. (41) The ratio can vary with T T , but the transformation contains no validity signal. Temperature can incidentally increase valid mass; it cannot guarantee selective recovery. Proposition 3 implies effective absence only when its stated condition M n ​ ε n , T ​ ( x ) → 0 M_{n}\varepsilon_{n,T}(x)\to 0 holds.

###### Example 2 (A two-peaked target temperature cannot reach) .

Take a fixed six-outcome context in which the model q = ( 0.50 , 0.25 , 0.12 , 0.07 , 0.04 , 0.02 ) q=(0.50,0.25,0.12,0.07,0.04,0.02) is monotone decreasing while the target p = ( 0.30 , 0.05 , 0.05 , 0.05 , 0.25 , 0.30 ) p=(0.30,0.05,0.05,0.05,0.25,0.30) is two-peaked: here N 2 ​ ( p ) = 4 ≤ S = 6 N_{2}(p)=4\leq S=6 , so the diversity value is attainable (Theorem, case (ii)), yet the shape of p p is not, because its ordering differs from that of q q (case (iii)).

Two-sided miscalibration. Temperature can leave diversity below the target or flatten beyond it. Neither direction alone establishes output validity: low-probability continuations can be valid or invalid. Even when a temperature matches N 2 ​ ( p ) N_{2}(p) , it matches p p only in the power-family case of Theorem 2 .

The rate of temperature-driven diversity change is determined by the spread of the model’s log-probabilities. Defining ψ ⁡ ( β ) = log ⁡ Z β \psi(\beta)=\log Z_{\beta} , we have ψ ′ ​ ( β ) = 𝔼 q β ​ [ log ⁡ q ] \psi^{\prime}(\beta)=\mathbb{E}_{q_{\beta}}[\log q] and ψ ′′ ​ ( β ) = Var q β ⁡ ( log ⁡ q ) ≥ 0 \psi^{\prime\prime}(\beta)=\Var_{q_{\beta}}(\log q)\geq 0 , where q β q_{\beta} is proportional to q β q^{\beta} . Given that log ⁡ C T = ψ ⁡ ( 2 ​ β ) − 2 ​ ψ ​ ( β ) \log C_{T}=\psi(2\beta)-2\psi(\beta) , the derivative with respect to log ⁡ T \log T is − β -\beta times the derivative with respect to β \beta . d ​ log ⁡ N 2 T ​ ( x ) d ​ log ⁡ T = − d ​ log ⁡ C T d ​ log ⁡ T = 2 ​ β ​ ∫ β 2 ​ β Var q s ⁡ ( log ⁡ q ) ​ 𝑑 s ≥ 0 . \frac{d\,\log N_{2}^{T}(x)}{d\,\log T}=-\frac{d\,\log C_{T}}{d\,\log T}=2\beta\!\int_{\beta}^{2\beta}\Var_{q_{s}}(\log q)\,ds\;\geq\;0. (42) The derivative is governed by the log-probability variance along the power family. For nonuniform finite-support q q , finite temperatures trace the open interval ( g ⋆ ​ ( x ) , S ​ ( x ) ) (g_{\star}(x),S(x)) and approach its endpoints only in the limits.

LLM decoding employs temperature locally, applying q T ​ ( a ∣ h ) = q ​ ( a ∣ h ) 1 / T / ∑ b q ​ ( b ∣ h ) 1 / T q_{T}(a\mid h)=q(a\mid h)^{1/T}/\sum_{b}q(b\mid h)^{1/T} , which influences the future contexts reached, as well as the probabilities of EOS, token length, format, and refusal, and the mixture of rare-valid and rare-invalid tokens. The monotonicity result for global sequence temperature therefore need not hold for token-by-token decoding.

Simple example. Suppose the root probabilities are 0.4 0.4 for EOS and 0.6 0.6 for CONTINUE, followed by 100 100 uniform endings after CONTINUE. Sequence collision is 0.4 2 + 0.6 2 ​ ( 0.01 ) = 0.1636 0.4^{2}+0.6^{2}(0.01)=0.1636 . Raising local temperature moves the root toward ( 0.5 , 0.5 ) (0.5,0.5) while leaving the uniform branch unchanged, so collision approaches 0.5 2 + 0.5 2 ​ ( 0.01 ) = 0.2525 0.5^{2}+0.5^{2}(0.01)=0.2525 . Thus local temperature can reduce sequence diversity even though global temperature of a fixed sequence distribution cannot.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .