LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics — part 6/10

Cleaned-text code points [74849, 90244). Bibliography omitted here; source.md includes it.

Fix x ∈ ℝ d x\in\mathbb{R}^{d} and write z ∈ B ⁡ ( 0 , r 0 ) z\in\mathrm{B}(0,r_{0}) for local displacements. Assume p ∈ C 3 p\in C^{3} , η ∈ C 2 \eta\in C^{2} with bounded derivatives on the region of interest, and expand a second-order Taylor expansion: p ⁡ ( x + z ) \displaystyle p(x+z) = p ( x ) + ∇ p ( x ) ⊤ z + 1 2 z ⊤ H p ( x ) z + O ( ∥ z ∥ 3 ) , \displaystyle=p(x)+\nabla p(x)^{\top}z+\tfrac{1}{2}z^{\top}Hp(x)z+O(\|z\|^{3}), η ⁡ ( x + z ) \displaystyle\eta(x+z) = η ( x ) + ∇ η ( x ) ⊤ z + 1 2 z ⊤ H η ( x ) z + O ( ∥ z ∥ 3 ) , \displaystyle=\eta(x)+\nabla\eta(x)^{\top}z+\tfrac{1}{2}z^{\top}H\eta(x)z+O(\|z\|^{3}), with remainders satisfying | R η ​ ( x , z ) | ≤ C η ​ ‖ z ‖ 3 |R_{\eta}(x;z)|\leq C_{\eta}\left\lVert z\right\rVert^{3} and | R p ​ ( x , z ) | ≤ C p ​ ‖ z ‖ 3 |R_{p}(x;z)|\leq C_{p}\left\lVert z\right\rVert^{3} uniformly for ‖ z ‖ ≤ r 0 \left\lVert z\right\rVert\leq r_{0} . Using the ball identities ∫ B ⁡ ( 0 , r ) z ​ 𝑑 z = 0 \int_{B(0,r)}zdz=0 and ∫ B ⁡ ( 0 , r ) z ​ z ⊤ ​ 𝑑 z = v d ​ r d + 2 d + 2 ​ I d \int_{B(0,r)}zz^{\top}dz=\frac{v_{d}r^{d+2}}{d+2}I_{d} and collecting terms up to order r 0 d + 2 r_{0}^{d+2} , we simplify the denominator as 𝒟 ⁡ ( x ) \displaystyle\mathcal{D}(x) \triangleq ​ ∫ B ⁡ ( 0 , r 0 ) p ⁡ ( x + z ) ​ 𝑑 z \displaystyle\triangleq\int_{\mathrm{B}(0,r_{0})}p(x+z)dz = ∫ B ⁡ ( 0 , r 0 ) [ p ( x ) + ∇ p ( x ) ⊤ z + 1 2 z ⊤ H p ( x ) z + R p ( x ; z ) ] d z \displaystyle=\int_{\mathrm{B}(0,r_{0})}\Big[p(x)+\nabla p(x)^{\top}z+\tfrac{1}{2}z^{\top}Hp(x)z+R_{p}(x;z)\Big]dz = Vol 0 d ​ p ​ ( x ) + Vol 0 d + 2 2 ​ ( d + 2 ) ​ tr ​ ( H ​ p ​ ( x ) ) + O ⁡ ( r 0 d + 3 ) , \displaystyle={\rm Vol}_{0}^{d}p(x)\;+\;\frac{{\rm Vol}_{0}^{d+2}}{2(d+2)}\mathrm{tr}\big(Hp(x)\big)\;+\;O(r_{0}^{d+3}), since ∫ z ​ 𝑑 z = 0 \int zdz=0 and ∫ z ⊤ ​ H ​ p ​ z ​ 𝑑 z = tr ⁡ ( H ​ p ) ​ v d ​ r 0 d + 2 d + 2 \int z^{\top}Hpzdz=\mathrm{tr}(Hp)\frac{v_{d}r_{0}^{d+2}}{d+2} and the denominator as 𝒩 ⁡ ( x ) \displaystyle\mathcal{N}(x) \triangleq ​ ∫ B ⁡ ( 0 , r 0 ) η ⁡ ( x + z ) ​ p ​ ( x + z ) ​ 𝑑 z \displaystyle\triangleq\int_{\mathrm{B}(0,r_{0})}\eta(x+z)p(x+z)dz = ∫ [ η ( x ) + ∇ η ( x ) ⊤ z + 1 2 z ⊤ H η ( x ) z ] [ p ( x ) + ∇ p ( x ) ⊤ z + 1 2 z ⊤ H p ( x ) z ] d z + O ( r 0 d + 3 ) \displaystyle=\int\Big[\eta(x)+\nabla\eta(x)^{\top}z+\tfrac{1}{2}z^{\top}H\eta(x)z\Big]\Big[p(x)+\nabla p(x)^{\top}z+\tfrac{1}{2}z^{\top}Hp(x)z\Big]dz+O(r_{0}^{d+3}) = η ( x ) p ( x ) v d r 0 d + η ( x ) v d ​ r 0 d + 2 2 ​ ( d + 2 ) tr ( H p ( x ) ) + v d ​ r 0 d + 2 d + 2 ∇ η ( x ) ⋅ ∇ p ( x ) + v d ​ r 0 d + 2 2 ​ ( d + 2 ) p ( x ) tr ( H η ( x ) ) + O ( r 0 d + 3 ) . \displaystyle=\eta(x)p(x)v_{d}r_{0}^{d}+\eta(x)\frac{v_{d}r_{0}^{d+2}}{2(d+2)}\mathrm{tr}\big(Hp(x)\big)+\frac{v_{d}r_{0}^{d+2}}{d+2}\nabla\eta(x)\cdot\nabla p(x)+\frac{v_{d}r_{0}^{d+2}}{2(d+2)}p(x)\mathrm{tr}\big(H\eta(x)\big)+O(r_{0}^{d+3}).

Cubic terms vanish by symmetry, and quartic terms are O ⁡ ( r 0 d + 4 ) O(r_{0}^{d+4}) . Subtract η ⁡ ( x ) ​ 𝒟 ​ ( x ) \eta(x)\mathcal{D}(x) to obtain the bias numerator: 𝒩 ⁡ ( x ) − η ⁡ ( x ) ​ 𝒟 ​ ( x ) = v d ​ r 0 d + 2 d + 2 ​ ( ∇ η ​ ( x ) ⋅ ∇ p ​ ( x ) + 1 2 ​ p ​ ( x ) ​ Δ ​ η ​ ( x ) ) + O ⁡ ( r 0 d + 3 ) . \mathcal{N}(x)-\eta(x)\mathcal{D}(x)=\frac{v_{d}r_{0}^{d+2}}{d+2}\Big(\nabla\eta(x)\cdot\nabla p(x)+\tfrac{1}{2}p(x)\Delta\eta(x)\Big)+O(r_{0}^{d+3}). Write 𝒟 ⁡ ( x ) = v d ​ r 0 d ​ p ​ ( x ) ​ ( 1 + α ⁡ ( x ) ​ r 0 2 + O ⁡ ( r 0 3 ) ) \mathcal{D}(x)=v_{d}r_{0}^{d}p(x)\big(1+\alpha(x)r_{0}^{2}+O(r_{0}^{3})\big) where α ⁡ ( x ) := 1 2 ​ ( d + 2 ) ​ p ​ ( x ) ​ tr ​ ( H ​ p ​ ( x ) ) \alpha(x):=\frac{1}{2(d+2)p(x)}\mathrm{tr}(Hp(x)) . Then 𝒩 ⁡ ( x ) 𝒟 ⁡ ( x ) − η ⁡ ( x ) \displaystyle\frac{\mathcal{N}(x)}{\mathcal{D}(x)}-\eta(x) = v d ​ r 0 d + 2 d + 2 ​ ( ∇ η ⋅ ∇ p + 1 2 ​ p ​ Δ ​ η ) + O ⁡ ( r 0 d + 3 ) v d ​ r 0 d ​ p ​ ( 1 + α ​ r 0 2 + O ⁡ ( r 0 3 ) ) \displaystyle=\frac{\frac{v_{d}r_{0}^{d+2}}{d+2}\left(\nabla\eta\cdot\nabla p+\frac{1}{2}p\Delta\eta\right)+O(r_{0}^{d+3})}{v_{d}r_{0}^{d}p\left(1+\alpha r_{0}^{2}+O(r_{0}^{3})\right)} = r 0 2 d + 2 ​ ( ∇ η ⋅ ∇ p p + 1 2 ​ Δ ​ η ) ​ ( 1 − α ​ r 0 2 + O ⁡ ( r 0 3 ) ) + O ⁡ ( r 0 3 ) \displaystyle=\frac{r_{0}^{2}}{d+2}\left(\frac{\nabla\eta\cdot\nabla p}{p}+\frac{1}{2}\Delta\eta\right)\Big(1-\alpha r_{0}^{2}+O(r_{0}^{3})\Big)\ +\ O(r_{0}^{3}) = r 0 2 d + 2 ​ ( ∇ η ​ ( x ) ⋅ ∇ log ⁡ p ​ ( x ) + 1 2 ​ Δ ​ η ​ ( x ) ) + o ⁡ ( r 0 2 ) , \displaystyle=\frac{r_{0}^{2}}{d+2}\Big(\nabla\eta(x)\cdot\nabla\log p(x)+\tfrac{1}{2}\Delta\eta(x)\Big)\ +\ o(r_{0}^{2}), uniformly on 𝒦 \mathcal{K} . This gives the bias formula 𝔼 ⁡ [ η ^ ​ ( x ) ] − η ⁡ ( x ) = r 0 2 d + 2 ​ ( ∇ η ​ ( x ) ⋅ ∇ log ⁡ p ​ ( x ) + 1 2 ​ Δ ​ η ​ ( x ) ) + o ⁡ ( r 0 2 ) , \mathbb{E}\big[\widehat{\eta}(x)\big]-\eta(x)=\frac{r_{0}^{2}}{d+2}\Big(\nabla\eta(x)\cdot\nabla\log p(x)+\tfrac{1}{2}\Delta\eta(x)\Big)\ +\ o(r_{0}^{2}),

completing the proof. ∎

### B.4 Proof of Section A.1

###### Proof.

Recall from Section B.3 that the bias term as sample 𝒙 {\bm{x}} is given by Bias ⁡ ( 𝒙 ) = \displaystyle\mathrm{Bias}({\bm{x}})= r 0 2 d + 2 ​ ( ∇ η ​ ( x ) ⋅ ∇ log ⁡ p ​ ( x ) ) + r 0 2 2 ​ ( d + 2 ) ​ Δ ​ η ​ ( x ) + o ⁡ ( r 0 2 ) \displaystyle\frac{r_{0}^{2}}{d+2}\Big(\nabla\eta(x)\cdot\nabla\log p(x)\Big)\;+\;\frac{r_{0}^{2}}{2(d+2)}\Delta\eta(x)\;+\;o(r_{0}^{2}) = \displaystyle= r 0 2 d + 2 ​ ( A ⁡ ( x ) + C ⁡ ( x ) ) + o ⁡ ( r 0 2 ) , \displaystyle\frac{r_{0}^{2}}{d+2}\big(A(x)+C(x)\big)+o(r_{0}^{2}), where we defined A ( x ) \triangleq ∇ η ( x ) ⋅ ∇ log p ( x ) A(x)\triangleq\nabla\eta(x)\cdot\nabla\log p(x) and C ⁡ ( x ) ​ \triangleq ​ 1 2 ​ Δ ​ η ​ ( x ) C(x)\triangleq\frac{1}{2}\Delta\eta(x) . We now square and take expectation of X ∼ p X\sim p and the isotropic gradient prior 𝔼 ⁡ [ Bias ​ ( X ) 2 ] \displaystyle\mathbb{E}\big[\mathrm{Bias}(X)^{2}\big] = 𝔼 ⁡ [ ( r 0 2 d + 2 ) 2 ​ ( A ​ ( x ) 2 + 2 ​ A ​ ( x ) ​ C ​ ( x ) + C ​ ( x ) 2 ) + o ⁡ ( r 0 4 ) ] \displaystyle=\mathbb{E}\big[\left(\frac{r_{0}^{2}}{d+2}\right)^{2}\big(A(x)^{2}+2A(x)C(x)+C(x)^{2}\big)+o(r_{0}^{4})\big] (14) = ( r 0 2 d + 2 ) 2 ​ { 𝔼 ⁡ [ A ​ ( X ) 2 ] ⏟ score-gradient term + 2 ​ 𝔼 ​ [ A ⁡ ( X ) ​ C ​ ( X ) ] ⏟ cross term + 𝔼 ⁡ [ C ​ ( X ) 2 ] ⏟ curvature term } + o ⁡ ( r 0 4 ) . \displaystyle=\left(\frac{r_{0}^{2}}{d+2}\right)^{2}\Big\{\underbrace{\mathbb{E}\big[A(X)^{2}\big]}_{\text{score-gradient term}}+\underbrace{2\mathbb{E}\big[A(X)C(X)\big]}_{\text{cross term}}+\underbrace{\mathbb{E}\big[C(X)^{2}\big]}_{\text{curvature term}}\Big\}+o(r_{0}^{4}). (15) We will derive each term separately, recalling that we assume an isotropic gradient prior for η \eta , i.e., 𝔼 ​ [ ∇ η ​ ( x ) ] = 0 \mathbb{E}\big[\nabla\eta(x)\big]=0 and 𝔼 [ ∇ η ( x ) ∇ η ( x ) ⊤ ] = τ g 2 I d \mathbb{E}\big[\nabla\eta(x)\nabla\eta(x)^{\top}\big]=\tau_{g}^{2}I_{d} , for some τ g 2 ∈ ( 0 , ∞ ) \tau_{g}^{2}\in(0,\infty) .

##### 1) The score-gradient term 𝔼 ⁡ [ A ​ ( X ) 2 ] \mathbb{E}[A(X)^{2}] .

Using v ​ ( x ) := ∇ log ⁡ p ​ ( x ) v(x):=\nabla\log p(x) for brevity: 𝔼 ⁡ [ A ​ ( X ) 2 ] = \displaystyle\mathbb{E}\big[A(X)^{2}\big]= 𝔼 X ​ [ 𝔼 η ​ [ A ​ ( X ) 2 ] ] \displaystyle\mathbb{E}_{X}\big[\mathbb{E}_{\eta}[A(X)^{2}]\big] = \displaystyle= 𝔼 X [ 𝔼 η [ ( ∇ η ( x ) ⊤ v ( x ) ) 2 ] ] \displaystyle\mathbb{E}_{X}\big[\mathbb{E}_{\eta}[\big(\nabla\eta(x)^{\top}v(x)\big)^{2}]\big] = \displaystyle= 𝔼 X [ 𝔼 η [ ∇ η ( x ) ⊤ ( v ( x ) v ( x ) ⊤ ) ∇ η ( x ) ] ] \displaystyle\mathbb{E}_{X}\big[\mathbb{E}_{\eta}[\nabla\eta(x)^{\top}\Big(v(x)v(x)^{\top}\Big)\nabla\eta(x)]\big] = \displaystyle= 𝔼 X [ 𝔼 η [ tr ( v ( x ) v ( x ) ⊤ ∇ η ( x ) ∇ η ( x ) ⊤ ) ] ] \displaystyle\mathbb{E}_{X}\big[\mathbb{E}_{\eta}[\mathrm{tr}\Big(v(x)v(x)^{\top}\nabla\eta(x)\nabla\eta(x)^{\top}\Big)]\big] = \displaystyle= 𝔼 X [ tr ( v ( x ) v ( x ) ⊤ 𝔼 η [ ∇ η ( x ) ∇ η ( x ) ⊤ ] ) ] \displaystyle\mathbb{E}_{X}\big[\mathrm{tr}\Big(v(x)v(x)^{\top}\mathbb{E}_{\eta}[\nabla\eta(x)\nabla\eta(x)^{\top}]\Big)\big] = \displaystyle= 𝔼 X ​ [ τ g 2 ​ ‖ v ⁡ ( x ) ‖ 2 ] \displaystyle\mathbb{E}_{X}\big[\tau_{g}^{2}\|v(x)\|^{2}\big] = \displaystyle= τ g 2 ​ 𝔼 X ​ [ ‖ v ⁡ ( X ) ‖ 2 ] \displaystyle\tau_{g}^{2}\mathbb{E}_{X}\big[\|v(X)\|^{2}\big] = \displaystyle= τ g 2 ​ ∫ ℝ d ‖ ∇ log ⁡ p ​ ( x ) ‖ 2 ​ p ​ ( x ) ​ 𝑑 x \displaystyle\tau_{g}^{2}\int_{\mathbb{R}^{d}}\|\nabla\log p(x)\|^{2}p(x)dx recovering the Fisher-information functional J ⁡ ( p ) J(p) , scaled by τ g 2 \tau_{g}^{2}

##### 2) The cross term 2 ​ 𝔼 ​ [ A ⁡ ( X ) ​ C ​ ( X ) ] 2\mathbb{E}[A(X)C(X)] .

We have A ( x ) C ( x ) = 1 2 ( ∇ η ( x ) ⊤ v ( x ) ) Δ η ( x ) . A(x)C(x)=\frac{1}{2}\big(\nabla\eta(x)^{\top}v(x)\big)\Delta\eta(x). Under the prior, ∇ η \nabla\eta is mean-zero and isotropic; if, additionally, Δ ​ η \Delta\eta is uncorrelated with ∇ η \nabla\eta and has zero mean (or is bounded and mean-zero after centering), then 𝔼 η ​ [ A ⁡ ( x ) ​ C ​ ( x ) ] = 0 \mathbb{E}_{\eta}[A(x)C(x)]=0 . If one does not assume the orthogonality/vanishing covariance above, then 𝔼 ⁡ [ A ⁡ ( X ) ​ C ​ ( X ) ] \mathbb{E}[A(X)C(X)] is a finite constant (depending on the joint law of derivatives of η \eta ), and the cross term contributes ( r 0 2 d + 2 ) 2 ⋅ 2 ​ 𝔼 ​ [ A ⁡ ( X ) ​ C ​ ( X ) ] = O ⁡ ( r 0 4 ) , \left(\frac{r_{0}^{2}}{d+2}\right)^{2}\cdot 2\mathbb{E}[A(X)C(X)]=\ O(r_{0}^{4}), not o ⁡ ( r 0 4 ) o(r_{0}^{4}) . In that general case, the leading p p -dependent term of 𝔼 ⁡ [ Bias ​ ( X ) 2 ] \mathbb{E}[\mathrm{Bias}(X)^{2}] is still the score-gradient τ g 2 ​ J ​ ( p ) \tau_{g}^{2}J(p) .

##### 3) The curvature term 𝔼 ⁡ [ C ​ ( X ) 2 ] \mathbb{E}[C(X)^{2}] .

𝔼 ⁡ [ C ​ ( X ) 2 ] = \displaystyle\mathbb{E}\big[C(X)^{2}\big]= 𝔼 X ​ [ 𝔼 η ​ [ C ​ ( X ) 2 ] ] \displaystyle\mathbb{E}_{X}\big[\mathbb{E}_{\eta}[C(X)^{2}]\big] = \displaystyle= 1 4 𝔼 X [ 𝔼 η [ ( Δ η ( X ) ) 2 ] \displaystyle\frac{1}{4}\mathbb{E}_{X}\big[\mathbb{E}_{\eta}[(\Delta\eta(X))^{2}\big] which is independent of p p , hence 𝔼 ⁡ [ C ​ ( X ) 2 ] = O ⁡ ( 1 ) \mathbb{E}\big[C(X)^{2}\big]=O(1)

##### Putting it together.

Substituting into ( 15 ): 𝔼 ⁡ [ Bias ​ ( X ) 2 ] \displaystyle\mathbb{E}\big[\mathrm{Bias}(X)^{2}\big] = ( r 0 2 d + 2 ) 2 ​ { τ g 2 ​ J ​ ( p ) + O ⁡ ( 1 ) } + o ⁡ ( r 0 4 ) \displaystyle=\left(\frac{r_{0}^{2}}{d+2}\right)^{2}\Big\{\tau_{g}^{2}J(p)+O(1)\Big\}+o(r_{0}^{4}) = r 0 4 ( d + 2 ) 2 ​ τ g 2 ​ J ​ ( p ) + O ⁡ ( r 0 4 ) , \displaystyle=\frac{r_{0}^{4}}{(d+2)^{2}}\tau_{g}^{2}J(p)\;+\;O(r_{0}^{4}),

We show that, among all mean-zero distributions p p on ℝ d \mathbb{R}^{d} with a given scalar constraint on the covariance (trace, determinant, Frobenius norm, or spectral radius), the density that minimizes the Fisher-information functional J ⁡ ( p ) := ∫ ℝ d ‖ ∇ log ⁡ p ​ ( x ) ‖ 2 ​ p ​ ( x ) ​ 𝑑 x J(p)\;:=\;\int_{\mathbb{R}^{d}}\|\nabla\log p(x)\|^{2}p(x)dx is the Gaussian with isotropic covariance satisfying the same scalar constraint.

We proceed in two steps: (i) for fixed covariance matrix Σ ≻ 0 \Sigma\succ 0 , J ⁡ ( p ) J(p) is minimized by the Gaussian 𝒩 ⁡ ( 0 , Σ ) \mathcal{N}(0,\Sigma) and attains the value tr ⁡ ( Σ − 1 ) \mathrm{tr}(\Sigma^{-1}) ; (ii) for each scalar constraint, tr ⁡ ( Σ − 1 ) \mathrm{tr}(\Sigma^{-1}) is minimized by Σ = s ​ I d \Sigma=sI_{d} for the appropriate scalar s > 0 s>0 .

###### Proof.

Consider the location family p θ ​ ( x ) := p ⁡ ( x − θ ) p_{\theta}(x):=p(x-\theta) , θ ∈ ℝ d \theta\in\mathbb{R}^{d} . Its Fisher-information matrix at θ \theta is ℐ ( θ ) = 𝔼 [ ∇ θ log p θ ( X ) ∇ θ log p θ ( X ) ⊤ ] = 𝔼 [ ∇ log p ( X ) ∇ log p ( X ) ⊤ ] , \mathcal{I}(\theta)\;=\;\mathbb{E}\big[\nabla_{\theta}\log p_{\theta}(X)\nabla_{\theta}\log p_{\theta}(X)^{\top}\big]\;=\;\mathbb{E}\big[\nabla\log p(X)\nabla\log p(X)^{\top}\big], so that J ⁡ ( p ) = tr ​ ℐ ​ ( θ ) J(p)=\mathrm{tr}\mathcal{I}(\theta) . The estimator T ⁡ ( X ) ≡ X T(X)\equiv X is unbiased for θ \theta under p θ p_{\theta} , with Cov ⁡ ( T ) = Σ \mathrm{Cov}(T)=\Sigma . The matrix Cramér–Rao bound gives Cov ⁡ ( T ) ⪰ ℐ ​ ( θ ) − 1 \mathrm{Cov}(T)\succeq\mathcal{I}(\theta)^{-1} , i.e., ℐ ⁡ ( θ ) ⪰ Σ − 1 \mathcal{I}(\theta)\succeq\Sigma^{-1} . Taking traces yields J ⁡ ( p ) ≥ tr ⁡ ( Σ − 1 ) J(p)\geq\mathrm{tr}(\Sigma^{-1}) . Equality in the matrix Cramér–Rao bound holds if and only if the score is an affine function of X − θ X-\theta , i.e., ∇ log ⁡ p θ ​ ( X ) = A ⁡ ( X − θ ) \nabla\log p_{\theta}(X)=A(X-\theta) a.s. for some matrix A A ; integrating this identity shows p θ p_{\theta} is Gaussian with precision matrix − A -A , hence p = 𝒩 ⁡ ( 0 , Σ ) p=\mathcal{N}(0,\Sigma) . ∎

### Step 2: Optimizing over covariance shapes under scalar constraints

Write the eigenvalues of Σ \Sigma as λ 1 , … , λ d > 0 \lambda_{1},\ldots,\lambda_{d}>0 . Then tr ⁡ ( Σ − 1 ) = ∑ i = 1 d 1 λ i . \mathrm{tr}(\Sigma^{-1})=\sum_{i=1}^{d}\frac{1}{\lambda_{i}}. We now solve min ∑ i 1 / λ i \min\sum_{i}1/\lambda_{i} under each scalar constraint; in every case the minimum is attained when all λ i \lambda_{i} are equal, i.e., Σ = s ​ I d \Sigma=sI_{d} .

##### (a) Trace constraint.

Given tr ⁡ ( Σ ) = ∑ i λ i = t > 0 \mathrm{tr}(\Sigma)=\sum_{i}\lambda_{i}=t>0 , by Cauchy–Schwarz, ( ∑ i = 1 d 1 λ i ) ​ ( ∑ i = 1 d λ i ) ≥ ( ∑ i = 1 d 1 ) 2 = d 2 , \left(\sum_{i=1}^{d}\frac{1}{\lambda_{i}}\right)\left(\sum_{i=1}^{d}\lambda_{i}\right)\;\geq\;\left(\sum_{i=1}^{d}1\right)^{2}=d^{2}, with equality if and only if λ 1 = ⋯ = λ d \lambda_{1}=\cdots=\lambda_{d} . Hence min Σ ≻ 0 : tr ⁡ ( Σ ) = t tr ( Σ − 1 ) = d 2 t , attained at Σ = t d I d . \min_{\Sigma\succ 0:\ \mathrm{tr}(\Sigma)=t}\ \mathrm{tr}(\Sigma^{-1})\;=\;\frac{d^{2}}{t},\quad\text{attained at}\quad\Sigma=\frac{t}{d}I_{d}.

##### (b) Determinant constraint.

Given det ( Σ ) = ∏ i λ i = δ > 0 \det(\Sigma)=\prod_{i}\lambda_{i}=\delta>0 , set μ i := 1 / λ i \mu_{i}:=1/\lambda_{i} so that ∏ i μ i = δ − 1 \prod_{i}\mu_{i}=\delta^{-1} . By the AM–GM inequality, 1 d ∑ i = 1 d μ i ≥ ( ∏ i = 1 d μ i ) 1 / d = δ − 1 / d , \frac{1}{d}\sum_{i=1}^{d}\mu_{i}\;\geq\;\left(\prod_{i=1}^{d}\mu_{i}\right)^{1/d}=\delta^{-1/d}, with equality iff μ 1 = ⋯ = μ d \mu_{1}=\cdots=\mu_{d} , i.e., λ 1 = ⋯ = λ d \lambda_{1}=\cdots=\lambda_{d} . Thus min Σ ≻ 0 : det ( Σ ) = δ tr ( Σ − 1 ) = d δ − 1 / d , attained at Σ = δ 1 / d I d . \min_{\Sigma\succ 0:\ \det(\Sigma)=\delta}\ \mathrm{tr}(\Sigma^{-1})\;=\;d\delta^{-1/d},\quad\text{attained at}\quad\Sigma=\delta^{1/d}I_{d}.

##### (c) Frobenius-norm constraint.

Given ‖ Σ ‖ F 2 = ∑ i λ i 2 = c 2 > 0 \|\Sigma\|_{F}^{2}=\sum_{i}\lambda_{i}^{2}=c^{2}>0 , minimize f ⁡ ( λ ) := ∑ i 1 / λ i f(\lambda):=\sum_{i}1/\lambda_{i} over λ i > 0 \lambda_{i}>0 subject to g ⁡ ( λ ) := ∑ i λ i 2 = c 2 g(\lambda):=\sum_{i}\lambda_{i}^{2}=c^{2} . The Lagrangian ℒ ⁡ ( λ , ν ) = ∑ i = 1 d 1 λ i + ν ⁡ ( ∑ i = 1 d λ i 2 − c 2 ) \mathcal{L}(\lambda,\nu)=\sum_{i=1}^{d}\frac{1}{\lambda_{i}}+\nu\left(\sum_{i=1}^{d}\lambda_{i}^{2}-c^{2}\right) has first-order conditions − λ i − 2 + 2 ​ ν ​ λ i = 0 -\lambda_{i}^{-2}+2\nu\lambda_{i}=0 for all i i , i.e., λ i 3 = 1 2 ​ ν \lambda_{i}^{3}=\frac{1}{2\nu} , so all λ i \lambda_{i} are equal. Imposing ∑ λ i 2 = c 2 \sum\lambda_{i}^{2}=c^{2} yields λ i = c / d \lambda_{i}=c/\sqrt{d} , hence min Σ ≻ 0 : ‖ Σ ‖ F = c tr ( Σ − 1 ) = ∑ i = 1 d 1 λ i = d 3 / 2 c , attained at Σ = c d I d . \min_{\Sigma\succ 0:\ \|\Sigma\|_{F}=c}\ \mathrm{tr}(\Sigma^{-1})\;=\;\sum_{i=1}^{d}\frac{1}{\lambda_{i}}\;=\;\frac{d^{3/2}}{c},\quad\text{attained at}\quad\Sigma=\frac{c}{\sqrt{d}}I_{d}.

##### (d) Spectral-radius constraint.