Parametrically Retargetable Decision-Makers Tend To Seek Power — part 6/13
Cleaned-text code points [77415, 92906). Bibliography omitted here; source.md includes it.
For 𝐄 \mathbf{E} which is a subset of a set acted on by S d S_{d} , let f : 𝐄 × ℝ d → ℝ f\mathrel{\mathop{\ordinarycolon}}\mathbf{E}\times\mathbb{R}^{d}\to\mathbb{R} be a bounded function which is measurable on its second argument, and let P ⊆ S d P\subseteq S_{d} . Then if f ( X ∣ 𝐮 ) f(X\mid\mathbf{u}) is increasing under joint permutation by P P , then f ′ ( X ∣ 𝒟 any ) ≔ 𝔼 𝐮 ∼ 𝒟 any [ f ( X ∣ 𝐮 ) ] f^{\prime}(X\mid\mathcal{D}_{\text{any}})\coloneqq\opE_{\mathbf{u}\sim\mathcal{D}_{\text{any}}}\left[f(X\mid\mathbf{u})\right] is increasing under joint permutation by P P . If f f is invariant under joint permutation by P P , then so is f ′ f^{\prime} .
###### Proof.
Let distribution 𝒟 any \mathcal{D}_{\text{any}} have probability measure F F , and let ϕ ⋅ 𝒟 any \phi\cdot\mathcal{D}_{\text{any}} have probability measure F ϕ F_{\phi} . f ( X ∣ 𝒟 any ) ≔ \displaystyle f\left(X\mid\mathcal{D}_{\text{any}}\right)\coloneqq{} 𝔼 𝐮 ∼ 𝒟 any [ f ( X ∣ 𝐮 ) ] \displaystyle\opE_{\mathbf{u}\sim\mathcal{D}_{\text{any}}}\left[f(X\mid\mathbf{u})\right] (10) ≔ \displaystyle\coloneqq{} ∫ ℝ d f ( X ∣ 𝐮 ) d F ( 𝐮 ) \displaystyle\int_{\mathbb{R}^{d}}f(X\mid\mathbf{u})\dif F(\mathbf{u}) (11) ≤ \displaystyle\leq{} ∫ ℝ d f ( ϕ ⋅ X ∣ 𝐏 ϕ 𝐮 ) d F ( 𝐮 ) \displaystyle\int_{\mathbb{R}^{d}}f(\phi\cdot X\mid\mathbf{P}_{\phi}\mathbf{u})\dif F(\mathbf{u}) (12) = \displaystyle={} ∫ ℝ d f ( ϕ ⋅ X ∣ 𝐮 ′ ) | det 𝐏 ϕ | d F ϕ ( 𝐮 ′ ) \displaystyle\int_{\mathbb{R}^{d}}f(\phi\cdot X\mid\mathbf{u}^{\prime})\left|\det\mathbf{P}_{\phi}\right|\dif F_{\phi}(\mathbf{u}^{\prime}) (13) = \displaystyle={} ∫ ℝ d f ( ϕ ⋅ X ∣ 𝐮 ′ ) d F ϕ ( 𝐮 ′ ) \displaystyle\int_{\mathbb{R}^{d}}f(\phi\cdot X\mid\mathbf{u}^{\prime})\dif F_{\phi}(\mathbf{u}^{\prime}) (14) ≕ \displaystyle\eqqcolon{} f ′ ( ϕ ⋅ X ∣ ϕ ⋅ 𝒟 any ) . \displaystyle f^{\prime}\left(\phi\cdot X\mid\phi\cdot\mathcal{D}_{\text{any}}\right). (15)
Equation 12 holds by assumption on f f : f ( X ∣ 𝐮 ) ≤ f ( ϕ ⋅ X ∣ 𝐏 ϕ 𝐮 ) f(X\mid\mathbf{u})\leq f(\phi\cdot X\mid\mathbf{P}_{\phi}\mathbf{u}) . Furthermore, f ( ϕ ⋅ X ∣ ⋅ ) f(\phi\cdot X\mid\cdot) is still measurable, and so the inequality holds. Equation 13 follows by the definition of F ϕ F_{\phi} (definition 6.3) and by substituting 𝐫 ′ ≔ 𝐏 ϕ 𝐫 \mathbf{r}^{\prime}\coloneqq\mathbf{P}_{\phi}\mathbf{r} . Equation 14 follows from the fact that all permutation matrices have unitary determinant. ∎
###### Lemma B.6 (Closure of orbit incentives under increasing functions) .
Suppose that S d S_{d} acts on sets 𝐄 1 , … , 𝐄 m \mathbf{E}_{1},\ldots,\mathbf{E}_{m} (with 𝐄 1 \mathbf{E}_{1} being a poset), and let P ⊆ S d P\subseteq S_{d} . Let f 1 , … , f n : ∏ i = 1 m 𝐄 i → ℝ f_{1},\ldots,f_{n}\mathrel{\mathop{\ordinarycolon}}\prod_{i=1}^{m}\mathbf{E}_{i}\to\mathbb{R} be increasing under joint permutation by P P on input ( X 1 , … , X m ) (X_{1},\ldots,X_{m}) , and suppose the f i f_{i} are order-preserving with respect to ⪯ 𝐄 1 \preceq_{\mathbf{E}_{1}} . Let g : ∏ j = 1 n ℝ → ℝ g\mathrel{\mathop{\ordinarycolon}}\prod_{j=1}^{n}\mathbb{R}\to\mathbb{R} be monotonically increasing on each argument. Then f ( X 1 , … , X m ) ≔ g ( f 1 ( X 1 , … , X m ) , … , f n ( X 1 , … , X m ) ) f\left(X_{1},\ldots,X_{m}\right)\coloneqq g\left(f_{1}\left(X_{1},\ldots,X_{m}\right),\ldots,f_{n}\left(X_{1},\ldots,X_{m}\right)\right) (16) is increasing under joint permutation by P P and order-preserving with respect to set inclusion on its first argument. Furthermore, if the f i f_{i} are invariant under joint permutation by P P , then so is f f .
###### Proof.
Let ϕ ∈ P \phi\in P . f ( X 1 , … , X m ) \displaystyle f\left(X_{1},\ldots,X_{m}\right) ≔ g ( f 1 ( X 1 , … , X m ) , … , f n ( X 1 , … , X m ) ) \displaystyle\coloneqq g\left(f_{1}\left(X_{1},\ldots,X_{m}\right),\ldots,f_{n}\left(X_{1},\ldots,X_{m}\right)\right) (17) ≤ g ( f 1 ( ϕ ⋅ X 1 , … , ϕ ⋅ X m ) , … , f n ( ϕ ⋅ X 1 , … , ϕ ⋅ X m ) ) \displaystyle\leq g\left(f_{1}\left(\phi\cdot X_{1},\ldots,\phi\cdot X_{m}\right),\ldots,f_{n}\left(\phi\cdot X_{1},\ldots,\phi\cdot X_{m}\right)\right) (18) ≕ f ( ϕ ⋅ X 1 , … , ϕ ⋅ X m ) . \displaystyle\eqqcolon f\left(\phi\cdot X_{1},\ldots,\phi\cdot X_{m}\right). (19) Equation 18 follows because we assumed that f i ( X 1 , … , X m ) ≤ f i ( ϕ ⋅ X 1 , … , ϕ ⋅ X m ) f_{i}\left(X_{1},\ldots,X_{m}\right)\leq f_{i}\left(\phi\cdot X_{1},\ldots,\phi\cdot X_{m}\right) , and because g g is monotonically increasing on each argument. If the f i f_{i} are all invariant, then eq. 18 is an equality.
Similarly, suppose X 1 ′ ⪯ 𝐄 1 X 1 X_{1}^{\prime}\preceq_{\mathbf{E}_{1}}X_{1} . The f i f_{i} are order-preserving on the first argument, and g g is monotonically increasing on each argument. Then f ( X 1 ′ , … , X m ) ≤ f ( X 1 , … , X m ) f\left(X_{1}^{\prime},\ldots,X_{m}\right)\leq f\left(X_{1},\ldots,X_{m}\right) . This shows that f f is order-preserving on its first argument. ∎
###### Remark .
g g could take the convex combination of its arguments, or multiply two f i f_{i} together and add them to a third f 3 f_{3} .
See 3.5 See 3.6
###### Proof.
Let θ ∈ Θ \theta\in\Theta , and let ϕ i ⋅ Orbit | Θ , A > B ( θ ) ≔ { ϕ i ⋅ θ A ∣ θ A ∈ Orbit | Θ , A > B ( θ ) } \phi_{i}\cdot\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right)\coloneqq\left\{\phi_{i}\cdot\theta^{A}\mid\theta^{A}\in\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right)\right\} . | Orbit | Θ , B > A ( θ ) ] | \displaystyle\left|\mathrm{Orbit}|_{\Theta,B>A}\left(\theta\right)]\right| ≥ | ⋃ i = 1 n ϕ i ⋅ Orbit | Θ , A > B ( θ ) | \displaystyle\geq\left|\bigcup_{i=1}^{n}\phi_{i}\cdot\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right)\right| (20) = ∑ i = 1 n | ϕ i ⋅ Orbit | Θ , A > B ( θ ) | \displaystyle=\sum_{i=1}^{n}\left|\phi_{i}\cdot\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right)\right| (21) = n | Orbit | Θ , A > B ( θ ) | . \displaystyle=n\left|\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right)\right|. (22) By item 1 and item 2 , ϕ i ⋅ ϕ i ⋅ Orbit | Θ , A > B ( θ ) ⊆ ϕ i ⋅ Orbit | Θ , B > A ( θ ) ] \phi_{i}\cdot\phi_{i}\cdot\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right)\subseteq\phi_{i}\cdot\mathrm{Orbit}|_{\Theta,B>A}\left(\theta\right)] for all i i . Therefore, eq. 20 holds. Equation 21 follows by the assumption that parameters are distinct, and so therefore the cosets ϕ i ⋅ Orbit | Θ , A > B ( θ ) \phi_{i}\cdot\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right) and ϕ j ⋅ Orbit | Θ , A > B ( θ ) \phi_{j}\cdot\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right) are pairwise disjoint for i ≠ j i\neq j . Equation 22 follows because each ϕ i \phi_{i} acts injectively on orbit elements.
Letting f A ( θ ) ≔ f ( A ∣ θ ) f_{A}(\theta)\coloneqq f(A\mid\theta) and f B ( θ ) ≔ f ( B ∣ θ ) f_{B}(\theta)\coloneqq f(B\mid\theta) , the shown inequality satisfies definition 3.2 . We conclude that f ( B ∣ θ ) ≥ most : Θ n f ( A ∣ θ ) f(B\mid\theta)\geq_{\text{{most}}\text{: }\Theta}^{n}f(A\mid\theta) . ∎
See 3.3
See 3.4
###### Proof.
Given that f f is a ( Θ , A → simple B ) (\Theta,A\overset{\text{simple}}{\to}B) -retargetable function ( definition 3.3 ), we want to show that f f is a ( Θ , A → 1 B ) (\Theta,A\overset{1}{\to}B) -retargetable function ( definition 3.5 when n = 1 n=1 ). Definition 3.5 ’s item 1 is true by assumption. Since Θ \Theta is acted on by S d S_{d} , Θ \Theta is closed under permutation and so definition 3.5 ’s item 2 holds. When n = 1 n=1 , there are no i ≠ j i\neq j , and so definition 3.5 ’s item 3 is tautologically true.
Then f f is a ( Θ , A → 1 B ) (\Theta,A\overset{1}{\to}B) -retargetable function; apply Lemma B.7 . ∎
### B.2 Helper results on retargetable functions
###### Lemma B.7 (Quantitative general orbit lemma) .
Let Θ \Theta be a subset of a set acted on by S d S_{d} , and let f : 𝐄 × Θ → ℝ f\mathrel{\mathop{\ordinarycolon}}\mathbf{E}\times\Theta\to\mathbb{R} . Consider A , B ∈ 𝐄 A,B\in\mathbf{E} .
For each θ ∈ Θ \theta\in\Theta , choose involutions ϕ 1 , … , ϕ n ∈ S d \phi_{1},\ldots,\phi_{n}\in S_{d} . Let θ ∗ ∈ Orbit | Θ ( θ ) \theta^{*}\in\mathrm{Orbit}|_{\Theta}\left(\theta\right) . 1. Retargetable under parameter permutation. There exist B i ⋆ ∈ 𝐄 B_{i}^{\star}\in\mathbf{E} such that if f ( B ∣ θ ∗ ) < f ( A ∣ θ ∗ ) f(B\mid\theta^{*})<f(A\mid\theta^{*}) , then ∀ i : f ( A ∣ θ ∗ ) ≤ f ( B i ⋆ ∣ ϕ i ⋅ θ ∗ ) \forall i\mathrel{\mathop{\ordinarycolon}}f\left(A\mid\theta^{*}\right)\leq f\left(B^{\star}_{i}\mid\phi_{i}\cdot\theta^{*}\right) .
2. Θ \Theta is closed under certain symmetries. f ( B ∣ θ ∗ ) < f ( A ∣ θ ∗ ) ⟹ ∀ i : ϕ i ⋅ θ ∗ ∈ Θ f(B\mid\theta^{*})<f(A\mid\theta^{*})\implies\forall i\mathrel{\mathop{\ordinarycolon}}\phi_{i}\cdot\theta^{*}\in\Theta .
3. f f is increasing on certain inputs. ∀ i : f ( B i ⋆ ∣ θ ∗ ) ≤ f ( B ∣ θ ∗ ) \forall i\mathrel{\mathop{\ordinarycolon}}f(B_{i}^{\star}\mid\theta^{*})\leq f(B\mid\theta^{*}) .
4. Increasing under alternate symmetries. For j = 1 , … , n j=1,\ldots,n and i ≠ j i\neq j , if f ( A ∣ θ ∗ ) < f ( B ∣ θ ∗ ) f(A\mid\theta^{*})<f(B\mid\theta^{*}) , then f ( B j ⋆ ∣ θ ∗ ) ≤ f ( B j ⋆ ∣ ϕ i ⋅ θ ∗ ) f\left(B_{j}^{\star}\mid\theta^{*}\right)\leq f\left(B_{j}^{\star}\mid\phi_{i}\cdot\theta^{*}\right) .
If these conditions hold for all θ ∈ Θ \theta\in\Theta , then
###### Proof.
Let θ \theta and θ ∗ \theta^{*} be as described in the assumptions, and let i ∈ { 1 , … , n } i\in\left\{1,\ldots,n\right\} . f ( A ∣ ϕ i ⋅ θ ∗ ) \displaystyle f(A\mid\phi_{i}\cdot\theta^{*}) = f ( A ∣ ϕ i − 1 ⋅ θ ∗ ) \displaystyle=f(A\mid\phi_{i}^{-1}\cdot\theta^{*}) (24) ≤ f ( B i ⋆ ∣ θ ∗ ) \displaystyle\leq f(B_{i}^{\star}\mid\theta^{*}) (25) ≤ f ( B ∣ θ ∗ ) \displaystyle\leq f(B\mid\theta^{*}) (26) < f ( A ∣ θ ∗ ) \displaystyle<f(A\mid\theta^{*}) (27) ≤ f ( B i ⋆ ∣ ϕ i ⋅ θ ∗ ) \displaystyle\leq f(B_{i}^{\star}\mid\phi_{i}\cdot\theta^{*}) (28) ≤ f ( B ∣ ϕ i ⋅ θ ∗ ) . \displaystyle\leq f(B\mid\phi_{i}\cdot\theta^{*}). (29) Equation 24 follows because ϕ i \phi_{i} is an involution. Equation 25 and eq. 28 follow by item 1 . Equation 26 and eq. 29 follow by item 3 . Equation 27 holds by assumption on θ ∗ \theta^{*} . Then eq. 29 shows that for any i i , f ( A ∣ ϕ i ⋅ θ ∗ ) < f ( B ∣ ϕ i ⋅ θ ∗ ) f(A\mid\phi_{i}\cdot\theta^{*})<f(B\mid\phi_{i}\cdot\theta^{*}) , satisfying definition 3.5 ’s item 1 .
This result’s item 2 satisfies definition 3.5 ’s item 2 . We now just need to show definition 3.5 ’s item 3 .
##### Disjointness.
Let θ ′ , θ ′′ ∈ Orbit | Θ , A > B ( θ ) \theta^{\prime},\theta^{\prime\prime}\in\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right) and let i ≠ j i\neq j . Suppose ϕ i ⋅ θ ′ = ϕ j ⋅ θ ′′ \phi_{i}\cdot\theta^{\prime}=\phi_{j}\cdot\theta^{\prime\prime} . We want to show that this leads to contradiction. f ( A ∣ θ ′′ ) \displaystyle f(A\mid\theta^{\prime\prime}) ≤ f ( B j ⋆ ∣ ϕ j ⋅ θ ′′ ) \displaystyle\leq f(B_{j}^{\star}\mid\phi_{j}\cdot\theta^{\prime\prime}) (30) = f ( B j ⋆ ∣ ϕ i − 1 ⋅ θ ′ ) \displaystyle=f(B_{j}^{\star}\mid\phi_{i}^{-1}\cdot\theta^{\prime}) (31) ≤ f ( B j ⋆ ∣ θ ′ ) \displaystyle\leq f(B_{j}^{\star}\mid\theta^{\prime}) (32) ≤ f ( B ∣ θ ′ ) \displaystyle\leq f(B\mid\theta^{\prime}) (33) < f ( A ∣ θ ′ ) \displaystyle<f(A\mid\theta^{\prime}) (34) ≤ f ( B i ⋆ ∣ ϕ i ⋅ θ ′ ) \displaystyle\leq f(B_{i}^{\star}\mid\phi_{i}\cdot\theta^{\prime}) (35) = f ( B i ⋆ ∣ ϕ j − 1 ⋅ θ ′′ ) \displaystyle=f(B_{i}^{\star}\mid\phi_{j}^{-1}\cdot\theta^{\prime\prime}) (36) ≤ f ( B i ⋆ ∣ θ ′′ ) \displaystyle\leq f(B_{i}^{\star}\mid\theta^{\prime\prime}) (37) ≤ f ( B ∣ θ ′′ ) \displaystyle\leq f(B\mid\theta^{\prime\prime}) (38) < f ( A ∣ θ ′′ ) . \displaystyle<f(A\mid\theta^{\prime\prime}). (39) Equation 30 follows by our assumption of item 1 . Equation 31 holds because we assumed that ϕ j ⋅ θ ′′ = ϕ i ⋅ θ ′ \phi_{j}\cdot\theta^{\prime\prime}=\phi_{i}\cdot\theta^{\prime} , and the involution ensures that ϕ i = ϕ i − 1 \phi_{i}=\phi_{i}^{-1} . Equation 32 is guaranteed by our assumption of item 4 , given that ϕ i − 1 ⋅ θ ′ = ϕ i ⋅ θ ′ ∈ Orbit | Θ , B > A ( θ ) ] \phi_{i}^{-1}\cdot\theta^{\prime}=\phi_{i}\cdot\theta^{\prime}\in\mathrm{Orbit}|_{\Theta,B>A}\left(\theta\right)] by the first half of this proof. Equation 33 follows by our assumption of item 3 . Equation 34 follows because we assumed that θ ′ ∈ Orbit | Θ , A > B ( θ ) \theta^{\prime}\in\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right) .
Equation 35 through eq. 39 follow by the same reasoning, switching the roles of θ ′ \theta^{\prime} and θ ′′ \theta^{\prime\prime} , and of i i and j j . But then we have demonstrated that a quantity is strictly less than itself, a contradiction. So for all θ ′ , θ ′′ ∈ Orbit | Θ , A > B ( θ ) \theta^{\prime},\theta^{\prime\prime}\in\mathrm{Orbit}|_{\Theta,A>B}\left(\theta\right) , when i ≠ j i\neq j , ϕ i ⋅ θ ′ ≠ ϕ j ⋅ θ ′′ \phi_{i}\cdot\theta^{\prime}\neq\phi_{j}\cdot\theta^{\prime\prime} .
Therefore, we have shown definition 3.5 ’s item 3 , and so f f is a ( Θ , A → 𝑛 B ) (\Theta,A\overset{n}{\to}B) -retargetable function. Apply Theorem 3.6 in order to conclude that eq. 23 holds. ∎
###### Definition B.8 (Superset-of-copy containment) .
Let A , B ⊆ ℝ d A,B\subseteq\mathbb{R}^{d} . B B contains n n superset-copies B i ⋆ B_{i}^{\star} of A A when there exist involutions ϕ 1 , … , ϕ n \phi_{1},\ldots,\phi_{n} such that ϕ i ⋅ A ⊆ B i ⋆ ⊆ B \phi_{i}\cdot A\subseteq B_{i}^{\star}\subseteq B , and whenever i ≠ j i\neq j , ϕ i ⋅ B j ⋆ = B j ⋆ \phi_{i}\cdot B_{j}^{\star}=B_{j}^{\star} .
###### Lemma B.9 (Looser sufficient conditions for orbit-level incentives) .
Suppose that Θ \Theta is a subset of a set acted on by S d S_{d} and is closed under permutation by S d S_{d} . Let A , B ∈ 𝐄 ⊆ 𝒫 ( ℝ d ) A,B\in\mathbf{E}\subseteq\mathcal{P}\left(\mathbb{R}^{d}\right) . Suppose that B B contains n n superset-copies B i ⋆ ∈ 𝐄 B_{i}^{\star}\in\mathbf{E} of A A via ϕ i \phi_{i} . Suppose that f ( X ∣ θ ) f(X\mid\theta) is increasing under joint permutation by ϕ 1 , … , ϕ n ∈ S d \phi_{1},\ldots,\phi_{n}\in S_{d} for all X ∈ 𝐄 , θ ∈ Θ X\in\mathbf{E},\theta\in\Theta , and suppose that ∀ i : ϕ i ⋅ A ∈ 𝐄 \forall i\mathrel{\mathop{\ordinarycolon}}\phi_{i}\cdot A\in\mathbf{E} . Suppose that f f is monotonically increasing on its first argument. Then f ( B ∣ θ ) ≥ most : Θ n f ( A ∣ θ ) . f(B\mid\theta)\geq_{\text{{most}}\text{: }\Theta}^{n}f(A\mid\theta).
###### Proof.
We check the conditions of Lemma B.7 . Let θ ∈ Θ \theta\in\Theta , and let θ ∗ ∈ ( S d ⋅ θ ) ∩ Θ \theta^{*}\in\left(S_{d}\cdot\theta\right)\cap\Theta be an orbit element. Item 1 . Holds since f ( A ∣ θ ∗ ) ≤ f ( ϕ i ⋅ A ∣ ϕ i ⋅ θ ∗ ) ≤ f ( B i ⋆ ∣ ϕ i ⋅ θ ∗ ) f(A\mid\theta^{*})\leq f(\phi_{i}\cdot A\mid\phi_{i}\cdot\theta^{*})\leq f(B^{\star}_{i}\mid\phi_{i}\cdot\theta^{*}) , with the first inequality by assumption of joint increasing under permutation, and the second following from monotonicity (as ϕ i ⋅ A ⊆ B i ⋆ \phi_{i}\cdot A\subseteq B^{\star}_{i} by superset copy definition B.8 ).
Item 2 . We have ∀ θ ∗ ∈ ( S d ⋅ θ ∗ ) ∩ Θ : f ( B ∣ θ ∗ ) < f ( A ∣ θ ∗ ) ⟹ ∀ i = 1 , … , n : ϕ i ⋅ θ ∗ ∈ Θ \forall\theta^{*}\in\left(S_{d}\cdot\theta^{*}\right)\cap\Theta\mathrel{\mathop{\ordinarycolon}}f(B\mid\theta^{*})<f(A\mid\theta^{*})\implies\forall i=1,...,n\mathrel{\mathop{\ordinarycolon}}\phi_{i}\cdot\theta^{*}\in\Theta since Θ \Theta is closed under permutation.
Item 3 . Holds because we assumed that f f is monotonic on its first argument.