On the Information Bottleneck Theory of Deep Learning (Saxe et al.) — part 2/6

Cleaned-text code points [14848, 29262). Bibliography omitted here; source.md includes it.

Published as a conference paper at ICLR 2018
-5 0 5
-1
-0.5
0
0.5
1
0 2 4 6 8 10
w1
0
1
2
3
4
5I(X;T)
0 2 4 6 8 10
w1
0
0.5
1
1.5
2
2.5I(X;T)
𝑥 ℎ
𝑓(⋅)
𝑦(
𝑤*
Net input (𝑤*𝑥)
Hidden activity ℎ
Continuous activity
Bin borders
A B
C D
Tanhnonlinearity ReLUnonlinearity
Figure 2: Nonlinear compression in a minimal model. (A) A three neuron nonlinear network which
receives Gaussian inputs x, multiplies by weight w1, and maps through neural nonlinearity f(·)
to produce hidden unit activityh. (B) The continuous activity h is binned into a discrete variable
T for the purpose of calculating mutual information. Blue: continuous tanh nonlinear activation
function. Grey: Bin borders for 30 bins evenly spaced between -1 and 1. Because of the saturation
in the sigmoid, a wide range of large magnitude net input values map to the same bin. (C) Mutual
information with the input as a function of weight size w1 for a tanh nonlinearity. Information
increases for small w1 and then decreases for large w1 as all inputs land in one of the two bins
corresponding to the saturation regions. (D) Mutual information with the input for the ReLU
nonlinearity increases without bound. Half of all inputs land in the bin corresponding to zero activity,
while the other half have information that scales with the size of the weights.
infinite (see Appendix C for extended discussion). Networks that include noise in their processing
(e.g., Kolchinsky et al. (2017)) can have finiteI(T ;X). Otherwise, to obtain a finite MI, one must
compute mutual information as though there were binning or added noise in the activations. But
this binning/noise is not actually a part of the operation of the network, and is therefore somewhat
arbitrary (different binning schemes can result in different mutual information with the input, as
shown in Fig. 14 of Appendix C).
We note that the binning procedure can be viewed as implicitly adding noise to the hidden layer
activity: a range of X values map to a single bin, such that the mapping between X andT is no
longer perfectly invertible (Laughlin, 1981). The binning procedure is therefore crucial to obtaining
a finite MI value, and corresponds approximately to a model where noise enters the system after
the calculation ofh, that is,T =h +ϵ, whereϵ is noise of fixed variance independent fromh and
X. This approach is common in information theoretic analyses of deterministic systems, and can
serve as a measure of the complexity of a system’s representation (see Sec 2.4 of Shwartz-Ziv &
Tishby (2017)). However, neither binning nor noise is present in the networks that Shwartz-Ziv &
Tishby (2017) considered, nor the ones in Fig. 2, either during training or testing. It therefore remains
unclear whether robustness of a representation to this sort of noise in fact influences generalization
performance in deep learning systems.
Furthermore, the addition of noise means that different architectures may no longer be compared in a
common currency of mutual information: the binning/noise structure is arbitrary, and architectures
that implement an identical input-output map can nevertheless have different robustness to noise added
in their internal representation. For instance, Appendix C describes a family of linear networks that
compute exactly the same input-output map and therefore generalize identically, but yield different
mutual information with respect to the input. Finally, we note that approaches which view the weights
obtained from the training process as the random variables of interest may sidestep this issue (Achille
& Soatto, 2017).
5

Published as a conference paper at ICLR 2018
TeacherA
B Student
𝑌"
𝑊$
𝑋
𝑊& … 𝑊(
ℎ$ ℎ& ℎ(*$ ℎ(
𝑊(+$
Depth 𝐷
…
𝑋
𝑌
𝜖
𝑊.
Train
Test
C
D
Figure 3: Generalization and information plane dynamics in deep linear networks. (A) A linear
teacher network generates a dataset by passing Gaussian inputsX through its weights and adding
noise. (B) A deep linear student network is trained on the dataset (here the network has 1 hidden
layer to allow comparison with Fig. 4A, see Supplementary Figure 18 for a deeper network). (C)
Training and testing error over time. (D) Information plane dynamics. No compression is observed.
Hence when a tanh network is initialized with small weights and over the course of training comes to
saturate its nonlinear units (as it must to compute most functions of practical interest, see discussion
in Appendix D), it will enter a compression period where mutual information decreases. Figures
16-17 of Appendix E show histograms of neural activity over the course of training, demonstrating
that activities in thetanh network enter the saturation regime during training. This nonlinearity-based
compression furnishes another explanation for the observation that training slows down as tanh
networks enter their compression phase (Shwartz-Ziv & Tishby, 2017): some fraction of inputs have
saturated the nonlinearities, reducing backpropagated error gradients.
3 I NFORMATION PLANE DYNAMICS IN DEEP LINEAR NETWORKS
The preceding section investigates the role of nonlinearity in the observed compression behavior,
tracing the source to double-saturating nonlinearities and the binning methodology used to calculate
mutual information. However, other mechanisms could lead to compression as well. Even without
nonlinearity, neurons could converge to highly correlated activations, or project out irrelevant direc-
tions of the input. These phenomena are not possible to observe in our simple three neuron minimal
model, as they require multiple inputs and hidden layer activities. To search for these mechanisms,
we turn to a tractable model system: deep linear neural networks (Baldi & Hornik (1989); Fukumizu
(1998); Saxe et al. (2014)). In particular, we exploit recent results on the generalization dynamics
in simple linear networks trained in a student-teacher setup (Seung et al., 1992; Advani & Saxe,
2017). In a student-teacher setting, one “student” neural network learns to approximate the output
of another “teacher” neural network. This setting is a way of generating a dataset with interesting
structure that nevertheless allows exact calculation of the generalization performance of the network,
exact calculation of the mutual information of the representation (without any binning procedure),
and, though we do not do so here, direct comparison to the IB bound which is already known for
linear Gaussian problems (Chechik et al., 2005).
We consider a scenario where a linear teacher neural network generates input and output examples
which are then fed to a deep linear student network to learn (Fig. 3A). Following the formulation
of (Advani & Saxe, 2017), we assume multivariate Gaussian inputsX∼N (0, 1
Ni
INi) and a scalar
outputY . The output is generated by the teacher network according to Y = W0X +ϵo, where
ϵo∼N (0,σ 2
o) represents aspects of the target function which cannot be represented by a neural
network (that is, the approximation error or bias in statistical learning theory), and the teacher weights
Wo are drawn independently fromN (0,σ 2
w). Here, the weights of the teacher define the rule to be
learned. The signal to noise ratio SNR =σ2
w/σ2
o determines the strength of the rule linking inputs to
6

Published as a conference paper at ICLR 2018
outputs relative to the inevitable approximation error. We emphasize that the “noise” added to the
teacher’s output is fundamentally different from the noise added for the purpose of calculating mutual
information:ϵo models the approximation error for the task–even the best possible neural network
may still make errors because the target function is not representable exactly as a neural network–and
is part of the construction of the dataset, not part of the analysis of the student network.
To train the student network, a dataset of P examples is generated using the teacher. The student
network is then trained to minimize the mean squared error between its output and the target output
using standard (batch or stochastic) gradient descent on this dataset. Here the student is a deep
linear neural network consisting of potentially many layers, but where the the activation function
of each neuron is simply f(u) = u. That is, a depth D deep linear network computes the output
ˆY =WD+1WD··· W2W1X. While linear activation functions stop the network from computing
complex nonlinear functions of the input, deep linear networks nevertheless show complicated
nonlinear learning trajectories (Saxe et al., 2014), the optimization problem remains nonconvex (Baldi
& Hornik, 1989), and the generalization dynamics can exhibit substantial overtraining (Fukumizu,
1998; Advani & Saxe, 2017).
Importantly, because of the simplified setting considered here, the true generalization error is easily
shown to be
Eg(t) =||Wo−Wtot(t)||2
F +σ2
o (5)
whereWtot(t) is the overall linear map implemented by the network at training epoch t (that is,
Wtot =WD+1WD··· W2W1).
Furthermore, the mutual information with the input and output may be calculated exactly, because
the distribution of the activity of any hidden layer is Gaussian. Let T be the activity of a specific
hidden layer, and let ¯W be the linear map from the input to this activity (that is, for layer l, ¯W =
Wl··· W2W1). Since T = ¯WX , the mutual information ofX andT calculated using differential
entropy is infinite. For the purpose of calculating the mutual information, therefore, we assume that
Gaussian noise is added to the hidden layer activity, T = ¯WX +ϵM I, with mean 0 and variance
σ2
M I = 1.0. This allows the analysis to apply to networks of any size, including overcomplete layers,
but as before we emphasize that we do not add this noise either during training or testing. With these
assumptions,T andX are jointly Gaussian and we have
I(T ;X) = log| ¯W ¯W T +σ2
M IINh|− log|σ2
M IINh| (6)
where|·| denotes the determinant of a matrix. Finally the mutual information with the output Y , also
jointly Gaussian, can be calculated similarly (see Eqns. (22)-(25) of Appendix G).
Fig. 3 shows example training and test dynamics over the course of learning in panel C, and the
information plane dynamics in panel D. Here the network has an input layer of 100 units, 1 hidden
layer of 100 units each and one output unit. The network was trained with batch gradient descent on a
dataset of 100 examples drawn from the teacher with signal to noise ratio of 1.0. The linear network
behaves qualitatively like the ReLU network, and does not exhibit compression. Nevertheless, it
learns a map that generalizes well on this task and shows minimal overtraining. Hence, in the setting
we study here, generalization performance can be acceptable without any compression phase.
The results in (Advani & Saxe (2017)) show that, for the case of linear networks, overtraining is
worst when the number of inputs matches the number of training samples, and is reduced by making
the number of samples smaller or larger. Fig. 4 shows learning dynamics with the number of samples
matched to the size of the network. Here overfitting is substantial, and again no compression is
seen in the information plane. Comparing to the result in Fig. 3D, both networks exhibit similar
information dynamics with respect to the input (no compression), but yield different generalization
performance.
Hence, in this linear analysis of a generic setting, there do not appear to be additional mechanisms that
cause compression over the course of learning; and generalization behavior can be widely different
for networks with the same dynamics of information compression regarding the input. We note that,
in the setting considered here, all input dimensions have the same variance, and the weights of the
teacher are drawn independently. Because of this, there are no special directions in the input, and
each subspace of the input contains as much information as any other. It is possible that, in real
world tasks, higher variance inputs are also the most likely to be relevant to the task (here, have large
weights in the teacher). We have not investigated this possibility here.
7

Published as a conference paper at ICLR 2018
A B
Error
C D
Epochs
Figure 4: Overtraining and information plane dynamics. (A) Average training and test mean square
error for a deep linear network trained with SGD. Overtraining is substantial. Other parameters:Ni =
100, P = 100, Number of hidden units = 100, Batch size = 5 (B) Information plane dynamics. No
compression is observed, and information about the labels is lost during overtraining. (C) Average
train and test accuracy (% correct) for nonlineartanh networks exhibiting modest overfitting (N = 8).
(D) Information plane dynamics. Overfitting occurs despite continued compression.
Figure 5: Stochastic training and the information plane. (A)tanh network trained with SGD. (B)tanh
network trained with BGD. (C) ReLU network trained with SGD. (D) ReLU network trained with
BGD. Both random and non-random training procedures show similar information plane dynamics.
To see whether similar behavior arises in nonlinear networks, we trained tanh networks in the same
setting as Section 2, but with 30% of the data, which we found to lead to modest overtraining.
Fig. 4C-D shows the resulting train, test, and information plane dynamics. Here the tanh networks
show substantial compression, despite exhibiting overtraining. This establishes a dissociation between
behavior in the information plane and generalization dynamics: networks that compress may (Fig. 1A)
or may not (Fig. 4C-D) generalize well, and networks that do not compress may (Figs.1B, 3A-B) or
may not (Fig. 4A-B) generalize well.
4 C OMPRESSION IN BATCH GRADIENT DESCENT AND SGD
Next, we test a core theoretical claim of the information bottleneck theory of deep learning, namely
that randomness in stochastic gradient descent is responsible for the compression phase. In particular,
because the choice of input samples in SGD is random, the weights evolve in a stochastic way during
training.
Shwartz-Ziv & Tishby (2017) distinguish two phases of SGD optimization: in the first “drift” phase,
the mean of the gradients over training samples is large relative to the standard deviation of the
gradients; in the second “diffusion” phase, the mean becomes smaller than the standard deviation
of the gradients. The authors propose that compression should commence following the transition
8