Neural spiking for causal inference and learning — part 5/5

Cleaned-text code points [61674, 65559). Bibliography omitted here; source.md includes it.

For the deep network ( Fig 5B ), a two-hidden layer neural network was simulated. Each layer had N = 10 neurons. The inputs to the first layer are as above: thus neural activity in this first layer is pairwise correlated with coefficient c . The second layer receives input from the first: thus noise in this layer is not correlated and any correlations between neural activity in the second layer come from correlated activity in the first layer. The matrix V weighs inputs from the first to second layer. It is generated from a normal distribution: . The same reward function is used as in the wide network simulations above, except here U is a vector of ones. The target is set to t = 0.02. To test the ability of spiking discontinuity learning to estimate causal effects in deep networks, the learning rule is used to estimate causal effects of neurons in the first layer on the reward function R .

## Supporting information

### S1 Text. Delayed XOR task description, methods and results.

https://doi.org/10.1371/journal.pcbi.1011005.s001

(PDF)

### S1 Fig. Error in causal effect estimation for LIF networks with different refractory periods.

As in Fig 3D , histograms plot error in causal effect over a range of network weights. LIF neurons have refractory period of 1,3 or 5 ms. Error is comparable for different refractory periods.

https://doi.org/10.1371/journal.pcbi.1011005.s002

(TIF)

### S2 Fig. One-sided discontinuity estimator.

The one-sided estimator makes updates for inputs placing the neuron p below threshold, and for any input that places the neuron above threshold. Statistically, the symmetric choice is the most sensible default. Regression discontinuity design, the related method in econometrics, has studied optimizing the underlying kernel, which may not be symmetric depending on the relevant distributions. To address this question, we ran extra simulations in which the window size is asymmetric. We tested, in particular, the case where p is some small value on the left of the threshold (sub-threshold inputs), and where p is large to the right of the threshold (above-threshold inputs). This deviates from the statistically correct choice, but in a way, this is a biologically plausible setting. This is because, with such a setup, the neuron does not need to distinguish between barely-above-threshold inputs and well-above-threshold inputs, which may be challenging. Instead, any spiking will result in an update to the estimate of the causal effect. We found that the asymmetric estimator performs worse when using the piecewise constant estimator of causal effect, but performs comparably to the symmetric version with using the piecewise linear estimator. Thus spiking discontinuity learning can operate using asymmetric update rules.

https://doi.org/10.1371/journal.pcbi.1011005.s003

(TIF)

### S3 Fig. Learning delayed XOR in the presence of correlated noise.

(A) Delayed XOR task setup, shown after training. Populations of input neurons sequentially encode binary inputs ( x 1 , x 2 ), and after a delay a population of neurons cues a response. Input value of 0 is indicated by the red population being active, and 1 is indicated by the blue population being active. Correlated Gaussian noise, with correlation coefficient c , is added to the neurons membrane potential. Softmax output above 0.5 indicates a network output of 1, and below 0.5 indicates 0. A sample raster of 20 neurons is shown here. (B) Mean validation error over 10 repeated training runs, for a range of correlation coefficients, c , and learning window sizes, p . (C) Mean number of iterations taken to reach training error below a stopping threshold of 0.07.

https://doi.org/10.1371/journal.pcbi.1011005.s004

(TIF)

## Acknowledgments

The authors would like to thank Roozbeh Farhoodi, Ari Benjamin and David Rolnick for valuable discussion and feedback.