Theory of nonstationary Hawkes processes

We expand the theory of Hawkes processes to the nonstationary case, in which the mutually exciting point processes receive time-dependent inputs. We derive an analytical expression for the time-dependent correlations, which can be applied to networks with arbitrary connectivity, and inputs with arbitrary statistics. The expression shows how the network correlations are determined by the interplay between the network topology, the transfer functions relating units within the network, and the pattern and statistics of the external inputs. We illustrate the correlation structure using several examples in which neural network dynamics are modeled as a Hawkes process. In particular, we focus on the interplay between internally and externally generated oscillations and their signatures in the spike and rate correlation functions.

Figure 1

Network structure and correlation function. (a) Network structure: a recurrent network with arbitrary connectivity, where each neuron receives a time-dependent external input. (b) The spike correlation structure in the time domain [Eq. (14)]. The external input affects the firing rate of all neurons via the propagator $G$. The firing rate of each one of the neurons, $〈{\lambda }_k\left(t^{\prime \prime }\right)〉$, affects the joint spiking activity of neurons $j$ and $i$ via the propagators $G_{jk}$ and $G_{ik}$. (c) The instantaneous firing rate correlation in the time domain [see Eqs. (14) and (16)]. The structure is similar to the spike correlation structure, with two additional convolutions with the synaptic current $J$.

Figure 2

Oscillatory external input. (a) The random connectivity matrix. Each element in the matrix represents the connection from the presynaptic neuron (horizontal) to the postsynaptic neuron (vertical). All the nonvanishing connections have equal magnitude and time course: $J_{ij}\left(t\right)=(0.85/M)a\left(t\right)$, where $M=6$, $a\left(t\right)=a_0{\mathrm{e}}^{-t/{\tau }_1}\left(1-{\mathrm{e}}^{-t/{\tau }_2}\right)$, ${\tau }_1=2\mathrm{ms}$, and ${\tau }_2=1\mathrm{s}$. The amplitude $a_0$ normalizes the synaptic current such that ${\int }_0^{\infty }dta\left(t\right)=1$. Each neuron receives input from six other randomly chosen neurons. (b) The analytical expression for the spike correlation function of a pair of neurons as a function of $t$ and $t^{\prime }$ (random connectivity). The pair of synapses between the two neurons for which the correlation is calculated are marked by red circles in panel (a). The external input satisfies Eq. (17), where $A_0=8\mathrm{Hz}$, $A=4\mathrm{Hz}$, and ${\omega }_0/\left(2\pi \right)=20\mathrm{Hz}$. (c) Contribution of the time-dependent input to the correlation function as a function of $t$ and $t^{\prime }$ (random connectivity). Arrows represent time points $t^{\prime }$ for which $C(t^{\prime },t)$, seen as a function of $t-t^{\prime }$, is equal to Re $f(t-t^{\prime })$ (blue) and Im $f(t-t^{\prime })$ (red). (d) Real (blue) and imaginary (red) parts of $f^{{\omega }_0}(t-t^{\prime })$. (e)–(h) Same as (a)–(d) but for the clustered connectivity. The external input, synaptic current, and the synaptic strength of the nonzero connections are same as in (a).

Figure 3

The interplay between internally generated oscillations and an external oscillatory input. (a) Network architecture. Two populations of neurons inhibit each other via all-to-all synaptic connections. Each population comprises $N=200$ neurons. Each neuron inhibits all neurons in the other population with a synaptic current $J_{ij}=-\frac{0.9}{N}a\left(t\right)$. (b) Time course of the synaptic current $a\left(t>d\right)=a_0\exp \left(-\frac{t-d}{{\tau }_1}\right)\left[1-\exp \left(-\frac{t-d}{{\tau }_2}\right)\right]$, where $d=5\mathrm{ms}$, ${\tau }_1=2\mathrm{ms}$, ${\tau }_2=2\mathrm{ms}$, and $a_0$ normalizes $a$ such that ${\int }_0^{\infty }dta\left(t\right)=1$. (c) The analytical expression for the spike correlation as a function of $t-t^{\prime }$. The amplitude of the constant input is $A_0=150\mathrm{Hz}$. (d) Power spectrum of the LFP: analytical expression (black line), compared to results from stochastic simulations, averaged over 100 realizations (gray line). Inset: power spectrum in the frequency range 0–100 Hz.

Figure 4

Two inhibitory populations that receive an oscillatory input. (a) Same architecture as in Fig. 3 but with an external input that satisfies Eq. (17), with $A_0=96\mathrm{Hz}$, $A=12\mathrm{Hz}$, and ${\omega }_0/2\pi =15\mathrm{Hz}$. (b) The time-dependent spike correlation function $C(t,t^{\prime })$. (c) Two cuts through the $(t,t^{\prime })$ plane: $C\left(t,t^{\prime }\right)$ for $t=t^{\prime }$ [top, blue arrow in panel (b)] and for $t^{\prime }=0$ [bottom, black arrow in (b)]. (d)-(e) Real (d) and imaginary (e) parts of $f^{{\omega }_0}\left(t\right)$ for different input frequencies ${\omega }_0$. The dashed gray line denotes the input frequency used in panels (b)-(c), ${\omega }_0=15\mathrm{Hz}$. The black dotted line denotes ${\omega }^{*}=63.5\mathrm{Hz}$. (f) Shifted correlation of the LFP in the frequency domain, Eq. (27): analytical expression (black line), compared to results from stochastic simulations, averaged over 1000 realizations (gray line). Inset: Shifted correlation of the LFP in the frequency range 0–100 Hz.

Figure 5

Correlations in networks driven by stochastic inputs. (a) Contribution of the mean input to the spike correlation function, first term on the right-hand side of Eq. (31). (b) Contribution of the input correlation to the spike correlation function, second term on the right-hand side of Eq. (31).

Figure 6

Correlations in networks driven by stochastic inputs. (a)–(d) Response of a neural network, consisting of two mutually inhibiting populations, to temporally uncorrelated, spatially correlated inputs. Network parameters are the same as in Fig. 4. (a, b) Contribution of the input correlation to the power spectrum of the LFP, for stationary white noise inputs that are fully correlated (a) or fully anticorrelated (b) between the two populations. This contribution is given by $\frac{1}{T}\tilde{J}\left(\omega \right)\tilde{G}\left(\omega \right)\tilde{B}\left(\omega ,-\omega \right){\tilde{G}}^T\left(-\omega \right){\tilde{J}}^T\left(-\omega \right)$. (c, d) Contribution of the input correlations to the spike correlation function of two neurons from the same population when the white noise is transiently injected to the network, for the fully correlated (c) and fully anticorrelated cases (d), as obtained from the second term on the right-hand side of Eq. (29). The white noise is injected for a duration of 50 ms, starting at $t=5\mathrm{ms}$ (marked by a gray bar above each panel). The input correlations, at all times in panels (a)-(b) and during the duration of the pulse in (c)-(d), are given by $B_{ij}\left(t,t^{\prime }\right)={\overline{B}}_{ij}\delta \left(t-t^{\prime }\right)$, where in panels (a) and (c) ${\overline{B}}_{ij}=1\mathrm{Hz}$ and in panels (b) and (d) ${\overline{B}}_{ij}=1$ for neurons that belong to the same group, and ${\overline{B}}_{ij}=-1\mathrm{Hz}$ for neurons that do not belong to the same group.