Switching Operation Modes in the Neocortex via Cholinergic Neuromodulation

In order to deal with the uncertainty in the world, our brains need to be able to flexibly switch between the exploration of new sensory representations and exploitation of previously acquired ones. This requires forming accurate estimations of what and how much something is expected. While modeling has allowed for the development of several ways to form predictions, how the brain could implement those is still under debate. Here, we recognize acetylcholine as one of the main neuromodulators driving learning based on uncertainty, promoting the exploration of new sensory representations. We identify its interactions with cortical inhibitory interneurons and derive a biophysically grounded computational model able to capture and learn from uncertainty. This model allows us to understand inhibition beyond gain control by suggesting that different interneuron subtypes either encode predictions or estimate their uncertainty, facilitating detection of unexpected cues. Moreover, we show how acetylcholine-like neuromodulation uniquely interacts with global and local sources of inhibition, disrupting perceptual certainty and promoting the rapid acquisition of new perceptual cues. Altogether, our model proposes that cortical acetylcholine favors sensory exploration over exploitation in a cortical microcircuit dedicated to estimating sensory uncertainty.


Introduction
Cortical microcircuits are the canonical building blocks of computations in the neocortex [5,9,18]. These canonical circuits across the neocortex are highly plastic and adaptable. Hence, they display remarkable flexibility in expressing a repertoire of complex cortical computations, including those used in higher order functions such as perception, attention, and decision-making, in both, humans and animals. There is growing evidence that the broad organization of these functions across the neocortex has been preserved across mammals [39], and that the microcircuitry across brain regions is remarkably similar. The latter is due to the anatomical composition of the neocortex. At the macroscopic scale (cerebral), the neocortex is a layered tissue comprising three to six distinct cortical layers that differ significantly in thickness but are similar in cellular composition. At the mesoscopic scale (cellular), the neocortex is thought to consist of 80% excitatory principal cells and 20% inhibitory interneurons. Each of these cell types can be further classified with respect to morphology (basket cells, Martinotti cells), spiking dynamics (fast or regular spiking), or molecular interactions (responsive to somatostatin, parvalbumin, serotonin, etc.) into many subtypes that are generally preserved across different cortical areas. At the microscopic scale (molecular), cells in the neocortex have receptors for a number of neuromodulators (cholinergic, noradrenergic, dopaminergic, serotonergic) that are c o m p a r a b l e a cr o s s a r e a s r eg a r d i n g t he t y p e o f neuromodulators, despite the marked differences across functional areas [52]. Hence, these microcircuits comprise the core workforce of cortical computations in human and animal brains.
How do these microcircuits express such a diverse range of cortical functions? It has been observed in humans and demonstrated in primates that following lesions which eliminate a perceptual modality (e.g., visual, auditory, or tactile) or a region of perceptual space (e.g., involving an arm or an eye), neurons that would process that sensory information would rapidly be recruited to support surrounding sensory processes that would subsequently become more relevant for the animal's actions [12,19,33,42,49,63]. In a similar fashion, other animal studies have shown that the implantation of a third eye in a particular species of frogs results in the recruitment of a portion of neurons from the optic tectum. In particular, those neurons will engage in the processing of incoming inputs from the new eye [14]. In a similar way, deprivation of sensory input, such as vision, results in the compensation for this loss by using the visual neocortex for processing auditory stimuli, as shown in cats [63]. These findings highlight three crucial characteristics of microcircuits in the neocortex and, maybe, the rest of the brain. First, despite having remarkably diverse abilities, different animals use these cortical circuits to process similar types of information. Moreover, while different types of information would seem to require radically different processing schemes, the neocortex preserves relatively similar layered, cellular and molecular structures in spite of different functions they are supposed to perform. And finally, that any area of the neocortex can flexibly process different kinds of information when exposed to them. This suggests that its unique structural characteristics might be more descriptive of the modality in which it operates rather than a predefined function. The picture that emerges of the neocortex is not one built of rigid functionally independent areas, but rather a plastic tissue involved in information processing across a wide range of sensory and nonsensory modalities. In this case, similarities across cortical regions, rather than the differences, would be useful for understanding complex mechanisms of information-processing across the neocortex.
The aforementioned flexibility in processing sensory information requires specific mechanisms of plasticity and adaptation, most importantly, synaptic plasticity and inhibition, respectively. Synaptic plasticity provides a way to tune the parameters of a system. Through inhibition, these parameters adapt to different input features, like dynamic range, gain, or sampling frequency [32]. This is, synaptic plasticity and inhibition together adapt or construct models of incoming sensory inputs, which refer to models of the external world as sampled by the agent. While adapting or updating existing models would be equivalent to preserving existing features of sensory inputs with minor modifications, constructing new models would imply overriding previous model features in order to address a new sensory scenario. This can be thought of as a form of exploitation-exploration trade-off, where at times preservation of a model is preferable and otherwise rapid incorporation of new features benefits future actions. This suggests a third important circuit mechanism of cortical microcircuits that enables rapid switching between exploratory modes and exploitatory modes, namely, neuromodulation. Behaviorally, the role of neuromodulation in switching between exploratory and exploitatory behaviors has been studied in animals [13]. An example of an exploitatory behavior is reward seeking, whereas avoidance of an aversive stimulus constitutes an exploratory behavior. Computationally, reinforcement learning mechanisms have been used to model exploitatory behaviors [21]. Less is known about computational mechanisms of involving avoidance of aversive stimuli.
In this article, we investigate the role of neuromodulation in learning at the level of cortical microcircuits. Neuromodulation in the brain, unsurprisingly, is released in situations related to uncertainty. Dopamine is released when reward prediction errors are found [68]. Acetylcholine (Ach) is associated with cues predictive of expected aversive events or expected changes in task conditions [28,54,78], while norepinephrine is associated with unexpected changes in conditions of an experiment [17]. In this work, we will focus on understanding the role of Ach in signaling and learning from expected changes or expected uncertainty [2], as a potential candidate to drive sensory exploration. Neuromodulation can be used as a mechanism to switch between exploitatory and exploratory learning. Following this, a range of different neuromodulators potentially signal different epochs, which are relevant for different types of learning. An example is learning to obtain a reward through a specific action or learning to avoid future punishment. Having different neuromodulators involved in these two scenarios suggests distinct learning mechanisms.
Compared to Bayesian approaches seeking to model computational mechanisms of predictive coding in the brain [5,61], in this article, we take a much more biologically grounded route to understanding mechanisms of cortical computation. Our work culminates in a biophysical dynamical systems model of how cortical microcircuits learn to estimate uncertainty in sensory stimuli. With Bayesian models, there is no clear consensus on how they can be realized in a biologically detailed way. Moreover, to the best of our knowledge, current Bayesian approaches do not address how uncertainty can be estimating within the model itself. In this sense, there still remains an explanatory gap between Bayesian models of predictive coding and biophysically realistic neural dynamics.
Instead, here we look for computations arising from the biophysics of cortical microcircuits.
The outline of this paper is as follows. We first review the literature on biophysics of learning from surprising or uncertain events, focusing on the interplay of inhibitory networks and acetylcholine. We then derive a neural model to estimate uncertainty in the sensory domain. Finally, we apply biologically plausible constraints to its dynamics and discuss the role of inhibition and neuromodulation in the signaling of and learning from uncertainty. We propose that specific interneuron subtypes play an essential role in capturing sensory uncertainty in the neocortex and that neuromodulation (particularly via Ach) provides a switching mechanism in cortical dynamics to favor exploration of sensory cues over exploitation of existing models by modulation of an intricate disinhibitory network.

Biophysics of Learning and Uncertainty
The neural processes underlying learning mechanisms have been studied for decades. Through conditioning paradigms, we can probe different brain areas to understand their role in associating perception to action. One of the best-studied cases is eye-blink conditioning, where a rabbit receives an air puff in the eye preceded by an auditory stimulus. These experiments demonstrate a slow learning process that, after many exposures to the conditioning stimuli, translates into a well-timed eye-blink which avoids the aversive air puff.
In 1990, Bakin and Weinberger discovered that during this paradigm, the cerebellum-mediated learning of an anticipatory eye-blink response would be preceded by a much faster process in which the auditory conditioning stimulus was learned and associated to a startle or fear-related response [4]. This was understood as a learning process that was actually separated into two phases: a slow, reflex-specific learning phase relying on the cerebellum and driving a well-timed motor response and a fast, non-specific learning phase taking place in the neocortex in order to re-shape sensory representations to facilitate (though not exclusively) cerebellar learning [41,47]. The fact that two learning processes act in a parallel and dissociated but complementary fashion suggests that they might serve two different functional purposes. In this case, in particular, we could interpret that before learning a specific behavioral response, we first need to identify what cues are potentially relevant for succeeding in that task. The two-phase modeling of conditioning [31,58] proposes that the differentiation between separating learning in these two phases allows to rapidly acquire sensory evidence that will later be essential to perform effective actions. Therefore, a form of exploratory learning to capture new task-dependent contingencies in the sensory space would be required in some tasks to facilitate a later, slower exploitation phase to refine and perfect behavior and performance.
It is not surprising that further studies showed that this faster, non-specific form of learning was mediated by a specific neuromodulator, in this case, Ach [8,79,80]. Cortical Ach stems from the topologically organized cholinergic neurons in the nucleus basalis of Meynert (NBM). This neuromodulator was found to produce long-lasting synaptic changes in the auditory neocortex when the conditioning stimulus (CS) was a sound [11], while it has been shown to actually refine cortical receptive fields of different sensory modalities through extensive evidence [6,20,50,62,67,83]. Many other neuromodulators have been shown to drive some form of learning: dopamine [55], noradrenaline [27], or even oxytocin [46] among others. What makes Ach different?
Cholinergic neurons in the NBM are topologically organized with different cortical areas. Higher levels of cortical acetylcholine have been found to be correlated with a number of brain processes, including increases in arousal, sleep-wake cycles, sensory information processing (visual, auditory, olfactory), memory, attention, and other cognitive processes [1,24,29,48,51,53,56]. The improved performance in detection tasks becomes particularly relevant in this case. Sensory detection paradigms are usually composed of a noisy context or mask that is presented at every trial and a detectable sensory pattern that is hidden in that context. By regulating the ratio between noise and signal (SNR), experimenters have control over the subjects' performance. This control of the SNR allows understanding what neural mechanisms influence such performance. Hence, it is obvious that if cortical Ach plays a role in regulating performance [28], it must have a way to either regulate the SNR in the neocortex or improve learning in situations of increased uncertainty or both [59]. While the possible interpretations of this observation are vast, it is undeniable that Ach is driven by a mechanism that detects or predicts uncertainty and that its modulation of cortical dynamics improves SNR, signal detection and influence gating of information. Through what mechanisms the brain can learn or evaluate the uncertainty of sensory inputs or other internal signals is left unexplained. Moreover, which are the particular differences between Ach-mediated learning and learning driven by other neuromodulators is still unclear.

Biology of Acetylcholine and Inhibitory Interneurons
Cholinergic neurons in the NBM are the main source of cortical Ach. Ach as a neuromodulator provides diffuse modulation and a broad innervation that reaches the vast majority of cortical areas of the brain. In contrast, glutamate and γaminobutyric acid are local and specific to one or few synapses. This property allows it to affect globally the dynamics of the cortical circuitry instead of merely transmitting a piece of information. Acetylcholine interacts with both excitatory and inhibitory interneurons via two types of receptors: muscarinic (mAchR) and nicotinic Ach receptors (nAchR). While both receptors depolarize the neurons in the presence of acetylcholine, metabotropic mAchR have slower and long-lasting effects while ionotropic nAchR rapidly open calcium channels [26,43]. Nonetheless, their overall impact in the network is markedly different. Muscarinic receptor agonists in the rat neocortex result in a global excitatory effect of both excitatory and inhibitory neurons [36]. In contrast, agonists of nicotinic receptors have a disinhibitory influence on the neural substrate, by preferentially targeting inhibitory neurons that hyperpolarize other inhibitory interneurons [81]. Altogether, acetylcholine affects the network level that depends on the balance and distribution of these two opposing cholinergic receptors [40]. The functional use of this segregation of receptors has not been identified yet, despite it seems that its influence on the inhibitory interneurons could be key for its understanding.
A fairly recent study showed that the cortical inhibitory interneuron network is composed of three types of interneurons distinct on their expression of three mutually exclusive markers [65,71]. These markers are the CA2+-binding protein parvalbumin, the neuropeptide somatostatin, and the serotonin receptor 5HT3a (serotonin). In turn, these three molecularly distinct types of interneurons are also morphologically, dynamically, and topologically distinct.
Parvalbumin-responsive inhibitory interneurons (PVi+) tend to be identified as fast-spiking interneurons or basket cells [37,70]. These cells typically have relatively small dendritic arbors that target one or few nearby cells [66]. These suggest a local reach of their inhibitory effect and have been associated with, e.g., feed-forward inhibition [69]. Somatostatin-responsive inhibitory interneurons (SOMi+) have larger dendritic arbors and tend to have synapses across cortical layers. In this case, somatostatin interneurons seem to play a critical role in more external cortical layers. These layers are the place where most top-down and bottom-up interactions take place [45]. The dynamics of these interneurons have been observed to be directly affected by locomotion, reducing surround suppression during movement via the external layers of V1 [3]. This modulation is known to be dependent on a disinhibitory network by vasoactive intestinal peptide interneurons (VIPi+) [25]. These observations suggest that they could play two crucial roles. First, extensive dendritic arbors would indicate a role on a more global form of gain control that integrates information from further areas. This is supported by the observations in reduced surround suppression driven by locomotion [3]. At the same time, their locations and interactions could suggest a role in receiving top-down information that could be more or less task-relevant, in the form of an attention modulating signal.
Finally, serotonin-responsive interneurons are usually coselective with vasoactive intestinal peptide-selective interneurons. These reside principally in the upper layers of the neocortex, where they are almost the only neuron type. VIP cells play an essential role in mediating interactions across areas. These cells are the ones signaling locomotion-related information to SOMi+ cells, being the driver of a disinhibitory effect [35,82] in cortical microcircuits. VIPi+ are involved in signaling top-down contextual information in order to gate [30] or modulate [3] sensory information in sensory cortical areas, while modulating executive control in prefrontal areas of the neocortex [34]. Strikingly, the results of this last study are behaviorally equivalent to those in [28]: while optogenetic stimulation of VIP cells favors Go over No Go responses in a memory-guided behavior task in the first, stimulation of cholinergic neurons in the NBM promoted false alarms in a detection task in the last, suggesting a significant overlap in the electrophysiology of the stimulation.
Altogether, this evidence draws a cortical motif of inhibitory interactions that have seems to be essential for sensing, learning, and acting under some degrees of uncertainty.

Biophysical Description
While the interactions between the interneuron subtypes mentioned above could be reasonably variable, there is increasing evidence for the existence of a circuit motif [3,9,73]. This microcircuit is composed of an intricate network of primary excitatory neurons interconnected with neighboring PVi+ and SOMi+ interneurons. VIPi+ cells, in turn, mostly reside in the upper layers of the neocortex, mediating top-down interactions with SOMi+, by the influence of signals from other cortical areas. The evidence shows that manipulating some of these interneurons affects performance under uncertain environments, suggesting a latent role of this network in capturing and modulating the perceived uncertainty in the world (Fig. 1).
While the plasticity in inhibitory networks is thought to hold the key to understanding perception and perceptual learning [23] via the regulation of the so-called excitatory-inhibitory balance, the debate is still open to identify what the particular inhibitory and disinhibitory interactions within the neocortex are. We here propose to start from a computational perspective and derive a model that captures the uncertainty of some input signal. Understanding the role of the different elements of such a model might help unveil the functions underlying cortical inhibition.

Derivation
In order to derive a model that captures sensory uncertainty, we define a simple perceptual problem. We consider the task of estimating the light intensity on a sensor in the retina, assuming that the sensor draws its values from a random normal distribution X with mean σ x and standard deviation σ x , the probability of a specific observation becomes: We can rephrase this problem as the problem of finding the estimates of μ x and σ x that maximize P(X = x | μ x , σ x ), or equivalently, maximizing its logarithm: Considering Θ x as the set of parameters μ x ,σ x we want to find the estimates of Θ * : In order to find the best estimate that maximizes the loss function L, we can use gradient ascent. Gradient ascent is an iterative process that allows us to find the parameters (Θ * ) that maximize a function (L) by finding analytically the slope of L and iteratively climbing through the steepest path. Assuming that μ x and σ x are independent variables, we obtain the following set of dynamical equations that represent our system from the partial derivatives that define the slopes of L: where Σ = σ 2 . These derived dynamic equations require two extra variables to estimate the relative and absolute errors e and ε:  [73] that includes the inhibitory network of three principal inhibitory cell types that compose the cortical inhibitory network [65]. Recently, models cortical microcircuitry have included VIP cells as the key player for gating and modulating information from other cortical areas [9] as attentional dynamics driven by contextual information [3]. Notice that the classical microcircuit would summarize all inhibitory subtypes (VIPi+, SOMi+, PVi+) into a single one, recurrently connected to excitatory pyramidal cells (PCs) [18]. b Predictive model of the visual neocortex that provides a functional interpretation of the cortical microcircuitry [61]. While these models provide a functional interpretation for the cortical microcircuits, the lack of sufficient biological constraints limits the scope of such models. c Model [61] derived to provide an estimation not just of the input but also of its statistical moments (mean and variance). While simpler in scope, the model can be constrained according to biology while identifying exclusive functions for each inhibitory interneuron subtype it includes (see derivation and results below) which converge to the steady states of the system: Namely the relative and absolute errors, respectively. Representing this system as a network with nodes (neurons) and vertices (synapses) (see Fig. 2) we can identify that our system has a few important properties. First of all, the changes in our nodes depend (by definition) exclusively on inputs in the form of weighted sums. Moreover, the synapses or vertices also change using purely local information, i.e., the changes in a vertex depend solely on the neighboring nodes or itself. Finally, the synaptic changes derived using gradient ascent are equivalent to a Hebbian learning rule with or without homeostatic decay. Altogether, this network can be interpreted as a neural network where each node represents the firing rate of a particular neuron. While this system converges to the desired solution (a good approximation of μ x and σ x as illustrated in Fig. 2 b and c) analitically, the fact that the error nodes can switch sign depending on the input and its prediction is not desirable when aiming to compare with a biologically plausible system. To address this concern, we constrain our model to have exclusively excitatory and exclusively inhibitory neurons, by using linear rectification: Because this will only consider one side of the error (e1 = f(x − wy)), we need to mirror the circuit in order to compute both halves: e1 and e2 = f(y − vx). This new circuit is illustrated in Fig. 2d as the biologically constrained version of the circuit in Fig. 2a.

Results
In order to see if the biologically constrained version of the network behaves as expected, we perform a series of tests. First, we check that the network parameters successfully estimate the mean and standard deviation of the network. Figure 2 shows that effectively, the network converges to a good estimation of the mean input as shown by the zero error obtained after training for a range of randomly selected initial conditions and parameters. Nonetheless, the variance is only partially estimated for large instances of σ x . In the cases where σ x is already small, the network does not fully converge, as illustrated in the first part of Fig. 3d. To understand this phenomenon, we analyze the stability of our network. Studying the distribution of poles of the dynamical system in the realimaginary space allows us to analyze the stability in the sense of Lyapunov, as shown in Fig. 3a. We can observe that the system is intrinsically oscillatory (not surprising to think of a neural network given [44]) due to the two imaginary poles close to zero in the real axis. To move the complex poles in along the real axis, we modulate inversely Σ and e. This modulation results in a continuous transformation of the system from unstable or marginally stable dynamics to a fully stable regime (poles with exclusively negative real), in which the system converges faster the further the negative poles are -One in which the relative error is too big, and the system diverges (not shown), corresponding to the states with a pole with a positive real part. -A marginal zone where the relative error is large despite the variance too small to account for it, that leads to oscillatory dynamics with extremely slow converging or diverging dynamics -A stable zone characterized by errors smaller than the variance or, equivalently, with errors close to zero and variance properly estimated One can notice that, while the last case is the most desired, more stable when the error has been already minimized, the second case corresponds to relative errors relatively larger than what the variance can account for. This last case can be seen as the case when an outlier is detected or when the model is no longer able to capture the statistics of the input. Our model reacts to that by entering an oscillatory mode where the absolute, objective error (measured as x − wy) cannot be reduced by the system and needs some external input to break this situation. Given that this abnormal state comes from the relative difference between Σ and e, we can either inject a transient increase in the conductivity of the synapse Σ or, equivalently, to inject current directly in the neuron e that enables the estimation of the variance. Figure 3d, e show how this injection rapidly brings the state to the stable region that quickly converges where otherwise the dynamics would have converged much slower, if at all. Fig. 3 Top-down modulation of uncertainty re-enables learning after convergence. a Illustration of the eigenvalues of our model in function of the relation between the relative error e and the estimated variance Σ. Red indicates greater values of Σ and blue greater values of e. b Illustration of the setup to simulate the top-down influence of uncertainty. While external input has been present in the form of a random normal variable that needs to be predicted, in this setup we include a top-down signal that illustrates the identification of an abnormal state in the network, i.e., there is a big error and a small predicted variance. In order to learn from this particular event, we inject external current in the units ε. c Graphical representation of the input signal x and the cholinergic equivalent input expressed as a Δ ε in function of time. d Single example of the effect of a short top-down stimulation on the absolute error, after reaching an abnormal oscillatory state due to a mismatch between estimated relative error e and variance Σ. e Same as d averaged over 500 trials with different random initial conditions. Notice that the signals before stimulation have increased variability (blue shadow) and a brief stimulation is sufficient to bring the system back to a stable regime We can understand this current injection as a top-down bias to the estimation of uncertainty. In our model, while our y inhibitory cells feed-forward information that can be used for prediction of x excitatory cells, ε inhibitory cells are more locally connected to estimate the variance of the system. In cortical anatomy, we can identify three main types of interneurons, PV, SOM, and VIP. PV cells are identified as feedforward interneurons that convey sensory information to regulate the excitatory input injected into the network (interpreted in our model as the relative error e). The second most noticeable type of inhibitory interneurons (SOM) are involved in gain control and are modulated by locomotion (in V1), acetylcholine (in other sensory areas), or other top-down signals to regulate the incorporation of new information to the network. We, therefore, propose that cholinergic modulation, via the intervention of VIP interneurons, transiently reduces the activity in SOM cells, which in turn brings the system temporarily to a nonstable regime that allows the exploration of sensory alternatives before the reshaping of internal models when the effect fades off.
In summary, our computational model and its analysis have provided us with a minimal microcircuit to perform three main functions: -To compute the uncertainty of an internal or external piece of information given its dynamics over time, in a biologically constrained neural network -To use an estimate of uncertainty to modulate learning, thus improving learning speed as a trade-off with accuracy -To provide a mechanism to switch between two regimes, one to rapidly learn when the system is unable to converge to a solution, and the opposite, to bring the system in an oscillatory state that can be beneficial for sensory exploration

Conclusions and Discussion
The role of excitatory-inhibitory interactions has been debated thoroughly in the past. However, this discussion has mostly considered inhibition as a homogeneous block. This block is sometimes responsible for performing some form of winnertake-all filtering or maintaining the balance in the output firing rates of excitatory cells. Our model in this article proposes that if a network of neurons has to perform some form of probabilistic computation, two types of interneurons, with different functions and connections, need to be considered: the first type is required to compute the mean of the input, while the second is necessary to estimate its variance. The first can be seen as a gain control system, aiming at balancing excitatory input to detect inconsistencies. In fact, in the reduced circuit we presented here, it estimates only the mean of the activity of the input data matches models of excitatory-inhibitory balance, winner-take-all and decision making [72], even producing receptive fields similar to those observed in different sensory regions of the neocortex [61]. The second type of interneurons extends the funtionality of the first by providing a mechanism to regulate learning through uncertainty. On one hand, it provides a faster convergence mechanism (not shown in this paper) by learning faster when uncertainty is increased.
On the other hand, it provides a mechanism to regulate learning in a top-down fashion. According to our model, this means that detection of an incongruency in the world raises the brain's estimate of uncertainty, which in turn increases learning rates transiently without biasing the actual content to be learned. Our model extends current models of cortical function by proposing a role for different inhibitory neuron types. Biophysical experiments have shown that two out of three biochemically segregated inhibitory populations are composed of around 60 to 90% of inhibition in most cortical layers [65,71]. While fast-spiking interneurons are typically associated with a gain-control kind of function, SOMi+, regularspiking interneurons are found to be involved in a range of functions that require top-down or cross-area information, like surround-suppression inhibition during locomotion in V1 [3]. These functions can be compared with those of the neural units presented in our microcircuit model. In that case, PVi+ would correspond to interneurons responsible for learning the mean of activity of the input and SOMi+ to the ones involved in estimating variance of input. External input to the second type of interneurons would then modulate both the perceived uncertainty and consequently the rate of synaptic change in the microcircuit.
To summarize, our model results in the following interpretation of cortical microcircuits: the recurrent microcircuitry of excitatory cells with two inhibitory populations inherently computes prediction errors and their uncertainty (through the variance). This can be seen as follows. Excitatory input from the thalamus comes to the principal excitatory cells of the neocortex. These neurons can be understood to be computing some form of prediction error via the inhibitory effect of parvalbumin cells. While PVi+ in different layers of the neocortex can be receiving excitatory input either from thalamocortical connections or from intracortical excitatory cells, they would be the candidates for providing predictions of the thalamocortical input. Finally, the broad dendritic range and long-term dynamics of SOMi+ allows them to integrate information effectively over both space and time. This integration of information would mediate the estimation of uncertainty on the PVi+ predictions. SOMi+ inhibition would be stronger for greater estimated uncertainty, in turn modulating the perceived errors signaled by PCs. If this were the case, our model would suggest that top-down inhibition of the same SOMi+ (via VIPi+) should increase the sensitivity of the errors, re-enabling learning of uncertainty by switching to a more exploitatory mode. This disinhibitory role could be played by both, the neuromodulatory action of Ach or topdown modulation of layer I VIPi+ from other cortical areas.
While the interpretation of the brain performing some form of predictive coding has long been posed [61], we go one step further by proposing an explicit hypothesis of how a neural circuit could implement predictions in a dynamical system and extending previous models with an additional interneuron subtype involved in monitoring uncertainty. By targeting explicitly some interneuron subtypes, we should be able to identify if their plastic changes correspond with those we propose.
Our model presents a new lens to understand the function of cortical microcircuits and neuromodulation. While the most common neuromodulators (Ach, DA, Se, NE) have been studied for decades, little is known about how they influence our behavior. Looking at cortical function through our microcircuit model allows us to bridge the gap between behavior and biophysics by presenting inhibitory interactions as a predictive process. Taking Ach as an example, the links between the role of Ach in perception and decision making at the behavioral level, and the interactions of Ach with different elements of cortical substrates is still unclear. Our model proposes that a phasic decrease on cortical SOMi+ activity (probably driven by top-down VIP control, induced via Ach (see [59] for details), functionally amounts to a decrease in -driven regulation of the relative error, consequently signaling a smaller variance which translates into larger response to outliers, triggering false alarm responses while further promoting learning. Finally, our model provides a mechanism to switch between perceptual and learning operational modes. Modulating uncertainty can be seen as driving exploration or exploitation by modulating how large an error will be perceived, relative to its expected variance. Going back to the examples of conditioning, the dopamine-dependent addiction circuit and the acetylcholine-dependent fear circuit can be seen as the axis between exploitation and exploration. While the first is involved in the repetition of particular behaviors and the formation of habits, Ach seems to be signaled to reshape and explore new potentially useful cues in the sensory domain. Our model suggests that while exploitation and exploration could be seen as two ends of the same axis, two separated circuits enable each of the operation modes. Moreover, in a detection task with white noise masking a signal that needs to be detected, our circuit would aim at filtering the expected noise in search of a stable cue that helps at detecting the target. Normal levels of Ach should regulate the optimal threshold of noise cancelation to optimize perception. Increased Ach levels would also increase the probability to identify the random noise as an outlier, or equivalently, the signal. This can be seen as a form of exploration by reducing the tolerance to noise and enabling more erratic behavior under the presence of predicted uncertainty. Misses, on the contrary, induced in [3] by reducing Ach levels, could then be associated to a more conservative, exploitatory mode in which a more robust performance is preferred, therefore fully inhibiting the predicted noise.
The phenomenology behind Alzheimer's disease, characterized by decreased levels of Ach due to loss of cholinergic innervation [15,16] and basal cholinergic neurons [76,77], can be understood as an abnormally decreased ability to compute prediction errors, translating into a reduced ability to learn from those and therefore form new memories. Alternatively, schizophrenia, correlated with generally with reduced amount of muscarinic and nicotinic acetylcholine receptors [22,60], indirectly increasing the levels of extracellular Ach, can be understood as a decreased ability to gate unreliable information [57], as it would be supported by the reported increases and reductions in sensory detection tasks, shown both in animals and humans [10,74,75]. Nonetheless, our model leaves some open questions. What drives the top-down modulation of SOMi+ or the modulation via Ach itself? These are still unanswered. While some papers suggest possible pathways, the picture is still fragmented and incomplete, and we are not much further from Yu and Dayan's [2] hypothesis of Ach signaling expected uncertainty, without knowing the mechanism for representing that expectation. Moreover, the complexity of the layered structure of the neocortex is not fully captured by our model. While our model only aims at explaining the role of the microcircuit per se, understanding how multiple instances of such microcircuits could be connected, forming different sorts of motifs could allow us to complete the picture of the predictive brain. Moreover, this motif is not found exclusively in the neocortex. Evidence suggests that other Ach targeted regions with excitatory principal cells like the hippocampus, the amygdala, and probably the olfactory bulb have the same kind of inhibitory microcircuits and equivalent interactions with Ach [7,38,64]. Further work should be done to identify how other neuromodulators interact with these microcircuits and affect behavior and cognitive function.