TY - JOUR
AB - Selection, mutation, and random drift affect the dynamics of allele frequencies and consequently of quantitative traits. While the macroscopic dynamics of quantitative traits can be measured, the underlying allele frequencies are typically unobserved. Can we understand how the macroscopic observables evolve without following these microscopic processes? This problem has been studied previously by analogy with statistical mechanics: the allele frequency distribution at each time point is approximated by the stationary form, which maximizes entropy. We explore the limitations of this method when mutation is small (4Nμ < 1) so that populations are typically close to fixation, and we extend the theory in this regime to account for changes in mutation strength. We consider a single diallelic locus either under directional selection or with overdominance and then generalize to multiple unlinked biallelic loci with unequal effects. We find that the maximum-entropy approximation is remarkably accurate, even when mutation and selection change rapidly.
AU - Bod'ová, Katarína
AU - Tkacik, Gasper
AU - Barton, Nicholas H
ID - 1420
IS - 4
JF - Genetics
TI - A general approximation for the dynamics of quantitative traits
VL - 202
ER -
TY - JOUR
AB - In this article the notion of metabolic turnover is revisited in the light of recent results of out-of-equilibrium thermodynamics. By means of Monte Carlo methods we perform an exact sampling of the enzymatic fluxes in a genome scale metabolic network of E. Coli in stationary growth conditions from which we infer the metabolites turnover times. However the latter are inferred from net fluxes, and we argue that this approximation is not valid for enzymes working nearby thermodynamic equilibrium. We recalculate turnover times from total fluxes by performing an energy balance analysis of the network and recurring to the fluctuation theorem. We find in many cases values one of order of magnitude lower, implying a faster picture of intermediate metabolism.
AU - De Martino, Daniele
ID - 1485
IS - 1
JF - Physical Biology
TI - Genome-scale estimate of the metabolic turnover of E. Coli from the energy balance analysis
VL - 13
ER -
TY - CONF
AB - In many applications, it is desirable to extract only the relevant aspects of data. A principled way to do this is the information bottleneck (IB) method, where one seeks a code that maximises information about a relevance variable, Y, while constraining the information encoded about the original data, X. Unfortunately however, the IB method is computationally demanding when data are high-dimensional and/or non-gaussian. Here we propose an approximate variational scheme for maximising a lower bound on the IB objective, analogous to variational EM. Using this method, we derive an IB algorithm to recover features that are both relevant and sparse. Finally, we demonstrate how kernelised versions of the algorithm can be used to address a broad range of problems with non-linear relation between X and Y.
AU - Chalk, Matthew J
AU - Marre, Olivier
AU - Tkacik, Gasper
ID - 1082
TI - Relevant sparse codes with variational information bottleneck
VL - 29
ER -
TY - CONF
AB - Jointly characterizing neural responses in terms of several external variables promises novel insights into circuit function, but remains computationally prohibitive in practice. Here we use gaussian process (GP) priors and exploit recent advances in fast GP inference and learning based on Kronecker methods, to efficiently estimate multidimensional nonlinear tuning functions. Our estimator require considerably less data than traditional methods and further provides principled uncertainty estimates. We apply these tools to hippocampal recordings during open field exploration and use them to characterize the joint dependence of CA1 responses on the position of the animal and several other variables, including the animal\'s speed, direction of motion, and network oscillations.Our results provide an unprecedentedly detailed quantification of the tuning of hippocampal neurons. The model\'s generality suggests that our approach can be used to estimate neural response properties in other brain regions.
AU - Savin, Cristina
AU - Tkacik, Gasper
ID - 1105
TI - Estimating nonlinear neural response functions using GP priors and Kronecker methods
VL - 29
ER -
TY - JOUR
AB - Continuous-time Markov chain (CTMC) models have become a central tool for understanding the dynamics of complex reaction networks and the importance of stochasticity in the underlying biochemical processes. When such models are employed to answer questions in applications, in order to ensure that the model provides a sufficiently accurate representation of the real system, it is of vital importance that the model parameters are inferred from real measured data. This, however, is often a formidable task and all of the existing methods fail in one case or the other, usually because the underlying CTMC model is high-dimensional and computationally difficult to analyze. The parameter inference methods that tend to scale best in the dimension of the CTMC are based on so-called moment closure approximations. However, there exists a large number of different moment closure approximations and it is typically hard to say a priori which of the approximations is the most suitable for the inference procedure. Here, we propose a moment-based parameter inference method that automatically chooses the most appropriate moment closure method. Accordingly, contrary to existing methods, the user is not required to be experienced in moment closure techniques. In addition to that, our method adaptively changes the approximation during the parameter inference to ensure that always the best approximation is used, even in cases where different approximations are best in different regions of the parameter space. © 2016 Elsevier Ireland Ltd
AU - Schilling, Christian
AU - Bogomolov, Sergiy
AU - Henzinger, Thomas A
AU - Podelski, Andreas
AU - Ruess, Jakob
ID - 1148
JF - Biosystems
TI - Adaptive moment closure for parameter inference of biochemical reaction networks
VL - 149
ER -
TY - JOUR
AB - The increasing complexity of dynamic models in systems and synthetic biology poses computational challenges especially for the identification of model parameters. While modularization of the corresponding optimization problems could help reduce the “curse of dimensionality,” abundant feedback and crosstalk mechanisms prohibit a simple decomposition of most biomolecular networks into subnetworks, or modules. Drawing on ideas from network modularization and multiple-shooting optimization, we present here a modular parameter identification approach that explicitly allows for such interdependencies. Interfaces between our modules are given by the experimentally measured molecular species. This definition allows deriving good (initial) estimates for the inter-module communication directly from the experimental data. Given these estimates, the states and parameter sensitivities of different modules can be integrated independently. To achieve consistency between modules, we iteratively adjust the estimates for inter-module communication while optimizing the parameters. After convergence to an optimal parameter set---but not during earlier iterations---the intermodule communication as well as the individual modules\' state dynamics agree with the dynamics of the nonmodularized network. Our modular parameter identification approach allows for easy parallelization; it can reduce the computational complexity for larger networks and decrease the probability to converge to suboptimal local minima. We demonstrate the algorithm\'s performance in parameter estimation for two biomolecular networks, a synthetic genetic oscillator and a mammalian signaling pathway.
AU - Lang, Moritz
AU - Stelling, Jörg
ID - 1170
IS - 6
JF - SIAM Journal on Scientific Computing
TI - Modular parameter identification of biomolecular networks
VL - 38
ER -
TY - JOUR
AU - Tkacik, Gasper
ID - 1171
JF - Physics of Life Reviews
TI - Understanding regulatory networks requires more than computing a multitude of graph statistics: Comment on "Drivers of structural features in gene regulatory networks: From biophysical constraints to biological function" by O. C. Martin et al.
VL - 17
ER -
TY - THES
AB - The process of gene expression is central to the modern understanding of how cellular systems
function. In this process, a special kind of regulatory proteins, called transcription factors,
are important to determine how much protein is produced from a given gene. As biological
information is transmitted from transcription factor concentration to mRNA levels to amounts of
protein, various sources of noise arise and pose limits to the fidelity of intracellular signaling.
This thesis concerns itself with several aspects of stochastic gene expression: (i) the mathematical
description of complex promoters responsible for the stochastic production of biomolecules,
(ii) fundamental limits to information processing the cell faces due to the interference from multiple
fluctuating signals, (iii) how the presence of gene expression noise influences the evolution
of regulatory sequences, (iv) and tools for the experimental study of origins and consequences
of cell-cell heterogeneity, including an application to bacterial stress response systems.
AU - Rieckh, Georg
ID - 1128
TI - Studying the complexities of transcriptional regulation
ER -
TY - JOUR
AU - Gilson, Matthieu
AU - Savin, Cristina
AU - Zenke, Friedemann
ID - 1564
IS - 11
JF - Frontiers in Computational Neuroscience
TI - Editorial: Emergent neural computation from the interaction of different forms of plasticity
VL - 9
ER -
TY - JOUR
AB - Grounding autonomous behavior in the nervous system is a fundamental challenge for neuroscience. In particular, self-organized behavioral development provides more questions than answers. Are there special functional units for curiosity, motivation, and creativity? This paper argues that these features can be grounded in synaptic plasticity itself, without requiring any higher-level constructs. We propose differential extrinsic plasticity (DEP) as a new synaptic rule for self-learning systems and apply it to a number of complex robotic systems as a test case. Without specifying any purpose or goal, seemingly purposeful and adaptive rhythmic behavior is developed, displaying a certain level of sensorimotor intelligence. These surprising results require no systemspecific modifications of the DEP rule. They rather arise from the underlying mechanism of spontaneous symmetry breaking,which is due to the tight brain body environment coupling. The new synaptic rule is biologically plausible and would be an interesting target for neurobiological investigation. We also argue that this neuronal mechanism may have been a catalyst in natural evolution.
AU - Der, Ralf
AU - Martius, Georg S
ID - 1570
IS - 45
JF - PNAS
TI - Novel plasticity rule can explain the development of sensorimotor intelligence
VL - 112
ER -
TY - JOUR
AB - Gene expression is controlled primarily by interactions between transcription factor proteins (TFs) and the regulatory DNA sequence, a process that can be captured well by thermodynamic models of regulation. These models, however, neglect regulatory crosstalk: the possibility that noncognate TFs could initiate transcription, with potentially disastrous effects for the cell. Here, we estimate the importance of crosstalk, suggest that its avoidance strongly constrains equilibrium models of TF binding, and propose an alternative nonequilibrium scheme that implements kinetic proofreading to suppress erroneous initiation. This proposal is consistent with the observed covalent modifications of the transcriptional apparatus and predicts increased noise in gene expression as a trade-off for improved specificity. Using information theory, we quantify this trade-off to find when optimal proofreading architectures are favored over their equilibrium counterparts. Such architectures exhibit significant super-Poisson noise at low expression in steady state.
AU - Cepeda Humerez, Sarah A
AU - Rieckh, Georg
AU - Tkacik, Gasper
ID - 1576
IS - 24
JF - Physical Review Letters
TI - Stochastic proofreading mechanism alleviates crosstalk in transcriptional regulation
VL - 115
ER -
TY - JOUR
AB - Quantifying behaviors of robots which were generated autonomously from task-independent objective functions is an important prerequisite for objective comparisons of algorithms and movements of animals. The temporal sequence of such a behavior can be considered as a time series and hence complexity measures developed for time series are natural candidates for its quantification. The predictive information and the excess entropy are such complexity measures. They measure the amount of information the past contains about the future and thus quantify the nonrandom structure in the temporal sequence. However, when using these measures for systems with continuous states one has to deal with the fact that their values will depend on the resolution with which the systems states are observed. For deterministic systems both measures will diverge with increasing resolution. We therefore propose a new decomposition of the excess entropy in resolution dependent and resolution independent parts and discuss how they depend on the dimensionality of the dynamics, correlations and the noise level. For the practical estimation we propose to use estimates based on the correlation integral instead of the direct estimation of the mutual information based on next neighbor statistics because the latter allows less control of the scale dependencies. Using our algorithm we are able to show how autonomous learning generates behavior of increasing complexity with increasing learning duration.
AU - Martius, Georg S
AU - Olbrich, Eckehard
ID - 1655
IS - 10
JF - Entropy
TI - Quantifying emergent behavior of autonomous robots
VL - 17
ER -
TY - CONF
AB - Continuous-time Markov chain (CTMC) models have become a central tool for understanding the dynamics of complex reaction networks and the importance of stochasticity in the underlying biochemical processes. When such models are employed to answer questions in applications, in order to ensure that the model provides a sufficiently accurate representation of the real system, it is of vital importance that the model parameters are inferred from real measured data. This, however, is often a formidable task and all of the existing methods fail in one case or the other, usually because the underlying CTMC model is high-dimensional and computationally difficult to analyze. The parameter inference methods that tend to scale best in the dimension of the CTMC are based on so-called moment closure approximations. However, there exists a large number of different moment closure approximations and it is typically hard to say a priori which of the approximations is the most suitable for the inference procedure. Here, we propose a moment-based parameter inference method that automatically chooses the most appropriate moment closure method. Accordingly, contrary to existing methods, the user is not required to be experienced in moment closure techniques. In addition to that, our method adaptively changes the approximation during the parameter inference to ensure that always the best approximation is used, even in cases where different approximations are best in different regions of the parameter space.
AU - Bogomolov, Sergiy
AU - Henzinger, Thomas A
AU - Podelski, Andreas
AU - Ruess, Jakob
AU - Schilling, Christian
ID - 1658
TI - Adaptive moment closure for parameter inference of biochemical reaction networks
VL - 9308
ER -
TY - JOUR
AB - Evolution of gene regulation is crucial for our understanding of the phenotypic differences between species, populations and individuals. Sequence-specific binding of transcription factors to the regulatory regions on the DNA is a key regulatory mechanism that determines gene expression and hence heritable phenotypic variation. We use a biophysical model for directional selection on gene expression to estimate the rates of gain and loss of transcription factor binding sites (TFBS) in finite populations under both point and insertion/deletion mutations. Our results show that these rates are typically slow for a single TFBS in an isolated DNA region, unless the selection is extremely strong. These rates decrease drastically with increasing TFBS length or increasingly specific protein-DNA interactions, making the evolution of sites longer than ∼ 10 bp unlikely on typical eukaryotic speciation timescales. Similarly, evolution converges to the stationary distribution of binding sequences very slowly, making the equilibrium assumption questionable. The availability of longer regulatory sequences in which multiple binding sites can evolve simultaneously, the presence of “pre-sites” or partially decayed old sites in the initial sequence, and biophysical cooperativity between transcription factors, can all facilitate gain of TFBS and reconcile theoretical calculations with timescales inferred from comparative genomics.
AU - Tugrul, Murat
AU - Paixao, Tiago
AU - Barton, Nicholas H
AU - Tkacik, Gasper
ID - 1666
IS - 11
JF - PLoS Genetics
TI - Dynamics of transcription factor binding site evolution
VL - 11
ER -
TY - JOUR
AB - Motion tracking is a challenge the visual system has to solve by reading out the retinal population. It is still unclear how the information from different neurons can be combined together to estimate the position of an object. Here we recorded a large population of ganglion cells in a dense patch of salamander and guinea pig retinas while displaying a bar moving diffusively. We show that the bar’s position can be reconstructed from retinal activity with a precision in the hyperacuity regime using a linear decoder acting on 100+ cells. We then took advantage of this unprecedented precision to explore the spatial structure of the retina’s population code. The classical view would have suggested that the firing rates of the cells form a moving hill of activity tracking the bar’s position. Instead, we found that most ganglion cells in the salamander fired sparsely and idiosyncratically, so that their neural image did not track the bar. Furthermore, ganglion cell activity spanned an area much larger than predicted by their receptive fields, with cells coding for motion far in their surround. As a result, population redundancy was high, and we could find multiple, disjoint subsets of neurons that encoded the trajectory with high precision. This organization allows for diverse collections of ganglion cells to represent high-accuracy motion information in a form easily read out by downstream neural circuits.
AU - Marre, Olivier
AU - Botella Soler, Vicente
AU - Simmons, Kristina
AU - Mora, Thierry
AU - Tkacik, Gasper
AU - Berry, Michael
ID - 1697
IS - 7
JF - PLoS Computational Biology
TI - High accuracy decoding of dynamical motion from a large retinal population
VL - 11
ER -
TY - JOUR
AB - The activity of a neural network is defined by patterns of spiking and silence from the individual neurons. Because spikes are (relatively) sparse, patterns of activity with increasing numbers of spikes are less probable, but, with more spikes, the number of possible patterns increases. This tradeoff between probability and numerosity is mathematically equivalent to the relationship between entropy and energy in statistical physics. We construct this relationship for populations of up to N = 160 neurons in a small patch of the vertebrate retina, using a combination of direct and model-based analyses of experiments on the response of this network to naturalistic movies. We see signs of a thermodynamic limit, where the entropy per neuron approaches a smooth function of the energy per neuron as N increases. The form of this function corresponds to the distribution of activity being poised near an unusual kind of critical point. We suggest further tests of criticality, and give a brief discussion of its functional significance.
AU - Tkacik, Gasper
AU - Mora, Thierry
AU - Marre, Olivier
AU - Amodei, Dario
AU - Palmer, Stephanie
AU - Berry Ii, Michael
AU - Bialek, William
ID - 1701
IS - 37
JF - PNAS
TI - Thermodynamics and signatures of criticality in a network of neurons
VL - 112
ER -
TY - JOUR
AB - Bow-tie or hourglass structure is a common architectural feature found in many biological systems. A bow-tie in a multi-layered structure occurs when intermediate layers have much fewer components than the input and output layers. Examples include metabolism where a handful of building blocks mediate between multiple input nutrients and multiple output biomass components, and signaling networks where information from numerous receptor types passes through a small set of signaling pathways to regulate multiple output genes. Little is known, however, about how bow-tie architectures evolve. Here, we address the evolution of bow-tie architectures using simulations of multi-layered systems evolving to fulfill a given input-output goal. We find that bow-ties spontaneously evolve when the information in the evolutionary goal can be compressed. Mathematically speaking, bow-ties evolve when the rank of the input-output matrix describing the evolutionary goal is deficient. The maximal compression possible (the rank of the goal) determines the size of the narrowest part of the network—that is the bow-tie. A further requirement is that a process is active to reduce the number of links in the network, such as product-rule mutations, otherwise a non-bow-tie solution is found in the evolutionary simulations. This offers a mechanism to understand a common architectural principle of biological systems, and a way to quantitate the effective rank of the goals under which they evolved.
AU - Friedlander, Tamar
AU - Mayo, Avraham
AU - Tlusty, Tsvi
AU - Alon, Uri
ID - 1827
IS - 3
JF - PLoS Computational Biology
TI - Evolution of bow-tie architectures in biology
VL - 11
ER -
TY - JOUR
AB - Continuous-time Markov chains are commonly used in practice for modeling biochemical reaction networks in which the inherent randomness of themolecular interactions cannot be ignored. This has motivated recent research effort into methods for parameter inference and experiment design for such models. The major difficulty is that such methods usually require one to iteratively solve the chemical master equation that governs the time evolution of the probability distribution of the system. This, however, is rarely possible, and even approximation techniques remain limited to relatively small and simple systems. An alternative explored in this article is to base methods on only some low-order moments of the entire probability distribution. We summarize the theory behind such moment-based methods for parameter inference and experiment design and provide new case studies where we investigate their performance.
AU - Ruess, Jakob
AU - Lygeros, John
ID - 1861
IS - 2
JF - ACM Transactions on Modeling and Computer Simulation
TI - Moment-based methods for parameter inference and experiment design for stochastic biochemical reaction networks
VL - 25
ER -
TY - JOUR
AB - The concept of positional information is central to our understanding of how cells determine their location in a multicellular structure and thereby their developmental fates. Nevertheless, positional information has neither been defined mathematically nor quantified in a principled way. Here we provide an information-theoretic definition in the context of developmental gene expression patterns and examine the features of expression patterns that affect positional information quantitatively. We connect positional information with the concept of positional error and develop tools to directly measure information and error from experimental data. We illustrate our framework for the case of gap gene expression patterns in the early Drosophila embryo and show how information that is distributed among only four genes is sufficient to determine developmental fates with nearly single-cell resolution. Our approach can be generalized to a variety of different model systems; procedures and examples are discussed in detail.
AU - Tkacik, Gasper
AU - Dubuis, Julien
AU - Petkova, Mariela
AU - Gregor, Thomas
ID - 1885
IS - 1
JF - Genetics
TI - Positional information, positional error, and readout precision in morphogenesis: A mathematical framework
VL - 199
ER -
TY - JOUR
AB - We typically think of cells as responding to external signals independently by regulating their gene expression levels, yet they often locally exchange information and coordinate. Can such spatial coupling be of benefit for conveying signals subject to gene regulatory noise? Here we extend our information-theoretic framework for gene regulation to spatially extended systems. As an example, we consider a lattice of nuclei responding to a concentration field of a transcriptional regulator (the "input") by expressing a single diffusible target gene. When input concentrations are low, diffusive coupling markedly improves information transmission; optimal gene activation functions also systematically change. A qualitatively new regulatory strategy emerges where individual cells respond to the input in a nearly step-like fashion that is subsequently averaged out by strong diffusion. While motivated by early patterning events in the Drosophila embryo, our framework is generically applicable to spatially coupled stochastic gene expression models.
AU - Sokolowski, Thomas R
AU - Tkacik, Gasper
ID - 1940
IS - 6
JF - Physical Review E Statistical Nonlinear and Soft Matter Physics
TI - Optimizing information flow in small genetic networks. IV. Spatial coupling
VL - 91
ER -