TY - CONF AB - We derive a tight lower bound on equivocation (conditional entropy), or equivalently a tight upper bound on mutual information between a signal variable and channel outputs. The bound is in terms of the joint distribution of the signals and maximum a posteriori decodes (most probable signals given channel output). As part of our derivation, we describe the key properties of the distribution of signals, channel outputs and decodes, that minimizes equivocation and maximizes mutual information. This work addresses a problem in data analysis, where mutual information between signals and decodes is sometimes used to lower bound the mutual information between signals and channel outputs. Our result provides a corresponding upper bound. AU - Hledik, Michal AU - Sokolowski, Thomas R AU - Tkačik, Gašper ID - 7606 SN - 9781538669006 T2 - IEEE Information Theory Workshop, ITW 2019 TI - A tight upper bound on mutual information ER - TY - JOUR AB - A cornerstone of statistical inference, the maximum entropy framework is being increasingly applied to construct descriptive and predictive models of biological systems, especially complex biological networks, from large experimental data sets. Both its broad applicability and the success it obtained in different contexts hinge upon its conceptual simplicity and mathematical soundness. Here we try to concisely review the basic elements of the maximum entropy principle, starting from the notion of ‘entropy’, and describe its usefulness for the analysis of biological systems. As examples, we focus specifically on the problem of reconstructing gene interaction networks from expression data and on recent work attempting to expand our system-level understanding of bacterial metabolism. Finally, we highlight some extensions and potential limitations of the maximum entropy approach, and point to more recent developments that are likely to play a key role in the upcoming challenges of extracting structures and information from increasingly rich, high-throughput biological data. AU - De Martino, Andrea AU - De Martino, Daniele ID - 306 IS - 4 JF - Heliyon TI - An introduction to the maximum entropy approach and its application to inference problems in biology VL - 4 ER - TY - JOUR AB - The hanging-drop network (HDN) is a technology platform based on a completely open microfluidic network at the bottom of an inverted, surface-patterned substrate. The platform is predominantly used for the formation, culturing, and interaction of self-assembled spherical microtissues (spheroids) under precisely controlled flow conditions. Here, we describe design, fabrication, and operation of microfluidic hanging-drop networks. AU - Misun, Patrick AU - Birchler, Axel AU - Lang, Moritz AU - Hierlemann, Andreas AU - Frey, Olivier ID - 305 JF - Methods in Molecular Biology TI - Fabrication and operation of microfluidic hanging drop networks VL - 1771 ER - TY - JOUR AB - Although cells respond specifically to environments, how environmental identity is encoded intracellularly is not understood. Here, we study this organization of information in budding yeast by estimating the mutual information between environmental transitions and the dynamics of nuclear translocation for 10 transcription factors. Our method of estimation is general, scalable, and based on decoding from single cells. The dynamics of the transcription factors are necessary to encode the highest amounts of extracellular information, and we show that information is transduced through two channels: Generalists (Msn2/4, Tod6 and Dot6, Maf1, and Sfp1) can encode the nature of multiple stresses, but only if stress is high; specialists (Hog1, Yap1, and Mig1/2) encode one particular stress, but do so more quickly and for a wider range of magnitudes. In particular, Dot6 encodes almost as much information as Msn2, the master regulator of the environmental stress response. Each transcription factor reports differently, and it is only their collective behavior that distinguishes between multiple environmental states. Changes in the dynamics of the localization of transcription factors thus constitute a precise, distributed internal representation of extracellular change. We predict that such multidimensional representations are common in cellular decision-making. AU - Granados, Alejandro AU - Pietsch, Julian AU - Cepeda Humerez, Sarah A AU - Farquhar, Isebail AU - Tkacik, Gasper AU - Swain, Peter ID - 281 IS - 23 JF - PNAS TI - Distributed and dynamic intracellular organization of extracellular information VL - 115 ER - TY - JOUR AB - Self-incompatibility (SI) is a genetically based recognition system that functions to prevent self-fertilization and mating among related plants. An enduring puzzle in SI is how the high diversity observed in nature arises and is maintained. Based on the underlying recognition mechanism, SI can be classified into two main groups: self- and non-self recognition. Most work has focused on diversification within self-recognition systems despite expected differences between the two groups in the evolutionary pathways and outcomes of diversification. Here, we use a deterministic population genetic model and stochastic simulations to investigate how novel S-haplotypes evolve in a gametophytic non-self recognition (SRNase/S Locus F-box (SLF)) SI system. For this model the pathways for diversification involve either the maintenance or breakdown of SI and can vary in the order of mutations of the female (SRNase) and male (SLF) components. We show analytically that diversification can occur with high inbreeding depression and self-pollination, but this varies with evolutionary pathway and level of completeness (which determines the number of potential mating partners in the population), and in general is more likely for lower haplotype number. The conditions for diversification are broader in stochastic simulations of finite population size. However, the number of haplotypes observed under high inbreeding and moderate to high self-pollination is less than that commonly observed in nature. Diversification was observed through pathways that maintain SI as well as through self-compatible intermediates. Yet the lifespan of diversified haplotypes was sensitive to their level of completeness. By examining diversification in a non-self recognition SI system, this model extends our understanding of the evolution and maintenance of haplotype diversity observed in a self recognition system common in flowering plants. AU - Bodova, Katarina AU - Priklopil, Tadeas AU - Field, David AU - Barton, Nicholas H AU - Pickup, Melinda ID - 316 IS - 3 JF - Genetics TI - Evolutionary pathways for the generation of new self-incompatibility haplotypes in a non-self recognition system VL - 209 ER - TY - GEN AB - File S1 contains figures that clarify the following features: (i) effect of population size on the average number/frequency of SI classes, (ii) changes in the minimal completeness deficit in time for a single class, and (iii) diversification diagrams for all studied pathways, including the summary figure for k = 8. File S2 contains the code required for a stochastic simulation of the SLF system with an example. This file also includes the output in the form of figures and tables. AU - Bod'ová, Katarína AU - Priklopil, Tadeas AU - Field, David AU - Barton, Nicholas H AU - Pickup, Melinda ID - 9813 TI - Supplemental material for Bodova et al., 2018 ER - TY - JOUR AB - Recent developments in automated tracking allow uninterrupted, high-resolution recording of animal trajectories, sometimes coupled with the identification of stereotyped changes of body pose or other behaviors of interest. Analysis and interpretation of such data represents a challenge: the timing of animal behaviors may be stochastic and modulated by kinematic variables, by the interaction with the environment or with the conspecifics within the animal group, and dependent on internal cognitive or behavioral state of the individual. Existing models for collective motion typically fail to incorporate the discrete, stochastic, and internal-state-dependent aspects of behavior, while models focusing on individual animal behavior typically ignore the spatial aspects of the problem. Here we propose a probabilistic modeling framework to address this gap. Each animal can switch stochastically between different behavioral states, with each state resulting in a possibly different law of motion through space. Switching rates for behavioral transitions can depend in a very general way, which we seek to identify from data, on the effects of the environment as well as the interaction between the animals. We represent the switching dynamics as a Generalized Linear Model and show that: (i) forward simulation of multiple interacting animals is possible using a variant of the Gillespie’s Stochastic Simulation Algorithm; (ii) formulated properly, the maximum likelihood inference of switching rate functions is tractably solvable by gradient descent; (iii) model selection can be used to identify factors that modulate behavioral state switching and to appropriately adjust model complexity to data. To illustrate our framework, we apply it to two synthetic models of animal motion and to real zebrafish tracking data. AU - Bod’Ová, Katarína AU - Mitchell, Gabriel AU - Harpaz, Roy AU - Schneidman, Elad AU - Tkacik, Gasper ID - 406 IS - 3 JF - PLoS One TI - Probabilistic models of individual and collective animal behavior VL - 13 ER - TY - JOUR AB - Temperate bacteriophages integrate in bacterial genomes as prophages and represent an important source of genetic variation for bacterial evolution, frequently transmitting fitness-augmenting genes such as toxins responsible for virulence of major pathogens. However, only a fraction of bacteriophage infections are lysogenic and lead to prophage acquisition, whereas the majority are lytic and kill the infected bacteria. Unless able to discriminate lytic from lysogenic infections, mechanisms of immunity to bacteriophages are expected to act as a double-edged sword and increase the odds of survival at the cost of depriving bacteria of potentially beneficial prophages. We show that although restriction-modification systems as mechanisms of innate immunity prevent both lytic and lysogenic infections indiscriminately in individual bacteria, they increase the number of prophage-acquiring individuals at the population level. We find that this counterintuitive result is a consequence of phage-host population dynamics, in which restriction-modification systems delay infection onset until bacteria reach densities at which the probability of lysogeny increases. These results underscore the importance of population-level dynamics as a key factor modulating costs and benefits of immunity to temperate bacteriophages AU - Pleska, Maros AU - Lang, Moritz AU - Refardt, Dominik AU - Levin, Bruce AU - Guet, Calin C ID - 457 IS - 2 JF - Nature Ecology and Evolution TI - Phage-host population dynamics promotes prophage acquisition in bacteria with innate immunity VL - 2 ER - TY - GEN AB - Implementation of the inference method in Matlab, including three applications of the method: The first one for the model of ant motion, the second one for bacterial chemotaxis, and the third one for the motion of fish. AU - Bod’Ová, Katarína AU - Mitchell, Gabriel AU - Harpaz, Roy AU - Schneidman, Elad AU - Tkačik, Gašper ID - 9831 TI - Implementation of the inference method in Matlab ER - TY - JOUR AB - Correlations in sensory neural networks have both extrinsic and intrinsic origins. Extrinsic or stimulus correlations arise from shared inputs to the network and, thus, depend strongly on the stimulus ensemble. Intrinsic or noise correlations reflect biophysical mechanisms of interactions between neurons, which are expected to be robust to changes in the stimulus ensemble. Despite the importance of this distinction for understanding how sensory networks encode information collectively, no method exists to reliably separate intrinsic interactions from extrinsic correlations in neural activity data, limiting our ability to build predictive models of the network response. In this paper we introduce a general strategy to infer population models of interacting neurons that collectively encode stimulus information. The key to disentangling intrinsic from extrinsic correlations is to infer the couplings between neurons separately from the encoding model and to combine the two using corrections calculated in a mean-field approximation. We demonstrate the effectiveness of this approach in retinal recordings. The same coupling network is inferred from responses to radically different stimulus ensembles, showing that these couplings indeed reflect stimulus-independent interactions between neurons. The inferred model predicts accurately the collective response of retinal ganglion cell populations as a function of the stimulus. AU - Ferrari, Ulisse AU - Deny, Stephane AU - Chalk, Matthew J AU - Tkacik, Gasper AU - Marre, Olivier AU - Mora, Thierry ID - 31 IS - 4 JF - Physical Review E SN - 24700045 TI - Separating intrinsic interactions from extrinsic correlations in a network of sensory neurons VL - 98 ER - TY - JOUR AB - A central goal in theoretical neuroscience is to predict the response properties of sensory neurons from first principles. To this end, “efficient coding” posits that sensory neurons encode maximal information about their inputs given internal constraints. There exist, however, many variants of efficient coding (e.g., redundancy reduction, different formulations of predictive coding, robust coding, sparse coding, etc.), differing in their regimes of applicability, in the relevance of signals to be encoded, and in the choice of constraints. It is unclear how these types of efficient coding relate or what is expected when different coding objectives are combined. Here we present a unified framework that encompasses previously proposed efficient coding models and extends to unique regimes. We show that optimizing neural responses to encode predictive information can lead them to either correlate or decorrelate their inputs, depending on the stimulus statistics; in contrast, at low noise, efficiently encoding the past always predicts decorrelation. Later, we investigate coding of naturalistic movies and show that qualitatively different types of visual motion tuning and levels of response sparsity are predicted, depending on whether the objective is to recover the past or predict the future. Our approach promises a way to explain the observed diversity of sensory neural responses, as due to multiple functional goals and constraints fulfilled by different cell types and/or circuits. AU - Chalk, Matthew J AU - Marre, Olivier AU - Tkacik, Gasper ID - 543 IS - 1 JF - PNAS TI - Toward a unified theory of efficient, predictive, and sparse coding VL - 115 ER - TY - JOUR AB - We study the Fokker-Planck equation derived in the large system limit of the Markovian process describing the dynamics of quantitative traits. The Fokker-Planck equation is posed on a bounded domain and its transport and diffusion coefficients vanish on the domain's boundary. We first argue that, despite this degeneracy, the standard no-flux boundary condition is valid. We derive the weak formulation of the problem and prove the existence and uniqueness of its solutions by constructing the corresponding contraction semigroup on a suitable function space. Then, we prove that for the parameter regime with high enough mutation rate the problem exhibits a positive spectral gap, which implies exponential convergence to equilibrium.Next, we provide a simple derivation of the so-called Dynamic Maximum Entropy (DynMaxEnt) method for approximation of observables (moments) of the Fokker-Planck solution, which can be interpreted as a nonlinear Galerkin approximation. The limited applicability of the DynMaxEnt method inspires us to introduce its modified version that is valid for the whole range of admissible parameters. Finally, we present several numerical experiments to demonstrate the performance of both the original and modified DynMaxEnt methods. We observe that in the parameter regimes where both methods are valid, the modified one exhibits slightly better approximation properties compared to the original one. AU - Bodova, Katarina AU - Haskovec, Jan AU - Markowich, Peter ID - 607 JF - Physica D: Nonlinear Phenomena TI - Well posedness and maximum entropy approximation for the dynamics of quantitative traits VL - 376-377 ER - TY - JOUR AB - Bacteria regulate genes to survive antibiotic stress, but regulation can be far from perfect. When regulation is not optimal, mutations that change gene expression can contribute to antibiotic resistance. It is not systematically understood to what extent natural gene regulation is or is not optimal for distinct antibiotics, and how changes in expression of specific genes quantitatively affect antibiotic resistance. Here we discover a simple quantitative relation between fitness, gene expression, and antibiotic potency, which rationalizes our observation that a multitude of genes and even innate antibiotic defense mechanisms have expression that is critically nonoptimal under antibiotic treatment. First, we developed a pooled-strain drug-diffusion assay and screened Escherichia coli overexpression and knockout libraries, finding that resistance to a range of 31 antibiotics could result from changing expression of a large and functionally diverse set of genes, in a primarily but not exclusively drug-specific manner. Second, by synthetically controlling the expression of single-drug and multidrug resistance genes, we observed that their fitness-expression functions changed dramatically under antibiotic treatment in accordance with a log-sensitivity relation. Thus, because many genes are nonoptimally expressed under antibiotic treatment, many regulatory mutations can contribute to resistance by altering expression and by activating latent defenses. AU - Palmer, Adam AU - Chait, Remy P AU - Kishony, Roy ID - 19 IS - 11 JF - Molecular Biology and Evolution SN - 0737-4038 TI - Nonoptimal gene expression creates latent potential for antibiotic resistance VL - 35 ER - TY - JOUR AB - Retina is a paradigmatic system for studying sensory encoding: the transformation of light into spiking activity of ganglion cells. The inverse problem, where stimulus is reconstructed from spikes, has received less attention, especially for complex stimuli that should be reconstructed “pixel-by-pixel”. We recorded around a hundred neurons from a dense patch in a rat retina and decoded movies of multiple small randomly-moving discs. We constructed nonlinear (kernelized and neural network) decoders that improved significantly over linear results. An important contribution to this was the ability of nonlinear decoders to reliably separate between neural responses driven by locally fluctuating light signals, and responses at locally constant light driven by spontaneous-like activity. This improvement crucially depended on the precise, non-Poisson temporal structure of individual spike trains, which originated in the spike-history dependence of neural responses. We propose a general principle by which downstream circuitry could discriminate between spontaneous and stimulus-driven activity based solely on higher-order statistical structure in the incoming spike trains. AU - Botella Soler, Vicent AU - Deny, Stephane AU - Martius, Georg S AU - Marre, Olivier AU - Tkacik, Gasper ID - 292 IS - 5 JF - PLoS Computational Biology TI - Nonlinear decoding of a complex movie from the mammalian retina VL - 14 ER - TY - JOUR AB - Which properties of metabolic networks can be derived solely from stoichiometry? Predictive results have been obtained by flux balance analysis (FBA), by postulating that cells set metabolic fluxes to maximize growth rate. Here we consider a generalization of FBA to single-cell level using maximum entropy modeling, which we extend and test experimentally. Specifically, we define for Escherichia coli metabolism a flux distribution that yields the experimental growth rate: the model, containing FBA as a limit, provides a better match to measured fluxes and it makes a wide range of predictions: on flux variability, regulation, and correlations; on the relative importance of stoichiometry vs. optimization; on scaling relations for growth rate distributions. We validate the latter here with single-cell data at different sub-inhibitory antibiotic concentrations. The model quantifies growth optimization as emerging from the interplay of competitive dynamics in the population and regulation of metabolism at the level of single cells. AU - De Martino, Daniele AU - Mc, Andersson Anna AU - Bergmiller, Tobias AU - Guet, Calin C AU - Tkacik, Gasper ID - 161 IS - 1 JF - Nature Communications TI - Statistical mechanics for metabolic networks during steady state growth VL - 9 ER - TY - JOUR AB - Gene regulatory networks evolve through rewiring of individual components—that is, through changes in regulatory connections. However, the mechanistic basis of regulatory rewiring is poorly understood. Using a canonical gene regulatory system, we quantify the properties of transcription factors that determine the evolutionary potential for rewiring of regulatory connections: robustness, tunability and evolvability. In vivo repression measurements of two repressors at mutated operator sites reveal their contrasting evolutionary potential: while robustness and evolvability were positively correlated, both were in trade-off with tunability. Epistatic interactions between adjacent operators alleviated this trade-off. A thermodynamic model explains how the differences in robustness, tunability and evolvability arise from biophysical characteristics of repressor–DNA binding. The model also uncovers that the energy matrix, which describes how mutations affect repressor–DNA binding, encodes crucial information about the evolutionary potential of a repressor. The biophysical determinants of evolutionary potential for regulatory rewiring constitute a mechanistic framework for understanding network evolution. AU - Igler, Claudia AU - Lagator, Mato AU - Tkacik, Gasper AU - Bollback, Jonathan P AU - Guet, Calin C ID - 67 IS - 10 JF - Nature Ecology and Evolution TI - Evolutionary potential of transcription factors for gene regulatory rewiring VL - 2 ER - TY - JOUR AB - Bacteria in groups vary individually, and interact with other bacteria and the environment to produce population-level patterns of gene expression. Investigating such behavior in detail requires measuring and controlling populations at the single-cell level alongside precisely specified interactions and environmental characteristics. Here we present an automated, programmable platform that combines image-based gene expression and growth measurements with on-line optogenetic expression control for hundreds of individual Escherichia coli cells over days, in a dynamically adjustable environment. This integrated platform broadly enables experiments that bridge individual and population behaviors. We demonstrate: (i) population structuring by independent closed-loop control of gene expression in many individual cells, (ii) cell-cell variation control during antibiotic perturbation, (iii) hybrid bio-digital circuits in single cells, and freely specifiable digital communication between individual bacteria. These examples showcase the potential for real-time integration of theoretical models with measurement and control of many individual cells to investigate and engineer microbial population behavior. AU - Chait, Remy P AU - Ruess, Jakob AU - Bergmiller, Tobias AU - Tkacik, Gasper AU - Guet, Calin C ID - 613 IS - 1 JF - Nature Communications SN - 20411723 TI - Shaping bacterial population behavior through computer interfaced control of individual cells VL - 8 ER - TY - CONF AB - We present an approach that enables robots to self-organize their sensorimotor behavior from scratch without providing specific information about neither the robot nor its environment. This is achieved by a simple neural control law that increases the consistency between external sensor dynamics and internal neural dynamics of the utterly simple controller. In this way, the embodiment and the agent-environment coupling are the only source of individual development. We show how an anthropomorphic tendon driven arm-shoulder system develops different behaviors depending on that coupling. For instance: Given a bottle half-filled with water, the arm starts to shake it, driven by the physical response of the water. When attaching a brush, the arm can be manipulated into wiping a table, and when connected to a revolvable wheel it finds out how to rotate it. Thus, the robot may be said to discover the affordances of the world. When allowing two (simulated) humanoid robots to interact physically, they engage into a joint behavior development leading to, for instance, spontaneous cooperation. More social effects are observed if the robots can visually perceive each other. Although, as an observer, it is tempting to attribute an apparent intentionality, there is nothing of the kind put in. As a conclusion, we argue that emergent behavior may be much less rooted in explicit intentions, internal motivations, or specific reward systems than is commonly believed. AU - Der, Ralf AU - Martius, Georg S ID - 652 SN - 978-150905069-7 TI - Dynamical self consistency leads to behavioral development and emergent social interactions in robots ER - TY - JOUR AB - With the accelerated development of robot technologies, control becomes one of the central themes of research. In traditional approaches, the controller, by its internal functionality, finds appropriate actions on the basis of specific objectives for the task at hand. While very successful in many applications, self-organized control schemes seem to be favored in large complex systems with unknown dynamics or which are difficult to model. Reasons are the expected scalability, robustness, and resilience of self-organizing systems. The paper presents a self-learning neurocontroller based on extrinsic differential plasticity introduced recently, applying it to an anthropomorphic musculoskeletal robot arm with attached objects of unknown physical dynamics. The central finding of the paper is the following effect: by the mere feedback through the internal dynamics of the object, the robot is learning to relate each of the objects with a very specific sensorimotor pattern. Specifically, an attached pendulum pilots the arm into a circular motion, a half-filled bottle produces axis oriented shaking behavior, a wheel is getting rotated, and wiping patterns emerge automatically in a table-plus-brush setting. By these object-specific dynamical patterns, the robot may be said to recognize the object's identity, or in other words, it discovers dynamical affordances of objects. Furthermore, when including hand coordinates obtained from a camera, a dedicated hand-eye coordination self-organizes spontaneously. These phenomena are discussed from a specific dynamical system perspective. Central is the dedicated working regime at the border to instability with its potentially infinite reservoir of (limit cycle) attractors "waiting" to be excited. Besides converging toward one of these attractors, variate behavior is also arising from a self-induced attractor morphing driven by the learning rule. We claim that experimental investigations with this anthropomorphic, self-learning robot not only generate interesting and potentially useful behaviors, but may also help to better understand what subjective human muscle feelings are, how they can be rooted in sensorimotor patterns, and how these concepts may feed back on robotics. AU - Der, Ralf AU - Martius, Georg S ID - 658 IS - MAR JF - Frontiers in Neurorobotics SN - 16625218 TI - Self organized behavior generation for musculoskeletal robots VL - 11 ER - TY - JOUR AB - Advances in multi-unit recordings pave the way for statistical modeling of activity patterns in large neural populations. Recent studies have shown that the summed activity of all neurons strongly shapes the population response. A separate recent finding has been that neural populations also exhibit criticality, an anomalously large dynamic range for the probabilities of different population activity patterns. Motivated by these two observations, we introduce a class of probabilistic models which takes into account the prior knowledge that the neural population could be globally coupled and close to critical. These models consist of an energy function which parametrizes interactions between small groups of neurons, and an arbitrary positive, strictly increasing, and twice differentiable function which maps the energy of a population pattern to its probability. We show that: 1) augmenting a pairwise Ising model with a nonlinearity yields an accurate description of the activity of retinal ganglion cells which outperforms previous models based on the summed activity of neurons; 2) prior knowledge that the population is critical translates to prior expectations about the shape of the nonlinearity; 3) the nonlinearity admits an interpretation in terms of a continuous latent variable globally coupling the system whose distribution we can infer from data. Our method is independent of the underlying system’s state space; hence, it can be applied to other systems such as natural scenes or amino acid sequences of proteins which are also known to exhibit criticality. AU - Humplik, Jan AU - Tkacik, Gasper ID - 720 IS - 9 JF - PLoS Computational Biology SN - 1553734X TI - Probabilistic models for neural populations that naturally capture global coupling and criticality VL - 13 ER -