--- _id: '6589' abstract: - lang: eng text: Distributed training of massive machine learning models, in particular deep neural networks, via Stochastic Gradient Descent (SGD) is becoming commonplace. Several families of communication-reduction methods, such as quantization, large-batch methods, and gradient sparsification, have been proposed. To date, gradient sparsification methods--where each node sorts gradients by magnitude, and only communicates a subset of the components, accumulating the rest locally--are known to yield some of the largest practical gains. Such methods can reduce the amount of communication per step by up to \emph{three orders of magnitude}, while preserving model accuracy. Yet, this family of methods currently has no theoretical justification. This is the question we address in this paper. We prove that, under analytic assumptions, sparsifying gradients by magnitude with local error correction provides convergence guarantees, for both convex and non-convex smooth objectives, for data-parallel SGD. The main insight is that sparsification methods implicitly maintain bounds on the maximum impact of stale updates, thanks to selection by magnitude. Our analysis and empirical validation also reveal that these methods do require analytical conditions to converge well, justifying existing heuristics. article_processing_charge: No author: - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Torsten full_name: Hoefler, Torsten last_name: Hoefler - first_name: Mikael full_name: Johansson, Mikael last_name: Johansson - first_name: Nikola H full_name: Konstantinov, Nikola H id: 4B9D76E4-F248-11E8-B48F-1D18A9856A87 last_name: Konstantinov - first_name: Sarit full_name: Khirirat, Sarit last_name: Khirirat - first_name: Cedric full_name: Renggli, Cedric last_name: Renggli citation: ama: 'Alistarh D-A, Hoefler T, Johansson M, Konstantinov NH, Khirirat S, Renggli C. The convergence of sparsified gradient methods. In: Advances in Neural Information Processing Systems 31. Vol Volume 2018. Neural Information Processing Systems Foundation; 2018:5973-5983.' apa: 'Alistarh, D.-A., Hoefler, T., Johansson, M., Konstantinov, N. H., Khirirat, S., & Renggli, C. (2018). The convergence of sparsified gradient methods. In Advances in Neural Information Processing Systems 31 (Vol. Volume 2018, pp. 5973–5983). Montreal, Canada: Neural Information Processing Systems Foundation.' chicago: Alistarh, Dan-Adrian, Torsten Hoefler, Mikael Johansson, Nikola H Konstantinov, Sarit Khirirat, and Cedric Renggli. “The Convergence of Sparsified Gradient Methods.” In Advances in Neural Information Processing Systems 31, Volume 2018:5973–83. Neural Information Processing Systems Foundation, 2018. ieee: D.-A. Alistarh, T. Hoefler, M. Johansson, N. H. Konstantinov, S. Khirirat, and C. Renggli, “The convergence of sparsified gradient methods,” in Advances in Neural Information Processing Systems 31, Montreal, Canada, 2018, vol. Volume 2018, pp. 5973–5983. ista: 'Alistarh D-A, Hoefler T, Johansson M, Konstantinov NH, Khirirat S, Renggli C. 2018. The convergence of sparsified gradient methods. Advances in Neural Information Processing Systems 31. NeurIPS: Conference on Neural Information Processing Systems vol. Volume 2018, 5973–5983.' mla: Alistarh, Dan-Adrian, et al. “The Convergence of Sparsified Gradient Methods.” Advances in Neural Information Processing Systems 31, vol. Volume 2018, Neural Information Processing Systems Foundation, 2018, pp. 5973–83. short: D.-A. Alistarh, T. Hoefler, M. Johansson, N.H. Konstantinov, S. Khirirat, C. Renggli, in:, Advances in Neural Information Processing Systems 31, Neural Information Processing Systems Foundation, 2018, pp. 5973–5983. conference: end_date: 2018-12-08 location: Montreal, Canada name: 'NeurIPS: Conference on Neural Information Processing Systems' start_date: 2018-12-02 date_created: 2019-06-27T09:32:55Z date_published: 2018-12-01T00:00:00Z date_updated: 2023-10-17T11:47:20Z day: '01' department: - _id: DaAl - _id: ChLa ec_funded: 1 external_id: arxiv: - '1809.10505' isi: - '000461852000047' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1809.10505 month: '12' oa: 1 oa_version: Preprint page: 5973-5983 project: - _id: 2564DBCA-B435-11E9-9278-68D0E5697425 call_identifier: H2020 grant_number: '665385' name: International IST Doctoral Program publication: Advances in Neural Information Processing Systems 31 publication_status: published publisher: Neural Information Processing Systems Foundation quality_controlled: '1' scopus_import: '1' status: public title: The convergence of sparsified gradient methods type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: Volume 2018 year: '2018' ... --- _id: '7' abstract: - lang: eng text: Animal social networks are shaped by multiple selection pressures, including the need to ensure efficient communication and functioning while simultaneously limiting disease transmission. Social animals could potentially further reduce epidemic risk by altering their social networks in the presence of pathogens, yet there is currently no evidence for such pathogen-triggered responses. We tested this hypothesis experimentally in the ant Lasius niger using a combination of automated tracking, controlled pathogen exposure, transmission quantification, and temporally explicit simulations. Pathogen exposure induced behavioral changes in both exposed ants and their nestmates, which helped contain the disease by reinforcing key transmission-inhibitory properties of the colony's contact network. This suggests that social network plasticity in response to pathogens is an effective strategy for mitigating the effects of disease in social groups. acknowledgement: This project was funded by two European Research Council Advanced Grants (Social Life, 249375, and resiliANT, 741491) and two Swiss National Science Foundation grants (CR32I3_141063 and 310030_156732) to L.K. and a European Research Council Starting Grant (SocialVaccines, 243071) to S.C. article_processing_charge: No article_type: original author: - first_name: Nathalie full_name: Stroeymeyt, Nathalie last_name: Stroeymeyt - first_name: Anna V full_name: Grasse, Anna V id: 406F989C-F248-11E8-B48F-1D18A9856A87 last_name: Grasse - first_name: Alessandro full_name: Crespi, Alessandro last_name: Crespi - first_name: Danielle full_name: Mersch, Danielle last_name: Mersch - first_name: Sylvia full_name: Cremer, Sylvia id: 2F64EC8C-F248-11E8-B48F-1D18A9856A87 last_name: Cremer orcid: 0000-0002-2193-3868 - first_name: Laurent full_name: Keller, Laurent last_name: Keller citation: ama: Stroeymeyt N, Grasse AV, Crespi A, Mersch D, Cremer S, Keller L. Social network plasticity decreases disease transmission in a eusocial insect. Science. 2018;362(6417):941-945. doi:10.1126/science.aat4793 apa: Stroeymeyt, N., Grasse, A. V., Crespi, A., Mersch, D., Cremer, S., & Keller, L. (2018). Social network plasticity decreases disease transmission in a eusocial insect. Science. AAAS. https://doi.org/10.1126/science.aat4793 chicago: Stroeymeyt, Nathalie, Anna V Grasse, Alessandro Crespi, Danielle Mersch, Sylvia Cremer, and Laurent Keller. “Social Network Plasticity Decreases Disease Transmission in a Eusocial Insect.” Science. AAAS, 2018. https://doi.org/10.1126/science.aat4793. ieee: N. Stroeymeyt, A. V. Grasse, A. Crespi, D. Mersch, S. Cremer, and L. Keller, “Social network plasticity decreases disease transmission in a eusocial insect,” Science, vol. 362, no. 6417. AAAS, pp. 941–945, 2018. ista: Stroeymeyt N, Grasse AV, Crespi A, Mersch D, Cremer S, Keller L. 2018. Social network plasticity decreases disease transmission in a eusocial insect. Science. 362(6417), 941–945. mla: Stroeymeyt, Nathalie, et al. “Social Network Plasticity Decreases Disease Transmission in a Eusocial Insect.” Science, vol. 362, no. 6417, AAAS, 2018, pp. 941–45, doi:10.1126/science.aat4793. short: N. Stroeymeyt, A.V. Grasse, A. Crespi, D. Mersch, S. Cremer, L. Keller, Science 362 (2018) 941–945. date_created: 2018-12-11T11:44:07Z date_published: 2018-11-23T00:00:00Z date_updated: 2023-10-17T11:50:05Z day: '23' department: - _id: SyCr doi: 10.1126/science.aat4793 ec_funded: 1 external_id: isi: - '000451124500041' intvolume: ' 362' isi: 1 issue: '6417' language: - iso: eng main_file_link: - open_access: '1' url: https://serval.unil.ch/resource/serval:BIB_E9228C205467.P001/REF.pdf month: '11' oa: 1 oa_version: Published Version page: 941 - 945 project: - _id: 25DC711C-B435-11E9-9278-68D0E5697425 call_identifier: FP7 grant_number: '243071' name: 'Social Vaccination in Ant Colonies: from Individual Mechanisms to Society Effects' publication: Science publication_identifier: issn: - 1095-9203 publication_status: published publisher: AAAS publist_id: '8049' quality_controlled: '1' related_material: link: - description: News on IST Homepage relation: press_release url: https://ist.ac.at/en/news/for-ants-unity-is-strength-and-health/ record: - id: '13055' relation: research_data status: public scopus_import: '1' status: public title: Social network plasticity decreases disease transmission in a eusocial insect type: journal_article user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 362 year: '2018' ... --- _id: '19' abstract: - lang: eng text: Bacteria regulate genes to survive antibiotic stress, but regulation can be far from perfect. When regulation is not optimal, mutations that change gene expression can contribute to antibiotic resistance. It is not systematically understood to what extent natural gene regulation is or is not optimal for distinct antibiotics, and how changes in expression of specific genes quantitatively affect antibiotic resistance. Here we discover a simple quantitative relation between fitness, gene expression, and antibiotic potency, which rationalizes our observation that a multitude of genes and even innate antibiotic defense mechanisms have expression that is critically nonoptimal under antibiotic treatment. First, we developed a pooled-strain drug-diffusion assay and screened Escherichia coli overexpression and knockout libraries, finding that resistance to a range of 31 antibiotics could result from changing expression of a large and functionally diverse set of genes, in a primarily but not exclusively drug-specific manner. Second, by synthetically controlling the expression of single-drug and multidrug resistance genes, we observed that their fitness-expression functions changed dramatically under antibiotic treatment in accordance with a log-sensitivity relation. Thus, because many genes are nonoptimally expressed under antibiotic treatment, many regulatory mutations can contribute to resistance by altering expression and by activating latent defenses. article_processing_charge: No article_type: original author: - first_name: Adam full_name: Palmer, Adam last_name: Palmer - first_name: Remy P full_name: Chait, Remy P id: 3464AE84-F248-11E8-B48F-1D18A9856A87 last_name: Chait orcid: 0000-0003-0876-3187 - first_name: Roy full_name: Kishony, Roy last_name: Kishony citation: ama: Palmer A, Chait RP, Kishony R. Nonoptimal gene expression creates latent potential for antibiotic resistance. Molecular Biology and Evolution. 2018;35(11):2669-2684. doi:10.1093/molbev/msy163 apa: Palmer, A., Chait, R. P., & Kishony, R. (2018). Nonoptimal gene expression creates latent potential for antibiotic resistance. Molecular Biology and Evolution. Oxford University Press. https://doi.org/10.1093/molbev/msy163 chicago: Palmer, Adam, Remy P Chait, and Roy Kishony. “Nonoptimal Gene Expression Creates Latent Potential for Antibiotic Resistance.” Molecular Biology and Evolution. Oxford University Press, 2018. https://doi.org/10.1093/molbev/msy163. ieee: A. Palmer, R. P. Chait, and R. Kishony, “Nonoptimal gene expression creates latent potential for antibiotic resistance,” Molecular Biology and Evolution, vol. 35, no. 11. Oxford University Press, pp. 2669–2684, 2018. ista: Palmer A, Chait RP, Kishony R. 2018. Nonoptimal gene expression creates latent potential for antibiotic resistance. Molecular Biology and Evolution. 35(11), 2669–2684. mla: Palmer, Adam, et al. “Nonoptimal Gene Expression Creates Latent Potential for Antibiotic Resistance.” Molecular Biology and Evolution, vol. 35, no. 11, Oxford University Press, 2018, pp. 2669–84, doi:10.1093/molbev/msy163. short: A. Palmer, R.P. Chait, R. Kishony, Molecular Biology and Evolution 35 (2018) 2669–2684. date_created: 2018-12-11T11:44:11Z date_published: 2018-08-28T00:00:00Z date_updated: 2023-10-17T11:51:06Z day: '28' department: - _id: CaGu - _id: GaTk doi: 10.1093/molbev/msy163 external_id: isi: - '000452567200006' pmid: - '30169679' intvolume: ' 35' isi: 1 issue: '11' language: - iso: eng main_file_link: - open_access: '1' url: https://www.ncbi.nlm.nih.gov/pubmed/30169679 month: '08' oa: 1 oa_version: Submitted Version page: 2669 - 2684 pmid: 1 publication: Molecular Biology and Evolution publication_identifier: issn: - 0737-4038 publication_status: published publisher: Oxford University Press publist_id: '8036' quality_controlled: '1' scopus_import: '1' status: public title: Nonoptimal gene expression creates latent potential for antibiotic resistance type: journal_article user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 35 year: '2018' ... --- _id: '13055' abstract: - lang: eng text: "Dataset for manuscript 'Social network plasticity decreases disease transmission in a eusocial insect'\r\nCompared to previous versions: - raw image files added\r\n \ - correction of URLs within README.txt file\r\n" article_processing_charge: No author: - first_name: Nathalie full_name: Stroeymeyt, Nathalie last_name: Stroeymeyt - first_name: Anna V full_name: Grasse, Anna V id: 406F989C-F248-11E8-B48F-1D18A9856A87 last_name: Grasse - first_name: Alessandro full_name: Crespi, Alessandro last_name: Crespi - first_name: Danielle full_name: Mersch, Danielle last_name: Mersch - first_name: Sylvia full_name: Cremer, Sylvia id: 2F64EC8C-F248-11E8-B48F-1D18A9856A87 last_name: Cremer orcid: 0000-0002-2193-3868 - first_name: Laurent full_name: Keller, Laurent last_name: Keller citation: ama: Stroeymeyt N, Grasse AV, Crespi A, Mersch D, Cremer S, Keller L. Social network plasticity decreases disease transmission in a eusocial insect. 2018. doi:10.5281/ZENODO.1322669 apa: Stroeymeyt, N., Grasse, A. V., Crespi, A., Mersch, D., Cremer, S., & Keller, L. (2018). Social network plasticity decreases disease transmission in a eusocial insect. Zenodo. https://doi.org/10.5281/ZENODO.1322669 chicago: Stroeymeyt, Nathalie, Anna V Grasse, Alessandro Crespi, Danielle Mersch, Sylvia Cremer, and Laurent Keller. “Social Network Plasticity Decreases Disease Transmission in a Eusocial Insect.” Zenodo, 2018. https://doi.org/10.5281/ZENODO.1322669. ieee: N. Stroeymeyt, A. V. Grasse, A. Crespi, D. Mersch, S. Cremer, and L. Keller, “Social network plasticity decreases disease transmission in a eusocial insect.” Zenodo, 2018. ista: Stroeymeyt N, Grasse AV, Crespi A, Mersch D, Cremer S, Keller L. 2018. Social network plasticity decreases disease transmission in a eusocial insect, Zenodo, 10.5281/ZENODO.1322669. mla: Stroeymeyt, Nathalie, et al. Social Network Plasticity Decreases Disease Transmission in a Eusocial Insect. Zenodo, 2018, doi:10.5281/ZENODO.1322669. short: N. Stroeymeyt, A.V. Grasse, A. Crespi, D. Mersch, S. Cremer, L. Keller, (2018). date_created: 2023-05-23T13:24:51Z date_published: 2018-10-23T00:00:00Z date_updated: 2023-10-17T11:50:04Z day: '23' ddc: - '570' department: - _id: SyCr doi: 10.5281/ZENODO.1322669 main_file_link: - open_access: '1' url: https://doi.org/10.5281/zenodo.1480665 month: '10' oa: 1 oa_version: Published Version publisher: Zenodo related_material: record: - id: '7' relation: used_in_publication status: public status: public title: Social network plasticity decreases disease transmission in a eusocial insect tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: research_data_reference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2018' ... --- _id: '22' abstract: - lang: eng text: Conventional ultra-high sensitivity detectors in the millimeter-wave range are usually cooled as their own thermal noise at room temperature would mask the weak received radiation. The need for cryogenic systems increases the cost and complexity of the instruments, hindering the development of, among others, airborne and space applications. In this work, the nonlinear parametric upconversion of millimeter-wave radiation to the optical domain inside high-quality (Q) lithium niobate whispering-gallery mode (WGM) resonators is proposed for ultra-low noise detection. We experimentally demonstrate coherent upconversion of millimeter-wave signals to a 1550 nm telecom carrier, with a photon conversion efficiency surpassing the state-of-the-art by 2 orders of magnitude. Moreover, a theoretical model shows that the thermal equilibrium of counterpropagating WGMs is broken by overcoupling the millimeter-wave WGM, effectively cooling the upconverted mode and allowing ultra-low noise detection. By theoretically estimating the sensitivity of a correlation radiometer based on the presented scheme, it is found that room-temperature radiometers with better sensitivity than state-of-the-art high-electron-mobility transistor (HEMT)-based radiometers can be designed. This detection paradigm can be used to develop room-temperature instrumentation for radio astronomy, earth observation, planetary missions, and imaging systems. article_processing_charge: No article_type: original author: - first_name: Gabriel full_name: Botello, Gabriel last_name: Botello - first_name: Florian full_name: Sedlmeir, Florian last_name: Sedlmeir - first_name: Alfredo R full_name: Rueda Sanchez, Alfredo R id: 3B82B0F8-F248-11E8-B48F-1D18A9856A87 last_name: Rueda Sanchez orcid: 0000-0001-6249-5860 - first_name: Kerlos full_name: Abdalmalak, Kerlos last_name: Abdalmalak - first_name: Elliott full_name: Brown, Elliott last_name: Brown - first_name: Gerd full_name: Leuchs, Gerd last_name: Leuchs - first_name: Sascha full_name: Preu, Sascha last_name: Preu - first_name: Daniel full_name: Segovia Vargas, Daniel last_name: Segovia Vargas - first_name: Dmitry full_name: Strekalov, Dmitry last_name: Strekalov - first_name: Luis full_name: Munoz, Luis last_name: Munoz - first_name: Harald full_name: Schwefel, Harald last_name: Schwefel citation: ama: Botello G, Sedlmeir F, Rueda Sanchez AR, et al. Sensitivity limits of millimeter-wave photonic radiometers based on efficient electro-optic upconverters. Optica. 2018;5(10):1210-1219. doi:10.1364/OPTICA.5.001210 apa: Botello, G., Sedlmeir, F., Rueda Sanchez, A. R., Abdalmalak, K., Brown, E., Leuchs, G., … Schwefel, H. (2018). Sensitivity limits of millimeter-wave photonic radiometers based on efficient electro-optic upconverters. Optica. https://doi.org/10.1364/OPTICA.5.001210 chicago: Botello, Gabriel, Florian Sedlmeir, Alfredo R Rueda Sanchez, Kerlos Abdalmalak, Elliott Brown, Gerd Leuchs, Sascha Preu, et al. “Sensitivity Limits of Millimeter-Wave Photonic Radiometers Based on Efficient Electro-Optic Upconverters.” Optica, 2018. https://doi.org/10.1364/OPTICA.5.001210. ieee: G. Botello et al., “Sensitivity limits of millimeter-wave photonic radiometers based on efficient electro-optic upconverters,” Optica, vol. 5, no. 10. pp. 1210–1219, 2018. ista: Botello G, Sedlmeir F, Rueda Sanchez AR, Abdalmalak K, Brown E, Leuchs G, Preu S, Segovia Vargas D, Strekalov D, Munoz L, Schwefel H. 2018. Sensitivity limits of millimeter-wave photonic radiometers based on efficient electro-optic upconverters. Optica. 5(10), 1210–1219. mla: Botello, Gabriel, et al. “Sensitivity Limits of Millimeter-Wave Photonic Radiometers Based on Efficient Electro-Optic Upconverters.” Optica, vol. 5, no. 10, 2018, pp. 1210–19, doi:10.1364/OPTICA.5.001210. short: G. Botello, F. Sedlmeir, A.R. Rueda Sanchez, K. Abdalmalak, E. Brown, G. Leuchs, S. Preu, D. Segovia Vargas, D. Strekalov, L. Munoz, H. Schwefel, Optica 5 (2018) 1210–1219. date_created: 2018-12-11T11:44:12Z date_published: 2018-10-20T00:00:00Z date_updated: 2023-10-17T12:12:40Z day: '20' department: - _id: JoFi doi: 10.1364/OPTICA.5.001210 external_id: isi: - '000447853100007' intvolume: ' 5' isi: 1 issue: '10' language: - iso: eng main_file_link: - open_access: '1' url: 'www.doi.org/10.1364/OPTICA.5.001210 ' month: '10' oa: 1 oa_version: Published Version page: 1210 - 1219 publication: Optica publication_identifier: issn: - '23342536' publication_status: published publist_id: '8033' quality_controlled: '1' scopus_import: '1' status: public title: Sensitivity limits of millimeter-wave photonic radiometers based on efficient electro-optic upconverters type: journal_article user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 5 year: '2018' ...