---
res:
bibo_abstract:
- "We show that statistical criticality, i.e. the occurrence of power law frequency
distributions, arises in samples that are maximally informative about the underlying
generating process. In order to reach this conclusion, we first identify the frequency
with which different outcomes occur in a sample, as the variable carrying useful
information on the generative process. The entropy of the frequency, that we call
relevance, provides an upper bound to the number of informative bits. This differs
from the entropy of the data, that we take as a measure of resolution. Samples
that maximise relevance at a given resolutionâ€”that we call maximally informative
samplesâ€”exhibit statistical criticality. In particular, Zipf's law arises at the
optimal trade-off between resolution (i.e. compression) and relevance. As a byproduct,
we derive a bound of the maximal number of parameters that can be estimated from
a dataset, in the absence of prior knowledge on the generative model.\r\n\r\nFurthermore,
we relate criticality to the statistical properties of the representation of the
data generating process. We show that, as a consequence of the concentration property
of the asymptotic equipartition property, representations that are maximally informative
about the data generating process are characterised by an exponential distribution
of energy levels. This arises from a principle of minimal entropy, that is conjugate
of the maximum entropy principle in statistical mechanics. This explains why statistical
criticality requires no parameter fine tuning in maximally informative samples.@eng"
bibo_authorlist:
- foaf_Person:
foaf_givenName: Ryan J
foaf_name: Cubero, Ryan J
foaf_surname: Cubero
foaf_workInfoHomepage: http://www.librecat.org/personId=850B2E12-9CD4-11E9-837F-E719E6697425
orcid: 0000-0003-0002-1867
- foaf_Person:
foaf_givenName: Junghyo
foaf_name: Jo, Junghyo
foaf_surname: Jo
- foaf_Person:
foaf_givenName: Matteo
foaf_name: Marsili, Matteo
foaf_surname: Marsili
- foaf_Person:
foaf_givenName: Yasser
foaf_name: Roudi, Yasser
foaf_surname: Roudi
- foaf_Person:
foaf_givenName: Juyong
foaf_name: Song, Juyong
foaf_surname: Song
bibo_doi: 10.1088/1742-5468/ab16c8
bibo_issue: '6'
bibo_volume: 2019
dct_date: 2019^xs_gYear
dct_isPartOf:
- http://id.crossref.org/issn/1742-5468
dct_language: eng
dct_publisher: IOP Publishing@
dct_subject:
- optimization under uncertainty
- source coding
- large deviation
dct_title: Statistical criticality arises in most informative representations@
...