A tight upper bound on mutual information

M. Hledik, T.R. Sokolowski, G. Tkačik, in:, IEEE Information Theory Workshop, ITW 2019, IEEE, 2019, p. 8989292.


Conference Paper | Published | English

Scopus indexed
Department
Abstract
We derive a tight lower bound on equivocation (conditional entropy), or equivalently a tight upper bound on mutual information between a signal variable and channel outputs. The bound is in terms of the joint distribution of the signals and maximum a posteriori decodes (most probable signals given channel output). As part of our derivation, we describe the key properties of the distribution of signals, channel outputs and decodes, that minimizes equivocation and maximizes mutual information. This work addresses a problem in data analysis, where mutual information between signals and decodes is sometimes used to lower bound the mutual information between signals and channel outputs. Our result provides a corresponding upper bound.
Publishing Year
Date Published
2019-08-01
Proceedings Title
IEEE Information Theory Workshop, ITW 2019
Article Number
8989292
Conference
Information Theory Workshop
Conference Location
Visby, Sweden
Conference Date
2019-08-25 – 2019-08-28
IST-REx-ID

Cite this

Hledik M, Sokolowski TR, Tkačik G. A tight upper bound on mutual information. In: IEEE Information Theory Workshop, ITW 2019. IEEE; 2019:8989292. doi:10.1109/ITW44776.2019.8989292
Hledik, M., Sokolowski, T. R., & Tkačik, G. (2019). A tight upper bound on mutual information. In IEEE Information Theory Workshop, ITW 2019 (p. 8989292). Visby, Sweden: IEEE. https://doi.org/10.1109/ITW44776.2019.8989292
Hledik, Michal, Thomas R Sokolowski, and Gašper Tkačik. “A Tight Upper Bound on Mutual Information.” In IEEE Information Theory Workshop, ITW 2019, 8989292. IEEE, 2019. https://doi.org/10.1109/ITW44776.2019.8989292.
M. Hledik, T. R. Sokolowski, and G. Tkačik, “A tight upper bound on mutual information,” in IEEE Information Theory Workshop, ITW 2019, Visby, Sweden, 2019, p. 8989292.
Hledik M, Sokolowski TR, Tkačik G. 2019. A tight upper bound on mutual information. IEEE Information Theory Workshop, ITW 2019. Information Theory Workshop 8989292.
Hledik, Michal, et al. “A Tight Upper Bound on Mutual Information.” IEEE Information Theory Workshop, ITW 2019, IEEE, 2019, p. 8989292, doi:10.1109/ITW44776.2019.8989292.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data IST Research Explorer

Sources

arXiv 1812.01475

Search this title in

Google Scholar
ISBN Search