The derivation of mutual information and covariance function using centered random variables

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Information theoretic measures such as Mutual Information are often said to be able to measure nonlinear dependencies whereas covariance (and correlation) are able to measure only linear dependencies. We aim to illustrate this claim using centered random variables. The set of centered random variable Fc={-q-12,-q-12+1,..,q-12-1,q-12} is mapped from F = {1,2,.., q - 1, q}. For q=2, we derive the relationship between the Mutual Information function, I, and the covariance function, Γ, and show that Γ=0→I=0. Furthermore we show that when q=3, the nonlinearities are captured by Mutual Information by highlighting a case where Γ=0 I=0.

Original languageEnglish
Title of host publicationAIP Conference Proceedings
PublisherAmerican Institute of Physics Inc.
Pages883-889
Number of pages7
Volume1635
ISBN (Print)9780735412743
DOIs
Publication statusPublished - 2014
Event3rd International Conference on Quantitative Sciences and Its Applications: Fostering Innovation, Streamlining Development, ICOQSIA 2014 - Langkawi, Kedah
Duration: 12 Aug 201414 Aug 2014

Other

Other3rd International Conference on Quantitative Sciences and Its Applications: Fostering Innovation, Streamlining Development, ICOQSIA 2014
CityLangkawi, Kedah
Period12/8/1414/8/14

Fingerprint

random variables
derivation
nonlinearity

Keywords

  • Centered random variable
  • Mutual Information and covariance
  • nonlinear dependence

ASJC Scopus subject areas

  • Physics and Astronomy(all)

Cite this

Abdul Razak, F. (2014). The derivation of mutual information and covariance function using centered random variables. In AIP Conference Proceedings (Vol. 1635, pp. 883-889). American Institute of Physics Inc.. https://doi.org/10.1063/1.4903687

The derivation of mutual information and covariance function using centered random variables. / Abdul Razak, Fatimah.

AIP Conference Proceedings. Vol. 1635 American Institute of Physics Inc., 2014. p. 883-889.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abdul Razak, F 2014, The derivation of mutual information and covariance function using centered random variables. in AIP Conference Proceedings. vol. 1635, American Institute of Physics Inc., pp. 883-889, 3rd International Conference on Quantitative Sciences and Its Applications: Fostering Innovation, Streamlining Development, ICOQSIA 2014, Langkawi, Kedah, 12/8/14. https://doi.org/10.1063/1.4903687
Abdul Razak F. The derivation of mutual information and covariance function using centered random variables. In AIP Conference Proceedings. Vol. 1635. American Institute of Physics Inc. 2014. p. 883-889 https://doi.org/10.1063/1.4903687
Abdul Razak, Fatimah. / The derivation of mutual information and covariance function using centered random variables. AIP Conference Proceedings. Vol. 1635 American Institute of Physics Inc., 2014. pp. 883-889
@inproceedings{5c5ca77ecd9647c9b9d26e4d6b90ad9b,
title = "The derivation of mutual information and covariance function using centered random variables",
abstract = "Information theoretic measures such as Mutual Information are often said to be able to measure nonlinear dependencies whereas covariance (and correlation) are able to measure only linear dependencies. We aim to illustrate this claim using centered random variables. The set of centered random variable Fc={-q-12,-q-12+1,..,q-12-1,q-12} is mapped from F = {1,2,.., q - 1, q}. For q=2, we derive the relationship between the Mutual Information function, I, and the covariance function, Γ, and show that Γ=0→I=0. Furthermore we show that when q=3, the nonlinearities are captured by Mutual Information by highlighting a case where Γ=0 I=0.",
keywords = "Centered random variable, Mutual Information and covariance, nonlinear dependence",
author = "{Abdul Razak}, Fatimah",
year = "2014",
doi = "10.1063/1.4903687",
language = "English",
isbn = "9780735412743",
volume = "1635",
pages = "883--889",
booktitle = "AIP Conference Proceedings",
publisher = "American Institute of Physics Inc.",

}

TY - GEN

T1 - The derivation of mutual information and covariance function using centered random variables

AU - Abdul Razak, Fatimah

PY - 2014

Y1 - 2014

N2 - Information theoretic measures such as Mutual Information are often said to be able to measure nonlinear dependencies whereas covariance (and correlation) are able to measure only linear dependencies. We aim to illustrate this claim using centered random variables. The set of centered random variable Fc={-q-12,-q-12+1,..,q-12-1,q-12} is mapped from F = {1,2,.., q - 1, q}. For q=2, we derive the relationship between the Mutual Information function, I, and the covariance function, Γ, and show that Γ=0→I=0. Furthermore we show that when q=3, the nonlinearities are captured by Mutual Information by highlighting a case where Γ=0 I=0.

AB - Information theoretic measures such as Mutual Information are often said to be able to measure nonlinear dependencies whereas covariance (and correlation) are able to measure only linear dependencies. We aim to illustrate this claim using centered random variables. The set of centered random variable Fc={-q-12,-q-12+1,..,q-12-1,q-12} is mapped from F = {1,2,.., q - 1, q}. For q=2, we derive the relationship between the Mutual Information function, I, and the covariance function, Γ, and show that Γ=0→I=0. Furthermore we show that when q=3, the nonlinearities are captured by Mutual Information by highlighting a case where Γ=0 I=0.

KW - Centered random variable

KW - Mutual Information and covariance

KW - nonlinear dependence

UR - http://www.scopus.com/inward/record.url?scp=84916613264&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84916613264&partnerID=8YFLogxK

U2 - 10.1063/1.4903687

DO - 10.1063/1.4903687

M3 - Conference contribution

SN - 9780735412743

VL - 1635

SP - 883

EP - 889

BT - AIP Conference Proceedings

PB - American Institute of Physics Inc.

ER -