### Abstract

Information theoretic measures such as Mutual Information are often said to be able to measure nonlinear dependencies whereas covariance (and correlation) are able to measure only linear dependencies. We aim to illustrate this claim using centered random variables. The set of centered random variable Fc={-q-12,-q-12+1,..,q-12-1,q-12} is mapped from F = {1,2,.., q - 1, q}. For q=2, we derive the relationship between the Mutual Information function, I, and the covariance function, Γ, and show that Γ=0→I=0. Furthermore we show that when q=3, the nonlinearities are captured by Mutual Information by highlighting a case where Γ=0 I=0.

Original language | English |
---|---|

Title of host publication | AIP Conference Proceedings |

Publisher | American Institute of Physics Inc. |

Pages | 883-889 |

Number of pages | 7 |

Volume | 1635 |

ISBN (Print) | 9780735412743 |

DOIs | |

Publication status | Published - 2014 |

Event | 3rd International Conference on Quantitative Sciences and Its Applications: Fostering Innovation, Streamlining Development, ICOQSIA 2014 - Langkawi, Kedah Duration: 12 Aug 2014 → 14 Aug 2014 |

### Other

Other | 3rd International Conference on Quantitative Sciences and Its Applications: Fostering Innovation, Streamlining Development, ICOQSIA 2014 |
---|---|

City | Langkawi, Kedah |

Period | 12/8/14 → 14/8/14 |

### Fingerprint

### Keywords

- Centered random variable
- Mutual Information and covariance
- nonlinear dependence

### ASJC Scopus subject areas

- Physics and Astronomy(all)

### Cite this

*AIP Conference Proceedings*(Vol. 1635, pp. 883-889). American Institute of Physics Inc.. https://doi.org/10.1063/1.4903687

**The derivation of mutual information and covariance function using centered random variables.** / Abdul Razak, Fatimah.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

*AIP Conference Proceedings.*vol. 1635, American Institute of Physics Inc., pp. 883-889, 3rd International Conference on Quantitative Sciences and Its Applications: Fostering Innovation, Streamlining Development, ICOQSIA 2014, Langkawi, Kedah, 12/8/14. https://doi.org/10.1063/1.4903687

}

TY - GEN

T1 - The derivation of mutual information and covariance function using centered random variables

AU - Abdul Razak, Fatimah

PY - 2014

Y1 - 2014

N2 - Information theoretic measures such as Mutual Information are often said to be able to measure nonlinear dependencies whereas covariance (and correlation) are able to measure only linear dependencies. We aim to illustrate this claim using centered random variables. The set of centered random variable Fc={-q-12,-q-12+1,..,q-12-1,q-12} is mapped from F = {1,2,.., q - 1, q}. For q=2, we derive the relationship between the Mutual Information function, I, and the covariance function, Γ, and show that Γ=0→I=0. Furthermore we show that when q=3, the nonlinearities are captured by Mutual Information by highlighting a case where Γ=0 I=0.

AB - Information theoretic measures such as Mutual Information are often said to be able to measure nonlinear dependencies whereas covariance (and correlation) are able to measure only linear dependencies. We aim to illustrate this claim using centered random variables. The set of centered random variable Fc={-q-12,-q-12+1,..,q-12-1,q-12} is mapped from F = {1,2,.., q - 1, q}. For q=2, we derive the relationship between the Mutual Information function, I, and the covariance function, Γ, and show that Γ=0→I=0. Furthermore we show that when q=3, the nonlinearities are captured by Mutual Information by highlighting a case where Γ=0 I=0.

KW - Centered random variable

KW - Mutual Information and covariance

KW - nonlinear dependence

UR - http://www.scopus.com/inward/record.url?scp=84916613264&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84916613264&partnerID=8YFLogxK

U2 - 10.1063/1.4903687

DO - 10.1063/1.4903687

M3 - Conference contribution

SN - 9780735412743

VL - 1635

SP - 883

EP - 889

BT - AIP Conference Proceedings

PB - American Institute of Physics Inc.

ER -