Journal
THEORETICAL COMPUTER SCIENCE
Volume 395, Issue 2-3, Pages 203-219Publisher
ELSEVIER
DOI: 10.1016/j.tcs.2008.01.012
Keywords
hidden Markov process; Shannon entropy; Renyi entropy; product of random matrices; top Lyapunov exponent; spectral representation of matrices
Categories
Funding
- NIGMS NIH HHS [R01 GM068959, R01 GM068959-01] Funding Source: Medline
Ask authors/readers for more resources
We study the entropy rate of a hidden Markov process (HMP) defined by observing the output of a binary symmetric channel whose input is a first-order binary Markov process. Despite the simplicity of the models involved, the characterization of this entropy is a long standing open problem. By presenting the probability of a sequence under the model as a product of random matrices, one can see that the entropy rate sought is equal to a top Lyapunov exponent of the product. This offers an explanation for the elusiveness of explicit expressions for the HMP entropy rate, as Lyapunov exponents are notoriously difficult to compute. Consequently, we focus on asymptotic estimates, and apply the same product of random matrices to derive an explicit expression for a Taylor approximation of the entropy rate with respect to the parameter of the binary symmetric channel. The accuracy of the approximation is validated against empirical simulation results. We also extend our results to higher-order Markov processes and to Renyi entropies of any order. (c) 2008 Elsevier B.V. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available