Organodynamics	Grant Holland, Apr 25, 2014
Slide: Constraints on too much uncertainty	Nav Refs Top Bottom Previous Next 90m 90m 45m 45m 25m 25m
The key to constraining the growth of the degree of chaos (uncertainty) over time in a stochastic process is the presence of statistical dependence. This occurs when a stochastic process has time steps that exhibit some mutual statistical dependence. When this happens, the stochastic process as a whole has less entropy than it would if there were no statistical dependence. Fundamentally, stochastic dependency reduces uncertainty, and entropy: H(Y|X) ≤ H(Y) …with equality when X and Y are stochastically independent.	When this relationship is applied to stochastic processes, we also see that stochastic dependency reduces the degree of uncertainty of a new time step added to the process. Entropy rate is the mean additional uncertainty added to the process by a new time step: Then, entropy rate = limn->∞ 1/n*H(X1, X2, …, Xn) Modified entropy rate is the entropy of this added time step by itself, given the previous time steps: Then, modified entropy rate = limn->∞ H(Xn | X1, X2,…, Xn-1) [Kleeman 2012, lecture 3, p. 4.]
It turns out that the amount of entropy added by the new time step, when conditioned on the previous time steps, is less than the entropy of the new time step by itself. That is: H’(X) ≤ H(Xn) …with equality when all the Xi earlier in the process are (mutually) stochastically independent. Thus, stochastic dependence constrains entropy and chaotic behavior.	Therefore, ODSPs have a limit on the amount of chaotic (uncertain) behavior they can exhibit. This limit is imposed by the presence of stochastic dependence. For example, if the process is Markov, then this limit can exists.

Notes: