Prove That for a Nonnegative Continuous Random Variable X We Have the Following Holds

Moments of Random Variables

Oliver C. Ibe , in Fundamentals of Applied Probability and Random Processes (Second Edition), 2014

3.3 Expectation of Nonnegative Random Variables

Some random variables assume only nonnegative values. For example, the time X until a component fails cannot be negative. In Chapter 1 we defined the reliability function R(t) of a component as the probability that the component has not failed by time t. Thus, if the PDF of X is f _X (x) and the CDF is F _X (x), we can define the reliability function of the component by R _X (t), which is related to the CDF and PDF as follows:

(3.3a) ${\mathit{R}}_{\mathit{X}}\left(\mathit{t}\right)=\mathit{P}\left[\mathit{X}>\mathit{t}\right]=1-\mathit{P}\left[\mathit{X}\le \mathit{t}\right]=1-{\mathit{F}}_{\mathit{X}}\left(\mathit{t}\right)$

(3.3b) ${\mathit{F}}_{\mathit{X}}\left(\mathit{t}\right)=1-{\mathit{R}}_{\mathit{X}}\left(\mathit{t}\right)$

Proposition 3.1:

For a nonnegative random variable X with CDF F _X (x), the expected value is given by

$\mathit{E}\left[\mathit{X}\right]=\left\{\begin{array}{l}{\displaystyle {\int }_0^{\mathit{\infty }}\mathit{P}\left[\mathit{X}>\mathit{x}\right]\mathit{dx}}={\displaystyle {\int }_0^{\mathit{\infty }}\left\{1-{\mathit{F}}_{\mathit{X}}\left(\mathit{x}\right)\right\}}\mathit{dx}\mathit{X}\mathrm{continuous}\\ {\displaystyle \sum _{\mathrm{x}=0}^{\mathit{\infty }}\mathit{P}\left[\mathit{X}>\mathit{x}\right]={\displaystyle \sum _{\mathrm{x}=0}^{\mathit{\infty }}\left\{1-{\mathit{F}}_{\mathit{X}}\left(\mathit{x}\right)\right\}\mathit{X}\mathrm{discrete}}}\end{array}\right.$

Proof:

We first prove the case for a continuous random variable. Since

$\mathit{P}\left[\mathit{X}>\mathit{x}\right]={\displaystyle {\int }_{\mathit{x}}^{\mathit{\infty }}{\mathit{f}}_{\mathit{X}}\left(\mathit{u}\right)\mathit{du}}$

we have that

${\displaystyle {\int }_0^{\mathit{\infty }}\mathit{P}\left[\mathit{X}>\mathit{x}\right]\mathit{dx}}={\displaystyle {\int }_0^{\mathit{\infty }}{\displaystyle {\int }_{\mathit{x}}^{\mathit{\infty }}{\mathit{f}}_{\mathit{X}}\left(\mathit{u}\right)\mathit{du}}\mathit{dx}}$

The region of integration {(x, u)|0   ≤ x  < ∞; x  ≤ u  < ∞} is as shown in Figure 3.2.

Figure 3.2. Region of Integration

From the figure we observe that the region of integration can be transformed into {(x, u)|0   ≤ x  ≤ u;   0   ≤ u  < ∞}, which gives

$\begin{array}{c}{\displaystyle {\int }_0^{\mathit{\infty }}\mathit{P}\left[\mathit{X}>\mathit{x}\right]\mathit{dx}}={\displaystyle {\int }_0^{\mathit{\infty }}{\displaystyle {\int }_{\mathit{x}}^{\mathit{\infty }}{\mathit{f}}_{\mathit{X}}\left(\mathit{u}\right)\mathit{du}}\mathit{dx}}={\displaystyle {\int }_{\mathit{u}=0}^{\mathit{\infty }}\left\{{\displaystyle {\int }_{\mathit{x}=0}^{\mathit{u}}\mathit{dx}}\right\}{\mathit{f}}_{\mathit{X}}\left(\mathit{u}\right)\mathit{du}}={\displaystyle {\int }_{\mathit{u}=0}^{\mathit{\infty }}\mathit{u}{\mathit{f}}_{\mathit{X}}\left(\mathit{u}\right)\mathit{du}}\\ =\mathit{E}\left[\mathit{X}\right]\end{array}$

This proves the proposition for the case of a continuous random variable. For a discrete random variable X that assumes only nonnegative values, we have that

$\begin{array}{c}\mathit{E}\left[\mathit{X}\right]={\displaystyle \sum _{\mathit{x}=0}^{\mathit{\infty }}\mathit{x}{\mathit{p}}_{\mathit{X}}\left(\mathit{x}\right)}=\sum _{\mathit{x}=1}^{\mathit{\infty }}\mathit{x}{\mathit{p}}_{\mathit{X}}\left(\mathit{x}\right)=1{\mathit{p}}_{\mathit{X}}(1)+2{\mathit{p}}_{\mathit{X}}(2)+3{\mathit{p}}_{\mathit{X}}(3)+4{\mathit{p}}_{\mathit{X}}(4)+\cdots \\ ={\mathit{p}}_{\mathit{X}}\left(1\right)\\ +{\mathit{p}}_{\mathit{X}}\left(2\right)+{\mathit{p}}_{\mathit{X}}(2)\\ +{\mathit{p}}_{\mathit{X}}\left(3\right)+{\mathit{p}}_{\mathit{X}}(3)+{\mathit{p}}_{\mathit{X}}(3)\\ +{\mathit{p}}_{\mathit{X}}\left(4\right)+{\mathit{p}}_{\mathit{X}}(4)+{\mathit{p}}_{\mathit{X}}(4)+{\mathit{p}}_{\mathit{X}}(4)\\ +\cdots \\ =\sum _{\mathit{x}=1}^{\mathit{\infty }}{\mathit{p}}_{\mathit{X}}\left(\mathit{x}\right)+\sum _{\mathit{x}=2}^{\mathit{\infty }}{\mathit{p}}_{\mathit{X}}\left(\mathit{x}\right)+{\displaystyle \sum _{\mathit{x}=3}^{\mathit{\infty }}{\mathit{p}}_{\mathit{X}}\left(\mathit{x}\right)}+\cdots \\ =\left\{1-{\mathit{F}}_{\mathit{X}}\left(0\right)\right\}+\left\{1-{\mathit{F}}_{\mathit{X}}\left(1\right)\right\}+\left\{1-{\mathit{F}}_{\mathit{X}}\left(2\right)\right\}+\cdots \\ =\sum _{\mathit{x}=0}^{\mathit{\infty }}\left\{1-{\mathit{F}}_{\mathit{X}}\left(\mathit{x}\right)\right\}={\displaystyle \sum _{\mathit{x}=0}^{\mathit{\infty }}\mathit{P}\left[\mathit{X}>\mathit{x}\right]}\end{array}$

Example 3.5

Use the above method to find the expected value of the random variable X whose PDF is given in Example 3.4.

Solution:

From Example 3.4, the PDF of X is given by

${\mathit{f}}_{\mathit{X}}\left(\mathit{x}\right)=\left\{\begin{array}{l}\mathit{\lambda }{\mathit{e}}^{-\mathit{\lambda x}}\mathit{x}\ge 0\\ 0\mathit{x}<0\end{array}\right.$

Thus, the CDF of X is given by

${\mathit{F}}_{\mathit{X}}\left(\mathit{x}\right)={\displaystyle {\int }_0^{\mathit{x}}{\mathit{f}}_{\mathit{X}}\left(\mathit{u}\right)\mathit{du}}={\displaystyle {\int }_0^{\mathit{x}}\mathit{\lambda }{\mathit{e}}^{-\mathit{\lambda u}}\mathit{du}}=1-{\mathit{e}}^{-\mathit{\lambda x}}$

Since X is a nonnegative random variable, its expected value is given by

$\mathit{E}\left[\mathit{X}\right]={\displaystyle {\int }_0^{\mathit{\infty }}\left\{1-{\mathit{F}}_{\mathit{X}}\left(\mathit{x}\right)\right\}\mathit{dx}}={\displaystyle {\int }_0^{\mathit{\infty }}{\mathit{e}}^{-\mathit{\lambda x}}\mathit{dx}}=\frac{1}{\mathit{\lambda }}$

which is the same result that we obtained in Example 3.4.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128008522000031

What is This Stuff Called Probability?

John K. Kruschke , in Doing Bayesian Data Analysis (Second Edition), 2015

4.3.2.2 The normal probability density function

Any function that has only nonnegative values and integrates to 1 (i.e., satisfies Equation 4.3) can be construed as a probability density function. Perhaps the most famous probability density function is the normal distribution, also known as the Gaussian distribution. A graph of the normal curve is a well-known bell shape; an example is shown in Figure 4.4.

Figure 4.4. A normal probability density function, shown with a comb of narrow intervals. The integral is approximated by summing the width times height of each interval.

The mathematical formula for the normal probability density has two parameters: μ (Greek mu) is called the mean of the distribution and σ (Greek sigma) is called the standard deviation. The value of μ governs where the middle of the bell shape falls on the x-axis, so it is called a location parameter, and the value of σ governs how wide the bell is, so it is called a scale parameter. As discussed in Section 2.2, you can think of the parameters as control knobs with which to manipulate the location and scale of the distribution. The mathematical formula for the normal probability density is

(4.4) $p\left(x\right)=\frac{1}{\sigma \sqrt{2\pi }}\text{exp}\left(-\frac{1}{2}{\left[\frac{x-\mu }{\sigma }\right]}^2\right).$

Figure 4.4 shows an example of the normal distribution for specific values of μ and σ as indicated. Notice that the peak probability density can be greater than 1.0 when the standard deviation, σ, is small. In other words, when the standard deviation is small, a lot of probability mass is squeezed into a small interval, and consequently the probability density in that interval is high.

Figure 4.4 also illustrates that the area under the normal curve is, in fact, 1. The x axis is divided into a dense comb of small intervals, with width denoted Δx. The integral of the normal density is approximated by summing the masses of all the tiny intervals as in Equation 4.2. As can be seen in the text within the graph, the sum of the interval areas is essentially 1.0. Only rounding error, and the fact that the extreme tails of the distribution are not included in the sum, prevent the sum from being exactly 1.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780124058880000040

Multiple Random Variables

Oliver C. Ibe , in Fundamentals of Applied Probability and Random Processes (Second Edition), 2014

5.5 Determining Probabilities from a Joint CDF

Suppose that X and Y are given random variables and we are required to determine the probability of a certain event defined in terms of X and Y for which the joint CDF is known. We start by sketching the event in the x-y plane. For example, assume we are required to find P[a  < X  ≤ b, c  < Y  ≤ d]. The region of interest is shown in Figure 5.2, which defines four partitions.

Figure 5.2. Domain Partitions

Consider the following events:

$\begin{array}{l}{\mathit{E}}_1=\left\{\left(\mathit{X}\le \mathit{b}\right)\cap \left(\mathit{Y}\le \mathit{d}\right)\right\}\\ {\mathit{E}}_2=\left\{\left(\mathit{X}\le \mathit{b}\right)\cap \left(\mathit{Y}\le \mathit{c}\right)\right\}\\ {\mathit{E}}_3=\left\{\left(\mathit{X}\le \mathit{a}\right)\cap \left(\mathit{Y}\le \mathit{d}\right)\right\}\\ {\mathit{E}}_4=\left\{\left(\mathit{X}\le \mathit{a}\right)\cap \left(\mathit{Y}\le \mathit{c}\right)\right\}\\ {\mathit{E}}_5=\left\{\left(\mathit{a}<\mathit{X}\le \mathit{b}\right)\cap \left(\mathit{c}<\mathit{Y}\le \mathit{d}\right)\right\}\end{array}$

The region of interest is B, which corresponds to event E ₅ that can be obtained as follows:

${\mathit{E}}_5={\mathit{E}}_1-{\mathit{E}}_2-{\mathit{E}}_3+{\mathit{E}}_4$

Thus,

$\mathit{P}\left[\mathit{a}<\mathit{X}\le \mathit{b},\mathit{c}<\mathit{Y}\le \mathit{d}\right]={\mathit{F}}_{\mathit{XY}}\left(\mathit{b},\mathit{d}\right)-{\mathit{F}}_{\mathit{XY}}\left(\mathit{b},\mathit{c}\right)-{\mathit{F}}_{\mathit{XY}}\left(\mathit{a},\mathit{d}\right)+{\mathit{F}}_{\mathit{XY}}\left(\mathit{a},\mathit{c}\right)$

Note that the probability is simply the joint CDF evaluated at the point where X and Y jointly have the larger of their two values plus the CDF evaluated at the point where they jointly have their smaller values minus the CDF evaluated at the two points where they have mixed smaller and larger values. When X and Y are independent random variables, the above result becomes

$\begin{array}{c}\mathit{P}\left[\mathit{a}<\mathit{X}\le \mathit{b},\mathit{c}<\mathit{Y}\le \mathit{d}\right]={\mathit{F}}_{\mathit{XY}}\left(\mathit{b},\mathit{d}\right)-{\mathit{F}}_{\mathit{XY}}\left(\mathit{b},\mathit{c}\right)-{\mathit{F}}_{\mathit{XY}}\left(\mathit{a},\mathit{d}\right)+{\mathit{F}}_{\mathit{XY}}\left(\mathit{a},\mathit{c}\right)\\ ={\mathit{F}}_{\mathit{X}}\left(\mathit{b}\right){\mathit{F}}_{\mathit{Y}}\left(\mathit{d}\right)-{\mathit{F}}_{\mathit{X}}\left(\mathit{b}\right){\mathit{F}}_{\mathit{Y}}\left(\mathit{c}\right)-{\mathit{F}}_{\mathit{X}}\left(\mathit{a}\right){\mathit{F}}_{\mathit{Y}}\left(\mathit{d}\right)+{\mathit{F}}_{\mathit{X}}\left(\mathit{a}\right){\mathit{F}}_{\mathit{Y}}\left(\mathit{c}\right)\\ ={\mathit{F}}_{\mathit{X}}\left(\mathit{b}\right)\left\{{\mathit{F}}_{\mathit{Y}}\left(\mathit{d}\right)-{\mathit{F}}_{\mathit{Y}}\left(\mathit{c}\right)\right\}-{\mathit{F}}_{\mathit{X}}\left(\mathit{a}\right)\left\{{\mathit{F}}_{\mathit{Y}}\left(\mathit{d}\right)-{\mathit{F}}_{\mathit{Y}}\left(\mathit{c}\right)\right\}\\ =\left\{{\mathit{F}}_{\mathit{X}}\left(\mathit{b}\right)-{\mathit{F}}_{\mathit{X}}\left(\mathit{a}\right)\right\}\left\{{\mathit{F}}_{\mathit{Y}}\left(\mathit{d}\right)-{\mathit{F}}_{\mathit{Y}}\left(\mathit{c}\right)\right\}\\ =\mathit{P}\left[\mathit{a}<\mathit{X}\le \mathit{b}\right]\mathit{P}[\mathit{c}<\mathit{Y}\le \mathit{d}]\end{array}$

Finally, for the case when X and Y are discrete random variables, the joint CDF can be obtained from the joint PMF as follows:

${\mathit{F}}_{\mathit{XY}}\left(\mathit{x},\mathit{y}\right)={\displaystyle \sum _{\mathit{m}\le \mathit{x}}{\displaystyle \sum _{\mathit{n}\le \mathit{y}}{\mathit{p}}_{\mathit{XY}}\left(\mathit{m},\mathit{n}\right)}}$

For example, if X and Y take only nonnegative values,

$\begin{array}{c}{\mathit{F}}_{\mathit{XY}}(1,2)={\displaystyle \sum _{\mathit{m}\le 1}{\displaystyle \sum _{\mathit{n}\le 2}{\mathit{p}}_{\mathit{XY}}\left(\mathit{m},\mathit{n}\right)}}\\ ={\mathit{p}}_{\mathit{XY}}(0,0)+{\mathit{p}}_{\mathit{XY}}(0,1)+{\mathit{p}}_{\mathit{XY}}(0,2)+{\mathit{p}}_{\mathit{XY}}(0,2)+{\mathit{p}}_{\mathit{XY}}(1,1)+{\mathit{p}}_{\mathit{XY}}(1,2)\end{array}$

Example 5.7

The joint CDF of two discrete random variables X and Y is given as follows:

${\mathit{F}}_{\mathit{XY}}\left(\mathit{x},\mathit{y}\right)=\left\{\begin{array}{l}\frac{1}{8}\mathit{x}=1,\mathit{y}=1\\ \frac{5}{8}\mathit{x}=1,\mathit{y}=2\\ \frac{1}{4}\mathit{x}=2,\mathit{y}=1\\ 1\mathit{x}=2,\mathit{y}=2\end{array}\right.$

Determine the following:

a.

Joint PMF of X and Y

b.

Marginal PMF of X

c.

Marginal PMF of Y

Solution:

The joint PMF is obtained from the relationship

${\mathit{F}}_{\mathit{XY}}\left(\mathit{x},\mathit{y}\right)={\displaystyle \sum _{\mathit{m}\le \mathit{x}}{\displaystyle \sum _{\mathit{n}\le \mathit{y}}{\mathit{p}}_{\mathit{XY}}\left(\mathit{m},\mathit{n}\right)}}$

Thus,

$\begin{array}{l}{\mathit{F}}_{\mathit{XY}}(1,1)={\mathit{p}}_{\mathit{XY}}(1,1)=1/8\\ {\mathit{F}}_{\mathit{XY}}(1,2)={\mathit{p}}_{\mathit{XY}}(1,1)+{\mathit{p}}_{\mathit{XY}}(1,2)=5/8\Rightarrow {\mathit{p}}_{\mathit{XY}}(1,2)=5/8-1/8=1/2\\ {\mathit{F}}_{\mathit{XY}}(2,1)={\mathit{p}}_{\mathit{XY}}(1,1)+{\mathit{p}}_{\mathit{XY}}(2,1)=1/4\Rightarrow {\mathit{p}}_{\mathit{XY}}(2,1)=1/4-1/8=1/8\\ {\mathit{F}}_{\mathit{XY}}(2,2)={\mathit{p}}_{\mathit{XY}}(1,1)+{\mathit{p}}_{\mathit{XY}}(1,2)+{\mathit{p}}_{\mathit{XY}}(2,1)+{\mathit{p}}_{\mathit{XY}}(2,2)=1\Rightarrow {\mathit{p}}_{\mathit{XY}}(2,2)=1/4\end{array}$

The joint PMF becomes

${\mathit{p}}_{\mathit{XY}}\left(\mathit{x},\mathit{y}\right)=\left\{\begin{array}{l}\frac{1}{8}\mathit{x}=1,\mathit{y}=1\\ \frac{1}{2}\mathit{x}=1,\mathit{y}=2\\ \frac{1}{8}\mathit{x}=2,\mathit{y}=1\\ \frac{1}{4}\mathit{x}=2,\mathit{y}=2\end{array}\right.$

The marginal PMF of X is given by

${\mathit{p}}_{\mathit{X}}\left(\mathit{x}\right)=\left\{\begin{array}{l}{\mathit{p}}_{\mathit{XY}}(1,1)+{\mathit{p}}_{\mathit{XY}}(1,2)=\frac{5}{8}\mathit{x}=1\\ {\mathit{p}}_{\mathit{XY}}(2,1)+{\mathit{p}}_{\mathit{XY}}(2,2)=\frac{3}{8}\mathit{x}=2\end{array}\right.$

The marginal PMF of Y is given by

${\mathit{p}}_{\mathit{Y}}\left(\mathit{y}\right)=\left\{\begin{array}{l}{\mathit{p}}_{\mathit{XY}}(1,1)+{\mathit{p}}_{\mathit{XY}}(2,1)=\frac{1}{4}\mathit{y}=1\\ {\mathit{p}}_{\mathit{XY}}(1,2)+{\mathit{p}}_{\mathit{XY}}(2,2)=\frac{3}{4}\mathit{y}=2\end{array}\right.$

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128008522000055

Basic Concepts in Probability

Oliver C. Ibe , in Markov Processes for Stochastic Modeling (Second Edition), 2013

1.9.1 Markov Inequality

The Markov inequality applies to random variables that take only nonnegative values. It can be stated as follows:

Proposition 1.1

If X is a random variable that takes only nonnegative values, then for any a>0,

$P[X\ge a]\le \frac{E\left[X\right]}{a}$

Proof

We consider only the case when X is a continuous random variable. Thus,

$\begin{array}{ll}\hfill E\left[X\right]& ={\int }_0^{\infty }x{f}_X(x)\mathrm{d}x={\int }_0^{a}x{f}_X(x)\mathrm{d}x+{\int }_a^{\infty }x{f}_X(x)\mathrm{d}x\ge {\int }_a^{\infty }x{f}_X(x)\mathrm{d}x\hfill \\ \hfill & \ge {\int }_a^{\infty }a{f}_X(x)\mathrm{d}x=a{\int }_a^{\infty }{f}_X(x)\mathrm{d}x=aP[X\ge a]\hfill \end{array}$

and the result follows.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780124077959000013

Special Random Variables

Sheldon M. Ross , in Introduction to Probability and Statistics for Engineers and Scientists (Fifth Edition), 2014

5.5 Normal Random Variables

A random variable is said to be normally distributed with parameters μ and σ ², and we write $X\sim \mathcal{N}\left(\mu ,{\sigma }^2\right)$ , if its density is

$\begin{array}{ll}f\left(x\right)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-{\left(x-\mu \right)}^2/2{\sigma }^2},\hfill & -\infty <x<\infty \hfill \end{array}$

* The normal density f (x) is a bell-shaped curve that is symmetric about μ and that attains its maximum value of $\frac{1}{\sqrt{2\pi }\sigma }\approx 0.399/\sigma$ at x  = μ (see Figure 5.7).

Figure 5.7. The normal density function (a) with μ   =   0, σ   =   1 and (b) with arbitrary μ and σ².

The normal distribution was introduced by the French mathematician Abraham de Moivre in 1733 and was used by him to approximate probabilities associated with binomial random variables when the binomial parameter n is large. This result was later extended by Laplace and others and is now encompassed in a probability theorem known as the central limit theorem, which gives a theoretical base to the often noted empirical observation that, in practice, many random phenomena obey, at least approximately, a normal probability distribution. Some examples of this behavior are the height of a person, the velocity in any direction of a molecule in gas, and the error made in measuring a physical quantity.

To compute E[X] note that

$E\left[X-\mu \right]=\frac{1}{\sqrt{2\pi }\sigma }{\int }_{-\infty }^{\infty }\left(x-\mu \right)e^{-{\left(x-\mu \right)}^2/2{\sigma }^2}dx$

Letting y  =   (x – μ)/σ gives that

$E\left[X-\mu \right]=\frac{\sigma }{\sqrt{2}\pi }{\displaystyle {\int }_{-\infty }^{\infty }y{e}^{-y^2/2}dy}$

But

${\displaystyle {\int }_{-\infty }^{\infty }y{e}^{-y^2/2}dy}=-e^{-y^2/2}{|}_{-\infty }^{\infty }=0$

showing that E[X – μ]   =   0, or equivalently that

$E\left[X\right]=\mu$

Using this, we now compute Var(X) as follows:

(5.5.1) $\begin{array}{ll}\text{Var}(X)\hfill & =E\left[{\left(X-\mu \right)}^2\right]\hfill \\ \hfill & =\frac{1}{\sqrt{2\pi }\sigma }{\displaystyle {\int }_{-\infty }^{\infty }{\left(x-\mu \right)}^2{e}^{-{\left(x-\mu \right)}^2/2{\sigma }^2}dx}\hfill \\ \hfill & =\frac{1}{\sqrt{2\pi }}{\displaystyle {\int }_{-\infty }^{\infty }{\sigma }^2{y}^2{e}^{-y^2/2}dy}\hfill \end{array}$

With u  = y and dv  = ye ^{−y 2/2}, the integration by parts formula

${\displaystyle \int udv}=uv-{\displaystyle \int vdu}$

yields that

$\begin{array}{ll}\underset{-\infty }{\overset{\infty }{{\displaystyle \int }}}{y}^2{e}^{-y^2/2}dy\hfill & =-y{e}^{-y^2/2}{|}_{-\infty }^{\infty }+{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-y^2/2}dy}\hfill \\ \hfill & ={\displaystyle {\int }_{-\infty }^{\infty }{e}^{-y^2/2}dy}\hfill \end{array}$

Hence, from (5.5.1)

$\begin{array}{ll}\text{V}\text{a}\text{r}\left(X\right)\hfill & ={\sigma }^2\frac{1}{\sqrt{2\pi }}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-y^2/2}}dy\hfill \\ \hfill & ={\sigma }^2\hfill \end{array}$

where the preceding used that $\frac{1}{\sqrt{2\pi }}{e}^{-y^2/2}dy$ is the density function of a normal random variable with parameters μ  =   0 and σ  =   1, so its integral must equal 1.

Thus μ and σ ² represent, respectively, the mean and variance of the normal distribution. A very important property of normal random variables is that if X is normal with mean μ and variance σ ², then for any constants a and b, b ≠ 0, the random variable Y  = a  + bX is also a normal random variable with parameters

$E\left[Y\right]=E\left[a+bX\right]=a+bE\left[X\right]=a+b\mu$

and variance

$\text{V}\text{a}\text{r}\left(Y\right)=\text{V}\text{a}\text{r}\left(a+bX\right)=b^2\text{V}\text{a}\text{r}\left(X\right)=b^2{\sigma }^2$

To verify this, let F_Y (y) be the distribution function of Y. Then, for b > 0

$\begin{array}{ll}{F}_Y\left(y\right)\hfill & =P\left(Y\le y\right)\hfill \\ \hfill & =P\left(a+bX\le y\right)\hfill \\ \hfill & =P\left(X\le \frac{y-a}{b}\right)\hfill \\ \hfill & =F_X\left(\frac{y-a}{b}\right)\hfill \end{array}$

where F_X is the distribution function of X. Similarly, if b < 0, then

$\begin{array}{ll}{F}_Y\left(y\right)\hfill & =P\left(a+bX\le y\right)\hfill \\ \hfill & =P\left(X\ge \frac{y-a}{b}\right)\hfill \\ \hfill & =1-F_X\left(\frac{y-a}{b}\right)\hfill \end{array}$

Differentiation yields that the density function of Y is

$f_Y\left(y\right)=\{\begin{array}{ll}\frac{1}{b}{f}_X\left(\frac{y-a}{b}\right),\hfill & \text{if}b>0\hfill \\ -\frac{1}{b}{f}_X\left(\frac{y-a}{b}\right),\hfill & \text{if}b<0\hfill \end{array}$

which can be written as

$\begin{array}{ll}{f}_Y\left(y\right)\hfill & =\frac{1}{\left|b\right|}{f}_X\left(\frac{y-a}{b}\right)\hfill \\ \hfill & =\frac{1}{\sqrt{2\pi }\sigma \left|b\right|}{e}^{-{\left(\frac{y-a}{b}-\mu \right)}^2/2{\sigma }^2}\hfill \\ \hfill & =\frac{1}{\sqrt{2\pi }\sigma \left|b\right|}{e}^{-{\left(y-a-b\mu \right)}^2/2b^2{\sigma }^2}\hfill \end{array}$

showing that Y  = a  + bX is normal with mean a  + bμ and variance b ² σ ².

It follows from the foregoing that if $X\sim \mathcal{N}\left(\mu ,{\sigma }^2\right)$ , then

$Z=\frac{X-\mu }{\sigma }$

is a normal random variable with mean 0 and variance 1. Such a random variable Z is said to have a standard, or unit, normal distribution. Let Φ(·) denote its distribution function. That is,

$\begin{array}{ll}\text{Φ}\left(x\right)=\frac{1}{\sqrt{2\pi }}{\displaystyle {\int }_{-\infty }^x{e}^{-y^2/2}dy,}\hfill & -\infty <x<\infty \hfill \end{array}$

This result that Z  =   (X – μ)/σ has a standard normal distribution when X is normal with parameters μ and σ ² is quite important, for it enables us to write all probability statements about X in terms of probabilities for Z. For instance, to obtain P{X < b}, we note that X will be less than b if and only if (X – μ)/σ is less than (b – μ)/σ, and so

$\begin{array}{ll}P\left\{X<b\right\}\hfill & =P\left\{\frac{X-\mu }{\sigma }<\frac{b-\mu }{\sigma }\right\}\hfill \\ \hfill & =\text{Φ}\left(\frac{b-\mu }{\sigma }\right)\hfill \end{array}$

Similarly, for any a < b,

$\begin{array}{ll}P\left\{a<X<b\right\}\hfill & =P\left\{\frac{a-\mu }{\sigma }<\frac{X-\mu }{\sigma }<\frac{b-\mu }{\sigma }\right\}\hfill \\ \hfill & =P\left\{\frac{a-\mu }{\sigma }<Z<\frac{b-\mu }{\sigma }\right\}\hfill \\ \hfill & =P\left\{Z<\frac{b-\mu }{\sigma }\right\}-P\left\{Z<\frac{a-\mu }{\sigma }\right\}\hfill \\ \hfill & =\text{Φ}\left(\frac{b-\mu }{\sigma }\right)-\text{Φ}\left(\frac{a-\mu }{\sigma }\right)\hfill \end{array}$

It remains for us to compute Φ(x). This has been accomplished by an approximation and the results are presented in Table A1 of the Appendix, which tabulates Φ(x ) (to a 4-digit level of accuracy) for a wide range of nonnegative values of x. In addition, Program 5.5a of the text disk can be used to obtain Φ(x).

While Table A1 tabulates Φ(x) only for nonnegative values of x, we can also obtain Φ(–x) from the table by making use of the symmetry (about 0) of the standard normal probability density function. That is, for x > 0, if Z represents a standard normal random variable, then (see Figure 5.8)

Figure 5.8. Standard normal probabilities.

$\begin{array}{ll}\text{Φ}\left(-x\right)\hfill & =P\left\{Z<-x\right\}\hfill \\ \hfill & \begin{array}{ll}=P\left\{Z>x\right\}\hfill & \text{by}\text{symmetry}\hfill \end{array}\hfill \\ \hfill & =1-\text{Φ}\left(x\right)\hfill \end{array}$

Thus, for instance,

$p\left\{Z<-1\right\}=\text{Φ}\left(-1\right)=1-\text{Φ}\left(1\right)=1-.8413=.1587$

Example 5.5a

If X is a normal random variable with mean μ  =   3 and variance σ ²  =   16, find

(a)

P{X < 11};

(b)

P{X > –1};

(c)

P{2 < X < 7}.

Solution

(a)

$\begin{array}{ll}P\left\{X<11\right\}\hfill & =P\left\{\frac{X-3}{4}<\frac{11-3}{4}\right\}\hfill \\ \hfill & =\text{Φ}\left(2\right)\hfill \\ \hfill & =.9772\hfill \end{array}$

(b)

$\begin{array}{ll}P\left\{X>-1\right\}\hfill & =P\left\{\frac{X-3}{4}>\frac{-1-3}{4}\right\}\hfill \\ \hfill & =P\left\{Z>-11\right\}\hfill \\ \hfill & =P\left\{Z<11\right\}\hfill \\ \hfill & =.8413\hfill \end{array}$

(c)

$\begin{array}{ll}P\left\{2<X<7\right\}\hfill & =P\left\{\frac{2-3}{4}<\frac{X-3}{4}<\frac{7-3}{4}\right\}\hfill \\ \hfill & =\text{Φ}\left(1\right)-\text{Φ}\left(-1/4\right)\hfill \\ \hfill & =\text{Φ}\left(1\right)-\left(1-\text{Φ}\left(1/4\right)\right)\hfill \\ \hfill & =.8413+.5987-1=.4400\hfill \end{array}$

■

Example 5.5b

Suppose that a binary message — either "0" or "1" — must be transmitted by wire from location A to location B. However, the data sent over the wire are subject to a channel noise disturbance and so to reduce the possibility of error, the value 2 is sent over the wire when the message is " 1" and the value –2 is sent when the message is "0." If x, x  =   ±2, is the value sent at location A then R, the value received at location B, is given by R  = x  + N, where N is the channel noise disturbance. When the message is received at location B, the receiver decodes it according to the following rule:

$\begin{array}{l}\text{if}R\ge \text{.5,\hspace{0.17em}then\hspace{0.17em}"1"\hspace{0.17em}is\hspace{0.17em}concluded}\hfill \\ \text{if}R<\text{.5,\hspace{0.17em}then\hspace{0.17em}"0"\hspace{0.17em}is\hspace{0.17em}concluded}\hfill \end{array}$

Because the channel noise is often normally distributed, we will determine the error probabilities when N is a standard normal random variable.

There are two types of errors that can occur: One is that the message "1" can be incorrectly concluded to be "0" and the other that "0" is incorrectly concluded to be "1." The first type of error will occur if the message is "1" and 2   + N < .5, whereas the second will occur if the message is "0" and –2   + N > .5.

Hence,

$\begin{array}{ll}P\left\{\text{e}\text{r}\text{r}\text{o}\text{r}|\text{m}\text{e}\text{s}\text{s}\text{a}\text{g}\text{e}\text{i}\text{s}"1"\right\}\hfill & =P\left\{N<-1.5\right\}\hfill \\ \hfill & =1-\text{Φ}\left(1.5\right)=.0668\hfill \end{array}$

and

$\begin{array}{ll}P\left\{\text{e}\text{r}\text{r}\text{o}\text{r}|\text{m}\text{e}\text{s}\text{s}\text{a}\text{g}\text{e}\text{i}\text{s}"0"\right\}\hfill & =P\left\{N>2.5\right\}\hfill \\ \hfill & =1-\text{Φ}\left(2.5\right)=.0662\hfill \end{array}$

■

Example 5.5c

The power W dissipated in a resistor is proportional to the square of the voltage V. That is,

$W=r{V}^2$

where r is a constant. If r  =   3, and V can be assumed (to a very good approximation) to be a normal random variable with mean 6 and standard deviation 1, find

(a)

E[W];

(b)

P{W > 120}.

Solution

(a)

$\begin{array}{ll}E\left[W\right]\hfill & =E\left[3V^2\right]\hfill \\ \hfill & =3E\left[V^2\right]\hfill \\ \hfill & =3\left(\text{V}\text{a}\text{r}\left[V\right]+E^2\left[V\right]\right)\hfill \\ \hfill & =3\left(1+36\right)=111\hfill \end{array}$

(b)

$\begin{array}{ll}P\left\{W>120\right\}\hfill & =P\left\{3V^2>120\right\}\hfill \\ \hfill & =P\left\{V>\sqrt{40}\right\}\hfill \\ \hfill & =P\left\{V-6>\sqrt{40}-6\right\}\hfill \\ \hfill & =P\left\{Z>.3246\right\}\hfill \\ \hfill & =1-\text{Φ}\left(.3246\right)\hfill \\ \hfill & =.3727\hfill \end{array}$ ■

Let us now compute the moment generating function of a normal random variable. To start, we compute the moment generating function of a standard normal random variable Z.

$\begin{array}{ll}E\left[e^{tZ}\right]\hfill & ={\displaystyle {\int }_{-\infty }^{\infty }{e}^{tx}\frac{1}{\sqrt{2\pi }}{e}^{-x^2/2}dx}\hfill \\ \hfill & =\frac{1}{\sqrt{2\pi }}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\left(x^2-2tx\right)/2}}dx\hfill \\ \hfill & =e^{-t^2/2}\frac{1}{\sqrt{2\pi }}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-{\left(x-t\right)}^2/2}dx}\hfill \\ \hfill & =e^{-t^2/2}\frac{1}{\sqrt{2\pi }}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-y^2/2}dy}\hfill \\ \hfill & =e^{-t^2/2}\hfill \end{array}$

Now, if Z is a standard normal, then X  = μ  + σZ is normal with mean μ and variance σ ². Using the preceding, its moment generating function is

$\begin{array}{ll}E\left[e^{tX}\right]\hfill & =E\left[e^{t\mu +t\sigma Z}\right]\hfill \\ \hfill & =E\left[e^{t\mu }{e}^{t\sigma Z}\right]\hfill \\ \hfill & =e^{t\mu }E\left[e^{t\sigma Z}\right]\hfill \\ \hfill & =e^{t\mu }{e}^{-{\left(\sigma t\right)}^2/2}\hfill \\ \hfill & =e^{\mu t-{\sigma }^2{t}^{2/2}}\hfill \end{array}$

Another important result is that the sum of independent normal random variables is also a normal random variable. To see this, suppose that X_i , i  =   1,…, n, are independent, with X_i being normal with mean μ_i and variance ${\sigma }_i^2$ . The moment generating function of ${\displaystyle {\sum }_{i=1}^n{X}_i}$ is as follows.

$\begin{array}{ll}E\left[e^t{\displaystyle {\sum }_{i=1}^n{X}_i}\right]\hfill & =E\left[e^{t{X}_1}{e}^{t{X}_2}\dots e^{t{X}_n}\right]\hfill \\ \hfill & \begin{array}{ll}={\displaystyle \prod _{i=1}^{n}E\left[e^{t{X}_i}\right]}\hfill & \text{by\hspace{0.17em}independence}\hfill \end{array}\hfill \\ \hfill & ={\displaystyle \prod _{i=1}^n{e}^{{\mu }_{i}t+{\sigma }_i^2{t}^2/2}}\hfill \\ \hfill & =e^{\mu t+{\sigma }^2{t}^2/2}\hfill \end{array}$

where

$\begin{array}{ll}\mu ={\displaystyle \sum _{i=1}^n{\mu }_i,}\hfill & {\sigma }^2={\displaystyle \sum _{i=1}^n{\sigma }_i^2}\hfill \end{array}$

Therefore, ${\displaystyle {\sum }_{i=1}^n{X}_i}$ has the same moment generating function as a normal random variable having mean μ and variance σ ². Hence, from the one-to-one correspondence between moment generating functions and distributions, we can conclude that ${\displaystyle {\sum }_{i=1}^n{X}_i}$ is normal with mean ${\displaystyle {\sum }_{i=1}^n{\mu }_i}$ and variance ${\displaystyle {\sum }_{i=1}^n{\sigma }_i^2}$ .

Example 5.5d

Data from the National Oceanic and Atmospheric Administration indicate that the yearly precipitation in Los Angeles is a normal random variable with a mean of 12.08 inches and a standard deviation of 3.1 inches.

(a)

Find the probability that the total precipitation during the next 2 years will exceed 25 inches.

(b)

Find the probability that next year's precipitation will exceed that of the following year by more than 3 inches.

Assume that the precipitation totals for the next 2 years are independent.

Solution

Let X ₁ and X ₂ be the precipitation totals for the next 2 years.

(a)

Since X ₁  + X ₂ is normal with mean 24.16 and variance 2(3.1)²  =   19.22, it follows that

$\begin{array}{ll}P\left\{X_1+X_2>25\right\}\hfill & =P\left\{\frac{X_1+X_2-24.16}{\sqrt{19.22}}>\frac{25-24.16}{\sqrt{19.22}}\right\}\hfill \\ \hfill & =P\left\{Z>.1916\right\}\hfill \\ \hfill & \approx .4240\hfill \end{array}$

(b)

Since –X ₂ is a normal random variable with mean –12.08 and variance (–1)²(3.1)², it follows that X ₁ – X ₂ is normal with mean 0 and variance 19.22.

$\begin{array}{ll}P\left\{X_1>X_2+3\right\}\hfill & =P\left\{X_1-X_2>3\right\}\hfill \\ \hfill & =P\left\{\frac{X_1-X_2}{\sqrt{19.22}}>\frac{3}{\sqrt{19.22}}\right\}\hfill \\ \hfill & =P\left\{Z>.6843\right\}\hfill \\ \hfill & \approx .2469\hfill \end{array}$

Thus there is a 42.4 percent chance that the total precipitation in Los Angeles during the next 2 years will exceed 25 inches, and there is a 24.69 percent chance that next year's precipitation will exceed that of the following year by more than 3 inches.■

For α ∈ (0, 1), let z_α be such that

$P\left\{Z>z_{\alpha }\right\}=1-\text{Φ}\left(z_{\alpha }\right)=\alpha$

That is, the probability that a standard normal random variable is greater than z_α is equal to α (see Figure 5.9).

Figure 5.9. P{Z &gt; z_α}   =   α.

The value of z_α can, for any α, be obtained from Table A1. For instance, since

$\begin{array}{ll}\hfill 1-\text{Φ}\left(1.645\right)& =.05\hfill \\ \hfill 1-\text{Φ}\left(1.96\right)& =.025\hfill \\ \hfill 1-\text{Φ}\left(2.33\right)& =.01\hfill \end{array}$

it follows that

$\begin{array}{lll}{z}_{.05}=1.645,\hfill & z_{.025}=1.96,\hfill & z_{.01}=2.33\hfill \end{array}$

Program 5.5b on the text disk can also be used to obtain the value of z_α .

Since

$P\left\{Z<z_{\alpha }\right\}=1-\alpha$

it follows that 100(1 – α) percent of the time a standard normal random variable will be less than z_α . As a result, we call z_α the 100(1 – α) percentile of the standard normal distribution.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123948113500058

Conditional Probability and Conditional Expectation

Mark A. Pinsky , Samuel Karlin , in An Introduction to Stochastic Modeling (Fourth Edition), 2011

2.5.3 The Maximal Inequality for Nonnegative Martingales

Because a martingale has constant mean, Markov's inequality applied to a nonnegative martingale immediately yields

$Pr\left\{X_n\ge \lambda \right\}\le \frac{E\left[X_0\right]}{\lambda },\lambda >0.$

We will extend the reasoning behind Markov's inequality to achieve an inequality of far greater power:

(2.49) $Pr\left\{{max}_{0\le n\le m}{X}_n\ge \lambda \right\}\le \frac{E\left[X_0\right]}{\lambda }.$

Instead of limiting the probability of a large value for a single observation X _n , the maximal inequality (2.49) limits the probability of observing a large value anywhere in the time interval 0, …, m, and since the right side of (2.49) does not depend on the length of the interval, the maximal inequality limits the probability of observing a large value at any time in the infinite future of the martingale!

In order to prove the maximal inequality for nonnegative martingales, we need but a single additional fact: If X and Y are jointly distributed random variables and B is an arbitrary set, then

(2.50) $E\left[X{1}_B\left(Y\right)\right]=E\left[E\left(X|Y\right)1_B\left(Y\right)\right]$

But (2.50) follows from the conditional expectation property (2.12), E [g (X)h (Y)] = E{h (Y)E [g (X)| Y]}, with g (x) = x and h (y) = 1 (y in B}. We will have need of (2.50) with X = X_m and Y = (X ₀, …, X _n), whereupon (2.50) followed by (2.48) then justifies

(2.51) $\begin{array}{l}E\left[X_{m}1\left\{X_0<\lambda ,\mathrm{\dots },X_{n-1}<\lambda ,X_n\ge \lambda \right\}\right]\\ =E\left[E\left\{X_m|X_0,\mathrm{\dots },X_n\right\}1\left\{X_0<\lambda ,\mathrm{\dots },X_{n-1}<\lambda ,X_n\ge \lambda \right\}\right]\\ =E\left[X_{n}1\left\{X_0<\lambda ,\mathrm{\dots },X_{n-1}<\lambda ,X_n\ge \lambda \right\}\right].\end{array}$

Theorem 2.1.

Let X ₀, X ₁, … be a martingale with nonnegative values; i.e., Pr{X _n > 0} = 1 for n = 0, 1, …. For any λ > 0,

(2.52) $Pr\left\{{max}_0{X}_n\ge \lambda \right\}\le \frac{E\left[X_0\right]}{\lambda },for0\le n\le m$

and

(2.53) $Pr\left\{{max}_{n\ge 0}{X}_n>\lambda \right\}\le \frac{E\left[X_0\right]}{\lambda },foralln.$

Proof. Inequality (2.53) follows from (2.52) because the right side of (2.52) does not depend on m. We begin with the law of total probability, as in Chapter 1, Section 1.2.1. Either the {X ₀, …, X _m} sequence rises above λfor the first time at some index n or else it remains always below λ. As these possibilities are mutually exclusive and exhaustive, we apply the law of total probability to obtain

$\begin{array}{l}E\left[X_m\right]=\sum _{n=0}^{m}E\left[X_{m}1\right]\left\{X_0<\lambda ,\mathrm{\dots },X_{n-1}<\lambda ,X_n\ge \lambda \right\}\\ +E\left[X_{m}1\left\{X_0<\lambda ,\mathrm{\dots },X_m<\lambda \right\}\right]\\ \ge \sum _{n=0}^{m}E\left[X_{m}1\left\{X_0<\lambda ,\mathrm{\dots },X_{n-1}<\lambda ,X_n\ge \lambda \right\}\right]\left(X_m\ge 0\right)\end{array}$

$\begin{array}{l}\sum _{n=0}^{m}E\left[X_{n}1\left\{X_0<\lambda ,\dots ,X_{n-1}<\lambda ,X_n\ge \lambda \right\}\right]\hfill \\ \ge \lambda \sum _{n=0}^m\text{Pr}\left\{X_0<\lambda ,\dots ,X_{n-1}<\lambda ,X_n\ge \lambda \right\}\hfill \\ =\lambda \text{pr}\left\{{max}_{0\le n\le m}{X}_n\ge \lambda \right\}.\hfill \end{array}$

Example A gambler begins with a unit amount of money and faces a series of independent fair games. Beginning with X ₀ = 1, the gambler bets the amount p, 0 < p < 1. If the first game is a win, which occurs with probability $\frac{1}{2}$ , the gambler's fortune is X ₁ = 1 + pX ₀ = 1 + p. If the first game is a loss, then X ₁ = 1 − pX ₀ = 1 − p. After the n th play and with a current fortune of X_n , the gambler wagers pX_n , and

$X_{n+1}\{\begin{array}{c}\left(1+p\right)X_n\text{with}\text{probability}\frac{1}{2}\\ \left(1-p\right)X_n\text{with}\text{probability}\frac{1}{2}.\end{array}$

Then, {X_n } is a nonnegative martingale, and the maximal inequality (2.52) with λ = 2, e.g., asserts that the probability that the gambler ever doubles his money is less than or equal to $\frac{1}{2}$ , and this holds no matter what the game is, as long as it is fair, and no matter what fraction p of his fortune is wagered at each play. Indeed, the fraction wagered may vary from play to play, as long as it is chosen without knowledge of the next outcome.

As amply demonstrated by this example, the maximal inequality is a very strong statement. Indeed, more elaborate arguments based on the maximal and other related martingale inequalities are used to show that a nonnegative martingale converges: If {X _n } is a nonnegative martingale, then there exists a random variable, let us call it X_∞, for which lim _{n →∞} X _n = X _∞. We cannot guarantee the equality of the expectations in the limit, but the inequality E [X ₀] ≥ E [X _∞] ≥ 0 can be established.

Example In Chapter 3, Section 3.8, we will introduce the branching process model for population growth. In this model, X _n is the number of individuals in the population in the n th generation, and μ > 0 is the mean family size or expected number of offspring of any single individual. The mean population size in the nth generation is X ₀μⁿ. In this branching process model, X_n /μ ⁿ is a nonnegative martingale (see Chapter 3, Problem 3.8.4), and the maximal inequality implies that the probability of the actual population ever exceeding 10 times the mean size is less than or equal to 1/10. The nonnegative martingale convergence theorem asserts that the evolution of such a population after many generations may be described by a single random variable X _∞ in the form

$X_n\approx X_{\infty }{\mu }^n,\text{for}\text{large}n$

Example How NOT to generate a uniformly distributed random variable An urn initially contains one red and one green ball. A ball is drawn at random and it is returned to the urn, together with another ball of the same color. This process is repeated indefinitely. After the nth play, there will be a total of n + 2 balls in the urn. Let R_n be the number of these balls that are red, and X_n = R _n /(n + 2) the fraction of red balls. We claim that {X _n } is a martingale. First, observe that

$R_{n+1}=\{\begin{array}{cc}{R}_{n+1}& \text{with}\text{probability}{X}_n\\ R_n& \text{with}\text{probability}1-X_n\end{array}$

so that

$E\left[R_{n+1}|X_n\right]=R_n+X_n=X_n\left(2+n+1\right),$

and finally,

$E\left[X_{n+1}|X_n\right]=\frac{1}{n+3}E\left[R_{n+1}|X_n\right]=\frac{2+n+1}{n+3}{X}_n=X_n.$

This verifies the martingale property, and because such a fraction is always nonnegative, indeed, between 0 and 1, there must be a random variable X _∞ to which the martingale converges. We will derive the probability distribution of the random limit. It is immediate that R ₁ is equally likely to be 1 or 2, since the first ball chosen is equally likely to be red or green. Continuing,

$\begin{array}{l}Pr\left\{R_2=3\right\}=Pr\left\{R_2=3|R_1=2\right\}Pr\left\{R_1=2\right\}\\ =\left(\frac{2}{3}\right)\left(\frac{1}{2}\right)=\frac{1}{3};\\ Pr\left\{R_2=2\right\}=Pr\left\{R_2=2|R_1=1\right\}Pr\left\{R_1=1\right\}\\ +Pr\left\{R_2=2|R_1=2\right\}Pr\left\{R_1=2\right\}\\ =\left(\frac{1}{3}\right)\left(\frac{1}{2}\right)+\left(\frac{1}{3}\right)\left(\frac{1}{2}\right)=\frac{1}{3};\end{array}$

and since the probabilities must sum to 1,

$Pr\left\{R_2=1\right\}=\frac{1}{3}.$

By repeating these simple calculations, it is easy to see that

$Pr\left\{R_n=k\right\}=\frac{1}{n+1}\text{for}k=1,2,\mathrm{\dots },n+1,$

and that, therefore, X_n is uniformly distributed over the values 1/(n + 2), 2/(n + 2), …, (n + 1)/(n + 2). This uniform distribution must prevail in the limit, which leads to

$Pr\left\{X_{\infty }\le x\right\}=x\text{for}0<x<1.$

Think about this remarkable result for a minute! If you sit down in front of such an urn and play this game, eventually the fraction of red balls in your urn will stabilize in the near vicinity of some value, call it U. If I play the game, the fraction of red balls in my urn will stabilize also, but at another value, U ′. Anyone who plays the game will find the fraction of red balls in the urn tending toward some limit, but everyone will experience a different limit. In fact, each play of the game generates a fresh, uniformly distributed random variable, in the limit. Of course, there may be faster and simpler ways to generate uniformly distributed random variables.

Martingale implications include many more inequalities and convergence theorems. As briefly mentioned at the start, there are so-called systems theorems that delimit the conditions under which a gambling system, such as doubling the bets until a win is secured, can turn a fair game into a winning game. A deeper discussion of martingale theory would take us well beyond the scope of this introductory text, and our aim must be limited to building an enthusiasm for further study. Nevertheless, a large variety of important martingales will be introduced in the Problems at the end of each section in the remainder of the book.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123814166000022

Other Generalised Functions

R.F. Hoskins Research Professor , in Delta Functions (Second Edition), 2011

7.1.4 Fractional differentiation

We have achieved a satisfactory definition of an integration operator I ^λ which makes sense for all nonnegative values of λ. The next step is to introduce a corresponding definition of a derivative operator of non-integer order. This will effectively extend the meaning of I ^λ to negative values of λ. In what follows we shall assume that all input signals x(t) are sufficiently well behaved to allow the necessary integrations and differentiations to be carried out; that is we assume not only that each x(t) vanishes outside some finite interval but also that it may be differentiated as often as we wish.

To begin with, let α be any real number such that 0   ≤ α  <   1. Then I ^{1   − α} is certainly well-defined and so we can define an operator D^α which represents the cascade connection of the fractional integrator represented by I ^{1   − α} and an ideal differentiator:

(7.14) $\begin{array}{l}x\left(t\right)\to y\left(t\right)\equiv D^{\alpha }x\left(t\right)=\frac{d}{\mathit{dt}}{I}^{1-\alpha }x\left(t\right)\hfill \\ =\frac{1}{\Gamma \left(1-\alpha \right)}\frac{d}{\mathit{dt}}{\displaystyle {\int }_{-\infty }^{t}x\left(\tau \right){\left(t-\tau \right)}^{-\alpha }\mathit{d\tau }}\text{.}\hfill \end{array}$

For α  =   0 this reduces to

$x\left(t\right)\to D^{0}x\left(t\right)=\frac{d}{\mathit{dt}}{\displaystyle {\int }_{-\infty }^{t}x\left(\tau \right)\mathit{d\tau }=x\left(t\right)}$

So that the impulse response is δ(t) and D ⁰ is identical with I ⁰. Further since we have the equivalent representation

$D^{\alpha }x\left(t\right)=\frac{1}{\Gamma \left(1-\alpha \right)}{\displaystyle {\int }_0^{+\infty }{\tau }^{-\alpha }{x}^{\prime }\left(t-\tau \right)\mathit{d\tau }}=I^{1-\alpha }{x}^{\prime }\left(t\right)\text{,}$

it follows at once that

$I^{\alpha }\left\{D^{\alpha }x\left(t\right)\right\}=I^{\alpha }\left\{I^{1-\alpha }{x}^{\prime }\left(t\right)\right\}=I^1{x}^{\prime }\left(t\right)=x\left(t\right)\text{.}$

Hence, for 0   ≤ α  <   1, D^α is the inverse of the fractional integration operator I^α . Finally, for an arbitrary λ   >   0 we can always write

$\mathrm{\lambda }=n-1+\alpha \mathrm{where}n\in \mathbb{N}\mathrm{and}0\le \alpha <1\text{,}$

and to complete the definition, we need only to set

(7.15) $D^{\mathrm{\lambda }}x\left(t\right)\equiv \frac{1}{\Gamma \left(1-\alpha \right)}\frac{d^n}{d{t}^n}{\displaystyle {\int }_{-\infty }^{t}x\left(\tau \right){\left(t-\tau \right)}^{-\alpha }\mathit{d\tau }}$

The operator D ^λ represents a (causal) time-invariant linear system which consists of a cascade combination of a fractional integrator of order 1   − α and n ideal differentiators. It should therefore have the transfer function H(s)   = s ^{n  −   1   + α} . However, for any value of n  >   1, there will exist no (ordinary) function h(t) of which H(s) is the Laplace transform, and so we cannot express (7.11) in the form of a classical convolution integral. If α  =   0 then D ^λ becomes D ^{n  −   1} and the problem is solved at least formally by the introduction of the generalised function δ ^{(n  −   1)}(t) To obtain a satisfactory development of a comprehensive fractional calculus we therefore need to define other types of generalised functions.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9781904275398500078

Elements of Probability

Sheldon Ross , in Simulation (Fifth Edition), 2013

2.7 Chebyshev's Inequality and the Laws of Large Numbers

We start with a result known as Markov's inequality.

Proposition 3 Markov's Inequality

If $X$ takes on only nonnegative values, then for any value $a>0$

$P\{X⩾a\}⩽\frac{E\left[X\right]}{a}$

Proof 3

Define the random variable $Y$ by

$Y=\left\{\begin{array}{cc}\hfill a\text{,}\hfill & \hfill \mathrm{if}X⩾a\hfill \\ \hfill 0\text{,}\hfill & \hfill \mathrm{if}X<a\hfill \end{array}\right.$

Because $X⩾0$ , it easily follows that

$X⩾Y$

Taking expectations of the preceding inequality yields

$E\left[X\right]⩾E\left[Y\right]=\mathit{aP}\{X⩾a\}$

and the result is proved. $\square$

As a corollary we have Chebyshev's inequality, which states that the probability that a random variable differs from its mean by more than $k$ of its standard deviations is bounded by $1/k^2\text{,}$ where the standard deviation of a random variable is defined to be the square root of its variance.

Corollary 2 Chebyshev's Inequality

If $X$ is a random variable having mean $\mu$ and variance ${\sigma }^2$ , then for any value $k>0$ ,

$P\{|X-\mu |⩾k\sigma \}⩽\frac{1}{k^2}$

Proof 4

Since $(X-\mu {)}^2/{\sigma }^2$ is a nonnegative random variable whose mean is

$E\left[\frac{(X-\mu {)}^2}{{\sigma }^2}\right]=\frac{E\left[(X-\mu {)}^2\right]}{{\sigma }^2}=1$

we obtain from Markov's inequality that

$P\left\{\frac{(X-\mu {)}^2}{{\sigma }^2}⩾k^2\right\}⩽\frac{1}{k^2}$

The result now follows since the inequality $(X-\mu {)}^2/{\sigma }^2⩾k^2$ is equivalent to the inequality $|X-\mu |⩾k\sigma$ . $\square$

We now use Chebyshev's inequality to prove the weak law of large numbers, which states that the probability that the average of the first $n$ terms of a sequence of independent and identically distributed random variables differs from its mean by more than $ϵ$ goes to 0 as $n$ goes to infinity.

Theorem 1 The Weak Law of Large Numbers

Let $X_1\text{,}{X}_2\text{,}\dots$ be a sequence of independent and identically distributed random variables having mean $\mu$ . Then, for any $ϵ>0$ ,

$P\left\{\left|\frac{X_1+\cdots +X_n}{n}-\mu \right|>ϵ\right\}\to 0\mathrm{as}n\to \infty$

Proof 5

We give a proof under the additional assumption that the random variables $X_i$ have a finite variance ${\sigma }^2$ . Now

$E\left[\frac{X_1+\cdots +X_n}{n}\right]=\frac{1}{n}(E\left[X_1\right]+\cdots +E\left[X_n\right])=\mu$

and

$\mathrm{Var}\left(\frac{X_1+\cdots +X_n}{n}\right)=\frac{1}{n^2}\left[\mathrm{Var}(X_1)+\cdots +\mathrm{Var}(X_n)\right]=\frac{{\sigma }^2}{n}$

where the above equation makes use of the fact that the variance of the sum of independent random variables is equal to the sum of their variances. Hence, from Chebyshev's inequality, it follows that for any positive $k$

$P\left\{\left|\frac{X_1+\cdots +X_n}{n}-\mu \right|⩾\frac{k\sigma }{\sqrt{n}}\right\}⩽\frac{1}{k^2}$

Hence, for any $ϵ>0$ , by letting $k$ be such that $k\sigma /\sqrt{n}=ϵ\text{,}$ that is, by letting $k^2=n{ϵ}^2/{\sigma }^2$ , we see that

$P\left\{\left|\frac{X_1+\cdots +X_n}{n}-\mu \right|⩾ϵ\right\}⩽\frac{{\sigma }^2}{n{ϵ}^2}$

which establishes the result. $\square$

A generalization of the weak law is the strong law of large numbers, which states that, with probability 1,

${\displaystyle \underset{n\to \infty }{\lim }}\frac{X_1+\cdots +X_n}{n}=\mu$

That is, with certainty, the long-run average of a sequence of independent and identically distributed random variables will converge to its mean.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780124158252000024

Random Variables and Expectation

Sheldon M. Ross , in Introduction to Probability and Statistics for Engineers and Scientists (Fifth Edition), 2014

4.9 Chebyshev's Inequality and the Weak Law of Large Numbers

We start this section by proving a result known as Markov's inequality.

Proposition 4.9.1 Markov's Inequality

If X is a random variable that takes only nonnegative values, then for any value a > 0

$P\left\{X\ge a\right\}\le \frac{E\left[X\right]}{a}$

Proof

We give a proof for the case where X is continuous with density f.

$\begin{array}{lll}E\left[X\right]\hfill & =\hfill & {\displaystyle {\int }_0^{\infty }xf\left(x\right)dx}\hfill \\ \hfill & =\hfill & {\displaystyle {\int }_0^{a}xf\left(x\right)dx+{\displaystyle {\int }_a^{\infty }xf\left(x\right)dx}}\hfill \\ \hfill & \ge \hfill & {\displaystyle {\int }_a^{\infty }xf\left(x\right)dx}\hfill \\ \hfill & \ge \hfill & {\displaystyle {\int }_a^{\infty }af\left(x\right)dx}\hfill \\ \hfill & =\hfill & a{\displaystyle {\int }_a^{\infty }f\left(x\right)dx}\hfill \\ \hfill & =\hfill & aP\left\{X\ge a\right\}\hfill \end{array}$

and the result is proved.■

As a corollary, we obtain Proposition 4.9.2.

Proposition 4.9.2 Chebyshev's Inequality

If X is a random variable with mean μ and variance σ ², then for any value k > 0

$P\left\{\left|X-\mu \right|\ge k\right\}\le \frac{{\sigma }^2}{k^2}$

Proof

Since (X – μ)² is a nonnegative random variable, we can apply Markov's inequality (with a  = k ²) to obtain

(4.9.1) $P\left\{{\left(X-\mu \right)}^2\ge k^2\right\}\le \frac{E\left[{\left(X-\mu \right)}^2\right]}{k^2}$

But since (X – μ) ≥ k ² if and only if $\left|X-\mu \right|\ge k$ , Equation 4.9.1 is equivalent to

$P\left\{\left|X-\mu \right|\ge k\right\}\le \frac{E\left[{\left(X-\mu \right)}^2\right]}{k^2}=\frac{{\sigma }^2}{k^2}$

and the proof is complete.■

The importance of Markov's and Chebyshev's inequalities is that they enable us to derive bounds on probabilities when only the mean, or both the mean and the variance, of the probability distribution are known. Of course, if the actual distribution were known, then the desired probabilities could be exactly computed and we would not need to resort to bounds.

Example 4.9a

Suppose that it is known that the number of items produced in a factory during a week is a random variable with mean 50.

(a)

What can be said about the probability that this week's production will exceed 75?

(b)

If the variance of a week's production is known to equal 25, then what can be said about the probability that this week's production will be between 40 and 60?

Solution

Let X be the number of items that will be produced in a week:

(a)

By Markov's inequality

$P\left\{X>75\right\}\le \frac{E\left[X\right]}{75}=\frac{50}{75}=\frac{2}{3}$

(b)

By Chebyshev's inequality

$P\left\{\left|X-50\right|\ge 10\right\}\le \frac{{\sigma }^2}{{10}^2}=\frac{1}{4}$

Hence

$P\left\{\left|X-50\right|<10\right\}\ge 1-\frac{1}{4}=\frac{3}{4}$

and so the probability that this week's production will be between 40 and 60 is at least .75.■

By replacing k by kσ in Equation 4.9.1, we can write Chebyshev's inequality as

$P\left\{\left|X-\mu \right|>k\sigma \right\}\le 1\text{/}{k}^2$

Thus it states that the probability a random variable differs from its mean by more than k standard deviations is bounded by 1/k ².

We will end this section by using Chebyshev's inequality to prove the weak law of large numbers, which states that the probability that the average of the first n terms in a sequence of independent and identically distributed random variables differs by its mean by more than ε goes to 0 as n goes to infinity.

Theorem 4.9.3 The Weak Law of Large Numbers

Let X ₁, X ₂,…, be a sequence of independent and identically distributed random variables, each having mean E[X_i ]   = μ. Then, for any ε > 0,

$\begin{array}{ll}P\left\{\left|\frac{X_1+\cdots +X_n}{n}-\mu \right|>\epsilon \right\}\to 0\hfill & \text{as}n\to \infty \hfill \end{array}$

Proof

We shall prove the result only under the additional assumption that the random variables have a finite variance σ ². Now, as

$\begin{array}{lll}E\left[\frac{X_1+\cdots +X_n}{n}\right]=\mu \hfill & \text{and}\hfill & \text{Var}\left(\frac{X_1+\cdots +X_n}{n}\right)\hfill \end{array}=\frac{{\sigma }^2}{n}$

it follows from Chebyshev's inequality that

$P\left\{\left|\frac{X_1+\cdots +X_n}{n}-\mu \right|>ϵ\right\}\le \frac{{\sigma }^2}{n{ϵ}^2}$

and the result is proved.■

For an application of the above, suppose that a sequence of independent trials is performed. Let E be a fixed event and denote by P(E) the probability that E occurs on a given trial. Letting

$X_i=\{\begin{array}{ll}1\hfill & \text{if}E\text{occurs\hspace{0.17em}on\hspace{0.17em}trial}i\hfill \\ 0\hfill & \text{if}E\text{does\hspace{0.17em}not\hspace{0.17em}occur\hspace{0.17em}on\hspace{0.17em}trial}i\hfill \end{array}$

it follows that X ₁  + X ₂  +   ⋯   + X_n represents the number of times that E occurs in the first n trials. Because E[X_i ]   = P(E), it thus follows from the weak law of large numbers that for any positive number ε, no matter how small, the probability that the proportion of the first n trials in which E occurs differs from P(E) by more than ε goes to 0 as n increases.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123948113500046

Random Variables

Sheldon Ross , in Introduction to Probability Models (Eleventh Edition), 2014

2.8 Limit Theorems

We start this section by proving a result known as Markov's inequality.

Proposition 2.6 Markov's Inequality

If $X$ is a random variable that takes only nonnegative values, then for any value $a>0$

$P\{X\ge a\}\le \frac{E[X]}{a}$

Proof

We give a proof for the case where $X$ is continuous with density $f$ .

$\begin{array}{cc}\hfill E[X]=& {\int }_0^{\infty }\mathit{xf}(x)\mathit{dx}\hfill \\ \hfill =& {\int }_0^a\mathit{xf}(x)\mathit{dx}+{\int }_a^{\infty }\mathit{xf}(x)\mathit{dx}\hfill \\ \hfill \ge & {\int }_a^{\infty }\mathit{xf}(x)\mathit{dx}\hfill \\ \hfill \ge & {\int }_a^{\infty }\mathit{af}(x)\mathit{dx}\hfill \\ \hfill =& a{\int }_a^{\infty }f(x)\mathit{dx}\hfill \\ \hfill =& \mathit{aP}\{X\ge a\}\hfill \end{array}$

and the result is proven.

As a corollary, we obtain the following.

Proposition 2.7 Chebyshev's Inequality

If $X$ is a random variable with mean $\mu$ and variance ${\sigma }^2$ , then, for any value $k>0$ ,

$P\{\mid X-\mu \mid \ge k\}\le \frac{{\sigma }^2}{k^2}$

Proof

Since ${(X-\mu )}^2$ is a nonnegative random variable, we can apply Markov's inequality (with $a=k^2$ ) to obtain

$P\{{(X-\mu )}^2\ge k^2\}\le \frac{E[{(X-\mu )}^2]}{k^2}$

But since ${(X-\mu )}^2\ge k^2$ if and only if $\mid X-\mu \mid \ge k$ , the preceding is equivalent to

$P\{\mid X-\mu \mid \ge k\}\le \frac{E[{(X-\mu )}^2]}{k^2}=\frac{{\sigma }^2}{k^2}$

and the proof is complete.

The importance of Markov's and Chebyshev's inequalities is that they enable us to derive bounds on probabilities when only the mean, or both the mean and the variance, of the probability distribution are known. Of course, if the actual distribution were known, then the desired probabilities could be exactly computed, and we would not need to resort to bounds.

Example 2.49

Suppose we know that the number of items produced in a factory during a week is a random variable with mean 500.

(a)

What can be said about the probability that this week's production will be at least 1000?

(b)

If the variance of a week's production is known to equal 100, then what can be said about the probability that this week's production will be between 400 and 600?

Solution:  Let $X$ be the number of items that will be produced in a week.

(a)

By Markov's inequality,

$P\{X\ge 1000\}\le \frac{E[X]}{1000}=\frac{500}{1000}=\frac{1}{2}$

(b)

By Chebyshev's inequality,

$P\{\mid X-500\mid \ge 100\}\le \frac{{\sigma }^2}{{(100)}^2}=\frac{1}{100}$

Hence,

$P\{\mid X-500\mid <100\}\ge 1-\frac{1}{100}=\frac{99}{100}$

and so the probability that this week's production will be between 400 and 600 is at least 0.99.   ■

The following theorem, known as the strong law of large numbers, is probably the most well-known result in probability theory. It states that the average of a sequence of independent random variables having the same distribution will, with probability 1, converge to the mean of that distribution.

Theorem 2.1 Strong Law of Large Numbers

Let $X_1\text{,}{X}_2\text{,}\dots$ be a sequence of independent random variables having a common distribution, and let $E[X_i]=\mu$ . Then, with probability 1,

$\frac{X_1+X_2+\cdots +X_n}{n}\to \mu \mathrm{as}n\to \infty$

As an example of the preceding, suppose that a sequence of independent trials is performed. Let $E$ be a fixed event and denote by $P(E)$ the probability that $E$ occurs on any particular trial. Letting

$X_i=\left\{\begin{array}{cc}1\text{,}\hfill & \mathrm{if}E\mathrm{occurs\; on\; the}i\mathrm{th\; trial}\hfill \\ 0\text{,}\hfill & \mathrm{if}E\mathrm{does\; not\; occur\; on\; the}i\mathrm{th\; trial}\hfill \end{array}\right.$

we have by the strong law of large numbers that, with probability 1,

(2.25) $\frac{X_1+\cdots +X_n}{n}\to E[X]=P(E)$

Since $X_1+\cdots +X_n$ represents the number of times that the event $E$ occurs in the first $n$ trials, we may interpret Equation (2.25) as stating that, with probability 1, the limiting proportion of time that the event $E$ occurs is just $P(E)$ .

Running neck and neck with the strong law of large numbers for the honor of being probability theory's number one result is the central limit theorem. Besides its theoretical interest and importance, this theorem provides a simple method for computing approximate probabilities for sums of independent random variables. It also explains the remarkable fact that the empirical frequencies of so many natural "populations" exhibit a bell-shaped (that is, normal) curve.

Theorem 2.2 Central Limit Theorem

Let $X_1\text{,}{X}_2\text{,}\dots$ be a sequence of independent, identically distributed random variables, each with mean $\mu$ and variance ${\sigma }^2$ . Then the distribution of

$\frac{X_1+X_2+\cdots +X_n-n\mu }{\sigma \sqrt{n}}$

tends to the standard normal as $n\to \infty$ . That is,

$P\left\{\frac{X_1+X_2+\cdots +X_n-n\mu }{\sigma \sqrt{n}}\le a\right\}\to \frac{1}{\sqrt{2\pi }}{\int }_{-\infty }^a{e}^{-x^2/2}\mathit{dx}$

as $n\to \infty$ .

Note that like the other results of this section, this theorem holds for any distribution of the $X_i$ s; herein lies its power.

If $X$ is binomially distributed with parameters $n$ and $p$ , then $X$ has the same distribution as the sum of $n$ independent Bernoulli random variables, each with parameter $p$ . (Recall that the Bernoulli random variable is just a binomial random variable whose parameter $n$ equals 1.) Hence, the distribution of

$\frac{X-E[X]}{\sqrt{\mathrm{Var}(X)}}=\frac{X-\mathit{np}}{\sqrt{\mathit{np}(1-p)}}$

approaches the standard normal distribution as $n$ approaches $\infty$ . The normal approximation will, in general, be quite good for values of $n$ satisfying $\mathit{np}(1-p)\ge 10$ .

Example 2.50 Normal Approximation to the Binomial

Let $X$ be the number of times that a fair coin, flipped 40 times, lands heads. Find the probability that $X=20$ . Use the normal approximation and then compare it to the exact solution.

Solution:  Since the binomial is a discrete random variable, and the normal a continuous random variable, it leads to a better approximation to write the desired probability as

$\begin{array}{cc}\hfill P\{X=20\}=& P\{19.5<X<20.5\}\hfill \\ \hfill =& P\left\{\frac{19.5-20}{\sqrt{10}}<\frac{X-20}{\sqrt{10}}<\frac{20.5-20}{\sqrt{10}}\right\}\hfill \\ \hfill =& P\left\{-0.16<\frac{X-20}{\sqrt{10}}<0.16\right\}\hfill \\ \hfill \approx & \mathrm{\Phi }(0.16)-\mathrm{\Phi }(-0.16)\hfill \end{array}$

where $\mathrm{\Phi }(x)$ , the probability that the standard normal is less than $x$ is given by

$\mathrm{\Phi }(x)=\frac{1}{\sqrt{2\pi }}{\int }_{-\infty }^x{e}^{-y^2/2}\mathit{dy}$

By the symmetry of the standard normal distribution

$\mathrm{\Phi }(-0.16)=P\{N(0\text{,}1)>0.16\}=1-\mathrm{\Phi }(0.16)$

where $N(0\text{,}1)$ is a standard normal random variable. Hence, the desired probability is approximated by

$P\{X=20\}\approx 2\mathrm{\Phi }(0.16)-1$

Using Table 2.3, we obtain

$P\{X=20\}\approx 0.1272$

The exact result is

$P\{X=20\}=\left(\begin{array}{c}\hfill 40\hfill \\ \hfill 20\hfill \end{array}\right){\left(\frac{1}{2}\right)}^{40}$

which can be shown to equal 0.1268.

 

■

Table 2.3. Area $\mathrm{\Phi }(x)$ under the Standard Normal Curve to the Left of $x$

$x$	0.00	0.01	0.02	0.03	0.04	0.05	0.06	0.07	0.08	0.09
0.0	0.5000	0.5040	0.5080	0.5120	0.5160	0.5199	0.5239	0.5279	0.5319	0.5359
0.1	0.5398	0.5438	0.5478	0.5517	0.5557	0.5597	0.5636	0.5675	0.5714	0.5753
0.2	0.5793	0.5832	0.5871	0.5910	0.5948	0.5987	0.6026	0.6064	0.6103	0.6141
0.3	0.6179	0.6217	0.6255	0.6293	0.6331	0.6368	0.6406	0.6443	0.6480	0.6517
0.4	0.6554	0.6591	0.6628	0.6664	0.6700	0.6736	0.6772	0.6808	0.6844	0.6879
0.5	0.6915	0.6950	0.6985	0.7019	0.7054	0.7088	0.7123	0.7157	0.7190	0.7224
0.6	0.7257	0.7291	0.7324	0.7357	0.7389	0.7422	0.7454	0.7486	0.7517	0.7549
0.7	0.7580	0.7611	0.7642	0.7673	0.7704	0.7734	0.7764	0.7794	0.7823	0.7852
0.8	0.7881	0.7910	0.7939	0.7967	0.7995	0.8023	0.8051	0.8078	0.8106	0.8133
0.9	0.8159	0.8186	0.8212	0.8238	0.8264	0.8289	0.8315	0.8340	0.8365	0.8389
1.0	0.8413	0.8438	0.8461	0.8485	0.8508	0.8531	0.8554	0.8557	0.8599	0.8621
1.1	0.8643	0.8665	0.8686	0.8708	0.8729	0.8749	0.8770	0.8790	0.8810	0.8830
1.2	0.8849	0.8869	0.8888	0.8907	0.8925	0.8944	0.8962	0.8980	0.8997	0.9015
1.3	0.9032	0.9049	0.9066	0.9082	0.9099	0.9115	0.9131	0.9147	0.9162	0.9177
1.4	0.9192	0.9207	0.9222	0.9236	0.9251	0.9265	0.9279	0.9292	0.9306	0.9319
1.5	0.9332	0.9345	0.9357	0.9370	0.9382	0.9394	0.9406	0.9418	0.9429	0.9441
1.6	0.9452	0.9463	0.9474	0.9484	0.9495	0.9505	0.9515	0.9525	0.9535	0.9545
1.7	0.9554	0.9564	0.9573	0.9582	0.9591	0.9599	0.9608	0.9616	0.9625	0.9633
1.8	0.9641	0.9649	0.9656	0.9664	0.9671	0.9678	0.9686	0.9693	0.9699	0.9706
1.9	0.9713	0.9719	0.9726	0.9732	0.9738	0.9744	0.9750	0.9756	0.9761	0.9767
2.0	0.9772	0.9778	0.9783	0.9788	0.9793	0.9798	0.9803	0.9808	0.9812	0.9817
2.1	0.9821	0.9826	0.9830	0.9834	0.9838	0.9842	0.9846	0.9850	0.9854	0.9857
2.2	0.9861	0.9864	0.9868	0.9871	0.9875	0.9878	0.9881	0.9884	0.9887	0.9890
2.3	0.9893	0.9896	0.9898	0.9901	0.9904	0.9906	0.9909	0.9911	0.9913	0.9916
2.4	0.9918	0.9920	0.9922	0.9925	0.9927	0.9929	0.9931	0.9932	0.9934	0.9936
2.5	0.9938	0.9940	0.9941	0.9943	0.9945	0.9946	0.9948	0.9949	0.9951	0.9952
2.6	0.9953	0.9955	0.9956	0.9957	0.9959	0.9960	0.9961	0.9962	0.9963	0.9964
2.7	0.9965	0.9966	0.9967	0.9968	0.9969	0.9970	0.9971	0.9972	0.9973	0.9974
2.8	0.9974	0.9975	0.9976	0.9977	0.9977	0.9978	0.9979	0.9979	0.9980	0.9981
2.9	0.9981	0.9982	0.9982	0.9983	0.9984	0.9984	0.9985	0.9985	0.9986	0.9986
3.0	0.9987	0.9987	0.9987	0.9988	0.9988	0.9989	0.9989	0.9989	0.9990	0.9990
3.1	0.9990	0.9991	0.9991	0.9991	0.9992	0.9992	0.9992	0.9992	0.9993	0.9993
3.2	0.9993	0.9993	0.9994	0.9994	0.9994	0.9994	0.9994	0.9995	0.9995	0.9995
3.3	0.9995	0.9995	0.9995	0.9996	0.9996	0.9996	0.9996	0.9996	0.9996	0.9997
3.4	0.9997	0.9997	0.9997	0.9997	0.9997	0.9997	0.9997	0.9997	0.9997	0.9998

Example 2.51

Let $X_i\text{,}i=1\text{,}2\text{,}\dots \text{,}10$ be independent random variables, each being uniformly distributed over (0, 1). Estimate $P\{{\sum }_1^{10}{X}_i>7\}$ .

Solution:  Since $E[X_i]=\frac{1}{2}$ , Var $(X_i)=\frac{1}{12}$ we have by the central limit theorem that

$\begin{array}{cc}\hfill P\left\{{\displaystyle \sum _1^{10}}{X}_i>7\right\}=& P\left\{\frac{{\displaystyle {\sum }_1^{10}}{X}_i-5}{\sqrt{10\left(\frac{1}{12}\right)}}>\frac{7-5}{\sqrt{10\left(\frac{1}{12}\right)}}\right\}\hfill \\ \hfill \approx & 1-\mathrm{\Phi }(2.2)\hfill \\ \hfill =& 0.0139■\hfill \end{array}$

Example 2.52

The lifetime of a special type of battery is a random variable with mean 40 hours and standard deviation 20 hours. A battery is used until it fails, at which point it is replaced by a new one. Assuming a stockpile of 25 such batteries, the lifetimes of which are independent, approximate the probability that over 1100 hours of use can be obtained.

Solution:  If we let $X_i$ denote the lifetime of the $i$ th battery to be put in use, then we desire $p=P\{X_1+\cdots +X_{25}>1100\}$ , which is approximated as follows:

$\begin{array}{cc}\hfill p=& P\left\{\frac{X_1+\cdots +X_{25}-1000}{20\sqrt{25}}>\frac{1100-1000}{20\sqrt{25}}\right\}\hfill \\ \hfill \approx & P\{N(0\text{,}1)>1\}\hfill \\ \hfill =& 1-\mathrm{\Phi }(1)\hfill \\ \hfill \approx & 0.1587■\hfill \end{array}$

We now present a heuristic proof of the central limit theorem. Suppose first that the $X_i$ have mean 0 and variance 1, and let $E[e^{\mathit{tX}}]$ denote their common moment generating function. Then, the moment generating function of $\frac{X_1+\cdots +X_n}{\sqrt{n}}$ is

$\begin{array}{cc}\hfill E\left[\exp \left\{t\left(\frac{X_1+\cdots +X_n}{\sqrt{n}}\right)\right\}\right]=& E[e^{{\mathit{tX}}_1/\sqrt{n}}{e}^{{\mathit{tX}}_2/\sqrt{n}}\cdots e^{{\mathit{tX}}_n/\sqrt{n}}]\hfill \\ \hfill =& {(E[e^{\mathit{tX}/\sqrt{n}}])}^n\mathrm{by\; independence}\hfill \end{array}$

Now, for $n$ large, we obtain from the Taylor series expansion of $e^y$ that

$e^{\mathit{tX}/\sqrt{n}}\approx 1+\frac{\mathit{tX}}{\sqrt{n}}+\frac{t^2{X}^2}{2n}$

Taking expectations shows that when $n$ is large

$\begin{array}{cc}\hfill E[e^{\mathit{tX}/\sqrt{n}}]\approx & 1+\frac{\mathit{tE}[X]}{\sqrt{n}}+\frac{t^{2}E[X^2]}{2n}\hfill \\ \hfill =& 1+\frac{t^2}{2n}\mathrm{because}E[X]=0\text{,}E[X^2]=1\hfill \end{array}$

Therefore, we obtain that when $n$ is large

$E\left[\exp \left\{t\left(\frac{X_1+\cdots +X_n}{\sqrt{n}}\right)\right\}\right]\approx {\left(1+\frac{t^2}{2n}\right)}^n$

When $n$ goes to $\infty$ the approximation can be shown to become exact and we have

${\displaystyle \underset{n\to \infty }{\lim }}E\left[\exp \left\{t\left(\frac{X_1+\cdots +X_n}{\sqrt{n}}\right)\right\}\right]=e^{t^2/2}$

Thus, the moment generating function of $\frac{X_1+\cdots +X_n}{\sqrt{n}}$ converges to the moment generating function of a (standard) normal random variable with mean 0 and variance 1. Using this, it can be proven that the distribution function of the random variable $\frac{X_1+\cdots +X_n}{\sqrt{n}}$ converges to the standard normal distribution function $\mathrm{\Phi }$ .

When the $X_i$ have mean $\mu$ and variance ${\sigma }^2$ , the random variables $\frac{X_i-\mu }{\sigma }$ have mean 0 and variance 1. Thus, the preceding shows that

$P\left\{\frac{X_1-\mu +X_2-\mu +\cdots +X_n-\mu }{\sigma \sqrt{n}}\le a\right\}\to \mathrm{\Phi }(a)$

which proves the central limit theorem.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780124079489000025

brownwereeld.blogspot.com

Source: https://www.sciencedirect.com/topics/mathematics/nonnegative-value

Share this post