Probability vector

In mathematics and statistics, a probability vector or stochastic vector is a vector with non-negative entries that add up to one.

Underlying every probability vector is an experiment that can produce an outcome. To connect this experiment with mathematics, one introduces a discrete random variable, a function that assigns a numerical value to each possible outcome. For example, if the experiment consists of rolling a single die, the random variable may be defined as the number of pips on the upward face. The possible values of this random variable are the integers 1 , 2 , … , 6 The associated probability vector then has six components, each representing the probability of obtaining the corresponding outcome. More generally, a probability vector of length 𝑛 represents the distribution of probabilities across the 𝑛 possible numerical outcomes of a discrete random variable.

The vector gives us the probability mass function of that random variable, which is the standard way of characterizing a discrete probability distribution.^[1]

Examples

Here are some examples of probability vectors. The vectors can be either columns or rows.

$x_{0}={\begin{bmatrix}0.5\\0.25\\0.25\end{bmatrix}},$
$x_{1}={\begin{bmatrix}0\\1\\0\end{bmatrix}},$
$x_{2}={\begin{bmatrix}0.65&0.35\end{bmatrix}},$
$x_{3}={\begin{bmatrix}0.3&0.5&0.07&0.1&0.03\end{bmatrix}}.$

Geometric interpretation

Writing out the vector components of a vector $p$ as

p={\begin{bmatrix}p_{1}\\p_{2}\\\vdots \\p_{n}\end{bmatrix}}\quad {\text{or}}\quad p={\begin{bmatrix}p_{1}&p_{2}&\cdots &p_{n}\end{bmatrix}}

the vector components must sum to one:

\sum _{i=1}^{n}p_{i}=1

Each individual component must have a probability between zero and one:

0\leq p_{i}\leq 1

for all $i$ . Therefore, the set of stochastic vectors coincides with the standard $(n-1)$ -simplex. It is a point if $n=1$ , a segment if $n=2$ , a (filled) triangle if $n=3$ , a (filled) tetrahedron if $n=4$ , etc.

Properties

The mean of the components of any probability vector is $1/n$ .
The shortest probability vector has the value $1/n$ as each component of the vector, and has a length of ${\textstyle 1/{\sqrt {n}}}$ .
The longest probability vector has the value 1 in a single component and 0 in all others, and has a length of 1.
The shortest vector corresponds to maximum uncertainty, the longest to maximum certainty.
The length of a probability vector is equal to ${\textstyle {\sqrt {n\sigma ^{2}+1/n}}}$ ; where $\sigma ^{2}$ is the variance of the elements of the probability vector.
The bounds on the variance are ${\textstyle \sigma ^{2}\in [0,(n-1)/n^{2}].}$
The derivative with respect to n of the maximum variance is ${\tfrac {d}{dn}}\sigma _{\max }^{2}={\tfrac {2-n}{n^{3}}}$

Significance of the bounds on variance

Many natural experiments have a large number of possible outcomes, but the bounds on variance show that as 𝑛 increases the variance decreases toward 0. This corresponds to increasing uncertainty, since the components become more nearly equal. As a result, probability vectors become less informative for large 𝑛, which motivates the common practice of binning outcomes to reduce the effective number of categories. While binning discards information from the original fine-grained outcomes, it reveals knowledge of the coarser structure that is otherwise obscured.

This notion of uncertainty is closely related to entropy as used in information theory and to entropy in statistical mechanics.

References

^ Jacobs, Konrad (1992), Discrete Stochastics, Basler Lehrbücher [Basel Textbooks], vol. 3, Birkhäuser Verlag, Basel, p. 45, doi:10.1007/978-3-0348-8645-1, ISBN 3-7643-2591-7, MR 1139766.

[1] Jacobs, Konrad (1992), Discrete Stochastics, Basler Lehrbücher [Basel Textbooks], vol. 3, Birkhäuser Verlag, Basel, p. 45, doi:10.1007/978-3-0348-8645-1, ISBN 3-7643-2591-7, MR 1139766.

[1]