A03: Probability, random variables, and simulation

A. Prerequisites and observable goals#

Level: Degree core.

B. The central question: what can be known before an uncertain outcome occurs?#

A probability model consists of a sample space of possible outcomes, events that are subsets of that space, and a probability assignment. Probability is not a synonym for a personal feeling of confidence: calculations are conditional on the specified model. A fair six-sided die has six equally probable elementary outcomes; a production server’s next latency need not have equally likely values, independence across requests, or an unchanging distribution.

For an event $A$, $0\le P(A)\le1$, and the whole sample space has probability one. For disjoint events, probabilities add. General addition requires subtracting overlap:

A count divided by a total gives a probability only when the counted elementary outcomes are equally probable. Ordered outcomes and unordered selections are different sample spaces. For example, selecting two distinct items from five gives $5\cdot4=20$ ordered pairs but $\binom52=10$ unordered pairs. Each unordered pair corresponds to two orders. Counting foundations are developed further in C05; probability uses those counts only under a justified sampling rule.

C. Conditioning changes the population under consideration#

For $P(B)>0$,

The vertical bar means “within outcomes where $B$ occurred.” It does not reverse the direction of a causal relationship. Events are independent when $P(A\cap B)=P(A)P(B)$. For positive $P(B)$, this is equivalent to $P(A\mid B)=P(A)$. Disjoint events with positive probabilities are not independent: learning that one occurred rules out the other.

Bayes’ theorem rearranges the same joint probability:

If $A$ and $A^c$ partition the possibilities, then $P(B)=P(B\mid A)P(A)+P(B\mid A^c)P(A^c)$. This denominator accounts for every route to observing $B$, including false alarms. A system’s sensitivity is not the probability that a flagged case is genuine.

D. A rare-event calculation with actual counts#

Suppose 1% of requests are genuinely defective, a detector flags 90% of defective requests, and it falsely flags 5% of nondefective requests. In a model population of 10,000 requests, expect 100 defective requests, 90 true flags, and $0.05(9900)=495$ false flags. Among 585 total flags, only 90 are genuine:

This does not make the detector useless. It shows why its operating threshold, base rate, and cost of false positives matter. A different population with a higher defect rate changes the posterior even when sensitivity and false-positive rate remain the same. The numerical result is conditional on those assumptions; it is not an empirical claim about a real system.

E. Random variables turn outcomes into quantities#

A random variable $X$ is a function from outcomes to numbers. For a discrete variable, its mass function is $p(x)=P(X=x)$ and

Expectation is a probability-weighted average, not necessarily a possible observed value. Variance has squared units; standard deviation has the original units. For a continuous density $f$, probabilities are integrals over intervals, and $E[X]=\int x f(x)\dd x$ when the expectation is well-defined. A density can exceed one over a short interval; its integral over the whole domain must equal one. A continuous variable’s probability at a single point is zero even when that point lies in its support.

Linearity of expectation does not require independence: $E[X+Y]=E[X]+E[Y]$ when the expectations exist. Variance does require attention to dependence:

Independence implies zero covariance when the relevant moments exist, but zero covariance does not generally imply independence. Shared server congestion can make two request latencies positively dependent, increasing the variability of their sum.

An indicator $I_A$ equals 1 if $A$ occurs and 0 otherwise. Then $E[I_A]=P(A)$. A binomial count $X=I_1+\cdots+I_n$ for independent, identical Bernoulli trials has mean $np$ and variance $np(1-p)$. The derivation exposes exactly where identical probabilities and independence enter; it is more useful than remembering a distribution name.

F. Select distributions by their generating assumptions#

A normal model permits negative values, so applying it to latency requires a justified approximation over the relevant range. An exponential model’s memorylessness is a substantive assumption, not a property of every wait. Fit and validate a model against the mechanism and observations; do not choose it only because its formula is convenient. [PROB]

For a synthetic latency $X$ equal to 10 ms with probability 0.95 and 110 ms with probability 0.05, $E[X]=15$ ms. But 15 ms is never observed. Its variance is $0.95(10^2)+0.05(110^2)-15^2=475\ \mathrm{ms}^2$. The average hides the 110 ms tail. A capacity or responsiveness requirement needs appropriate tail behavior as well as a mean.

G. Independent practice#

F foundation C standard course work T transfer S optional stretch L runnable lab or project. Attempt the complete set before opening its solutions.

H. Complete solutions: A03#

Compare the first step where your work differs. A corrected answer is not yet an independent success.

Open every solution

"""A03.05: seeded binomial experiment with analytic comparison."""
import random
import statistics

def binomial_experiment(n: int = 10, p: float = 0.3,
                        trials: int = 10000, seed: int = 41) -> dict:
    if type(n) is not int or n < 0 or not 0 <= p <= 1:
        raise ValueError("invalid binomial parameters")
    if type(trials) is not int or trials < 2:
        raise ValueError("at least two trials required")
    rng = random.Random(seed)
    data = [sum(rng.random() < p for _ in range(n)) for _ in range(trials)]
    return {"sample_mean": statistics.mean(data),
            "sample_variance": statistics.variance(data),
            "theory_mean": n * p, "theory_variance": n * p * (1 - p),
            "trials": trials, "seed": seed}

if __name__ == "__main__":
    print(binomial_experiment())

I. Retention and readiness#

Retrieve fractions, inclusion–exclusion, and the difference between a density and an area. Advance when you can name the conditioning population, reject independence when evidence does not support it, and explain a mean that is never itself observed.