Probability, begins with discrete events

Path: janda.org/c10 > Syllabus > Topics and Readings > Statistical Inference > Conceptualizing Probability Theory

Statistical Inference:
Conceptualizing Probability Theory

Descriptive v. Inferential Statistics

DESCRIPTIVE statistics describe and summarize OBSERVED data:

Univariate measures: central tendency and dispersion for observed data

Inferential statistics permit statements about UNOBSERVED data:

Predicting from samples to populations

Predict central tendencies: population means

Predict dispersion: population variance

The place of probability in the study of statistics

It is fundamental to statistical inference

Predicting population parameters from sample data or predicting the likelihood of of random processes generating observed relationships (e.g., monkeys typing all the works in English literature) is an uncertain business, involving estimates of error in the predictions.

Calculation of error estimates involve probability considerations.

Probability is of prime importance of to mathematical statistics

Concerned largely with inferential statistics

Therefore concerned with estimation techniques

Mathematical statistics probes deeply into probability theory

Relevance of probability to our course

Need to grasp some fundamental concepts

Will only cover essentials in this course

Discrete v. continuous probability distributions

Outcomes of discrete variables

The possible outcomes are countable.

The "probability" of an outcome is the proportion of times that it is expected to occur over the long run -- over an infinite number of trials.

These expectations are of two types:

a priori -- come prior to observation

formed from examination of the nature of the event

outcomes are presumably determined by chance events and are thus equally likely

sometimes described as "theoretical probability distributions"

Example: coin flips, dice toss, or card draws

empirical -- expectations based on past observations: relative frequency

sometimes called "empirical probability distributions"

Predicting the sex of a new-born child: only two outcomes but expect more males than females

A great deal of probability theory deals with discrete outcomes, but we will slight this area in favor of probability distributions of continuous variables.

Outcomes of continuous variables (to be taken up later this week)

In theory, continuous variables have "unique" (i.e., infinitesimally small) outcomes and therefore cannot be aggregated in categories in an empirical frequency distribution.

While one cannot therefore predict the likelihood of any particular outcome of a continuous variable, one can predict the likelihood of a range of outcomes in a distribution of outcomes:

Between any two values

Beyond any value -- as in the "tail" of a distribution

Computing probabilities for discrete events

Simple experiments resulting in single, well-defined outcomes:

The list of outcomes must be exhaustive.

All outcomes must be mutually exclusive events.

Probability of occurrence is simply a proportion:

Number of elementary events that are A

p(A) = Number of possible events / number of outcomes

Combined experiments involving multiple elementary events:

Example:

p(royal flush)= ^{Number
of 5-card hands that are royal
flushes}

^{Number
of possible 5-card
hands}

One usually knows the numerator in this case but has trouble determining the denominator.

Two "counting" rules for determining the number of complex events in the denominator (Schmidt, p. 219):

Permutations: unique orderings of a set of objects:

Possible permutations of N objects = N! <--- "factorial"

where: N! = (N)(N-1)(N-2) ...(3)(2)(1)

EXAMPLE:

All possible orderings of A, B, and C = 3! = 3 x 2 x 1 = 6

Combinations: Occurrences of r objects from N objects, without regard to the order of their occurrence:

EXAMPLE:

Possible combinations of 3 letters from the set A, B, C

3 x 2 x 1 3 x 2 x 1 N = 3 r = 3 --------------- = --------------- = 1 3 x 2 x 1 (3-3)! 3 x 2 x 1 (1)!

EXAMPLE:

Combinations of 3 letters from A, B, C, D

4 x 3 x 2 x 1 24 N = 4 r = 3 --------------------- = -------- = 4 3 x 2 x 1 (4-3)! 6

Number of possible 5 card hands in a deck of 52 cards, applying the formula for combinations:

Computing probabilities for COMPOUND discrete events

Compound event involve the simultaneous occurrence of different elementary events.

Compound events differ by being

independent or dependent events

alternative or joint events

independent events: occurrence of one has no influence on the occurrence of the other.

Independent alternative events: the "OR" connection

Occurrence of one event OR another in one trial-- the "OR" connection

Rule of Addtion applies:

the probability of an ALTERNATIVE event is equal to the SUM of the probabilities of the simple event

EXAMPLE: standard deck of American playing cards--

52 cards p(any particular card) = 1/52 = .02

13 cards in 4 suits p(card of given suit) = 13/52 = .25

p(any card, no suit) = 4/52 = .08
p(Heart OR Diamond) = .25 + .25 = .50

Independent JOINT events in two trials: Occurrence of one event AND another-- the "AND" connection

Rule of Multiplications applies:

the probability of a JOINT event (if the events are independent) is equal to the product of their separate probabilities.

p(Heart AND diamond) = .25 x .25 = .06

DEPENDENT events: the occurrence of one has an influence on the occurrence of the other -- i.e., they are not MUTUALLY EXCLUSIVE

Alternative events:
general case: p(A or B) = p(A) + p(B) - p(A and B)

When A and B are MUTUALLY EXCLUSIVE (something cannot be both A and B), the last term drops out and the computation is equal to the rule of addition for independent alternative events.

When A and B are not mutually exclusive, the last term operates:

p(Ace or Diamond) = p(Ace) + p(Diamond) - p(Ace and Diamond)

= .08 + .25 - .08 x .25
= .33 - .02
= .31

Joint events:

general case: p(A and B) = p(A) p(B|A) = p(B) p(A|B)

In the case of independence, the conditional probabilities are the same as the unconditional probabilities.

See the boxes below for examples of probability theory in baseball and poker.

"Just no explaining those no-hitters,"
Chicago Tribune

A Chicago Tribune article once looked at pitching performance in terms of probability theory. The article sought to determine the probability of no-hitting a team of all 250 hitters. In that case, the probability of an out for each player is .75.

Use the multiplication rule to compute the probablity of three outs in a single inning:
.75 x .75 x .75 for one inning = .422

Use the rule again to compute the probablity of three outs each inning:for all nine innings:

(.42)*(.42)*(.42)*(.42)*(.42)*(.42)*(.42)*(.42)*(.42) = .0004

Since 1900, a no-hitter has been pitched 7.5 times for every 10,000 games -- about the same as the probability of pitching a no-hitter against a team of .235 hitters. Some "lucky" no-hitters, explainable by chance?

Dick Fowler (his only win in 1945 was a no-hitter)

Virgil Trucks (pitched 2 no-hitters in 1952 in only 5 wins)

Bobo Holloman (pitched a no-hitter in his 1st major-league start 1953 and won only 3 games the rest of the season)

Reproduced from Ivars Peterson's MathLand on the WWW
Trouble with Wild-Card Poker
September 9, 1996

Poker originated in the Louisiana territory around the year 1800. Ever since, this addictive card game has occupied the time and teased the minds of generations of gamblers. It has also attracted the attention of mathematicians and statisticians.

The standard game and its many variants involve a curious mixture of luck and skill. Given a deck of 52 cards, there are 2,598,960 ways to select a subset of five cards. So, the probability of getting any one hand is 1 in 2,598,960.

One of the first things a novice player learns is the relative value of different sets of five cards. At the top of the heap is the straight flush, which consists of any sequence of five cards of the same suit. There are 40 ways of getting such a hand, so the probability of being dealt a straight flush is 40/2,598,960, or .000015.

The next most valuable type of hand is four of a kind, and so on. The table below lists the number of possible ways that different types of hands can arise and their probability of occurrence.

Rankings of Poker Hands and Their Frequencies of Occurrence:

Hand

No. of Ways

Probability

Straight flush

40

.000015

Four of a kind

624

.000240

Full house

3,744

.001441

Flush

5,108

.001965

Straight

10,200

.003925

Three of a kind

54,912

.021129

Two pair

123,552

.047539

One pair

1,098,240

.422569

The rules of poker specify that a straight flush beats four of a kind, which tops a full house, which bests a flush, and so on through a straight, three of kind, two pair, and one pair. Whatever your hand, you can still bet and bluff your way into winning the pot, but the ranking (and value) of the hands truly reflects the probabilities of obtaining various combinations by random selections of five cards from a deck.

Many people, however, play a livelier version of poker. They salt the deck with wild cards -- deuces, jokers, one-eyed jacks, or whatever. The presence of wild cards brings a new element into the game, allowing a player to use such a card to stand for any card of the player's choosing. It increases the chances of drawing more valuable hands.

It also potentially alters the ranking of different hands. One can even draw a five of a kind, which typically goes to the top of the rankings. Just how much wild cards alter the game is recounted in an article in the current issue of Chance, written by mathematician John Emert and statistician Dale Umbach of Ball State University in Muncie, Ind. They analyze wild-card poker and conclude, "When wild cards are allowed, there is no ranking of the hands that can be formed for which more valuable hands occur less frequently."

return to place

Descriptive v. Inferential Statistics
DESCRIPTIVE statistics describe and summarize OBSERVED data: Univariate measures: central tendency and dispersion for observed data Inferential statistics permit statements about UNOBSERVED data: Predicting from samples to populations Predict central tendencies: population means Predict dispersion: population variance The place of probability in the study of statistics It is fundamental to statistical inference Predicting population parameters from sample data or predicting the likelihood of of random processes generating observed relationships (e.g., monkeys typing all the works in English literature) is an uncertain business, involving estimates of error in the predictions. Calculation of error estimates involve probability considerations. Probability is of prime importance of to mathematical statistics Concerned largely with inferential statistics Therefore concerned with estimation techniques Mathematical statistics probes deeply into probability theory Relevance of probability to our course Need to grasp some fundamental concepts Will only cover essentials in this course Discrete v. continuous probability distributions Outcomes of discrete variables The possible outcomes are countable. The "probability" of an outcome is the proportion of times that it is expected to occur over the long run -- over an infinite number of trials. These expectations are of two types: a priori -- come prior to observation formed from examination of the nature of the event outcomes are presumably determined by chance events and are thus equally likely sometimes described as "theoretical probability distributions" Example: coin flips, dice toss, or card draws empirical -- expectations based on past observations: relative frequency sometimes called "empirical probability distributions" Predicting the sex of a new-born child: only two outcomes but expect more males than females A great deal of probability theory deals with discrete outcomes, but we will slight this area in favor of probability distributions of continuous variables. Outcomes of continuous variables (to be taken up later this week) In theory, continuous variables have "unique" (i.e., infinitesimally small) outcomes and therefore cannot be aggregated in categories in an empirical frequency distribution. While one cannot therefore predict the likelihood of any particular outcome of a continuous variable, one can predict the likelihood of a range of outcomes in a distribution of outcomes: Between any two values Beyond any value -- as in the "tail" of a distribution Computing probabilities for discrete events Simple experiments resulting in single, well-defined outcomes: The list of outcomes must be exhaustive. All outcomes must be mutually exclusive events. Probability of occurrence is simply a proportion: Number of elementary events that are A p(A) = Number of possible events / number of outcomes Combined experiments involving multiple elementary events: Example: p(royal flush)= ^{Number of 5-card hands that are royal flushes} ^{Number of possible 5-card hands} One usually knows the numerator in this case but has trouble determining the denominator. Two "counting" rules for determining the number of complex events in the denominator (Schmidt, p. 219): Permutations: unique orderings of a set of objects: Possible permutations of N objects = N! <--- "factorial" where: N! = (N)(N-1)(N-2) ...(3)(2)(1) EXAMPLE: All possible orderings of A, B, and C = 3! = 3 x 2 x 1 = 6 Combinations: Occurrences of r objects from N objects, without regard to the order of their occurrence: EXAMPLE: Possible combinations of 3 letters from the set A, B, C 3 x 2 x 1 3 x 2 x 1 N = 3 r = 3 --------------- = --------------- = 1 3 x 2 x 1 (3-3)! 3 x 2 x 1 (1)! EXAMPLE: Combinations of 3 letters from A, B, C, D 4 x 3 x 2 x 1 24 N = 4 r = 3 --------------------- = -------- = 4 3 x 2 x 1 (4-3)! 6 Number of possible 5 card hands in a deck of 52 cards, applying the formula for combinations: Computing probabilities for COMPOUND discrete events Compound event involve the simultaneous occurrence of different elementary events. Compound events differ by being independent or dependent events alternative or joint events independent events: occurrence of one has no influence on the occurrence of the other. Independent alternative events: the "OR" connection Occurrence of one event OR another in one trial-- the "OR" connection Rule of Addtion applies: the probability of an ALTERNATIVE event is equal to the SUM of the probabilities of the simple event EXAMPLE: standard deck of American playing cards-- 52 cards p(any particular card) = 1/52 = .02 13 cards in 4 suits p(card of given suit) = 13/52 = .25 p(any card, no suit) = 4/52 = .08 p(Heart OR Diamond) = .25 + .25 = .50 Independent JOINT events in two trials: Occurrence of one event AND another-- the "AND" connection Rule of Multiplications applies: the probability of a JOINT event (if the events are independent) is equal to the product of their separate probabilities. p(Heart AND diamond) = .25 x .25 = .06 DEPENDENT events: the occurrence of one has an influence on the occurrence of the other -- i.e., they are not MUTUALLY EXCLUSIVE Alternative events: general case: p(A or B) = p(A) + p(B) - p(A and B) When A and B are MUTUALLY EXCLUSIVE (something cannot be both A and B), the last term drops out and the computation is equal to the rule of addition for independent alternative events. When A and B are not mutually exclusive, the last term operates: p(Ace or Diamond) = p(Ace) + p(Diamond) - p(Ace and Diamond) = .08 + .25 - .08 x .25 = .33 - .02 = .31 Joint events: general case: p(A and B) = p(A) p(B\|A) = p(B) p(A\|B) In the case of independence, the conditional probabilities are the same as the unconditional probabilities. See the boxes below for examples of probability theory in baseball and poker.