Path: > Syllabus > Topics and Readings > Statistical Inference > Conceptualizing Probability Theory

Statistical Inference:
Conceptualizing Probability Theory
Descriptive v. Inferential Statistics
DESCRIPTIVE statistics describe and summarize OBSERVED data:
Univariate measures: central tendency and dispersion for observed data
Inferential statistics permit statements about UNOBSERVED data:
Predicting from samples to populations
Predict central tendencies: population means
Predict dispersion: population variance

The place of probability in the study of statistics

It is fundamental to statistical inference
Predicting population parameters from sample data or predicting the likelihood of of random processes generating observed relationships (e.g., monkeys typing all the works in English literature) is an uncertain business, involving estimates of error in the predictions.
Calculation of error estimates involve probability considerations. 
Probability is of prime importance of to mathematical statistics
Concerned largely with inferential statistics
Therefore concerned with estimation techniques
Mathematical statistics probes deeply into probability theory 
Relevance of probability to our course
Need to grasp some fundamental concepts
Will only cover essentials in this course

Discrete v. continuous probability distributions
Outcomes of discrete variables
The possible outcomes are countable.
The "probability" of an outcome is the proportion of times that it is expected to occur over the long run -- over an infinite number of trials.
These expectations are of two types:
a priori -- come prior to observation
formed from examination of the nature of the event
outcomes are presumably determined by chance events and are thus equally likely
sometimes described as "theoretical probability distributions"
Example: coin flips, dice toss, or card draws
empirical -- expectations based on past observations: relative frequency
sometimes called "empirical probability distributions"
Predicting the sex of a new-born child: only two outcomes but expect more males than females
A great deal of probability theory deals with discrete outcomes, but we will slight this area in favor of probability distributions of continuous variables.
Outcomes of continuous variables (to be taken up later this week)
In theory, continuous variables have "unique" (i.e., infinitesimally small) outcomes and therefore cannot be aggregated in categories in an empirical frequency distribution.
While one cannot therefore predict the likelihood of any particular outcome of a continuous variable, one can predict the likelihood of a range of outcomes in a distribution of outcomes:
Between any two values
Beyond any value -- as in the "tail" of a distribution

Computing probabilities for discrete events
Simple experiments resulting in single, well-defined outcomes:
  • The list of outcomes must be exhaustive.
  • All outcomes must be mutually exclusive events. 
Probability of occurrence is simply a proportion: 
  • Number of elementary events that are A
    • p(A) = Number of possible events / number of outcomes
Combined experiments involving multiple elementary events: 
p(royal flush)= Number of 5-card hands that are royal flushes
Number of possible 5-card hands
One usually knows the numerator in this case but has trouble determining the denominator.

Two "counting" rules for determining the number of complex events in the denominator (Schmidt, p. 219):

Permutations: unique orderings of a set of objects:
Possible permutations of N objects = N! <--- "factorial" 
where: N! = (N)(N-1)(N-2) ...(3)(2)(1)
All possible orderings of A, B, and C = 3! = 3 x 2 x 1 = 6 
Combinations: Occurrences of r objects from N objects, without regard to the order of their occurrence:
Possible combinations of 3 letters from the set A, B, C 
					                3 x 2 x 1           3 x 2 x 1
				N = 3    r = 3    ---------------   = ---------------  = 1
					             3 x 2 x 1 (3-3)!     3 x 2 x 1 (1)!
Combinations of 3 letters from A, B, C, D 
					              4 x 3 x 2 x 1            24
				N  = 4   r = 3  ---------------------  =  --------    =  4
					             3 x 2 x 1 (4-3)!           6
Number of possible 5 card hands in a deck of 52 cards, applying the formula for combinations:

Computing probabilities for COMPOUND discrete events

Compound event involve the simultaneous occurrence of different elementary events.
Compound events differ by being
independent or dependent events
alternative or joint events
independent events: occurrence of one has no influence on the occurrence of the other.
Independent alternative events: the "OR" connection 
  • Occurrence of one event OR another in one trial-- the "OR" connection
  • Rule of Addtion applies:
    • the probability of an ALTERNATIVE event is equal to the SUM of the probabilities of the simple event 
    • EXAMPLE: standard deck of American playing cards-- 
      • 52 cards p(any particular card) = 1/52 = .02
      • 13 cards in 4 suits p(card of given suit) = 13/52 = .25
      • p(any card, no suit) = 4/52 = .08
        p(Heart OR Diamond) = .25 + .25 = .50
  • Independent JOINT events in two trials: Occurrence of one event AND another-- the "AND" connection
  • Rule of Multiplications applies:  
  • the probability of a JOINT event (if the events are independent) is equal to the product of their separate probabilities. 
    • p(Heart AND diamond) = .25 x .25 = .06 

DEPENDENT events: the occurrence of one has an influence on the occurrence of the other -- i.e., they are not MUTUALLY EXCLUSIVE

Alternative events: 
general case: p(A or B) = p(A) + p(B) - p(A and B) 
When A and B are MUTUALLY EXCLUSIVE (something cannot be both A and B), the last term drops out and the computation is equal to the rule of addition for independent alternative events. 
When A and B are not mutually exclusive, the last term operates:
p(Ace or Diamond) = p(Ace) + p(Diamond) - p(Ace and Diamond)
= .08 + .25 - .08 x .25
= .33 - .02
= .31
Joint events:
general case: p(A and B) = p(A) p(B|A) = p(B) p(A|B) 
In the case of independence, the conditional probabilities are the same as the unconditional probabilities. 
See the boxes below for examples of probability theory in baseball and poker.

"Just no explaining those no-hitters,"

Chicago Tribune

A Chicago Tribune article once looked at pitching performance in terms of probability theory. The article sought to determine the probability of no-hitting a team of all 250 hitters. In that case, the probability of an out for each player is .75.

Use the multiplication rule to compute the probablity of three outs in a single inning:
.75 x .75 x .75 for one inning = .422

Use the rule again to compute the probablity of three outs each inning:for all nine innings:

(.42)*(.42)*(.42)*(.42)*(.42)*(.42)*(.42)*(.42)*(.42) = .0004

Since 1900, a no-hitter has been pitched 7.5 times for every 10,000 games -- about the same as the probability of pitching a no-hitter against a team of .235 hitters. Some "lucky" no-hitters, explainable by chance?

Dick Fowler (his only win in 1945 was a no-hitter)
Virgil Trucks (pitched 2 no-hitters in 1952 in only 5 wins)
Bobo Holloman (pitched a no-hitter in his 1st major-league start 1953 and won only 3 games the rest of the season)
Reproduced from Ivars Peterson's MathLand on the WWW
Trouble with Wild-Card Poker
September 9, 1996

Poker originated in the Louisiana territory around the year 1800. Ever since, this addictive card game has occupied the time and teased the minds of generations of gamblers. It has also attracted the attention of mathematicians and statisticians.

The standard game and its many variants involve a curious mixture of luck and skill. Given a deck of 52 cards, there are 2,598,960 ways to select a subset of five cards. So, the probability of getting any one hand is 1 in 2,598,960.

One of the first things a novice player learns is the relative value of different sets of five cards. At the top of the heap is the straight flush, which consists of any sequence of five cards of the same suit. There are 40 ways of getting such a hand, so the probability of being dealt a straight flush is 40/2,598,960, or .000015.

The next most valuable type of hand is four of a kind, and so on. The table below lists the number of possible ways that different types of hands can arise and their probability of occurrence.

Rankings of Poker Hands and Their Frequencies of Occurrence:


No. of Ways


Straight flush



Four of a kind



Full house









Three of a kind



Two pair



One pair



The rules of poker specify that a straight flush beats four of a kind, which tops a full house, which bests a flush, and so on through a straight, three of kind, two pair, and one pair. Whatever your hand, you can still bet and bluff your way into winning the pot, but the ranking (and value) of the hands truly reflects the probabilities of obtaining various combinations by random selections of five cards from a deck.

Many people, however, play a livelier version of poker. They salt the deck with wild cards -- deuces, jokers, one-eyed jacks, or whatever. The presence of wild cards brings a new element into the game, allowing a player to use such a card to stand for any card of the player's choosing. It increases the chances of drawing more valuable hands.

It also potentially alters the ranking of different hands. One can even draw a five of a kind, which typically goes to the top of the rankings. Just how much wild cards alter the game is recounted in an article in the current issue of Chance, written by mathematician John Emert and statistician Dale Umbach of Ball State University in Muncie, Ind. They analyze wild-card poker and conclude, "When wild cards are allowed, there is no ranking of the hands that can be formed for which more valuable hands occur less frequently."

return to place