###
**Multiple t tests and Type I error**

###
·
As a simple example, you know that there is a **0.50 probability** of
obtaining “heads” in a coin flip. If I flip the coin **four times**, what is the
probability of obtaining a heads *one or more times* across all four flips?

###
·
For two coin flips, the probability of __not obtaining__ at
least one heads (i.e., getting tails both times) is 0.50 × 0.50 = **0.25**.

###
·
The
**probability of one or more heads in two coin flips is 1 – 0.25 = 0.75**.
Three-fourths of "two coin flips" will have at least one heads.

###
·
So, if I flip the coin **four times**, the probability of one or more
heads is **1 – (0.50 × 0.50 × 0.50 × 0.50) = 1 – (0.50)**^{4} = 1 – 0.625 =
0.9375; you will get one or more heads in about 94% of sets of "four coin flips".

###
·
Similarly, for a statistical test (such as a t test) with α= 0.05,
__if the null hypothesis
is true__ then the probability of __not__ obtaining a significant result is 1 –
0.05 = 0.95.

###
·
**Multiply 0.95 by the number of tests** to calculate the probability
of __not__ obtaining one or more significant results across all tests. For
**two tests**, the probability of not obtaining one or more significant results is
0.95 × 0.95 = **0.9025**.

###
·
**Subtract that result from 1.00** to calculate the probability of
making **at least one type I error** with multiple tests: 1 – 0.9025 = **0.0975**.

###
·
Example (p. 162): You are comparing **4 groups** (A, B, C, D). You
compare these six pairs (α= 0.05 for each): A vs B, B vs C, C vs D, A vs C, A vs
D, and B vs D.

###
·
Using the convenient formula (see p. 162), the probability of __
not__ obtaining a significant result is 1 – (1 – 0.05)^{6} = **0.265**,
which means your chances of __incorrectly rejecting__ the null hypothesis (a
type I error) is about **1 in 4** instead of 1 in 20!!

###
·
**ANOVA compares all means simultaneously** and maintains the type I
error probability at the designated level.