When determining the standard error (SE) in the 1-sample z-test section, we assumed that the population standard deviation (denoted by \(\sigma\)) is known:
$$\begin{align*} \text{SE} &= \frac{\color{red}{\sigma}}{\sqrt{n}} \end{align*}$$However, in practice, the population standard deviation is usually unknown. Hence, we often use the sample standard deviation (denoted by \(s\)) instead:
$$\begin{align*} \hat{\text{SE}} &= \frac{\color{red}{s}}{\sqrt{n}} \end{align*}$$
One observation is that when we substitute the sample standard deviation for the population standard deviation,
we now write \(\hat{\text{SE}}\) instead of \(\text{SE}\).
The \(\hat{}\) represents that this standard error is an estimate.
We don’t have the true \(\text{SE}\) anymore, as we are using the sample,
which is a subset of the population.
As the sample size increases, the sample standard deviation (\(s\)) converges to the
population standard deviation (\(\sigma\)), so \(\hat{\text{SE}}\) becomes a closer and
closer approximation of \(\text{SE}\).
Conceptually, this is because as the sample becomes larger,
it is more representative of the population.
Also, the above is only true if we are sampling at random!
Firstly, let’s remind ourselves of the test statistic (\(\text{TS}\)) calculation for a 1-sample z-test:
$$\begin{align*} \text{Test Statistic (TS)} &= \frac{\text{OV} - \text{EV}}{\color{red}{\text{SE}}} \end{align*}$$As a reminder, \(\text{OV}\) stands for observed value and \(\text{EV}\) stands for expected value. As mentioned in the above section, the \(\text{SE}\) (indicated in red) requires that the population standard deviation is known. When this is not known, we can instead use the estimate standard error (using the sample’s standard deviation):
$$\begin{align*} \text{Test Statistic (TS)} &= \frac{\text{OV} - \text{EV}}{\color{red}{\hat{\text{SE}}}} \end{align*}$$
In both cases above, the test statistic formulas look the same. The only difference is that the latter one has an estimate for \(\text{SE}\)
(i.e. \(\hat{\text{SE}}\)). Because \(\hat{\text{SE}}\) is itself uncertain — it changes from sample to sample — the second test statistic
has extra variability.
How do we account for this extra variability? We change the curve we use to find the p-value: where the 1-sample z-test uses the normal curve,
we instead use the t-curve. The t-curve (or t-distribution) is similar in
appearance to the normal curve, except it contains an extra parameter called degrees of freedom, which adjusts the ‘fatness’ of the curve’s tails.
This is evident when playing with the app to the right. When degrees of freedom increases, the tail fatness decreases. You can also see that
as the degrees of freedom value increases, the curve approaches the normal distribution.
The slider below changes the degrees of freedom of the t-curve in the graph below.
Previously, it was mentioned that the parameter called “degrees of freedom” adjusts the ‘fatness’ of the curve’s tails. This can be easily
verified in the app above when setting the slider to 1 and then 10. At 1 degree of freedom, the black t-curve has its tails well and truly above
the red dashed normal curve. However, when looking at 10 degrees of freedom, the black t-curve’s tails have shrunk to be closer to that of the
red dashed normal curve. Notice also that the fatter the tails, the lower the peak in the middle: the total area under any density curve has to
be 1, so the extra area out in the tails has to be paid for somewhere.
There is a very logical reason for this! Recall from the ‘1-sample z-test’ exercise that when it comes to finding the p-value, we plot our test
statistic and find the area under the curve covering the shaded region. If we have a p-value below our significance level (which is typically
\(\alpha = 0.05\)), we reject the null hypothesis.
As mentioned previously, if we do not have the population standard deviation, we need to account for the extra variability introduced by using
the sample standard deviation. Degrees of freedom allow us to do just that! The value of degrees of freedom is directly linked to sample size,
meaning larger degrees of freedom are associated with larger sample sizes. When degrees of freedom are equal to 1, this indicates that our sample
is tiny (in the 1-sample case, just 2 observations), meaning there is lots of variability/uncertainty, and that is why the tails of the t-curve are
so fat. The result of this is that the shaded tail area — the p-value — is much larger for the same test statistic, meaning that a more extreme test
statistic is needed to reject the null hypothesis.
On the other hand, when degrees of freedom are equal to a larger value (such as 25), the t-curve tails appear much more closely aligned with the
normal curve’s tails. This is because we are now taking a much larger sample, and there is less variability to account for.
The demo to the right allows you to compare the p-values from a normal and t-distribution. You can see that for low values of degrees of freedom,
the t-curve’s p-value is much larger than the normal curve’s. However, as you increase degrees of freedom, the two p-values become more similar.
Note that each of the two plots is scaled to the height of its own curve, so compare the p-values printed underneath them rather than the
heights of the curves themselves.
Comparison of the p-values for a two-sided alternate hypothesis test using a normal and t-curve.
The purpose of this exercise was to develop a conceptual understanding of what the t-distribution is and why we need it. We did this through discussing the 1-sample z-test, and specifically, how things change when we don't know the population standard deviation. I have left some things vague (such as how we know which value to set degrees of freedom to), but we will go into greater detail in further exercises.