2-Sample t-Test


Input Sample Data

What data source will you be using?


The second step in choosing our samples involves selecting a categorical variable that we will split the samples from:

With the categorical variable set, we now choose the 2 samples we would like to analyse:

Finally, we select the dependent variable. This is the variable that each obervation in the samples are being measured against.

Warning: Sample 1 and Sample 2 should be different.

Enter data into the text box below for each sample.

  • All values must be numeric, and numbers should contain no spaces or any characters other than a '.' for a decimal place.
  • Each value must be on it's own line, or comma seperated.
  • Don't forget to press 'Upload' once you are finished!

Sample 1:

Sample 2:


Warning: You must have at least two unique values in your manually specified data.

In order to proceed, you must select some data to act as your sample.


The 'NULL' Hypothesis - Setting up the Box

Similar to the 1-sample tests, we will be using the box model to represent the null hypothesis. The key difference here, however, is that we don't have input any values. The null hypothesis is set by default! Take some time to read the points below to ensure you understand why the boxes are arranged as they are.

The first thing you may wonder is why we have two box models for a 2-sample t-test. Unlike the 1-sample t-test, which only had ONE sample, the 2-sample t-test has TWO samples, and so we need to have two box models to represent each sample.

More concretely, these 2 samples are different, and so they are likely to come from slightly different distributions. Hence, we need to have 2 different box models (note that the standard deviation \(s\) for each is different). The samples are not coming from the same distribution.

For a 2-sample t-test, under the null hypothesis, we claim that the mean of sample 1 (\(\mu_1\)) is equal to the mean of sample 2 (\(\mu_2\)). More formally, we write this as \(H_0: \mu_1 = \mu_2\).

Hence, the mean of sample 1 (\(\mu_1\)) is equal to the mean of sample 2 (\(\mu_2\)).
Likewise, the mean of sample 2 (\(\mu_2\)) is equal to the mean of sample 1 (\(\mu_1\)).

In case you are confused about where all the values for each box model have come from, these come from the sample 1 and sample 2 data that you specified above. In particular:

  • \(s_1\) represents the sample standard deviation for sample 1 (similar for sample 2). Recall that in a t-test, we estimate the population standard devation as the sample standard deviation.
  • \(n_1 \) is the number of observation in sample 1 (simlar for sample 2).
  • \(\text{OV}_1 \) is the observed sample mean for sample 1 (similar for sample 2).
Sample 1
Sample 2



The Alternate Hypothesis

Specify what type of alternate hypothesis you will be using below:


Null Hypothesis

\( H_0: \) \( \mu_1 = \mu_2 \)

Alternate Hypothesis




Assumptions

For the hypothesis test to be valid, we need to check the following assumptions:

The first assumption is that our 2 samples are independent and randomly chosen.

How do we check? We check by investigating the experimental setup.

For example, consider we were doing an experiment where we were interested in the effect of a treatment on resting heart rate, and had a control and treatment group. Independence would be violated if people in the sample were related (family members would share similar genetics, and similar resting heart rates). Randomness could be violated if we only choose people with certain characteristics for one of the groups, such as healthy people for the treatment group. It could appear the treatment is reducing these people's heart rate, but it is actually that they are healthy so already have a comparably low heart rate.

The second assumption is that each sample's means are normally distributed.

How do we check?

Firstly, it is important to realise that here, we want the distribution of sample means to be normally distributed. Hence, we do not necessarily need the underlying data to be normally distributed, as we can rely on the Central Limit Theorem (CLT) to apply.

That being said, knowing whether the CLT applies or not is not straightforward. A common convention used online is that if there are more than 30 points in the sample, then the CLT will 'kick in'. But this is not necessarily true! If the underlying distribution is very skewed and/or asymmetric, then you will likely need many more than 30 points. Hence, checking whether the underlying data is nearly normally distributed will give us some insight. This is because if data is nearly normally distributed, you need fewer points for the CLT to 'kick in'.

There are a few different ways that you can check whether the underlying data is normally distributed. For each of the following, you must look at both samples separately:

  • Boxplots: look to see if the plot is symmetric with few outliers.
  • Histograms: look to see if the frequency (or density) is normally distributed.
  • QQ-plots: see if the points follow the diagonal QQ-line (see below).
  • Number of points: You can also make reference to the sample size when making your decision. For example, if you see some slight deviations in normality, but have a large sample size, you have more confidence that the CLT will apply.

The third assumption is that sample 1 and sample 2's population spreads are equal.

How do we check?

There are 2 main ways we can check this visually:

  • Histograms: look to see if the histograms for each sample have similar spreads.
  • Boxplots: look to see if the boxplots have similar spread (done in the section below).

Unlike the other tests, for a 2-sample t-test, if this assumption is not satisfied, we can make an easy change to how the test statistic is calculated to relax this assumption. If the assumption is not satisfied, we can move from a regular 2-sample t-test to a Welch 2-sample t-test. In the next section, you will determine whether we do a Welch test or not!




Checking Assumption 3 - Is the Spread Equal?

As mentioned in assumption 3, an assumption of a standard 2-sample t-test with equal variance is that the population spreads of sample 1 and sample 2 are equal. However, if this assumption is not satisfied, it is not bad news! We can instead use a different type of 2-sample t-test which relaxes this assumption. In particular, the two types of test we can use include:

  • 2-Sample T-Test With Equal Variance: When the 2 samples have the same (approximately the same) spread.
  • Welch 2-Sample T-Test: When the 2 samples have different spreads.

Your goal is to decide whether we will use a 2-sample t-test with equal variance, or a Welch 2-sample t-test. To make this determination, use a combination of the outputs below to make your decision.

Indicators that the spreads are similar:

  • Is the IQR (the size of the box) similar?
  • Are the whiskers of the box a similar size?

Indicators that the spreads are similar:

  • Does the distribution of vlaues look similar?
  • Does the spread (standard deviation) look similar?

Indicators that the spreads are similar:

  • Do both samples have a similar sample standard deviation?
Equal Spread?

Toggle the switch below to determine whether the spread is the same or not.




Test Statistic

Step 1) Calculate Expected Value (SE) and Standard Error (SE)

Step 2) Test Statistic Calculation




p-value



Conclusion (p-value)

Step 1) What is your significance level?

\( \alpha = \)

Error: The value for α must be between 0 and 1.

Step 2) Final Conclusion




Conclusion (Confidence Interval)

A confidence interval in a two-sample t-test shows the range of plausible values for the difference between the population means at the chosen confidence level. If 0 falls outside this range, we reject the null hypothesis.

We can also use the confidence interval to guide hypothesis testing: if 0 (representing no difference between the two population means) does not lie within the confidence interval, then we reject the null hypothesis. If 0 does lie within the interval, we do not reject the null.

Step 1) What is your confidence level?

\( \alpha = \)

Error: The value for confidence level must be between 0 and 1.

Step 2) Final Conclusion