Module 3: Inference & Asymptotics

What p-values Actually Mean

Author

Austin Tucker

Published

July 23, 2026

1 From Standard Errors to Inference

In Modules 1 and 2 you learned that an OLS estimate \(\hat{\beta}_1\) comes packaged with a standard error: the standard deviation of its sampling distribution. The standard error answers the question, “if I re-ran this study on a fresh sample, how much would my estimate wobble?”

This module is about the next step: using that wobble to make statements about the world. Is the effect we estimated distinguishable from zero, or is it the kind of number that pure sampling noise produces all the time?

1.1 The running example: bednets and malaria

Throughout this module we work with a simulated population of 10,000 children under five in a malaria-endemic region. The outcome is the number of malaria episodes a child experiences per year, and the key regressor is whether the child’s household owns an insecticide-treated bednet (ITN). The true data-generating process (DGP) is:

\[ \text{episodes}_i = 2.4 - 0.5 \cdot \text{bednet}_i + 0.02 \cdot \text{age}_i + u_i \]

Owning a bednet reduces expected episodes by 0.5 per year — a substantial protective effect. Age in months enters with a small positive coefficient (older children in this simulated region spend more time outdoors at dusk).

An honest simplification

Malaria episodes are a count (0, 1, 2, …), and a real analysis might use a Poisson or negative binomial model. Here we treat episodes as a continuous variable with normal errors, so a handful of simulated values even dip below zero. This continuous approximation keeps the focus on inference mechanics — everything you learn here about t-statistics, p-values, and confidence intervals carries over to count models with only cosmetic changes. We also make bednet ownership randomly assigned (independent of age and \(u\)), so OLS identifies the causal effect; Module 5 covers what happens when it does not.

1.2 The t-statistic: how many SEs from zero?

Every inference procedure in this module is built from one quantity. For a coefficient \(\hat{\beta}_1\) with standard error \(\widehat{\text{SE}}(\hat{\beta}_1)\), the t-statistic for the null hypothesis \(\beta_1 = 0\) is:

\[ t = \frac{\hat{\beta}_1 - 0}{\widehat{\text{SE}}(\hat{\beta}_1)} \]

Read it literally: the t-statistic is how many standard errors the estimate sits away from zero. An estimate of \(-0.5\) with a standard error of \(0.1\) is five SEs below zero (\(t = -5\)). An estimate of \(-0.5\) with a standard error of \(0.4\) is barely more than one SE below zero (\(t = -1.25\)) — the kind of distance noise covers routinely.

That single intuition — distance measured in units of noise — powers everything that follows.

2 Hypothesis Testing Mechanics

2.1 Null and alternative

A hypothesis test formalizes the question “could this be noise?” We specify:

Null hypothesis \(H_0: \beta_1 = 0\) — bednets have no effect on episodes.
Alternative hypothesis \(H_A: \beta_1 \neq 0\) — bednets have some effect (two-sided).

We then ask: if the null were true, how surprising would our t-statistic be? Under \(H_0\) (and the OLS assumptions), the t-statistic follows a \(t\) distribution with \(n - k - 1\) degrees of freedom, which is essentially standard normal once \(n\) is a few hundred.

2.2 Critical values and the p-value

Two equivalent ways to run the test at significance level \(\alpha\):

Critical value: reject \(H_0\) if \(|t|\) exceeds the critical value \(c_\alpha\). For \(\alpha = 0.05\) and large \(n\), \(c_{0.05} \approx 1.96\).
p-value: compute \(p = 2 \cdot P(T \geq |t|)\), the probability of a t-statistic at least this extreme if the null were true. Reject if \(p < \alpha\).

Table 1: Common two-sided critical values

\(\alpha\) (two-sided)	Critical value (large \(n\))	Reject when
0.10	1.645	\(\lvert t \rvert > 1.645\)
0.05	1.960	\(\lvert t \rvert > 1.960\)
0.01	2.576	\(\lvert t \rvert > 2.576\)

What a p-value is (and is not)

The p-value is the probability of data at least as extreme as ours, computed under the assumption that the null is true. It is:

not the probability that the null is true,
not the probability that the result is a fluke,
not a measure of the effect’s size or importance.

It answers exactly one question: “how often would noise alone produce a t-statistic this large?” Section Section 3 makes this concrete by simulation.

2.3 Exercise 1: Compute the t-statistic and p-value by hand

Draw a random sample of 1,000 children and fit the correctly specified regression. Then reconstruct the t-statistic and two-sided p-value for the bednet coefficient by hand, and verify they match summary(lm) exactly.

Solution

set.seed(123456789)
data_sample <- pop[sample(nrow(pop), 1000, replace = FALSE), ]

mod <- lm(episodes ~ bednet + age_months, data = data_sample)

# Extract the estimate and its standard error for bednet
b  <- coef(summary(mod))["bednet", "Estimate"]
se <- coef(summary(mod))["bednet", "Std. Error"]

# t-statistic: how many SEs is the estimate from zero?
t_stat <- b / se

# Two-sided p-value from the t distribution
df    <- mod$df.residual
p_val <- 2 * pt(abs(t_stat), df = df, lower.tail = FALSE)

cat("By hand:\n")
cat("  estimate =", round(b, 4), "\n")
cat("  SE       =", round(se, 4), "\n")
cat("  t-stat   =", round(t_stat, 3), "\n")
cat("  p-value  =", format.pval(p_val, digits = 3), "\n\n")

cat("From summary(lm):\n")
round(coef(summary(mod))["bednet", ], 6)

Interpretation question

The t-statistic should be around \(-5\). In words, what does “\(t \approx -5\)” say about the bednet estimate, and why does it produce such a tiny p-value?

Discussion: The estimate sits about five standard errors below zero. Under the null, t-statistics beyond \(\pm 1.96\) occur only 5% of the time and beyond \(\pm 5\) essentially never (about 1 in 3.5 million). So either we witnessed an absurd fluke, or the null is wrong. The p-value quantifies exactly that: the probability that sampling noise alone, with no true effect, would throw an estimate this many SEs from zero. Note also that the by-hand numbers match summary(lm) to the last decimal — the regression table is not doing anything mysterious.

3 What a p-value Actually Is: Simulation Under the Null

Here is the intuition anchor of this entire module.

A p-value is a statement about a hypothetical world where the null is true. The cleanest way to understand it is to build that world and watch what p-values do there. Suppose bednets truly had zero effect. We run the same study 500 times — 500 field teams, 500 samples, 500 regressions — and record the bednet p-value from each one.

What should we see? If the test is working correctly, then under the null:

\[ p \sim \text{Uniform}(0, 1) \]

Every p-value is equally likely. A p-value of 0.03 is exactly as probable as a p-value of 0.53 or 0.93. And that is precisely why the test “works”: the event \(p < 0.05\) then happens exactly 5% of the time — by construction. The 5% false-positive rate is not an empirical discovery about nature; it is a property we engineered into the procedure.

3.1 Exercise 2: 500 studies of a bednet that does nothing

Simulate 500 samples of 500 children each from a DGP where the bednet coefficient is zero (everything else unchanged). Store the bednet p-value from each regression, check the rejection rate at \(\alpha = 0.05\), and plot the histogram of p-values.

Solution

set.seed(2026)
n_reps <- 500
n_samp <- 500
p_vals <- numeric(n_reps)

for (i in 1:n_reps) {
  bednet     <- rbinom(n_samp, size = 1, prob = 0.5)
  age_months <- runif(n_samp, min = 6, max = 59)
  u          <- rnorm(n_samp, mean = 0, sd = 1.5)
  episodes   <- 2.4 + 0 * bednet + 0.02 * age_months + u  # true effect is ZERO

  mod <- lm(episodes ~ bednet + age_months)
  p_vals[i] <- coef(summary(mod))["bednet", "Pr(>|t|)"]
}

cat("Share of p-values below 0.05:", mean(p_vals < 0.05), "\n")
cat("(Theoretical value: 0.05 -- by construction)\n")

ggplot(data.frame(p = p_vals), aes(x = p)) +
  geom_histogram(breaks = seq(0, 1, by = 0.05),
                 fill = slate_blue, color = "white", alpha = 0.8) +
  geom_hline(yintercept = n_reps * 0.05, color = terracotta,
             linewidth = 1, linetype = "dashed") +
  labs(
    title = "500 p-values When the True Effect Is Zero",
    subtitle = "Terracotta dashed = expected count per bin under uniformity (25)",
    x = "p-value for the bednet coefficient",
    y = "Count"
  ) +
  theme_house()

Interpretation question

The histogram is flat — small p-values are no rarer than large ones when the null is true. About 5% of your 500 null studies “found” a significant bednet effect. What does this imply for a literature where journals preferentially publish \(p < 0.05\) results?

Discussion: Under the null, \(p < 0.05\) happens 5% of the time by design — the flat histogram means the leftmost bin (0 to 0.05) captures exactly one-twentieth of the studies. Now imagine 500 research teams each testing a truly ineffective intervention: roughly 25 of them obtain “statistically significant” results by pure chance, and those 25 are the ones most likely to be written up and published. This is the engine of publication bias and the replication crisis. A single \(p < 0.05\) is weak evidence precisely because the procedure guarantees a steady background rate of false positives.

4 Confidence Intervals

4.1 Construction

A hypothesis test gives a binary verdict; a confidence interval reports the full range of parameter values compatible with the data. The 95% CI for \(\beta_1\) is:

\[ \hat{\beta}_1 \pm 1.96 \times \widehat{\text{SE}}(\hat{\beta}_1) \]

(Using the large-sample normal critical value; R’s confint() uses the exact \(t\) critical value, which differs only in the third decimal once \(n\) is in the hundreds.)

The CI and the test are two views of the same object: the 95% CI is exactly the set of null values \(\beta_1^0\) that a two-sided test at \(\alpha = 0.05\) would fail to reject. If zero is outside the CI, \(p < 0.05\); if zero is inside, \(p > 0.05\).

What “95% confident” means

The 95% refers to the procedure, not any single interval. If we repeated the study many times and built a CI each time, about 95% of those intervals would contain the true \(\beta_1\). Any particular interval either contains the truth or it does not — the probability statement is about the long-run behavior of the recipe, which Exercise 4 verifies by brute force.

4.2 Exercise 3: Build the 95% CI by hand

Using the same sample as Exercise 1, construct the 95% confidence interval for the bednet coefficient from the estimate and SE, then compare with confint().

Solution

set.seed(123456789)
data_sample <- pop[sample(nrow(pop), 1000, replace = FALSE), ]

mod <- lm(episodes ~ bednet + age_months, data = data_sample)

b  <- coef(summary(mod))["bednet", "Estimate"]
se <- coef(summary(mod))["bednet", "Std. Error"]

# 95% CI: estimate plus/minus 1.96 standard errors
ci_lower <- b - 1.96 * se
ci_upper <- b + 1.96 * se

cat("By hand:      [", round(ci_lower, 4), ",", round(ci_upper, 4), "]\n")
cat("True beta_1:   -0.5\n\n")

cat("From confint() (exact t critical value):\n")
confint(mod, "bednet", level = 0.95)

Interpretation question

Does your interval contain the true value \(-0.5\)? Does it contain zero? How would you report this interval to a health minister deciding on a bednet program?

Discussion: The interval should contain \(-0.5\) (this is one of the lucky 95%) and exclude zero (consistent with the tiny p-value from Exercise 1). For a policy audience, the CI is far more useful than the p-value: “the data are compatible with bednets preventing between roughly 0.3 and 0.7 episodes per child per year” tells the minister both the direction and the plausible magnitude of the effect — enough to weigh against program costs. “p < 0.001” tells them only that the effect is unlikely to be exactly zero.

4.3 Exercise 4: The coverage simulation

Now verify the “95%” claim directly. Draw 500 samples of 500 children from the true DGP (\(\beta_1 = -0.5\)), build the 95% CI in each, and count how often the interval contains the truth. Then draw the classic coverage plot for the first 100 intervals.

Solution

set.seed(3141)
n_reps <- 500
n_samp <- 500
ci_lo  <- numeric(n_reps)
ci_hi  <- numeric(n_reps)

for (i in 1:n_reps) {
  bednet     <- rbinom(n_samp, size = 1, prob = 0.5)
  age_months <- runif(n_samp, min = 6, max = 59)
  u          <- rnorm(n_samp, mean = 0, sd = 1.5)
  episodes   <- 2.4 - 0.5 * bednet + 0.02 * age_months + u

  mod <- lm(episodes ~ bednet + age_months)
  b   <- coef(summary(mod))["bednet", "Estimate"]
  se  <- coef(summary(mod))["bednet", "Std. Error"]

  ci_lo[i] <- b - 1.96 * se
  ci_hi[i] <- b + 1.96 * se
}

covers <- (ci_lo <= -0.5) & (-0.5 <= ci_hi)
cat("Coverage rate:", mean(covers), "\n")
cat("(Target: 0.95)\n")

plot_df <- data.frame(
  rep    = 1:100,
  lo     = ci_lo[1:100],
  hi     = ci_hi[1:100],
  covers = ifelse(covers[1:100], "Covers -0.5", "Misses -0.5")
)

ggplot(plot_df, aes(x = rep, ymin = lo, ymax = hi, color = covers)) +
  geom_linerange(linewidth = 0.7) +
  geom_hline(yintercept = -0.5, color = charcoal,
             linewidth = 0.8, linetype = "dashed") +
  scale_color_manual(values = c("Covers -0.5" = sage, "Misses -0.5" = terracotta)) +
  labs(
    title = "The First 100 Confidence Intervals",
    subtitle = "Charcoal dashed = true beta (-0.5) | Terracotta intervals miss it",
    x = "Sample number",
    y = "95% CI for the bednet coefficient",
    color = NULL
  ) +
  theme_house()

Interpretation question

Your coverage rate should land near 0.95 (typically 0.93–0.97 with 500 replications). Look at the terracotta intervals in the plot: is there anything visibly “wrong” with the samples that produced them?

Discussion: Nothing is wrong with those samples — they are ordinary random draws that happened to land in the tails. That is the uncomfortable core of frequentist inference: roughly 1 study in 20 produces a CI that misses the truth entirely, and nothing in that study’s own data reveals which kind it is. The researcher holding a “missing” interval sees a perfectly normal-looking regression table. This is why replication, meta-analysis, and humility about any single estimate are structural necessities, not just good manners.

5 Type I/II Errors and Power

Two things can go wrong in a test, and they trade off against each other:

Table 2: The four outcomes of a hypothesis test

	\(H_0\) true (no effect)	\(H_0\) false (real effect)
Reject \(H_0\)	Type I error (\(\alpha\))	Correct rejection (power \(= 1 - \beta\))
Fail to reject \(H_0\)	Correct	Type II error (\(\beta\))

Type I error rate (\(\alpha\)): rejecting a true null. We choose this (usually 5%) — Exercise 2 showed it holds by construction.
Type II error rate (\(\beta\)): failing to detect a real effect. This depends on the effect size, the noise, and above all the sample size.
Power \(= 1 - \beta\): the probability of detecting an effect that is really there.

Power grows with \(|\beta_1|\) (bigger effects are easier to see) and with \(n\) (the SE shrinks like \(1/\sqrt{n}\), so the same effect sits more SEs from zero). An underpowered study is a lose-lose proposition: it will probably miss a real effect, and when it does reject, the estimated magnitude is often inflated (a “Type M” error) because only exaggerated draws clear the significance bar.

5.1 Exercise 5: Power at n = 100 vs. n = 1,000

The true bednet effect is \(-0.5\). Estimate the power to detect it at \(\alpha = 0.05\) with samples of 100 children versus 1,000 children, using 300 simulated studies at each size.

Solution

run_power <- function(n_samp, n_reps = 300) {
  reject <- logical(n_reps)
  for (i in 1:n_reps) {
    bednet     <- rbinom(n_samp, size = 1, prob = 0.5)
    age_months <- runif(n_samp, min = 6, max = 59)
    u          <- rnorm(n_samp, mean = 0, sd = 1.5)
    episodes   <- 2.4 - 0.5 * bednet + 0.02 * age_months + u  # the effect IS real

    mod <- lm(episodes ~ bednet + age_months)
    p   <- coef(summary(mod))["bednet", "Pr(>|t|)"]
    reject[i] <- (p < 0.05)
  }
  mean(reject)
}

set.seed(1789)
power_100  <- run_power(100)
power_1000 <- run_power(1000)

cat("Power at n = 100:  ", power_100, "\n")
cat("Power at n = 1000: ", power_1000, "\n")

Interpretation question

You should find power of roughly 0.35–0.45 at \(n = 100\) but essentially 1.00 at \(n = 1{,}000\). A colleague runs the \(n = 100\) study, gets \(p = 0.21\), and concludes “bednets don’t work here.” What went wrong?

Discussion: At \(n = 100\) the study misses this real, policy-relevant effect more often than it finds it — a coin flip would nearly match its detection rate. The colleague’s non-significant result is exactly what an underpowered study of an effective intervention is expected to produce; it is evidence of a small sample, not of an ineffective bednet. The theoretical SE at \(n=100\) is about \(1.5/\sqrt{100 \times 0.25} = 0.30\), so the true effect sits only \(0.5/0.30 \approx 1.7\) SEs from zero — below the 1.96 bar even before sampling noise. Power calculations belong before data collection, and “we failed to reject” must never be flattened into “there is no effect.” The confidence interval (Section Section 7) makes this distinction visible.

6 Asymptotics: Why Any of This Works

Everything above leaned on two large-sample facts that deserve their own spotlight. Together they are the reason OLS inference is so robust in practice.

6.1 Consistency: the Law of Large Numbers at work

An estimator is consistent if it converges in probability to the truth as the sample grows:

\[ \hat{\beta}_1 \xrightarrow{p} \beta_1 \quad \text{as } n \to \infty \]

Under the OLS assumptions (crucially, exogeneity: \(E[u \mid x] = 0\)), \(\hat{\beta}_1\) is consistent, and its standard error shrinks at rate \(1/\sqrt{n}\):

\[ \text{SE}(\hat{\beta}_1) \approx \frac{\sigma_u}{\sqrt{n \cdot \text{Var}(x_1)}} \]

Quadrupling the sample halves the SE. The sampling distribution doesn’t just stay centered on the truth (unbiasedness) — it collapses onto it.

6.2 Exercise 6: Watch the sampling distribution tighten

Simulate the sampling distribution of the bednet coefficient at \(n = 50\), \(n = 500\), and \(n = 5{,}000\) (300 replications each) and plot the three distributions on one axis.

Solution

set.seed(2718)
n_reps  <- 300
sizes   <- c(50, 500, 5000)
results <- data.frame()

for (n_samp in sizes) {
  bhat <- numeric(n_reps)
  for (i in 1:n_reps) {
    bednet     <- rbinom(n_samp, size = 1, prob = 0.5)
    age_months <- runif(n_samp, min = 6, max = 59)
    u          <- rnorm(n_samp, mean = 0, sd = 1.5)
    episodes   <- 2.4 - 0.5 * bednet + 0.02 * age_months + u

    mod     <- lm(episodes ~ bednet + age_months)
    bhat[i] <- coef(mod)["bednet"]
  }
  results <- rbind(results, data.frame(n = n_samp, estimate = bhat))
}

results |>
  group_by(n) |>
  summarise(mean_estimate = mean(estimate),
            sd_estimate   = sd(estimate))

results$n_label <- factor(paste0("n = ", results$n),
                          levels = paste0("n = ", sizes))

ggplot(results, aes(x = estimate, fill = n_label)) +
  geom_density(alpha = 0.55, color = NA) +
  geom_vline(xintercept = -0.5, color = charcoal,
             linewidth = 0.8, linetype = "dashed") +
  scale_fill_manual(values = c(terracotta, dusty_gold, slate_blue)) +
  labs(
    title = "Consistency: The Sampling Distribution Collapses onto the Truth",
    subtitle = "Charcoal dashed = true beta (-0.5) | 300 replications per sample size",
    x = "Estimated bednet coefficient",
    y = "Density",
    fill = NULL
  ) +
  theme_house()

Interpretation question

Compare the simulated sd_estimate values to the theoretical prediction \(\text{SE} \approx 1.5 / \sqrt{n \times 0.25} = 3/\sqrt{n}\). Do they match? Contrast this behavior with the biased estimator from Module 5’s Exercise 5.

Discussion: The theory predicts SEs of about \(0.42\), \(0.13\), and \(0.042\) at \(n = 50\), \(500\), and \(5{,}000\) — and the simulated standard deviations should land right on those values. Each tenfold increase in \(n\) shrinks the SE by \(\sqrt{10} \approx 3.2\). Crucially, all three distributions are centered on \(-0.5\): more data buys precision because the estimator was aimed at the truth to begin with. Contrast the omitted-variable case (Module 5), where the sampling distribution tightens around the wrong value — there, more data just makes you more confidently wrong. Consistency is a property of the design plus assumptions, not of sample size alone.

6.3 Asymptotic normality: the Central Limit Theorem at work

A widely believed myth: “OLS requires normally distributed errors.” It does not — not for large-sample inference. What normal errors buy you is an exact \(t\) distribution for the t-statistic at any \(n\). Without them, the Central Limit Theorem delivers the same thing asymptotically: because \(\hat{\beta}_1\) is (approximately) a weighted average of the errors,

\[ \sqrt{n}\,(\hat{\beta}_1 - \beta_1) \xrightarrow{d} N\!\left(0,\; \frac{\sigma_u^2}{\text{Var}(x_1)}\right) \]

for any error distribution with finite variance — skewed, lumpy, heavy-shouldered, it doesn’t matter. Averaging washes the shape away. So the 1.96-SE confidence intervals and p-values from the previous sections remain valid at moderate \(n\) even when the errors are wildly non-normal.

Let’s torture-test this claim. First, look at how ugly an exponential error distribution is:

Hard floor on the left, long tail on the right — nothing like a bell curve. Now watch what happens to the sampling distribution of \(\hat{\beta}_1\).

6.4 Exercise 7: The CLT rescues inference from ugly errors

Draw 500 samples of 300 children each, with errors from the recentered exponential distribution above. Plot the sampling distribution of the bednet coefficient against the normal curve that asymptotic theory predicts.

Solution

set.seed(1618)
n_reps <- 500
n_samp <- 300
bhat   <- numeric(n_reps)

for (i in 1:n_reps) {
  bednet     <- rbinom(n_samp, size = 1, prob = 0.5)
  age_months <- runif(n_samp, min = 6, max = 59)
  u          <- rexp(n_samp, rate = 1/1.5) - 1.5  # skewed errors, mean 0, sd 1.5

  episodes   <- 2.4 - 0.5 * bednet + 0.02 * age_months + u
  mod        <- lm(episodes ~ bednet + age_months)
  bhat[i]    <- coef(mod)["bednet"]
}

# Normal curve predicted by asymptotic theory: SE = sigma / sqrt(n * Var(x))
theory_se <- 1.5 / sqrt(n_samp * 0.25)
cat("Theoretical SE:", round(theory_se, 4), "\n")
cat("Simulated SD:  ", round(sd(bhat), 4), "\n")

ggplot(data.frame(estimate = bhat), aes(x = estimate)) +
  geom_histogram(aes(y = after_stat(density)),
                 bins = 30, fill = dusty_gold, color = "white", alpha = 0.8) +
  stat_function(fun = dnorm, args = list(mean = -0.5, sd = theory_se),
                color = charcoal, linewidth = 1) +
  geom_vline(xintercept = -0.5, color = terracotta,
             linewidth = 1, linetype = "dashed") +
  labs(
    title = "Skewed Errors, Normal Sampling Distribution",
    subtitle = "Charcoal curve = normal predicted by the CLT | Terracotta dashed = true beta (-0.5)",
    x = "Estimated bednet coefficient",
    y = "Density"
  ) +
  theme_house()

Interpretation question

The errors were violently skewed, yet the histogram of estimates hugs the normal curve — and the simulated SD matches the theoretical SE. Why does the skewness vanish, and what does this license us to do?

Discussion: Each \(\hat{\beta}_1\) is essentially an average of 300 error draws (weighted by the regressor), and averages of many independent draws are approximately normal regardless of the shape of what is being averaged — that is the CLT. The skewness of any individual error is diluted 300-fold. The practical license is enormous: we may use normal critical values (1.96 and friends) for tests and CIs on health outcomes that are themselves skewed — costs, hospital days, parasite counts — as long as the sample is reasonably large and the errors have finite variance. Checking histograms of residuals for normality before trusting a CI at \(n = 1{,}000\) is mostly wasted worry; checking exogeneity is not.

The asymptotics payoff

Two theorems quietly underwrite every regression table you will ever read:

LLN \(\Rightarrow\) consistency: with exogeneity, \(\hat{\beta} \to \beta\) as \(n\) grows; the SE shrinks at rate \(1/\sqrt{n}\).
CLT \(\Rightarrow\) asymptotic normality: the sampling distribution of \(\hat{\beta}\) is approximately normal whatever the error distribution, so 1.96-SE intervals and normal-based p-values are valid in large samples.

Normality of errors is not an OLS requirement for large-sample inference. Exogeneity is. Big samples fix noise; they never fix bias.

7 Reading Regression Tables in Published Papers

You now have every ingredient needed to read a regression table the way a referee does. Here is a table in the style of a health-policy journal, reporting our bednet study at two sample sizes:

Table 3: Illustrative regression table. Standard errors in parentheses. \(^{*} p<0.05\), \(^{**} p<0.01\), \(^{***} p<0.001\).

Outcome: malaria episodes/year	(1) Full sample	(2) Pilot subsample
Bednet ownership	\(-0.48^{***}\)	\(-0.21\)
	\((0.09)\)	\((0.13)\)
Age (months)	\(0.021^{***}\)	\(0.018^{**}\)
	\((0.003)\)	\((0.006)\)
Constant	\(2.37^{***}\)	\(2.45^{***}\)
	\((0.11)\)	\((0.21)\)
Observations	1,000	250

7.1 The conventions

Stars encode p-value thresholds; the legend beneath the table defines them (conventions vary — some fields start stars at \(p < 0.10\); always check).
Parentheses usually hold standard errors, but some papers (and many older economics papers) put t-statistics there instead. The table notes tell you which. A quick sanity check: divide the coefficient by the parenthetical value — if the result looks like a t-statistic (e.g., \(-0.48/0.09 \approx -5.3\)), the parentheses hold SEs.
Everything else is reconstructible: \(t = \hat{\beta}/\text{SE}\), the 95% CI is \(\hat{\beta} \pm 1.96 \times \text{SE}\), and stars follow from Table 1.

7.2 The classic misreadings

“Not significant” \(\neq\) “no effect.” Column (2)’s bednet coefficient has no stars (\(t = -0.21/0.13 \approx -1.6\), \(p \approx 0.11\)). But look at what the CI says:

Stars are not effect sizes. Three stars on the age coefficient (\(0.021\)) do not make it important — 10 months of age is associated with 0.2 extra episodes. Significance is about precision; importance is about magnitude in real-world units.
Comparing “significant” to “not significant” is not a test. Column (1) is significant and column (2) is not, but the two estimates (\(-0.48\) vs. \(-0.21\)) are not significantly different from each other. “The effect appeared in the full sample but not the pilot” is a statement about sample sizes, not about heterogeneity.
A tiny p-value does not validate the design. Every p-value in this module was computed assuming exogeneity holds. If bednet ownership were confounded — wealthier households owning more nets and living in lower-transmission areas — the p-value would be a precise answer to the wrong question. Inference quantifies noise; it is silent about bias.

A reader’s checklist for any coefficient

What is the point estimate, in real-world units? Is that magnitude meaningful?
What is the 95% CI (\(\hat{\beta} \pm 1.96 \times \text{SE}\))? What is the range of effects the data are compatible with?
Is the identifying assumption (exogeneity) plausible — or is this a precisely estimated confounded association?

The stars come last, if at all.

8 Summary

Table 4: Module 3 at a glance

Concept	Definition	The simulation that proves it
t-statistic	\(\hat{\beta}/\text{SE}\): distance from zero in units of noise	Exercise 1 (matches `summary(lm)`)
p-value	\(P(\text{data this extreme} \mid H_0)\); Uniform(0,1) under \(H_0\)	Exercise 2 (flat histogram; 5% rejections by construction)
95% CI	\(\hat{\beta} \pm 1.96 \times \text{SE}\); the set of nulls not rejected	Exercises 3–4 (coverage \(\approx\) 95%)
Power	\(P(\text{reject} \mid \text{effect is real})\); rises with \(n\) and \(\lvert\beta\rvert\)	Exercise 5 (\(\approx\) 0.4 at \(n=100\); \(\approx\) 1 at \(n=1{,}000\))
Consistency (LLN)	\(\hat{\beta} \xrightarrow{p} \beta\); SE shrinks like \(1/\sqrt{n}\)	Exercise 6 (collapsing densities)
Asymptotic normality (CLT)	\(\hat{\beta}\) approximately normal for any finite-variance errors	Exercise 7 (skewed errors, normal estimates)

What comes next?

Every result in this module assumed the estimator was aimed at the truth — that \(E[u \mid x] = 0\). Module 5 (When OLS Fails) studies what happens when that assumption breaks: omitted variable bias and measurement error, where \(\hat{\beta}\) converges confidently to the wrong number and no p-value can warn you. (Module 4, on multiple regression and functional form, is coming later and slots in between.) Keep this module’s lesson in your pocket as you go: large samples shrink standard errors, but they are powerless against bias.

References

Wooldridge, Jeffrey M. 2020. Introductory Econometrics: A Modern Approach. 7th ed. Cengage Learning. (Chapters 4–5: inference and asymptotics.)
Angrist, Joshua D., and Jorn-Steffen Pischke. 2009. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press.
Wasserstein, Ronald L., and Nicole A. Lazar. 2016. “The ASA Statement on p-Values: Context, Process, and Purpose.” The American Statistician 70(2): 129–133.
Greenland, Sander, et al. 2016. “Statistical Tests, P Values, Confidence Intervals, and Power: A Guide to Misinterpretations.” European Journal of Epidemiology 31: 337–350.
Gelman, Andrew, and John Carlin. 2014. “Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors.” Perspectives on Psychological Science 9(6): 641–651.