Module 1: OLS Foundations

Fitting Lines, Reading Coefficients

Author

Austin Tucker

Published

July 23, 2026

1 Regression as a Conditional Expectation

Before any formulas, let’s fix what a regression line is. Suppose you are a clinician and a 60-year-old patient walks in. Before you inflate the cuff, what is your best guess of their systolic blood pressure (SBP)? A sensible answer: the average SBP among 60-year-olds. If a 30-year-old walks in next, your best guess changes — it is the average among 30-year-olds.

That mapping — from a value of \(x\) to the average of \(y\) among people with that value of \(x\) — is the conditional expectation function (CEF):

\[ E[y \mid x] = \text{the mean of } y \text{ among observations with a given value of } x \]

Regression is, at its heart, an attempt to describe this function with a simple curve. When we write the population model

\[ \text{sbp}_i = \beta_0 + \beta_1 \, \text{age}_i + u_i \]

we are claiming that the conditional mean of SBP moves linearly with age, and that everything else — diet, genetics, stress, measurement wobble — is bundled into the error term \(u_i\), which averages to zero at every age.

The zero conditional mean assumption

The population model is only a model of the CEF if

\[ E[u \mid x] = 0 \]

In words: at every value of \(x\), the unobserved stuff averages out to zero. This is the single most important assumption in this course. When it holds, the regression line passes through the conditional means. When it fails (Module 5), the line describes something, but not what we intended.

1.1 Our running example: blood pressure in adults

Throughout this module we work with a simulated population of 10,000 adults, where we (unusually, and usefully) know the true data-generating process (DGP):

\[ \text{sbp}_i = 90 + 0.55 \cdot \text{age}_i + 1.2 \cdot \text{bmi}_i + u_i \]

age is in years, roughly Normal(45, 12) truncated to \([18, 85]\)
bmi is in kg/m\(^2\), centered near 26, and drifts upward with age
u is Normal with standard deviation 12 mmHg — the biological and measurement noise around the conditional mean

Because we built this world ourselves, we can always compare what OLS estimates to what is true. That is the entire pedagogical trick of this module.

1.2 Exercise 1: See the conditional means

Draw a random sample of 1,000 adults. Compute the mean SBP within 5-year age bands, then plot the raw data, the band means, and the OLS line together. If regression is doing its job, the line should track the band means closely.

Solution

set.seed(123456789)
data_sample <- pop[sample(nrow(pop), 1000, replace = FALSE), ]

# Assign each person to a 5-year age band (midpoint stored)
data_sample$age_band <- floor(data_sample$age / 5) * 5 + 2.5

# Mean SBP within each band
band_means <- data_sample %>%
  group_by(age_band) %>%
  summarise(mean_sbp = mean(sbp), n = n())

ggplot(data_sample, aes(x = age, y = sbp)) +
  geom_point(color = slate_blue, alpha = 0.25, size = 0.9) +
  geom_point(data = band_means, aes(x = age_band, y = mean_sbp),
             color = terracotta, size = 3) +
  geom_smooth(method = "lm", se = FALSE, color = dusty_gold,
              linewidth = 1) +
  labs(
    title = "SBP by Age: Conditional Means vs. the OLS Line",
    subtitle = "Terracotta = 5-year band means | Gold = OLS fit",
    x = "Age (years)", y = "Systolic blood pressure (mmHg)"
  ) +
  theme_house()

Interpretation question

The terracotta dots (band means) should fall almost exactly on the gold OLS line, except perhaps at the extreme ages. Why do the extremes wobble more?

Discussion: The band means at the youngest and oldest ages are computed from very few observations (check the n column of band_means), so they are noisy estimates of the true conditional mean. In the middle of the age distribution, where data are plentiful, the band means and the line agree beautifully. This is the CEF idea made visible: the regression line is a smoothed, parsimonious summary of “average SBP at each age.”

2 How OLS Works

2.1 The minimization problem

A line is defined by two numbers: an intercept and a slope. OLS chooses them by a simple criterion — make the sum of squared vertical misses as small as possible:

\[ (\hat{\beta}_0, \hat{\beta}_1) = \arg\min_{b_0,\, b_1} \; \sum_{i=1}^{n} \left( y_i - b_0 - b_1 x_i \right)^2 \]

Each term in the sum is a residual squared — how far person \(i\)’s actual SBP sits above or below the candidate line. Squaring does two jobs: it treats misses above and below symmetrically, and it punishes large misses more than proportionally. (It also makes the calculus clean, which is why Legendre and Gauss liked it two centuries ago.)

2.2 The formulas

Take the derivative with respect to \(b_0\) and \(b_1\), set both to zero, and solve. Two famous formulas fall out. For the slope:

\[ \hat{\beta}_1 = \frac{\widehat{\text{Cov}}(x, y)}{\widehat{\text{Var}}(x)} \]

and for the intercept:

\[ \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x} \]

Read the slope formula slowly

\[ \hat{\beta}_1 = \frac{\widehat{\text{Cov}}(x, y)}{\widehat{\text{Var}}(x)} \]

The numerator asks: when \(x\) is above its mean, does \(y\) tend to be above its mean too? That co-movement is the covariance.
The denominator rescales by how much \(x\) varies, converting co-movement into a rate: “mmHg per year of age.”
The intercept formula says the OLS line always passes through the point of means \((\bar{x}, \bar{y})\). The line pivots around the center of the data.

No matrix algebra required. If you can compute a covariance and a variance, you can compute a regression slope.

2.3 Exercise 2: Compute \(\hat{\beta}_1\) by hand, then check against `lm()`

Using the same sample of 1,000 adults, compute the slope and intercept of the regression of SBP on age using only cov(), var(), and mean(). Then verify that lm() produces the identical numbers.

Solution

set.seed(123456789)
data_sample <- pop[sample(nrow(pop), 1000, replace = FALSE), ]

# Slope: Cov(x, y) / Var(x)
beta1_hat <- cov(data_sample$age, data_sample$sbp) / var(data_sample$age)

# Intercept: ybar - beta1 * xbar
beta0_hat <- mean(data_sample$sbp) - beta1_hat * mean(data_sample$age)

# The same regression via lm()
mod_simple <- lm(sbp ~ age, data = data_sample)

cat("Manual slope (cov/var):    ", round(beta1_hat, 4), "\n")
cat("lm() slope:                ", round(coef(mod_simple)["age"], 4), "\n\n")
cat("Manual intercept:          ", round(beta0_hat, 4), "\n")
cat("lm() intercept:            ", round(coef(mod_simple)["(Intercept)"], 4), "\n")

Interpretation question

The manual and lm() results should match to every decimal place. But notice something else: the slope you estimated is larger than the true DGP coefficient on age (0.55). Any guess why?

Discussion: The match is exact because lm() is the cov/var formula (generalized to matrices) — there is no magic inside. The slope exceeds 0.55 because this simple regression omits BMI, and in our population BMI rises with age. Some of BMI’s effect on SBP is being smuggled into the age coefficient. We will resolve this precisely in Section 5, and Module 5 devotes itself to the general problem.

3 Fitted Values and Residuals

Once we have \(\hat{\beta}_0\) and \(\hat{\beta}_1\), each observation splits into two pieces:

\[ y_i = \underbrace{\hat{y}_i}_{\text{fitted value}} + \underbrace{\hat{u}_i}_{\text{residual}} \qquad \text{where} \qquad \hat{y}_i = \hat{\beta}_0 + \hat{\beta}_1 x_i, \qquad \hat{u}_i = y_i - \hat{y}_i \]

The fitted value is the model’s best guess for person \(i\); the residual is the part of their SBP the model cannot explain — how far they sit off the line.

3.1 Mechanical properties

Because OLS solves a minimization problem, its residuals are not just small — they are structured. The first-order conditions of the minimization force, in every sample, no assumptions needed:

\[ \sum_{i=1}^{n} \hat{u}_i = 0 \qquad \text{and} \qquad \sum_{i=1}^{n} x_i \hat{u}_i = 0 \]

Mechanics, not miracles

These two facts hold by construction — they are algebra, not evidence that the model is good.

Residuals sum to zero: the line’s misses above and below cancel exactly. (Corollary: the mean of the fitted values equals the mean of \(y\).)
Residuals are orthogonal to \(x\): the residuals have zero sample covariance with the regressor. OLS has squeezed every drop of linear information about \(y\) out of \(x\); whatever is left over is uncorrelated with \(x\).

A common trap: “my residuals average zero and are uncorrelated with \(x\), so my model is fine.” No — your residuals do that in every OLS fit, including terrible ones. The assumptions worth worrying about concern the population error \(u\), which you never observe.

3.2 Exercise 3: Verify the mechanical properties

Fit the simple regression of SBP on age, extract the residuals and fitted values, and confirm both properties numerically.

Solution

set.seed(123456789)
data_sample <- pop[sample(nrow(pop), 1000, replace = FALSE), ]

mod_simple <- lm(sbp ~ age, data = data_sample)

# Extract residuals and fitted values
resid_vals  <- residuals(mod_simple)
fitted_vals <- fitted(mod_simple)

cat("Property 1: residuals sum to zero\n")
cat("  sum(residuals) =", format(sum(resid_vals), scientific = TRUE), "\n\n")

cat("Property 2: residuals orthogonal to x\n")
cat("  cov(residuals, age) =", format(cov(resid_vals, data_sample$age),
                                      scientific = TRUE), "\n\n")

cat("Corollary: mean of fitted values equals mean of y\n")
cat("  mean(fitted) =", round(mean(fitted_vals), 4), "\n")
cat("  mean(sbp)    =", round(mean(data_sample$sbp), 4), "\n")

Interpretation question

The sums are not exactly zero — they are numbers like 2e-12. Is something wrong?

Discussion: Nothing is wrong. Computers store numbers in finite binary precision, so quantities that are exactly zero in algebra come out as values on the order of \(10^{-12}\) in floating-point arithmetic. Anything that small is “zero, up to rounding.” The deeper point stands: these zeros are guaranteed by the OLS first-order conditions in every sample, whether or not the model is correctly specified.

3.3 Exercise 4: Draw the residuals

Numbers are convincing; pictures are memorable. Take a small subsample of 60 adults so individual points are visible, fit the line, and draw each residual as a vertical segment from the point to the line.

Solution

set.seed(2468)
mini <- pop[sample(nrow(pop), 60, replace = FALSE), ]

mod_mini    <- lm(sbp ~ age, data = mini)
mini$fitted <- fitted(mod_mini)

ggplot(mini, aes(x = age, y = sbp)) +
  geom_segment(aes(x = age, xend = age, y = sbp, yend = fitted),
               color = dusty_gold, linewidth = 0.6, alpha = 0.8) +
  geom_line(aes(y = fitted), color = terracotta, linewidth = 1) +
  geom_point(color = slate_blue, size = 2) +
  labs(
    title = "Residuals: What OLS Minimizes (Squared)",
    subtitle = "Gold segments = residuals | Terracotta = fitted line",
    x = "Age (years)", y = "Systolic blood pressure (mmHg)"
  ) +
  theme_house()

Interpretation question

Imagine tilting the terracotta line slightly steeper or shallower. What would happen to the total squared length of the gold segments?

Discussion: It would increase. The OLS line is, by definition, the unique line for which the sum of squared segment lengths is smallest. Any rotation or shift makes some segments shorter but lengthens others by more (in squared terms). This is also why single outlying points can be so influential: a point far from the line contributes its residual squared, so OLS will tilt noticeably to appease it.

4 Interpreting Coefficients

Fitting the line is mechanical. Reading it is where judgment enters. Take our fitted simple regression from Exercise 2, roughly:

\[ \widehat{\text{sbp}} = 115.6 + 0.70 \cdot \text{age} \]

4.1 The slope: units first, always

The slope is a rate: its units are always (units of \(y\)) per (unit of \(x\)). Here: 0.70 mmHg of systolic blood pressure per year of age. Comparing two people who differ by one year of age, we predict the older one’s SBP to be 0.70 mmHg higher. Comparing people a decade apart: about 7 mmHg.

Note the careful language: “comparing two people who differ in age” — not “if a person ages one year, their SBP rises by 0.70.” The regression describes differences across the population, not changes within a person over time. Whether the descriptive slope has a causal reading is a question about the DGP and the zero conditional mean assumption, not about the arithmetic.

4.2 The intercept: often a fiction

The intercept, 115.6, is the predicted SBP at \(\text{age} = 0\). Our sample contains no newborns — the youngest adult is 18 — so this number describes a point far outside the data. It is a geometric anchor for the line, not a clinical statement about infant blood pressure (which, for the record, averages around 70 mmHg — the extrapolation is not even close).

Three habits for reading a coefficient

Say the units out loud. “0.65 mmHg per year.” Half of all interpretation errors die at this step.
Ask whether the intercept’s \(x = 0\) is inside the data. If not, treat the intercept as scaffolding. (Centering — regressing on \(\text{age} - 45\) instead of age — makes the intercept the predicted SBP at age 45, a number worth reading.)
Separate statistically detectable from clinically meaningful. With \(n\) large enough, a coefficient of 0.02 mmHg per year would earn three stars of significance while being clinically irrelevant. Conversely, a noisy study can fail to detect a 10 mmHg effect that matters enormously. Significance is about precision; meaning is about magnitude.

4.3 Clinically meaningful vs. statistically detectable

A useful benchmark: in large trials and meta-analyses, a sustained 5 mmHg reduction in SBP is associated with roughly a 10% reduction in major cardiovascular events. So when you read a coefficient in this module, ask: how many units of \(x\) does it take to move SBP by 5 mmHg?

Table 1: Magnitude, not stars, carries the clinical meaning

Coefficient (mmHg per unit)	Change in \(x\) needed for 5 mmHg	Verdict
age: \(\approx 0.55\) per year	\(\approx\) 9 years	Meaningful over a decade of aging
bmi: \(\approx 1.2\) per kg/m\(^2\)	\(\approx\) 4 BMI units	Meaningful — attainable with weight change
(hypothetical) 0.02 per unit	250 units	Statistically detectable, clinically inert

We defer how precise these estimates are — standard errors, confidence intervals, and the machinery of inference — entirely to Module 2. Today is about what the numbers mean; next time is about how much to trust them.

5 Multiple Regression: “Holding Constant”

5.1 Why one regressor was not enough

In Exercise 2 the simple regression handed us an age slope near 0.70, though the true DGP coefficient is 0.55. The culprit is BMI: in our population (as in real cohorts), BMI drifts upward with age, and BMI independently raises SBP. The simple regression cannot tell “older” apart from “older and heavier,” so its age coefficient absorbs both.

The multiple regression model separates them:

\[ \text{sbp}_i = \beta_0 + \beta_1 \, \text{age}_i + \beta_2 \, \text{bmi}_i + u_i \]

Now \(\beta_1\) answers a sharper question: comparing two people with the same BMI who differ by one year of age, how much higher is the older one’s predicted SBP? That is what “holding BMI constant” (or “adjusting for BMI,” in epidemiology-speak) means. No one physically held anything constant — the phrase describes a comparison within levels of BMI.

When does adding a covariate change a coefficient?

Adding \(x_2\) moves the coefficient on \(x_1\) only if both of these hold:

(a) \(x_2\) predicts \(y\) conditional on \(x_1\) (\(\beta_2 \neq 0\)), and
(b) \(x_2\) is correlated with \(x_1\) in the sample.

If either fails, the coefficient on \(x_1\) stays put (though precision may change). You will meet this pair of conditions again in Module 5, where their violation has a name: omitted variable bias.

5.2 Exercise 5: Fit both models and compare

Fit the simple regression (SBP on age) and the multiple regression (SBP on age and BMI) on the same sample, and compare the age coefficients to each other and to the truth.

Solution

set.seed(123456789)
data_sample <- pop[sample(nrow(pop), 1000, replace = FALSE), ]

mod_simple <- lm(sbp ~ age, data = data_sample)
mod_multi  <- lm(sbp ~ age + bmi, data = data_sample)

cat("Age coefficient, simple regression:  ",
    round(coef(mod_simple)["age"], 4), "\n")
cat("Age coefficient, multiple regression:",
    round(coef(mod_multi)["age"], 4), "\n")
cat("True DGP coefficient on age:          0.55\n\n")
cat("BMI coefficient, multiple regression:",
    round(coef(mod_multi)["bmi"], 4), "\n")
cat("True DGP coefficient on bmi:          1.2\n\n")
cat("Sample correlation of age and bmi:   ",
    round(cor(data_sample$age, data_sample$bmi), 3), "\n")

Interpretation question

The simple-regression age coefficient should sit near 0.70, and the multiple-regression one near 0.58 — much closer to the true 0.55. Where did the extra \(\approx 0.11\) mmHg/year go?

Discussion: It was never age’s to begin with. In the DGP, each year of age carries about \(+0.08\) BMI units on average, and each BMI unit carries \(+1.2\) mmHg, so the simple regression’s age slope picks up roughly \(1.2 \times 0.08 \approx 0.10\) mmHg/year of BMI’s effect (a bit more in this particular sample). Adding BMI to the model returns that piece to its rightful owner. Neither estimate hits 0.55 exactly — that residual gap is ordinary sampling variability, the subject of Module 2. This decomposition — short coefficient = long coefficient + (omitted effect \(\times\) association) — is exactly the omitted variable bias formula you will study in Module 5.

5.3 Frisch–Waugh–Lovell: what “holding constant” literally does

There is a beautiful, concrete answer to how multiple regression holds BMI constant. The Frisch–Waugh–Lovell (FWL) theorem says the multiple-regression coefficient on age can be obtained in two steps:

Residualize: regress age on bmi and keep the residuals, \(\tilde{a}_i\). These residuals are the part of each person’s age that is not linearly predictable from their BMI — age, purged of BMI.
Regress: run a simple regression of sbp on \(\tilde{a}_i\). Its slope equals the multiple-regression coefficient on age. Exactly. Every decimal.

\[ \hat{\beta}_1^{\text{multi}} \;=\; \frac{\widehat{\text{Cov}}(\tilde{a}, \, \text{sbp})}{\widehat{\text{Var}}(\tilde{a})} \]

So “adjusting for BMI” is not hand-waving: multiple regression compares SBP across the BMI-purged variation in age. If age and BMI were perfectly correlated, no purged variation would remain and the coefficient would be unidentifiable — which is exactly the intuition behind multicollinearity.

5.4 Exercise 6: Recover the age coefficient via FWL

Implement the two steps and confirm the slope matches the multiple regression’s age coefficient.

Solution

set.seed(123456789)
data_sample <- pop[sample(nrow(pop), 1000, replace = FALSE), ]

mod_multi <- lm(sbp ~ age + bmi, data = data_sample)

# Step 1: residualize age on bmi
step1     <- lm(age ~ bmi, data = data_sample)
age_resid <- residuals(step1)

# Step 2: regress sbp on the purged age variation
step2 <- lm(sbp ~ age_resid, data = data_sample)

cat("FWL two-step age coefficient:        ",
    round(coef(step2)["age_resid"], 6), "\n")
cat("Multiple regression age coefficient: ",
    round(coef(mod_multi)["age"], 6), "\n")

Interpretation question

The two coefficients should agree to six decimal places. What does the FWL result tell you about which people “drive” an adjusted coefficient?

Discussion: The adjusted age coefficient is estimated from the residualized variation — people whose age is surprisingly high or low given their BMI. Someone whose age is exactly what their BMI would predict contributes an \(\tilde{a}_i\) near zero and thus little information. This is worth internalizing for applied work: after heavy adjustment, your effective comparison group can be a small, unusual slice of the sample, even though the regression output looks like it used everyone.

6 Goodness of Fit: \(R^2\) and Its Misuse

6.1 The decomposition

Every OLS fit splits the total variation in \(y\) into an explained piece and a leftover piece:

\[ \underbrace{\sum_i (y_i - \bar{y})^2}_{\text{SST (total)}} \;=\; \underbrace{\sum_i (\hat{y}_i - \bar{y})^2}_{\text{SSE (explained)}} \;+\; \underbrace{\sum_i \hat{u}_i^2}_{\text{SSR (residual)}} \]

The coefficient of determination is the explained share:

\[ R^2 = \frac{\text{SSE}}{\text{SST}} = 1 - \frac{\text{SSR}}{\text{SST}}, \qquad 0 \le R^2 \le 1 \]

In words: the fraction of the sample variance of SBP that the regressors account for.

6.2 What \(R^2\) is not

Two misreadings to retire today

High \(R^2 \neq\) causal. Regress SBP on antihypertensive prescriptions across clinics and you may get a splendid \(R^2\) with a positive slope — sicker populations get more prescriptions. \(R^2\) measures fit, and confounded relationships can fit beautifully.
Low \(R^2 \neq\) useless. Individual-level epidemiology routinely lives with \(R^2\) below 0.10. A regression of lung cancer incidence on smoking explains a small share of individual variation — most smokers never develop lung cancer — yet the slope captures one of the largest, best-established harms in public health. The policy-relevant quantity is the coefficient (and later, its confidence interval), not the \(R^2\).

\(R^2\) answers a prediction question: “how much of the variation do I capture?” Most of this course asks causal and descriptive questions: “what is the slope, and can I trust it?” Those are different questions with different report cards.

6.3 Adjusted \(R^2\), briefly

Plain \(R^2\) can never fall when you add a regressor — even a column of random noise — because OLS can always use the extra coefficient to shave a bit off SSR. Adjusted \(R^2\) charges a small penalty per regressor:

\[ \bar{R}^2 = 1 - \frac{\text{SSR} / (n - k - 1)}{\text{SST} / (n - 1)} \]

where \(k\) is the number of regressors. It can decrease when a new variable adds less than its degrees-of-freedom cost, which makes it modestly useful for comparing specifications. It inherits every interpretive limitation of \(R^2\): it, too, says nothing about causality.

6.4 Exercise 7: Compute \(R^2\) from its pieces

Fit the multiple regression, build \(R^2\) manually from SSR and SST, and check it against summary().

Solution

set.seed(123456789)
data_sample <- pop[sample(nrow(pop), 1000, replace = FALSE), ]

mod_multi <- lm(sbp ~ age + bmi, data = data_sample)

# Total sum of squares: variation of sbp around its mean
sst <- sum((data_sample$sbp - mean(data_sample$sbp))^2)

# Residual sum of squares: what the model leaves unexplained
ssr <- sum(residuals(mod_multi)^2)

# R^2 = 1 - SSR/SST
r2_manual <- 1 - ssr / sst

cat("Manual R^2:            ", round(r2_manual, 4), "\n")
cat("summary() R^2:         ", round(summary(mod_multi)$r.squared, 4), "\n")
cat("summary() adjusted R^2:", round(summary(mod_multi)$adj.r.squared, 4), "\n")

Interpretation question

Your \(R^2\) should be around 0.35–0.40: the model leaves most of the variation in SBP unexplained — and yet we know this model is the true DGP. What does that tell you?

Discussion: Even the correct model cannot beat irreducible noise. In our DGP, \(u\) has a standard deviation of 12 mmHg — genuine biological variability that no regressor can absorb. A correctly specified model of a noisy outcome will have a modest \(R^2\), and that is fine: the coefficient estimates are excellent (compare them to 0.55 and 1.2). Conversely, an \(R^2\) near 1 in observational health data should make you suspicious — it often signals a tautology (regressing an outcome on a near-copy of itself) rather than insight.

7 Summary

Table 2: OLS foundations at a glance

Concept	The one-sentence version
Regression line	A parsimonious description of the conditional mean \(E[y \mid x]\)
OLS criterion	Choose the line minimizing the sum of squared residuals
Slope formula	\(\hat{\beta}_1 = \widehat{\text{Cov}}(x,y) / \widehat{\text{Var}}(x)\); the line passes through \((\bar{x}, \bar{y})\)
Residual mechanics	\(\sum \hat{u}_i = 0\) and \(\sum x_i \hat{u}_i = 0\), by construction, in every sample
Coefficient reading	A rate, in units of \(y\) per unit of \(x\); check the intercept’s \(x = 0\) against the data range
Holding constant	Comparison within levels of the covariate; FWL: regress on residualized \(x\)
\(R^2\)	Share of variance fitted — a prediction score, not a truth score

What comes next?

Everything in this module treated our estimates as if they were the answer. But Exercise 1’s wobbly band means already hinted at the real situation: a different sample of 1,000 adults would have produced slightly different coefficients. Module 2 (Standard Errors and Inference) takes that wobble seriously — we will build the sampling distribution of \(\hat{\beta}_1\) by simulation, derive its standard error, and learn what confidence intervals and hypothesis tests do (and do not) tell you about a health effect. Bring your fitted lines; we are about to put error bars on them.

References

Angrist, Joshua D., and Jorn-Steffen Pischke. 2009. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press.
Wooldridge, Jeffrey M. 2020. Introductory Econometrics: A Modern Approach. 7th ed. Cengage Learning.
Angrist, Joshua D., and Jorn-Steffen Pischke. 2015. Mastering ’Metrics: The Path from Cause to Effect. Princeton University Press.
Ettehad, Dena, et al. 2016. “Blood Pressure Lowering for Prevention of Cardiovascular Disease and Death: A Systematic Review and Meta-Analysis.” The Lancet 387 (10022): 957–967.

1 Regression as a Conditional Expectation

1.1 Our running example: blood pressure in adults

1.2 Exercise 1: See the conditional means

2 How OLS Works

2.1 The minimization problem

2.2 The formulas

2.3 Exercise 2: Compute \(\hat{\beta}_1\) by hand, then check against lm()

3 Fitted Values and Residuals

3.1 Mechanical properties

3.2 Exercise 3: Verify the mechanical properties

3.3 Exercise 4: Draw the residuals

4 Interpreting Coefficients

4.1 The slope: units first, always

4.2 The intercept: often a fiction

4.3 Clinically meaningful vs. statistically detectable

5 Multiple Regression: “Holding Constant”

5.1 Why one regressor was not enough

5.2 Exercise 5: Fit both models and compare

5.3 Frisch–Waugh–Lovell: what “holding constant” literally does

5.4 Exercise 6: Recover the age coefficient via FWL

6 Goodness of Fit: \(R^2\) and Its Misuse

6.1 The decomposition

6.2 What \(R^2\) is not

6.3 Adjusted \(R^2\), briefly

6.4 Exercise 7: Compute \(R^2\) from its pieces

7 Summary

References

2.3 Exercise 2: Compute \(\hat{\beta}_1\) by hand, then check against `lm()`