Code Audit

LLM Adoption and Labor Market Outcomes

82/100 Reduced-form 2026-04-10

Code-Strategy Alignment

Code Quality

#CategoryStatusIssues
#CategorySTATUSIssues
1Project layoutOKScripts numbered 01-05, clear pipeline
2Script headersOKPurpose, inputs, outputs, runtime estimates documented
3Console outputOKProgress bars via `cli::cli_progress_bar()`; no raw `print()`
4ReproducibilityOK`set.seed(20250301)` in `01_build_exposure.R`; `here()` for all paths
5Numerical disciplineOKExposure clamped; no float `==`; log(x+1) documented
6Function designOK`compute_aioe()` modular; `estimate_did()` reused across specifications
7Figure qualityWARNEvent study plot uses `ggtitle("Event Study: Employment")` — violates INV-12, title belongs in LaTeX `\caption{}`
8Table qualityOKBare `tabular` via `modelsummary`; no `\begin{table}` wrapper (INV-13 compliant)
9RDS/checkpointFAILNo RDS save after `01_build_exposure.R` which calls GPT API (45 min, $12.40). Re-running `03_estimate.R` requires full rebuild including API calls
10Comment qualityOKKey methodology decisions annotated with paper references
11Error handlingOK`stopifnot(all(aioe_score >= 0 & aioe_score <= 1))` after API scoring; retry logic for API timeouts
12Prohibited patternsOKNo `setwd()`, `rm(list=ls())`, `install.packages()`, `attach()`

Score Breakdown

Starting100
Code-strategy alignment (missing SEs)-8
Missing robustness (Rambachan & Roth)-3
Missing robustness (placebo)-2
Missing robustness (leave-one-out)-2
Figure title in ggplot (INV-12)-3
No RDS checkpoint after API calls-5
Magnitude flag (3x historical rate)+5
Final82/100

Recommendations

CRITICAL

Add Adao et al. (2019) SEs via ShiftShareSE package. Three lines of code: library(ShiftShareSE); reg_ss <- ivreg_ss(...); summary(reg_ss). This resolves the strategy alignment deduction and may change inference.

MAJOR

Add saveRDS(exposure_panel, here("data/cleaned/exposure_index.rds")) at end of 01_build_exposure.R. This prevents re-running 45 minutes of GPT API calls (and spending $12.40) every time downstream scripts change.

MAJOR

Implement the three missing robustness checks. The Rambachan & Roth analysis is 10 lines with HonestDiD. The placebo is a copy of 03_estimate.R with a date filter. The leave-one-out is a loop over unique(soc2).

MINOR

Replace ggtitle() with empty string in all figure scripts. Panel labels inside facets are fine per INV-12.