DPO simplifies: removes the explicit reward model, trains directly on preferences

Reinforcement learning from human feedback

RLHF optimizes a reward model trained on human preference pairs

the back-door criterion identifies: sufficient adjustment sets for causal estimation

Causal models use formal notation like DAGs for causal inference

score matching does: learns the gradient of the log-density without normalizing

Propensity score matching reduces bias in treatment effect estimates

Greedy vs dynamic programming: greedy makes locally optimal choices, DP considers all subproblems

Greedy: locally optimal choices; DP: considers all subproblems

classifier-free guidance does: interpolates between conditional and unconditional generation

"Classifies samples as either conditioned or unconditioned, guiding generation towards desired outcomes."

MoE models have more parameters but similar compute cost

MoE models distribute parameters across k experts, reducing active experts' compute cost