DPO simplifies: removes the explicit reward model, trains directly on preferences

Ever wished your bank could be both secure and flexible?

Imagine you're saving money for a vacation but need access to your funds quickly if an unexpected expense arises.

Decentralized finance (DeFi) lets you save money in a secure way on a public blockchain, and then borrow it when you need it without going through a bank.

Example

You deposit 1,000 in a DeFi savings account (like a bank savings account) and later borrow 500 when you need it for an emergency, all through smart contracts on a blockchain.

Remember this

DeFi simplifies financial services by removing the need for traditional banks and intermediaries.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

Reinforcement learning from human feedback

RLHF optimizes a reward model trained on human preference pairs

Greedy vs dynamic programming: greedy makes locally optimal choices, DP considers all subproblems

Greedy: locally optimal choices; DP: considers all subproblems

classifier-free guidance does: interpolates between conditional and unconditional generation

"Classifies samples as either conditioned or unconditioned, guiding generation towards desired outcomes."

ReLU and Leaky ReLU

Why do computers sometimes struggle with simple decisions?

the back-door criterion identifies: sufficient adjustment sets for causal estimation

Can we trust studies without random experiments?

the condition number κ(A) measures: sensitivity of Ax=b to perturbations

How small changes affect big outcomes

Swipe through 100 ML concepts daily

Open Pocket Polymath