Contrastive loss function: L = (1/2N) Σ [max(0, margin - y_i * (z_i - z_j))² + max(0, y_i * (z_i - z_j) - margin)²]

Image: Neoclassicism Enthusiast, CC BY-SA 4.0, via Wikimedia Commons

Write the contrastive loss function for SimCLR

Contrastive loss function: L = (1/2N) Σ [max(0, margin - y_i * (z_i - z_j))² + max(0, y_i * (z_i - z_j) - margin)²]

Related concepts

Write the triplet loss formula: max(d(a,p) - d(a,n) + margin, 0)

Can you tell friends apart even if they've changed a lot over time?

Cross-entropy

Cross-entropy loss equation: H(p, q) = -Σ(p(x) * log(q(x)))

Jensen–Shannon divergence

Jensen-Shannon divergence formula: D_JS(P||Q) = 1/2 * D_KL(P||(M)) + 1/2 * D_KL(Q||(M))

Normalization (machine learning)

L2 normalization equation: x_i' = x_i / ||x||_2

Softmax function

Softmax converts real numbers into a probability distribution

Gradient descent

Gradient descent weight update equation: w := w - α * ∇J(w)

Swipe through 100 ML concepts daily