Ever wondered how to blend two colors perfectly?
Image: MaedaAkihiko This photo was taken with Panasonic Lumix DC-FZ1000 II, CC BY-SA 4.0, via Wikimedia Commons
mixup does: trains on convex combinations of pairs: x̃=λx_i+(1-λ)x_j
Ever wondered how to blend two colors perfectly?
Imagine you're mixing paint colors for a painting, wanting to get a shade that's halfway between two other shades.
Think about taking half of one color and adding half of another color. You're creating a new shade that smoothly transitions between the two original shades.
Example
If you have red (255,0,0) and blue (0,0,255), mixing them equally gives you purple (127,0,127).
Remember this
This is the concept of a convex combination, blending colors smoothly.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
convolution (f * g)(t) = ∫f(τ)g(t-τ)dτ
How do different shapes combine to create new patterns?
AdaGrad does: divides learning rate by sqrt of sum of squared gradients
How do we avoid overshooting in learning?
random projection to O(log n/ε²) dimensions preserves pairwise distances within 1±ε
Can you shrink a 3D object into 2D without losing its shape?
quantization to INT8 doubles throughput
Quantization to INT8 doubles throughput because tensor cores process INT8 2x faster
Convex optimization
Convex functions have only one global minimum
Mixture of experts
Mixture of experts (MoE) divides problem space into homogeneous regions
Swipe through 100 ML concepts daily
Open Pocket Polymath