Can you not see what's right in front of you?
Masking (behavior)
Can you not see what's right in front of you?
Imagine you're at a party, and someone you don't know well starts talking to you. You feel uncomfortable but don't want to be rude. So, you smile and nod, even though you're not interested.
You're subconsciously hiding your true feelings to fit in, a process known as masking.
Example
You smile and nod, even though you're feeling awkward and uninterested.
Remember this
Masking helps us hide our true emotions to fit into social situations.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Encoder vs decoder: encoder sees all tokens bidirectionally, decoder sees only past tokens
Encoder: Sees all tokens bidirectionally; Decoder: Sees only past tokens
Large language model
LLMs can generate, summarize, translate, and analyze text in many contexts
the tokenizer's special tokens do: [CLS], [SEP], [PAD], [MASK] have specific roles
[CLS] marks the start of input, [SEP] denotes separation, [PAD] fills space, [MASK] hides words for prediction
Adam has bias correction: divides by (1-β^t) in early steps
Why do we sometimes need to fix mistakes in computer decisions?
gradient checkpointing trades: recomputes activations to save memory
Gradient checkpointing trades off computation time for memory savings by recomputing activations
Transformer (deep learning)
Transformers use multi-head attention for contextualizing tokens
Swipe through 100 ML concepts daily
Open Pocket Polymath