Can you not see what's right in front of you?
Masking (behavior)
Can you not see what's right in front of you?
Imagine you're at a party, and someone you don't know well starts talking to you. You feel uncomfortable but don't want to be rude. So, you smile and nod, even though you're not interested.
You're subconsciously hiding your true feelings to fit in, a process known as masking.
Example
You smile and nod, even though you're feeling awkward and uninterested.
Remember this
Masking helps us hide our true emotions to fit into social situations.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Encoder vs decoder: encoder sees all tokens bidirectionally, decoder sees only past tokens
Encoder: Sees all tokens bidirectionally; Decoder: Sees only past tokens
Large language model
LLMs can generate, summarize, translate, and analyze text in many contexts
the tokenizer's special tokens do: [CLS], [SEP], [PAD], [MASK] have specific roles
[CLS] marks the start of input, [SEP] denotes separation, [PAD] fills space, [MASK] hides words for prediction
2022 in science
Why do Transformers sometimes seem to 'ignore' irrelevant parts of the input?
Adam has bias correction: divides by (1-β^t) in early steps
Why do we sometimes need to fix mistakes in computer decisions?
ring attention does: distributes long sequences across multiple devices
How can a machine understand and generate human language?
Swipe through 100 ML concepts daily
Open Pocket Polymath