300-dim word2vec encodes: trained on word co-occurrence with skip-gram window

Ever wondered how computers understand words?

Imagine you're trying to teach your smart speaker to recognize different ways to say "I will go to the store."

The speaker uses a special method to link words that mean the same thing based on how they're used together.

Example

If you say "I will go to the store," the speaker learns to connect it with "I will shop."

Remember this

Word2vec helps computers grasp language nuances by finding patterns in word usage.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

word error rate (WER) measures: edit distance between predicted and reference transcriptions

Ever wondered how machines understand speech as we do?

ring attention does: distributes long sequences across multiple devices

How can a machine understand and generate human language?

384-dim all-MiniLM-L6-v2 optimizes: fast sentence similarity with 6 layers

All-MiniLM-L6-v2 optimizes fast sentence similarity with 6 layers

weight tying does in language models: shares embedding and output projection matrices

Ever wonder how machines understand the sequence of words in a sentence?

BPE tokenization does: iteratively merges the most frequent adjacent byte pairs

How do we make computers understand language better?

[CLS] pooling does: uses the first token's embedding as the sentence representation

CLS pooling: uses the first token's embedding as the sentence representation

Swipe through 100 ML concepts daily