Embeddings and Neural NLP

Why this module belongs in the course

Neural sequence models explain the path to modern attention. They remain useful foundations, but a larger neural model should earn its cost by beating a suitable baseline.

Before you begin

The concepts and project evidence from Module 12. You should also be able to create a Python virtual environment and keep private or employer data out of the exercise.

Four lessons, one connected result

1. Lesson 1Word Embeddings: Word2Vec, GloVe and fastText75 min · Intermediate
2. Lesson 2Sentence Embeddings and Semantic Similarity70 min · Intermediate
3. Lesson 3RNN, GRU and LSTM Sequence Models Explained80 min · Advanced
4. Lesson 4Project: Build a Semantic Duplicate Detector105 min · Advanced

How to know you are ready to continue

Complete the checkpoint without copying the worked example. Keep the code, output and a short decision note. Your note should explain one choice, one failure you observed and one limitation a reviewer should know.

Primary references for this module

The lessons explain the ideas in original wording. Use these primary or official sources when a library interface, standard or research claim needs verification.

• https://radimrehurek.com/gensim/models/word2vec.html
• https://fasttext.cc/docs/en/unsupervised-tutorial.html
• https://huggingface.co/docs/hub/model-cards
• https://www.sbert.net/
• https://docs.pytorch.org/docs/stable/nn.html#recurrent-layers
• https://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html
• https://scikit-learn.org/stable/modules/model_evaluation.html