Graphs, Recommenders and Scalable Data Mining

Why this module belongs in the course

Scale changes engineering choices, but it does not remove the need for a clear question, valid evaluation and understandable failure handling.

Before you begin

The concepts and project evidence from Module 7. You should also be able to create a Python virtual environment and keep private or employer data out of the exercise.

Four lessons, one connected result

1. Lesson 1Graph Mining with PageRank and HITS95 min · Intermediate
2. Lesson 2Recommender Systems: Collaborative and Content-Based Methods100 min · Intermediate
3. Lesson 3Scalable Data Mining with MapReduce and Apache Spark105 min · Intermediate to advanced
4. Lesson 4Capstone: Build a Production-Ready Data Mining System240 min · Intermediate to advanced

How to know you are ready to continue

Complete the checkpoint without copying the worked example. Keep the code, output and a short decision note. Your note should explain one choice, one failure you observed and one limitation a reviewer should know.

Primary references for this module

The lessons explain the ideas in original wording. Use these primary or official sources when a library interface, standard or research claim needs verification.

• NetworkX PageRank
• NetworkX HITS
• scikit-learn cosine similarity
• scikit-learn TF-IDF
• Apache Spark RDD programming guide
• Apache Spark SQL performance tuning
• Python defaultdict
• NIST AI Risk Management Framework