Distance and Similarity Measures for Data Mining

Choose closeness deliberately

You will calculate three common measures by hand and with NumPy, then explain why the same pair can look close under one definition and far under another.

• Calculate Euclidean and Manhattan distance for numerical vectors.
• Use cosine similarity when direction matters more than magnitude.
• Recognise when scaling or mixed data types invalidate a naive distance.
• Connect a distance choice to the geometry assumed by a mining method.

Every metric carries an assumption

Euclidean distance is the straight-line length between points. Squared differences give a large single gap strong influence. It works naturally for continuous features on comparable scales and underlies k-means. Manhattan distance sums absolute differences and can be easier to interpret as movement along feature axes.

Cosine similarity compares the angle between vectors. Two documents with the same pattern of terms can be similar even when one is much longer. Cosine ignores overall magnitude, which can be useful for text but wrong when magnitude contains the signal, such as total spending.

Scaling can completely change numerical neighbours. A one-year age difference disappears beside a salary difference measured in thousands. Standardization or another domain-justified transformation can balance the axes, but it also changes the question. Keep units visible and compare neighbour stability before and after scaling.

Binary and categorical fields need their own reasoning. Treating colour codes 1, 2 and 3 as continuous creates fictional distances. Jaccard similarity can suit sets where shared presences matter. Gower-style approaches can combine mixed types, but each field’s contribution still needs a meaningful weight.

High-dimensional distances often become less discriminating as many noisy features accumulate. Feature selection, dimensionality reduction and domain structure may matter more than switching between two metrics. Always inspect actual neighbours, not just an aggregate score.

Calculate three views of the same records

The vectors represent two product-interest profiles. The third has the same direction as the first but twice the activity. Compare what each measure preserves.

Install the lesson dependencies

python -m pip install numpy

Inspect magnitude and direction

import numpy as np

a = np.array([1.0, 2.0, 0.0])
b = np.array([2.0, 1.0, 0.0])
c = np.array([2.0, 4.0, 0.0])

def euclidean(x, y):
    return float(np.sqrt(np.sum((x - y) ** 2)))

def manhattan(x, y):
    return float(np.sum(np.abs(x - y)))

def cosine(x, y):
    denominator = np.linalg.norm(x) * np.linalg.norm(y)
    if denominator == 0:
        raise ValueError("Cosine similarity is undefined for a zero vector")
    return float(np.dot(x, y) / denominator)

for name, vector in [("b", b), ("c", c)]:
    print(name)
    print(" euclidean:", round(euclidean(a, vector), 3))
    print(" manhattan:", round(manhattan(a, vector), 3))
    print(" cosine:", round(cosine(a, vector), 3))

Expected comparison

b
 euclidean: 1.414
 manhattan: 2.0
 cosine: 0.8
c
 euclidean: 2.236
 manhattan: 3.0
 cosine: 1.0

Vector c is farther from a than b under Euclidean and Manhattan distance because its magnitude differs. It is perfectly aligned under cosine similarity. Neither answer is universally correct. The task decides whether magnitude should count.

Neighbour problems to diagnose

When results feel arbitrary, print the contributing features for a few neighbour pairs. A single unit or placeholder value often explains the surprise.

• A zero vector makes cosine undefined. Decide whether it means no activity, missing data or an invalid record.
• One-hot categories with many possible levels can dominate distance through sparse mismatches.
• Imputed values can create a dense group of artificial neighbours.
• Distances calculated before fitting a scaler on the training set can leak information from evaluation data.

Design a similarity test, not just a formula

Create six synthetic customer profiles with age, monthly orders and three binary interests. Define two candidate representations and compare the nearest neighbour of each customer.

• Scale only continuous features and keep the transformation parameters.
• Calculate a numerical distance and a set-based Jaccard similarity.
• Print the features that make one neighbour pair close.
• Ask whether a domain expert would regard that pair as meaningfully similar.

Evidence to include

• A table of neighbour results under both definitions.
• A short justification for which properties should affect similarity.
• One documented failure case where the chosen measure produces a poor neighbour.

Official references and further reading

• SciPy distance functions (Official definitions for common distance and similarity measures)
• scikit-learn pairwise metrics (Official pairwise distance and kernel API)
• scikit-learn preprocessing (Official scaling guidance)

Review note for Distance and Similarity Measures for Data Mining: recheck the linked documentation after a dependency changes the relevant API, metric or modelling assumption, then record the tested version beside your result.