Introduction
A shopper adds coffee to their basket. Should your store recommend biscuits, milk or bread? Looking at which products sell most often will not answer that question. Popular products can appear together simply because many shoppers buy them anyway.
Market basket analysis identifies products that occur in the same transactions. Support, confidence and lift help you judge how common a pairing is and whether it occurs more often than its baseline popularity suggests.
This project takes you from a small transaction dataset to association rules in Python, then to a recommendation worth investigating. The data is invented for teaching, so the findings demonstrate the method rather than represent customer behaviour at a real retailer.
Define the Basket and the Business Question
Use one completed order as one basket. Count each product once within that order, regardless of quantity. A shopper buying three packets of biscuits contributes one basket containing biscuits.
The business question is: which companion product should we investigate for a basket recommendation?
An association rule such as “coffee to biscuits” has two parts. Coffee is the antecedent, the item already present. Biscuits are the consequent, the possible companion. The direction helps frame a recommendation, but does not establish which item was bought first or whether one purchase caused the other.
Prepare the Transaction Data
For real order data, start with order ID and product ID. Quantity, order date and transaction status help you check eligibility and investigate patterns.
Before grouping orders:
- Standardise product names or map them to consistent product IDs.
- Investigate missing identifiers instead of silently assigning them.
- Consolidate repeated product lines within each order.
- Exclude cancelled transactions and define how returns are handled. A refund record should not become a new shopping basket.
If you need help cleaning and grouping data with pandas, apply those checks before counting product combinations. Keep legitimate single-product orders in the dataset because removing them inflates pair support.
Use these ten illustrative baskets. Identical lists represent separate orders, not accidental duplicate records:
baskets = [
["bread", "milk", "coffee", "biscuits"],
["bread", "milk", "coffee", "biscuits"],
["bread", "milk", "coffee", "biscuits"],
["bread", "milk"],
["bread", "coffee"],
["bread"],
["bread"],
["bread"],
["milk"],
["tea"],
]
Bread appears in eight baskets, milk in five, coffee in four and biscuits in three. Coffee and biscuits occur together three times.
Calculate Support Confidence and Lift
Support Measures How Often the Pair Appears
Support = baskets containing both products ÷ all eligible baskets.
Coffee and biscuits appear together in three of ten orders, giving support of 30%. This measures the pair’s frequency across the dataset. It does not tell you whether recommending biscuits would change a shopper’s behaviour.
Confidence Measures the Conditional Purchase Rate
Confidence = baskets containing both products ÷ baskets containing the antecedent.
Three of the four coffee baskets contain biscuits, so coffee-to-biscuits confidence is 75%.
Reverse the rule and the denominator changes. All three biscuit baskets contain coffee, so biscuits-to-coffee confidence is 100%. Confidence is a conditional frequency, not a statistical confidence level.
Lift Compares Confidence With Baseline Popularity
Lift = confidence ÷ support of the consequent.
Biscuits appear in 30% of all baskets. Divide the coffee-to-biscuits confidence of 0.75 by 0.30 to get lift of 2.5.
In this sample, biscuits occur 2.5 times as frequently within coffee baskets as they do across all baskets. Lift above 1 indicates positive association; 1 matches the independence baseline; below 1 indicates negative association.
Compare three rules:
| Rule | Support | Confidence | Lift |
| Milk to bread | 40% | 80% | 1.00 |
| Coffee to biscuits | 30% | 75% | 2.50 |
| Biscuits to coffee | 30% | 100% | 2.50 |
The milk-to-bread rule has higher confidence than coffee-to-biscuits. However, bread already appears in 80% of all baskets. Knowing that a basket contains milk adds no positive association with bread in this sample.
Reversing a rule changes confidence when the products have different frequencies, while support and lift remain the same.
Generate Association Rules in Python
In your project environment, install the packages used below:
python -m pip install pandas==2.2.3 mlxtend==0.23.4
Run the following after defining baskets. The example was executed with Python 3.12.14, pandas 2.2.3 and mlxtend 0.23.4, and its selected results were checked against direct transaction counts.
import pandas as pd
from mlxtend.preprocessing import TransactionEncoder
from mlxtend.frequent_patterns import apriori, association_rules
encoder = TransactionEncoder()
encoded = pd.DataFrame(
encoder.fit_transform(baskets),
columns=encoder.columns_
)
itemsets = apriori(
encoded, min_support=0.20,
use_colnames=True, max_len=2
)
rules = association_rules(
itemsets, num_itemsets=len(encoded),
metric="confidence", min_threshold=0.50
)
rules = rules[
rules["antecedents"].map(len).eq(1)
& rules["consequents"].map(len).eq(1)
].copy()
The encoder creates one row per basket and one column per product. Each cell is True when that basket contains the product and False otherwise.
Apriori finds itemsets occurring in at least 20% of baskets. Setting max_len=2 limits this exercise to individual products and pairs. Rule generation then keeps directions with confidence of at least 50%.
Format the results for inspection:
rules["from_item"] = rules["antecedents"].map(
lambda items: next(iter(items))
)
rules["to_item"] = rules["consequents"].map(
lambda items: next(iter(items))
)
rules["pair_orders"] = (
rules["support"] * len(encoded)
).round().astype(int)
columns = [
"from_item", "to_item", "pair_orders",
"support", "confidence", "lift"
]
print(rules[columns].sort_values(
["lift", "confidence"], ascending=False
).to_string(index=False))
The comparison table above shows three selected rules from this output. The program also returns other qualifying pairs. Support and confidence appear as decimals, so 0.30 means 30%.
These thresholds are teaching choices, not universal standards. Raising minimum support to 0.40 removes the coffee-and-biscuits pair, even though its lift is high. In a real project, balance occurrence counts, coverage and practical relevance.
CompeteX is PangaeaX’s platform for practical data competitions. Its data analytics challenges provide a way to practise making analytical decisions on unfamiliar problems after completing a guided exercise like this.
Check Whether the Association Holds Up
Ten baskets are enough to verify the arithmetic, but too few to justify a commercial rollout. Three joint purchases can produce an impressive lift while offering little evidence about future shoppers.
With a larger dataset, investigate whether a promotion, stock availability or seasonal demand explains the pairing. Compare relevant channels or customer groups rather than assuming one overall rule fits everyone.
Discover rules in an earlier period, then recalculate their metrics in a later period. A rule that disappears may reflect temporary conditions. High lift neither proves causation nor predicts a sales uplift.
Turn a Rule Into a Testable Recommendation
If coffee-to-biscuits remains useful in larger, later data, propose showing biscuits to shoppers whose baskets contain coffee but not biscuits. Check stock availability, product relevance and margin before testing.
Randomly assign eligible shoppers to the recommendation or an unchanged experience, keeping their assignment consistent. Define eligibility and the measurement window before launch.
Measure biscuit attachment as the proportion of all assigned eligible shoppers who complete a purchase containing biscuits. Compare contribution profit per eligible shopper, including discounts and relevant variable costs. Monitor overall conversion so extra biscuit sales do not hide lost orders.
Use the controlled test to assess incremental impact. The association rule identifies a candidate; it does not establish the recommendation’s effect.
Present the Project So Others Can Review It
Include your notebook, dataset description, cleaning decisions, dependency versions and rule table. Explain why you selected a candidate, which tempting rule you rejected and what evidence would change your recommendation.
When presenting a data project as a case study, make the route from question to decision visible. Label invented data clearly and separate observed patterns from proposed business outcomes.
Extend this project with another dataset or time period. Recalculate the metrics, document what changes and explain which product association you would test first.