Methodology & Data
Marketplace Compass is built to be auditable: every KPI, segment, and model metric traces back to a public dataset through a reproducible pipeline. This page documents the sources, the method, the measured numbers, and the limits.
Data sources
| Source | Role | License |
|---|---|---|
| UCI Online Retail IIUK online retailer transactions 2009-12 to 2011-12, two Excel sheets concatenated | keyless demo dataset (direct download, no credentials) | CC BY 4.0 |
| Olist Brazilian E-CommerceHonest substitution: Online Retail II used for the keyless demo because Olist needs Kaggle credentials unavailable in this environment. Olist adds order status/delivery timestamps and geolocation enabling the proposed models below. | richer PRIMARY target (NOT used here) -- requires Kaggle authentication | CC BY-NC-SA 4.0 |
The live demo uses UCI Online Retail II because it downloads without credentials. Olist is the richer intended target but requires Kaggle authentication (CC BY-NC-SA 4.0, non-commercial); its models are listed as proposed, not shown as results.
Method
Repeat-purchase model. customer-level repeat-purchase (churn) prediction. The label is 1 if customer made >=1 purchase in [cutoff, cutoff+90d), else 0. We compare a HistGradientBoostingClassifier against a LogisticRegression baseline on eight RFM-style features: recency_days, frequency, monetary, tenure_days, avg_basket_value, unique_products, is_uk, country_freq.
Temporal split & leakage discipline. temporal forward-in-time holdout (no random shuffle). Features are computed only from invoices dated before the cutoff, the label window sits strictly after it, and the test cutoff (2011-09-01) is after the train label window ends (2011-07-30) — so no information from the future leaks into training. features derived only from InvoiceDate < cutoff; country encoding learned on train only. There is no random shuffle; the holdout is genuinely forward-in-time (train n=4,822, test n=5,249).
RFM segmentation. Every customer is scored on recency, frequency, and monetary value into quantile buckets (1–4), then rule-labeled into Champions, Loyal, Potential, At-Risk, and Hibernating segments.
CLV proxy. A coarse 90-day value estimate: (historical monetary / tenure_days × 90) × predicted_repeat_prob, tiered into Platinum/Gold/Silver/Bronze. It is a proxy, not a survival or BG/NBD model.
Honest metrics
All numbers are measured on the temporal holdout (42.7% base rate), reported as-is:
- ROC-AUC: HGB 0.783 vs logistic baseline 0.788 — the baseline slightly wins (uplift -0.005).
- PR-AUC: HGB 0.751 vs 0.744 — HGB wins on the precision-recall metric most relevant to ranking who to retain.
- Brier score: HGB 0.195 vs 0.193 (lower is better) — near-parity calibration.
- Top-decile lift: HGB 2.14× vs 2.09× — targeting the top 10% finds repeat buyers at roughly twice the base rate.
On this dataset a well-regularized linear model is a strong baseline. We show it beating the gradient-boosted model on ROC-AUC rather than hiding it — the gain from added model complexity is small here, and saying so is the point.
Limitations (read before acting)
- One retailer, one window. UCI Online Retail II is a single UK-based online retailer, 2009–2011; patterns may not generalize.
- CLV is a proxy. Coarse expected-value heuristic, not a survival model; tenure < 1 day is clipped to avoid divide-by-zero inflation.
- Model uplift is marginal. The gradient-boosted model does not clearly beat the linear baseline on this feature set; treat the choice as a tie and prefer the simpler, better-calibrated model in production.
- Proposed models not shown as results: Order-level late-delivery risk model (needs Olist delivery timestamps).; Brazil geospatial expansion / regional demand model (needs Olist geolocation). — these require Olist data.
Reproducibility
The pipeline is one command — npm run data (python3 scripts/build_index.py) — downloading UCI Online Retail II, cleaning it, computing the temporal-holdout model, RFM segments, and CLV tiers, then writing validated JSON that npm run validate checks (schema, metric ranges, and leakage-token guards on feature names). No manual steps, no hand-edited numbers.