Deep Data Science and Business Decisions

Spurious relationships: from correlation to causal inference

"Correlation is not causation" is the iron law of statistics and empirical research. A spurious relationship arises when two variables display significant covariance or correlation not because one drives the other, but because an unobserved third variable — a confounder — influences both at once.

Turing Award laureate Judea Pearl, in his foundational work on causal inference, observed that traditional statistics and machine learning largely operate at the level of joint probability distributions over observed data, and cannot properly reason about interventions or counterfactuals. Pearl introduced directed acyclic graphs (DAGs) and the do-calculus to visualise and compute causal structure, transforming how science handles confounding. In econometrics, spurious relationships typically arise from omitted-variable bias (OVB) or simultaneity. If a linear model is specified as Y = βX + ε and an unobserved confounder C influences both the regressor X and the outcome Y, the error term becomes correlated with X (cov(X, ε) ≠ 0) and the estimated coefficient β is badly over- or under-stated. The classic meteorological illustration: a falling barometer reading correlates reliably with an approaching storm, yet moving the needle by hand summons no weather — both are driven by a common cause, the true atmospheric pressure. Likewise, noticing that bad breath correlates with lung cancer (because smokers tend to have both) and then handing out mints to reduce cancer incidence would be an absurd strategy.

In digital marketing and resource allocation across Hong Kong and international markets, spurious relationships are the number-one destroyer of advertising budgets. Modern businesses lean heavily on multi-touch attribution (MTA) to grade their digital channels, but such observational data routinely conflates consumer self-selection with genuine ad effectiveness. A multinational may find that audiences who saw its display ads convert at a spectacular rate and pour budget into the channel. But the ad platform's algorithm exists precisely to show ads to people already most likely to buy. Here the consumer's intrinsic purchase intent (say, an interest in surfing) is a powerful confounder that determines both whether the ad is seen and whether the purchase happens. Trust the raw correlation and a simple regression will wildly overstate the causal effect of ad spend on sales.

A professional data team never stops at surface correlation; it reaches for marketing econometrics and causal-inference technique. When randomised controlled trials are commercially infeasible, analysts turn to quasi-experimental designs: difference-in-differences (DiD) comparing regions that launched campaigns at different times, regression discontinuity designs (RDD), and instrumental variables (IV) to isolate exogenous variation. With disciplined geo-experiments and holdout control groups, a company can recover the true incrementality of its advertising — ensuring cross-market budgets buy real growth rather than harvesting traffic that would have arrived anyway.

Simpson's paradox: the fatal blind spot of aggregated data

Simpson's paradox, formalised by the statistician Edward H. Simpson in 1951, is a counter-intuitive and highly destructive phenomenon: examine the data group by group and every group shows one clear, consistent trend; merge the groups into a single aggregate and the trend weakens, vanishes — or reverses outright.

Mechanically, the paradox arises when subgroup sample sizes (the weights) are badly unbalanced and the grouping variable is itself a strong confounder of both treatment and outcome. The most famous academic case is the UC Berkeley graduate admissions analysis. In aggregate, men were admitted at a markedly higher rate than women — on its face, serious evidence of discrimination. Conditioned on department, however, nearly every department admitted women at a rate equal to or higher than men. The aggregate distortion arose because women disproportionately applied to fiercely competitive departments with low admission rates, while men favoured departments with high ones.

For a Hong Kong-headquartered company expanding a global e-commerce footprint, Simpson's paradox is the single easiest statistical trap to fall into in omnichannel A/B testing. Industry benchmarks for 2025–2026 put average desktop conversion at 3.93% against mobile's 2.46% — roughly a 1.56× gap — while mobile carries an overwhelming 78.4% of traffic.

Suppose this platform redesigns its checkout flow and runs a two-week A/B test. Careful tracking reveals the following paradox, driven purely by unbalanced traffic mix:

The new design wins in every segment yet loses in aggregate — Simpson's reversal in action.

In this scenario, an executive or junior analyst reading only the aggregate row of a BI dashboard would see the new design converting at 4.3% against the old design's 5.0% — and kill a genuinely superior product iteration. Drill into the segments and the truth is plain: on mobile and on desktop alike, the new design is strictly better. The aggregate flipped only because the variant received a far larger share of mobile traffic — whose baseline conversion is inherently lower — and that asymmetric weighting dragged down the blended average.

The same paradox recurs in international expansion. When a Hong Kong company takes a high-ticket business into Southeast Asia or Latin America, the new markets' lower initial conversion and enormous traffic can drag the company's blended margin or conversion rate down in the short run. A board that does not understand Simpson's paradox may misread the macro strategy and cut — or close — high-potential business lines that are in fact beating their local competitors in every individual market. Elite data teams defend against this with standardised rates, strict consistency in experimental traffic allocation, and automated causal engines that detect and neutralise confounders — keeping decisions anchored to true causal effects.

Selection bias: flawed samples and counterfactual recovery

Selection bias is an insidious statistical fallacy that appears whenever the process that selects data into the sample is not random, so that the observed sample differs systematically from the population. Any inference drawn directly from such a sample, without rigorous econometric correction, simply does not generalise — and decisions built on it invite disaster.

The most authoritative framework for handling selection bias in modern econometrics is the two-step correction of James Heckman, the 2000 Nobel laureate in economics. Heckman's insight: when the outcome we care about (wages, customer satisfaction, default rates) is only observable once some precondition is met — that is, when data are truncated or censored — running ordinary least squares directly amounts to omitting a latent variable that is highly correlated with the error term: a serious specification error. His remedy proceeds in two steps. First, fit a probit selection equation estimating each unit's probability of entering the observed sample, and from it compute an adjustment term known as the inverse Mills ratio. Second, insert that ratio as an additional control in the outcome equation and re-run OLS. This mathematically strips out the endogeneity introduced by non-random selection.

Across retail and online platforms in Hong Kong and worldwide, selection bias wreaks havoc on customer-satisfaction measurement. Research on online review systems consistently finds ratings distributed in an extreme, asymmetric "J-shape" or bimodal pattern rather than anything like a normal curve. Two entrenched selection mechanisms produce the distortion: acquisition bias — only customers already favourably disposed toward the product buy it and thereby qualify to review it at all; and underreporting bias — after purchase, only customers at the emotional extremes (the furious and the delighted) are motivated enough to overcome the cost of writing a review.

Platform heterogeneity compounds the measurement problem. Empirical work on global travel platforms shows that TripAdvisor and Google Travel exhibit measurably different non-response and measurement biases: users with genuinely negative experiences on Google Travel tend not to post at all (worsening the non-random missingness), but when they do post they score more harshly than they would on an internal satisfaction survey; TripAdvisor's users show less non-response bias but rate negative experiences comparatively leniently. If a hotel group or Hong Kong e-commerce executive uses uncorrected online rating averages as the KPI for regional general managers, or as the sole basis for pricing and service investment, they are effectively being held hostage by a tiny fringe of emotional extremists — while the silent majority of customers who transacted smoothly and felt "fine" go entirely unheard.

The same challenge now cuts deep into Hong Kong's fast-rising virtual banks and FinTech sector. Rolling out cashless payments, open banking or online micro-lending, risk teams often notice that users who complete the full, tedious eKYC flow show remarkably low subsequent default rates — and management may congratulate itself on a credit model of near-perfect predictive power. That comfort is usually a by-product of selection: the financially fragile, the credit-impaired and the digitally struggling tend to abandon the application at the liveness check or the data-authorisation step. The systematically filtered-out group leaves behind an unusually clean sample — one that says nothing about the true credit risk the bank will face at mass-market scale.

Facing these scenarios, an academically grounded data team refuses to take observed data at face value. It builds virtual control groups with propensity score matching (PSM) to estimate the true inclusion effects of an open-banking platform across customer segments. It applies Heckman-style corrections that model the latent probability of "writing a review" or "completing registration" jointly with the outcome, recovering the needs and quality expectations of the silent, filtered-out majority beneath the waterline. This respect for the data-generating process is what gives a company's global pricing strategy and credit models their predictive stability.

The myth of p-values: statistical significance vs business value

For nearly a century, null hypothesis significance testing (NHST) and its central artefact — the p-value — have played the role of final arbiter in empirical science and quantitative business research. Strictly defined, the p-value is the probability, assuming the null hypothesis is exactly true, of observing the current sample statistic or something more extreme. Yet journals and businesses alike have long worshipped p < 0.05 as an inviolable threshold, breeding widespread p-hacking (torturing the analysis until significance confesses) and severe publication bias.

The American Statistical Association responded with a landmark official statement warning, in effect, that statistical significance must never be equated with scientific, human or economic significance. On one hand, in the age of big data, when samples swell to the millions, random error is squeezed so hard that even differences with zero operational relevance sail past p < 0.05. On the other, in fast-paced e-commerce A/B testing, product managers habitually "peek" at interim results and stop the moment p dips below 0.05 — an undisciplined sequential practice that makes the false-positive rate explode, so that the celebrated "winning variant" is very often pure noise.

To escape the rigidity of classical fixed-sample testing, the large experimentation platforms (Optimizely, VWO and peers) have rebuilt their statistical engines around three main strategies:

Mishandle the p-value and the cost lands as wasted opportunity and engineering effort. Suppose a global SaaS company tests new copy on its APAC subscription page. After twenty million impressions the new copy lifts conversion by just 0.05% — at p < 0.01, overwhelmingly significant. Viewed through practical significance, though, the annualised revenue from that 0.05% may not come close to covering the cross-team coordination, multilingual translation, code refactoring and long-term maintenance required to roll the design out platform-wide. That is the classic trap of "statistically significant, commercially meaningless."

The reverse failure is just as costly. When optimising high-value continuous financial metrics — ARPU, average order value, lifetime value — the natural variance across users is enormous. Even when a feature genuinely lifts the metric by 5%, a conventional A/B test may need months to scrape past p < 0.05. In Hong Kong's fast-moving e-commerce and digital finance environment, that wait alone can strangle the product iteration rhythm.

An elite data science team therefore builds an experimentation culture that outgrows single-number worship. Before any experiment launches, it sets a minimum detectable effect (MDE) grounded in business logic, guaranteeing that anything labelled a win is large enough to move the P&L. It deploys CUPED — pioneered at Microsoft and Netflix — to soak up systematic noise with users' own history, compressing required samples and timelines by 20–40% and enabling genuinely agile global releases. And it switches engines to fit the stakes: strict sequential frequentist testing to lock down risk on core pricing changes; Bayesian inference for traffic-starved B2B interfaces and early exploration of new international markets, converting institutional experience into explicit priors and making the best risk-reward call under uncertainty.

Survivorship bias: the invisible failures and underpriced risk

Survivorship bias is among the most classic and lethal biases in data science and cognitive psychology. It occurs when analysis fixates on the individuals that survived some harsh filter — funds still trading, companies still operating — while ignoring everyone who failed, went bankrupt or dropped out along the way. Ignoring this truncation makes models wildly optimistic about the odds of success and fatally blind to catastrophic risk.

In modern finance and empirical asset pricing, survivorship bias has catastrophically distorted the measured performance of hedge funds, mutual funds and quantitative strategies. The authoritative fund databases (TASS, HFR, Eurekahedge and others) are built largely on voluntary reporting by managers. When a fund blows up and is liquidated or absorbed, its ugly final numbers are frequently expunged from the active database. Academic reconstructions show that analysing only surviving funds — excluding the dead — inflates median fund alpha under the Fama–French five-factor model by roughly 0.60% (60 basis points) up to 3.0% per year, without any basis in reality. The industry's peculiar backfill bias ("instant history") makes it worse: many funds report nothing in their early years and join a commercial database only after a lucky, high-return incubation period — backfilling the flattering history in one stroke. Strip out the backfilled records and measured performance drops by a further 400 to 500 basis points.

Hong Kong — an international financial centre ranked alongside New York and London, and Asia's asset-management hub — concentrates the world's family offices, institutional investors and sovereign wealth funds. In their APAC asset allocation, algorithmic backtests and manager due diligence, survivorship bias is a core driver of ten-figure systematic errors.

When a Hong Kong-based institutional investor evaluates a novel "Asian emerging-markets long-short" quant strategy, the glossy pitch book shows five turbulent years survived at a steady 12–15% annualised return. What the book does not show are the fifty-odd funds running near-identical leveraged strategies that were margin-called into oblivion during a sudden liquidity drought over the same period. Capital thus flows to managers who appear to possess extraordinary alpha and market timing, but who in truth carried concealed, extremely dangerous tail risk and simply happened to survive. When the political-economic regime turns, these over-priced survivors face an annihilating reversion.

Beyond investing, survivorship bias corrupts best-practice analysis and venture-capital judgement alike. Business media and MBA case studies obsessively dissect the handful of Silicon Valley unicorns and breakout Asian e-commerce giants, distilling winning formulas — radical flat management, wolf-culture overtime, scorched-earth subsidies for market share. The thousands of companies that adopted precisely the same playbook and died when the funding dried up become silent evidence nobody studies.

A professional data architect building global financial models or competitive benchmarks makes survivorship-bias-free training data the first requirement. High-frequency strategies, ML credit scoring and churn models are built on strict point-in-time data architecture, forcing the system to incorporate the complete life cycles of every bankruptcy, delisting, liquidation and default — and slamming the door on look-ahead leakage that lets backtests flatter themselves. Applied to executive decision-making, the same discipline means that for every glowing survivor story the board absorbs, it commits equal or greater analytical resources to excavating, dissecting and quantifying the trajectories of the failures — building an operating architecture with genuine antifragility and resilience.

Regression to the mean: extreme performance and management error

Regression to the mean was first identified and named by Francis Galton in his studies of heredity. Its core statistical meaning: when a random variable posts an extreme value on first measurement — far above or below the population average — subsequent independent measurements will, with high probability, drift back toward the true mean.

Through the lens of linear regression, any observed figure — monthly sales, daily conversion, quarterly performance — decomposes into a stable true score (signal) plus volatile random error. An extreme peak usually means the underlying ability is genuinely high and the observation caught an enormous positive shock of luck. Because random error has an expected value of zero in the long run, the odds of the same extreme shock repeating are minuscule; deprived of that tailwind, the next observation inevitably "falls back" toward the true-ability line. This is pure probability at work — yet the human mind's hard-wired causal instinct insists on reading it as real decline, slacking off, or proof that some intervention "worked."

In Hong Kong's hyper-competitive premium retail, in matrix-managed B2B sales teams, and in the tangled KPI regimes of multinationals, ignorance of regression to the mean routinely produces strategic misjudgement, useless intervention and morale-destroying management.

Take luxury retail in a prime Hong Kong district, or regional FMCG operations. Suppose a Tsim Sha Tsui flagship posts a record-shattering third quarter. Regional headquarters, elated, credits the store manager's brilliant leadership and raises the Q4 target by a punishing 20%. Careful dissection shows the Q3 explosion was a perfect stack of uncontrollable positive shocks: unusually good weather, a competitor's supply-chain failure creating a temporary stock-out, and a coincidental surge of high-net-worth tourist spending. Statistically, a textbook extreme value. In Q4, with the one-off factors gone, sales regress — while the team works exactly as hard as before — back down to the long-run structural growth curve.

At this point, executives without statistical grounding attribute the inevitable to "complacency after the bonus", "incentive fatigue" or "sudden managerial failure" — and rashly replace a good manager or impose destructive pressure tactics, damaging an operation that was in fact healthy.

The mirror-image absurdity plays out at the bottom of the league table. A regional sales team battered by consecutive negative shocks gets publicly excoriated and force-marched through an expensive external restructuring and training programme. Next month, performance duly rebounds. Management rejoices: the harsh medicine worked instantly! The brutal statistical truth is that a team at the bottom of its random fluctuation range would very likely have bounced back had headquarters done nothing at all. This beautiful illusion fills companies with costly, useless interventions and redundant training.

Breaking the illusion requires the data team to separate deep structural trend from surface randomness with precision. Simple linear projections cannot cope with today's retail complexity; data scientists build multivariate regression and GLM models that jointly control for baseline sales, marketing spend, fine-grained seasonality, competitor dynamics and macro indicators — isolating the true causal drivers of performance. By computing standard errors and dynamic prediction intervals, the analytics platform can tell management clearly whether a violent swing in regional performance sits inside normal random variation or marks a genuine structural break. And future KPIs should be set not by hostage-taking to one extreme quarter, but from algorithmically smoothed long-run trends, reinforced by robust machine-learning time-series models (random-forest regression, LSTM networks) — an objective, scientific and durable global performance system.