Introduction
Statistics questions in data analyst interviews are rarely limited to definitions. You may be given a business problem and asked which test you would use, what assumptions you would check, how you would interpret the result, and what action you would recommend.
The goal is not to memorise every formula. It is to connect the question, data, method, uncertainty, and business decision. Use this cheat sheet to revise the essential concepts and practise explaining them clearly.
A Simple Framework for Statistics Questions
Before selecting a test, work through this sequence:
| Step | Ask yourself |
| Question | What business decision are we trying to make? |
| Outcome | Is the variable continuous, binary, categorical, a count, or a time value? |
| Groups | How many groups are involved? |
| Design | Are observations independent, paired, observational, or randomly assigned? |
| Assumptions | Does the method fit how the data was collected and distributed? |
| Result | What do the effect size, confidence interval, and p-value show? |
| Decision | Is the result practically meaningful enough to act on? |
A strong response sounds like this:
“I would use [method] because the outcome is [data type], the groups are [independent or paired], and the objective is to [compare, test, or estimate]. I would check [assumptions], report [effect and uncertainty], and then relate the result to the business decision.”
Essential Descriptive Statistics
| Measure | Formula or meaning | Best used for |
| Mean | x̄ = Σxᵢ / n | Average of numerical values |
| Median | Middle ordered value | Skewed data or data containing outliers |
| Mode | Most frequent value | Common category or repeated value |
| Sample variance | s² = Σ(xᵢ − x̄)² / (n − 1) | Squared spread within a sample |
| Standard deviation | s = √s² | Typical variation around the mean |
| Interquartile range | IQR = Q3 − Q1 | Spread of the middle 50% |
| Standard error | SE = s / √n | Uncertainty in the estimated mean |
Standard Deviation vs Standard Error
- Standard deviation describes how individual observations vary.
- Standard error describes how precisely a sample statistic estimates a population parameter.
Increasing the sample size usually reduces the standard error. It does not necessarily reduce the variability among individual observations.
Confidence Interval
A confidence interval is commonly calculated as:
estimate ± critical value × standard error
It shows a range of parameter values compatible with the data and method. Under the frequentist interpretation, a 95% confidence procedure would capture the true parameter in approximately 95% of repeated samples under its assumptions. It does not mean there is a 95% probability that the fixed parameter lies inside one already-calculated interval.
Probability Rules to Remember
| Concept | Formula |
| Complement | P(Aᶜ) = 1 − P(A) |
| Addition | P(A ∪ B) = P(A) + P(B) − P(A ∩ B) |
| Conditional probability | P(A|B) = P(A ∩ B) / P(B) |
| Multiplication | P(A ∩ B) = P(A|B)P(B) |
| Independence | P(A ∩ B) = P(A)P(B) |
| Expected value | E[X] = ΣxP(X=x) |
Two events are mutually exclusive if they cannot happen together. They are independent if one event occurring does not change the probability of the other.
Bayes’ Theorem
P(A\|B) = [P(B\|A) × P(A)] / P(B)
Bayes’ theorem combines prior probability with new evidence. The important interview lesson is to consider the base rate. A fraud rule can detect most fraudulent transactions but still produce many false alerts when fraud is rare.
Common Probability Distributions
| Distribution | Data type | Example |
| Bernoulli | One binary outcome | A user converts or does not convert |
| Binomial | Number of successes in fixed trials | Conversions among 1,000 users |
| Poisson | Event counts within an interval | Support tickets received per hour |
| Normal | Symmetrical continuous values | Measurement errors around a target |
| Uniform | Equally likely values in a range | Random assignment values |
| Exponential | Time between independent events | Time between incoming requests |
Normal Distribution and Z-Score
A z-score shows how many standard deviations a value is from the mean:
z = (x − μ) / σ
A high absolute z-score may indicate an unusual observation, but it does not prove the value is incorrect. Investigate its source and business context before removing it.
Central Limit Theorem
Under suitable conditions, the sampling distribution of a sum or mean becomes approximately normal as sample size grows, even when the individual observations are not normally distributed.
The Central Limit Theorem concerns the distribution of a statistic across repeated samples. It does not make the raw data normal.
Hypothesis Testing Explained Simply
- Define the business question.
- State the null hypothesis, H₀, and alternative hypothesis, H₁.
- Select the significance level, α, before examining the result.
- Choose a test that matches the data and design.
- Check assumptions and data quality.
- Calculate the effect, confidence interval, test statistic, and p-value.
- Translate the result into a business recommendation.
What Does a P-Value Mean?
A p-value is the probability of obtaining a result at least as extreme as the observed result, assuming the null hypothesis and statistical model are correct.
A p-value does not show:
- The probability that the null hypothesis is true
- The size or importance of the effect
- Whether the data or experiment are free from bias
- Whether the result will replicate
The American Statistical Association advises against making decisions only because a p-value crosses a threshold. Consider the design, effect size, confidence interval, assumptions, and business consequences.
Type I and Type II Errors
| Decision | No real effect | Real effect exists |
| Reject H₀ | Type I error, false positive | Correct detection |
| Do not reject H₀ | Correct non-rejection | Type II error, false negative |
Statistical power is 1 − β, the probability of detecting a specified effect when it exists. Error thresholds should reflect the consequences of making the wrong decision.
Statistical Test Selection Cheat Sheet
| Question | Common method | Main checks |
| Is one sample mean different from a reference? | One-sample t-test | Independence, outliers, approximate normality for small samples |
| Are the means of two independent groups different? | Welch’s t-test | Independent groups and meaningful mean comparison |
| Did the same subjects change before and after? | Paired t-test | Correct pairing and distribution of differences |
| Are three or more group means different? | ANOVA | Independence, residual behaviour, variance assumptions |
| Are two conversion rates different? | Two-proportion z-test | Independent groups and sufficient expected counts |
| Are two categorical variables associated? | Chi-square test | Independent observations and adequate expected cell counts |
| Are two ranked or non-normal groups different? | Mann-Whitney U test | Independent observations and careful rank interpretation |
| Are two numerical variables linearly related? | Pearson correlation | Linearity, independence, and influential outliers |
| Are two variables monotonically related? | Spearman correlation | Monotonic relationship and independent observations |
Use a t-test to compare numerical means. Use a chi-square test for categorical counts or associations. Correlation measures association, but it does not establish causation.
A/B Testing Cheat Sheet
An A/B test compares a control with a treatment using random assignment. Before launch, define:
- Primary metric and guardrail metrics
- Randomisation unit, such as user, account, or device
- Baseline performance and minimum detectable effect
- Significance level, statistical power, and required sample size
- Test duration and stopping rule
Before interpreting the result, verify assignment, exposure, and tracking. A sample ratio mismatch occurs when observed group allocation differs unexpectedly from the planned split. It can indicate assignment or data-quality problems and should be investigated before trusting the outcome.
Worked Example
| Variant | Users | Conversions | Conversion rate |
| Control | 1,000 | 100 | 10% |
| Treatment | 1,000 | 120 | 12% |
- Absolute increase: 12% − 10% = 2 percentage points
- Relative lift: (12% − 10%) / 10% = 20%
- Approximate two-sided p-value: 0.15
- Approximate 95% confidence interval: −0.7 to +4.7 percentage points
The treatment has a higher observed conversion rate, but the interval includes both a small decrease and a meaningful increase. At the 5% significance level, the evidence is not strong enough to declare a winner. Check experiment quality, guardrails, and whether the planned sample was reached before deciding what to do next.
Common A/B Testing Mistakes
| Mistake | Better approach |
| Stopping as soon as p < 0.05 | Set the sample and stopping rule in advance |
| Testing many metrics and segments | Predefine the primary analysis |
| Ignoring sample ratio mismatch | Diagnose assignment and tracking first |
| Reporting only relative lift | Include baseline, absolute change, and confidence interval |
| Treating non-significance as equality | Examine which effects remain plausible |
| Ignoring guardrails | Check whether another important metric deteriorated |
Final Interview Checklist
- Identify the business question before choosing a test.
- Confirm the outcome type, number of groups, and study design.
- Explain why the chosen method fits.
- Check independence, outliers, missing data, and distributional assumptions.
- Report effect size and confidence interval, not only the p-value.
- Separate statistical significance from practical importance.
- End with a recommendation and limitations.
Put Statistical Knowledge Into Practice
Reading formulas helps with revision, but applying them to realistic problems develops stronger analytical judgement. CompeteX gives data professionals opportunities to work through practical data and scenario-based challenges, receive performance feedback, and demonstrate how they approach problems.
Within the wider PangaeaX ecosystem, professionals can use AuthenX to authenticate relevant skills through portfolio screening and an AI-led interview, while ConnectX supports continued learning through a data-focused community.
The progression is simple: learn the concepts, apply them to practical problems, and build credible evidence of what you can do.
Conclusion
A strong statistics answer makes the reasoning visible. Start with the business question, identify the data type and design, choose an appropriate method, check its assumptions, and interpret the effect with its uncertainty.
Use this cheat sheet for quick revision, then practise explaining each decision in plain language. That is what turns statistical knowledge into analytical judgement.
Sources