Statistics for Data Science in 2026: Probability, Hypothesis Testing and Interview Questions

Statistics forms the mathematical backbone of every data science role. Interviewers at companies like Google, Meta, and Netflix test statistical reasoning because it separates candidates who can build models from those who understand why those models work.

Probability Distributions Every Data Scientist Must Know

Probability distributions describe how data points spread across possible values. Interviewers expect fluency with these distributions because they underpin sampling, A/B testing, and model assumptions.

The normal distribution (Gaussian) appears everywhere: measurement errors, height, test scores. Its bell curve centers on the mean, with 68% of data within one standard deviation, 95% within two. When sample sizes exceed 30, the Central Limit Theorem guarantees that sample means follow a normal distribution regardless of the original population shape.

The binomial distribution models success/failure trials. Click-through rates, conversion events, and pass/fail quality checks follow binomial patterns. For n trials with probability p of success, the expected value is np and variance is np(1-p).

The Poisson distribution counts rare events over fixed intervals: website visits per minute, defects per batch, calls per hour. Its single parameter λ represents both mean and variance. When events occur independently at a constant average rate, Poisson fits.

# distributions_demo.py
import numpy as np
from scipy import stats

# Normal: customer spend follows N(μ=50, σ=15)
normal_samples = stats.norm.rvs(loc=50, scale=15, size=1000)
prob_above_80 = 1 - stats.norm.cdf(80, loc=50, scale=15)  # P(X > 80)

# Binomial: 100 users, 5% conversion rate
binom_samples = stats.binom.rvs(n=100, p=0.05, size=1000)
prob_at_least_10 = 1 - stats.binom.cdf(9, n=100, p=0.05)  # P(X >= 10)

# Poisson: 3 support tickets per hour on average
poisson_samples = stats.poisson.rvs(mu=3, size=1000)
prob_zero_tickets = stats.poisson.pmf(0, mu=3)  # P(X = 0)

These three distributions handle most practical scenarios. Interviewers often ask which distribution fits a given business context, so practice mapping real problems to the right model.

Hypothesis Testing: The Framework Behind A/B Tests

Hypothesis testing provides a formal structure for making decisions from data. The framework starts with a null hypothesis (H₀) representing no effect or no difference, and an alternative hypothesis (H₁) representing what the experiment aims to detect.

A typical A/B test null hypothesis states: "The new feature has no effect on conversion rate." The alternative claims an effect exists. The test calculates how likely the observed data would be if the null hypothesis were true.

The p-value quantifies this likelihood. A p-value of 0.03 means there's a 3% chance of seeing results this extreme if the null hypothesis holds. When the p-value falls below the significance level (commonly α = 0.05), the null hypothesis is rejected.

# ab_test_analysis.py
import numpy as np
from scipy import stats

# A/B test data: control vs treatment conversion rates
control_conversions = 120
control_visitors = 2000
treatment_conversions = 145
treatment_visitors = 2000

# Proportions
p_control = control_conversions / control_visitors  # 0.060
p_treatment = treatment_conversions / treatment_visitors  # 0.0725

# Pooled proportion under null hypothesis
p_pooled = (control_conversions + treatment_conversions) / (control_visitors + treatment_visitors)

# Standard error
se = np.sqrt(p_pooled * (1 - p_pooled) * (1/control_visitors + 1/treatment_visitors))

# Z-statistic
z_stat = (p_treatment - p_control) / se

# Two-tailed p-value
p_value = 2 * (1 - stats.norm.cdf(abs(z_stat)))

print(f"Z-statistic: {z_stat:.3f}")  # Z-statistic: 1.681
print(f"P-value: {p_value:.4f}")  # P-value: 0.0928

With a p-value of 0.0928, this test does not reject the null hypothesis at α = 0.05. The observed difference might be random variation. More data or a larger effect size would be needed to reach statistical significance.

Type I and Type II Errors: The Tradeoffs Interviewers Probe

Statistical tests balance two types of errors. A Type I error (false positive) occurs when rejecting a true null hypothesis. A Type II error (false negative) occurs when failing to reject a false null hypothesis.

The significance level α directly controls Type I error probability. Setting α = 0.05 means accepting a 5% chance of false positives. Lowering α to 0.01 reduces false positives but increases false negatives.

Statistical power (1 - β) measures the probability of correctly rejecting a false null hypothesis. Power depends on effect size, sample size, and significance level. Industry standard targets 80% power, meaning a 20% chance of missing real effects.

Sample size calculations balance these concerns. For an A/B test targeting a 2% lift in conversion (from 5% to 7%) with 80% power and α = 0.05:

# sample_size_calculation.py
from statsmodels.stats.power import zt_ind_solve_power

# Baseline conversion: 5%, expected lift: 2 percentage points
baseline = 0.05
expected = 0.07
effect_size = (expected - baseline) / np.sqrt(baseline * (1 - baseline))  # Cohen's h

# Required sample size per group
n_per_group = zt_ind_solve_power(
    effect_size=effect_size,
    alpha=0.05,
    power=0.80,
    alternative='two-sided'
)

print(f"Sample size per group: {int(np.ceil(n_per_group))}")  # ~1,570 per group

Under-powered tests waste resources by running experiments that cannot detect realistic effects.

Confidence Intervals: Quantifying Uncertainty

A 95% confidence interval means: if the same sampling process were repeated many times, 95% of the constructed intervals would contain the true parameter. Confidence intervals communicate uncertainty better than point estimates alone.

For means with known population variance, the interval uses the z-distribution. For small samples or unknown variance, the t-distribution applies. The t-distribution has heavier tails than the normal, producing wider intervals that account for additional uncertainty.

# confidence_intervals.py
import numpy as np
from scipy import stats

# Sample data: response times in milliseconds
response_times = np.array([245, 312, 278, 299, 256, 334, 287, 301, 265, 289])

n = len(response_times)
mean = np.mean(response_times)  # 286.6
std_err = stats.sem(response_times)  # Standard error of the mean

# 95% confidence interval using t-distribution (df = n-1)
t_critical = stats.t.ppf(0.975, df=n-1)
margin_of_error = t_critical * std_err

ci_lower = mean - margin_of_error
ci_upper = mean + margin_of_error

print(f"Mean: {mean:.1f}ms")  # Mean: 286.6ms
print(f"95% CI: [{ci_lower:.1f}, {ci_upper:.1f}]ms")  # 95% CI: [265.4, 307.8]ms

Narrower intervals indicate more precise estimates. Sample size drives precision: quadrupling n halves the margin of error.

Bayesian vs Frequentist: Interview Questions on Statistical Philosophy

Interviewers sometimes probe understanding of competing statistical paradigms. Frequentist statistics treats probability as long-run frequency: parameters are fixed, and data varies across repeated experiments. P-values and confidence intervals come from this tradition.

Bayesian statistics treats probability as degree of belief: prior knowledge combines with data to produce posterior beliefs. The framework allows incorporating domain expertise and produces probability statements about parameters directly.

A/B testing increasingly uses Bayesian methods because they answer the question stakeholders actually ask: "What is the probability that treatment B outperforms treatment A?" rather than "How surprising is this data if there were no difference?"

# bayesian_ab_test.py
import numpy as np
from scipy import stats

# Prior: Beta(1, 1) = uniform prior for conversion rate
# Posterior: Beta(alpha + successes, beta + failures)

control_successes, control_failures = 120, 1880
treatment_successes, treatment_failures = 145, 1855

# Posterior distributions
posterior_control = stats.beta(1 + control_successes, 1 + control_failures)
posterior_treatment = stats.beta(1 + treatment_successes, 1 + treatment_failures)

# Monte Carlo estimation: P(treatment > control)
samples_control = posterior_control.rvs(100000)
samples_treatment = posterior_treatment.rvs(100000)
prob_treatment_better = np.mean(samples_treatment > samples_control)

print(f"P(treatment > control): {prob_treatment_better:.2%}")  # ~95%

Bayesian results communicate directly: "There is a 95% probability that the treatment conversion rate exceeds the control rate." Frequentist results require more careful interpretation.

Common Interview Questions on Statistical Inference

These questions appear frequently in data science interviews at tech companies. Each tests a specific statistical concept.

"Explain the Central Limit Theorem and why it matters."

The CLT states that sample means from any population (with finite variance) approach a normal distribution as sample size increases. This matters because normality assumptions in hypothesis tests hold for large samples regardless of the underlying distribution. Sample sizes of 30+ typically suffice.

"What is p-hacking and how do you prevent it?"

P-hacking inflates false positive rates by testing multiple hypotheses until finding significance, or by stopping data collection when results become significant. Prevention strategies: pre-register analysis plans, apply multiple comparison corrections (Bonferroni, FDR), and set sample sizes before collecting data.

"When would you use a t-test vs a Mann-Whitney U test?"

The t-test assumes normally distributed data (or large samples via CLT) and compares means. Mann-Whitney U is a non-parametric alternative that compares distributions without normality assumptions. Use Mann-Whitney for small samples with clearly non-normal data or when comparing ordinal measurements.

"How do you handle multiple comparisons in A/B testing?"

Testing multiple variants inflates Type I error. With 20 comparisons at α = 0.05, the probability of at least one false positive exceeds 60%. The Bonferroni correction divides α by the number of tests. The Benjamini-Hochberg procedure controls false discovery rate while maintaining more power than Bonferroni.

Simpson's Paradox: A Favorite Interview Topic

Simpson's paradox occurs when trends in aggregated data reverse at the subgroup level. Interviewers love this topic because it tests causal reasoning, not just statistical mechanics.

A classic example: Hospital A has a higher overall mortality rate than Hospital B. But Hospital A has lower mortality for both mild and severe cases. The paradox arises because Hospital A treats more severe cases, which have higher mortality regardless of hospital quality.

# simpsons_paradox.py
import pandas as pd

# Hospital mortality data
data = {
    'Hospital': ['A', 'A', 'B', 'B'],
    'Severity': ['Mild', 'Severe', 'Mild', 'Severe'],
    'Patients': [200, 800, 800, 200],
    'Deaths': [10, 160, 48, 50]
}
df = pd.DataFrame(data)
df['Mortality'] = df['Deaths'] / df['Patients']

# Subgroup mortality
print(df[['Hospital', 'Severity', 'Mortality']])
# Hospital A: Mild 5%, Severe 20%
# Hospital B: Mild 6%, Severe 25%

# Aggregate mortality
agg = df.groupby('Hospital').agg({'Deaths': 'sum', 'Patients': 'sum'})
agg['Mortality'] = agg['Deaths'] / agg['Patients']
print(agg['Mortality'])
# Hospital A: 17%, Hospital B: 9.8%

The resolution requires identifying the confounding variable (severity) and analyzing subgroups separately. Aggregate analysis misleads when subgroup proportions differ.

Statistical Concepts for Feature Engineering

Statistical knowledge directly improves feature engineering. Understanding distributions guides transformations; understanding relationships guides feature creation.

Skewness measures distribution asymmetry. Right-skewed data (income, prices) benefits from log transformations that normalize the distribution and improve model performance. Left-skewed data uses square or exponential transforms.

Correlation quantifies linear relationships. Pearson correlation assumes normality and linearity. Spearman correlation handles monotonic non-linear relationships. Highly correlated features cause multicollinearity in linear models, so one is typically dropped.

Outliers affect means and standard deviations disproportionately. Robust statistics (median, IQR) resist outlier influence. Decision: remove, cap, or transform outliers depending on whether they represent errors or genuine extreme values.

For more on applying these concepts to machine learning models, see the feature engineering interview guide and explore the inferential statistics practice module.

What Sets Apart Strong Candidates in Statistics Questions

Statistical competence in data science interviews requires more than textbook definitions. Strong candidates demonstrate these abilities:

• Connect statistics to business outcomes: frame A/B test results in terms of revenue impact or user engagement, not just p-values
• Know the assumptions: every test has conditions (independence, normality, equal variance) and candidates should state when those conditions might fail
• Quantify uncertainty: point estimates without confidence intervals or credible intervals leave stakeholders guessing about reliability
• Recognize causation limits: observational data shows correlation, and candidates should articulate what experimental design would establish causation
• Choose the right test: match the data type (continuous vs categorical), sample size, and research question to the appropriate statistical method
• Communicate results clearly: translate statistical findings into actionable recommendations that non-technical stakeholders can understand and act upon