
When analyzing data, a single number rarely tells the whole story. A sample mean, for example, is a useful estimate — but how much should you trust it? Confidence intervals answer that question by providing a range of plausible values for a population parameter, giving your estimate both context and credibility.
Used across medicine, economics, psychology, and data science, confidence intervals are one of the most practical tools in inferential statistics. Rather than pinning your conclusion to a single point estimate, a confidence interval acknowledges the uncertainty inherent in sampling and quantifies it in a way that is easy to communicate and interpret.
A confidence interval (CI) is a range of values, calculated from sample data, that is likely to contain the true value of a population parameter. Instead of reporting a single point estimate — such as a sample mean — a confidence interval provides an upper and lower bound within which the true parameter is expected to fall, given a specified level of confidence.
For example, if a study reports a 95% confidence interval of [112, 128] for the average systolic blood pressure of adults in a city, it means the analysis supports the conclusion that the true population mean likely falls somewhere between 112 and 128 mmHg.
The Confidence Level Explained
The confidence level is the probability, expressed as a percentage, that the interval produced by a given method will contain the true parameter across repeated sampling. The most commonly used confidence levels are:
| Confidence Level | Meaning |
|---|---|
| 90% | 90 out of 100 similarly constructed intervals contain the true parameter |
| 95% | 95 out of 100 similarly constructed intervals contain the true parameter |
| 99% | 99 out of 100 similarly constructed intervals contain the true parameter |
The 95% confidence level is the standard choice in most academic and professional research.
It is important to understand what a confidence level does not mean. A 95% confidence interval does not mean there is a 95% probability that the true parameter lies within that specific interval. Once an interval is calculated, the true parameter either falls within it or it does not. The 95% refers to the long-run reliability of the method, not the probability attached to any single interval.
Key Components of a Confidence Interval
Every confidence interval is built from three core components:
Point estimate — The sample statistic used as the central value of the interval, such as the sample mean () or sample proportion ().
Margin of error — The value added to and subtracted from the point estimate to create the interval’s upper and lower bounds. It reflects how much the estimate is expected to vary due to sampling variability.
Critical value — A multiplier derived from the chosen confidence level and the relevant probability distribution (typically the Z-distribution or t-distribution). It determines how wide the interval extends around the point estimate.
The general structure of any confidence interval is:
Or, written in full:
Where is the critical value and SE is the standard error of the estimate.
Why Confidence Intervals Matter
Confidence intervals are more informative than p-values or point estimates alone. A result can be statistically significant yet practically meaningless if the confidence interval is very wide, signaling high uncertainty. Conversely, a narrow confidence interval around a meaningful estimate provides strong, actionable evidence.
Reporting confidence intervals alongside point estimates is now a standard requirement in medical research, social sciences, and data-driven industries — precisely because they communicate both the direction and the precision of an effect.
The formula for a confidence interval varies slightly depending on what population parameter you are estimating and what information is available from your sample. The two most common cases are estimating a population mean and estimating a population proportion.
Confidence Interval for a Population Mean
When estimating a population mean, the formula is:
Where:
This version of the formula applies when the population standard deviation () is known and the sample size is large (typically n). The term is the standard error (SE), which measures how much the sample mean is expected to vary from the true population mean.
When the population standard deviation is unknown — which is the more common situation in practice — the sample standard deviation (s) is used in its place, and the critical value is drawn from the t-distribution rather than the Z-distribution:
Where:
Confidence Interval for a Population Proportion
When the parameter of interest is a proportion rather than a mean — for example, the percentage of voters who support a policy — the formula is:
Where:
The expression is the standard error of the proportion, playing the same role as does in the mean formula.
Critical Values for Common Confidence Levels
The critical value is determined by the confidence level and is read from the standard normal (Z) distribution. The critical value depends on both the confidence level and the degrees of freedom ().
For large samples, the most frequently used Z critical values are:
| Confidence Level | Alpha () | |
|---|---|---|
| 90% | 0.10 | 1.645 |
| 95% | 0.05 | 1.960 |
| 99% | 0.01 | 2.576 |
For smaller samples where the t-distribution applies, values vary by degrees of freedom and must be looked up in a t-distribution table or calculated using statistical software.
The Margin of Error
The margin of error (MOE) is the second half of the confidence interval formula — everything to the right of the sign:
The margin of error captures the combined influence of the confidence level, the variability in the data, and the sample size. A wider margin of error produces a wider interval, reflecting greater uncertainty. A narrower margin of error produces a tighter interval, reflecting greater precision.
Three factors directly control the width of a confidence interval:
Confidence level — Increasing the confidence level raises the critical value , which widens the interval. A 99% CI is always wider than a 95% CI calculated from the same data.
Sample size — Increasing n reduces the standard error , which narrows the interval. Larger samples produce more precise estimates.
Variability — Higher standard deviation in the data increases the standard error, which widens the interval. More variable data produces less precise estimates.
A researcher measures the resting heart rate of 40 randomly selected adults. The sample produces a mean of 72 beats per minute (bpm). The population standard deviation is known to be 8 bpm. Calculate a 95% confidence interval for the true population mean.
Given:
Step 1: Identify the point estimate
The point estimate is the sample mean:
Step 2: Calculate the standard error
Step 3: Find the critical value
For a 95% confidence level using the Z-distribution:
Step 4: Calculate the margin of error
Step 5: Construct the confidence interval
Interpretation: Based on this sample, you can be 95% confident that the true mean resting heart rate of the adult population falls between 69.52 bpm and 74.48 bpm.
A quality control analyst tests 15 randomly selected bolts and records their diameters in millimeters. The sample mean diameter is 10.4 mm and the sample standard deviation is 0.6 mm. Calculate a 95% confidence interval for the true population mean diameter.
Given:
Step 1: Identify the point estimate
Step 2: Calculate the standard error
Step 3: Find the critical value
Because σ is unknown and n<30, use the t-distribution with df=14:
Step 4: Calculate the margin of error
Step 5: Construct the confidence interval
Interpretation: Based on this sample, you can be 95% confident that the true mean bolt diameter falls between 10.07 mm and 10.73 mm.
A marketing team surveys 200 customers and finds that 134 would recommend the product to a friend. Calculate a 95% confidence interval for the true proportion of customers who would recommend the product.
Given:
Step 1: Identify the point estimate
Step 2: Verify the normal approximation conditions
Before applying the Z-based proportion formula, confirm that:
Both conditions are satisfied, so the normal approximation is appropriate.
Step 3: Calculate the standard error
Step 4: Find the critical value
Step 5: Calculate the margin of error
Step 6: Construct the confidence interval
Interpretation: Based on this survey, you can be 95% confident that between 60% and 74% of all customers would recommend the product to a friend.
| Step | Mean (SD Known) | Mean (SD Unknown) | Proportion |
|---|---|---|---|
| 1 | Identify | Identify | Calculate |
| 2 | Calculate | Calculate | Verify normal approximation |
| 3 | Find | Find using | Calculate |
| 4 | Calculate | Calculate | Find |
| 5 | Construct CI | Construct CI | Calculate |
| 6 | — | — | Construct CI |
Calculating a confidence interval is only half the work. Interpreting it correctly is equally important — and it is where many students and researchers go wrong. Confidence intervals are frequently misread, even in published research. This section clarifies what a confidence interval does and does not tell you.
The Correct Interpretation
The formal interpretation of a confidence interval follows this structure:
“We are [confidence level]% confident that the true population [parameter] falls between [lower bound] and [upper bound].”
Using the heart rate example from the previous section:
“We are 95% confident that the true mean resting heart rate of the adult population falls between 69.52 bpm and 74.48 bpm.”
This phrasing is precise because it reflects what a confidence level actually represents: the long-run reliability of the estimation method, not a probability statement about a single interval.
What a Confidence Interval Does Not Mean
Several common misinterpretations persist across textbooks, classrooms, and research reports. Each one reflects a subtle but meaningful misunderstanding of the underlying concept.
Misconception 1: “There is a 95% probability that the true parameter lies within this interval.”
This is the most widespread misinterpretation. Once a confidence interval has been calculated from a specific sample, the true population parameter is fixed — it either falls inside the interval or it does not. There is no probability involved at that point. The 95% refers to the proportion of all intervals, constructed using the same method across repeated samples, that would contain the true parameter.
Misconception 2: “95% of the data values fall within this interval.”
A confidence interval is not a description of the data distribution. It is an estimate of where the population parameter lies. The spread of individual data points is described by the standard deviation, not the confidence interval.
Misconception 3: “A wider confidence interval means the results are unreliable and should be discarded.”
Width reflects uncertainty, not failure. A wide interval simply means the estimate is less precise — often due to a small sample size or high variability in the data. It is still a valid and informative result. Reporting a wide interval honestly is preferable to artificially narrowing it by increasing the confidence level or cherry-picking data.
Misconception 4: “If two confidence intervals overlap, the difference between groups is not statistically significant.”
Overlapping confidence intervals do not automatically indicate a non-significant difference. Formal hypothesis testing is required to determine statistical significance when comparing two groups. Overlap is suggestive, but not conclusive.
Confidence Level vs. Interval Width
The relationship between confidence level and interval width is direct and predictable. Holding sample size and variability constant:
| Confidence Level | Critical Value () | Interval Width | Precision |
|---|---|---|---|
| 90% | 1.645 | Narrowest | Highest |
| 95% | 1.960 | Moderate | Moderate |
| 99% | 2.576 | Widest | Lowest |
Choosing a higher confidence level does not make your estimate more accurate — it makes it less precise. A 99% confidence interval is wider because it must capture the true parameter under a broader range of possible samples. The tradeoff between confidence and precision is a fundamental consideration in study design.
Practical Significance vs. Statistical Significance
A confidence interval communicates more than a p-value alone because it conveys the magnitude of an effect, not just whether one exists. Consider two hypothetical study results comparing a new drug to a placebo for reducing systolic blood pressure:
| Study | 95% Confidence Interval | Conclusion |
|---|---|---|
| Study A | [0.1 mmHg, 0.9 mmHg] | Statistically significant, but clinically trivial |
| Study B | [8.2 mmHg, 14.6 mmHg] | Statistically significant and clinically meaningful |
| Study C | [−1.2 mmHg, 11.4 mmHg] | Includes zero — effect uncertain |
Study A shows a real but negligible effect. Study B demonstrates a result large enough to matter in practice. Study C cannot rule out the possibility of no effect at all, since its interval crosses zero. None of this information would be apparent from a p-value alone.
When a confidence interval for a difference includes zero (for means) or includes one (for ratios such as odds ratios or relative risks), it indicates that the null hypothesis value is plausible — which corresponds to a non-significant result at the chosen confidence level.
One-Sided Confidence Intervals
All examples discussed so far have used two-sided confidence intervals, which extend in both directions from the point estimate. In some situations, only one bound is of interest — for example, when a researcher wants to confirm that a mean is at least a certain value, or no greater than a certain threshold.
A one-sided confidence interval places the entire alpha (α) in one tail of the distribution rather than splitting it equally between both tails:
For a 95% one-sided interval, the critical value is rather than , because all 5% of alpha is concentrated in a single tail. One-sided intervals are narrower than their two-sided counterparts at the same confidence level, but they are only appropriate when there is a strong, pre-specified directional hypothesis.
Sample size has a direct and mathematically predictable effect on interval width. As sample size increases, the standard error decreases, which narrows the confidence interval and produces a more precise estimate.
The relationship is defined by the standard error formula:
Because n appears under a square root, the effect is not linear. To cut the standard error — and therefore the margin of error — in half, you must quadruple the sample size.
Consider a population with and a 95% confidence level ():
| Sample Size (n) | Standard Error | Margin of Error | CI Width |
|---|---|---|---|
| 25 | 2.000 | 3.920 | 7.840 |
| 100 | 1.000 | 1.960 | 3.920 |
| 400 | 0.500 | 0.980 | 1.960 |
| 1600 | 0.250 | 0.490 | 0.980 |
Each time the sample size quadruples, the CI width halves. This square root relationship has direct implications for study design: collecting more data always improves precision, but the returns diminish as sample size grows larger.
Variability — measured by the standard deviation — reflects how spread out individual values are in the population. Higher variability means greater uncertainty about where the true parameter lies, which produces a wider confidence interval.
This relationship is also captured in the standard error formula. For a fixed sample size of and a 95% confidence level:
| Standard Deviation (σ) | Standard Error | Margin of Error | CI Width |
|---|---|---|---|
| 5 | 0.707 | 1.386 | 2.772 |
| 10 | 1.414 | 2.771 | 5.542 |
| 20 | 2.828 | 5.543 | 11.086 |
| 40 | 5.657 | 11.087 | 22.174 |
Unlike sample size, variability is often not within the researcher’s control — it is a property of the population being studied. However, it can sometimes be reduced through careful study design. Strategies such as using homogeneous samples, standardizing measurement procedures, and controlling for extraneous variables all reduce unexplained variability and tighten confidence intervals.
The confidence level determines the critical value or used in the margin of error calculation. A higher confidence level demands a larger critical value, which directly widens the interval.
For a fixed dataset with , , and :
| Confidence Level | Critical Value (z∗) | Margin of Error | Confidence Interval |
|---|---|---|---|
| 90% | 1.645 | 2.549 | [47.45, 52.55] |
| 95% | 1.960 | 3.036 | [46.96, 53.04] |
| 99% | 2.576 | 3.991 | [46.01, 53.99] |
The tradeoff is fundamental: increasing confidence requires accepting less precision. A 99% confidence interval captures the true parameter more reliably across repeated samples, but it does so by providing a wider, less informative range. Researchers must choose a confidence level that balances the need for certainty against the need for a precise, actionable estimate.
When calculating a confidence interval for a proportion, the variability in the data is not a separate input — it is determined by the sample proportion itself, through the standard error formula:
The expression reaches its maximum value of 0.25 when , meaning that proportions near 50% produce the widest confidence intervals for a given sample size. Proportions closer to 0 or 1 produce narrower intervals because there is less variability in the binary outcome.
For a fixed sample size of and a 95% confidence level:
| Sample Proportion () | Standard Error | Margin of Error | CI Width |
|---|---|---|---|
| 0.10 | 0.0212 | 0.0416 | 0.0832 |
| 0.30 | 0.0324 | 0.0635 | 0.1270 |
| 0.50 | 0.0354 | 0.0693 | 0.1386 |
| 0.70 | 0.0324 | 0.0635 | 0.1270 |
| 0.90 | 0.0212 | 0.0416 | 0.0832 |
This symmetry around is a direct consequence of the formula’s structure. It also has practical implications for survey design: when the expected proportion is unknown, researchers conservatively assume to ensure the sample size is large enough to achieve the desired margin of error under the worst-case scenario.
In practice, sample size, variability, and confidence level act simultaneously, and changes to one factor can be used to offset another. The table below summarizes how each factor influences interval width:
| Factor | Change | Effect on CI Width |
|---|---|---|
| Sample size () | Increases | Narrows |
| Sample size () | Decreases | Widens |
| Variability ( or ) | Increases | Widens |
| Variability ( or ) | Decreases | Narrows |
| Confidence level | Increases | Widens |
| Confidence level | Decreases | Narrows |
| Proportion () | Moves toward 0.50 | Widens |
| Proportion () | Moves away from 0.50 | Narrows |
When designing a study, the most actionable lever is nearly always sample size. Variability is often fixed by the nature of the research question, and the confidence level is typically set by disciplinary convention. Collecting a sufficiently large sample is the most reliable way to achieve a confidence interval narrow enough to be practically useful.

Excel does not have a single function that outputs a complete confidence interval, but it provides all the building blocks needed to construct one efficiently. The approach depends on whether you are working with a known population standard deviation or estimating from sample data.
The CONFIDENCE.NORM Function (Known σ)
When the population standard deviation is known, Excel’s CONFIDENCE.NORM function calculates the margin of error directly.
Syntax:
=CONFIDENCE.NORM(alpha, standard_dev, size)
| Argument | Description |
|---|---|
alpha | The significance level, equal to 1 − confidence level. For a 95% CI, alpha = 0.05 |
standard_dev | The known population standard deviation (σ) |
size | The sample size (n) |
The function returns the margin of error only — not the full interval. You must add and subtract it from the sample mean manually.
Worked example using the heart rate dataset (x̄ = 72, σ = 8, n = 40, 95% confidence):
| Cell | Formula | Result |
|---|---|---|
| B1 | =AVERAGE(data_range) | 72.00 |
| B2 | =CONFIDENCE.NORM(0.05, 8, 40) | 2.479 |
| B3 | =B1 - B2 | 69.521 |
| B4 | =B1 + B2 | 74.479 |
The 95% confidence interval is [69.52, 74.48] — matching the manual calculation from the step-by-step section exactly.
The CONFIDENCE.T Function (Unknown σ)
When the population standard deviation is unknown and must be estimated from sample data, use CONFIDENCE.T, which applies the t-distribution rather than the Z-distribution.
Syntax:
=CONFIDENCE.T(alpha, standard_dev, size)
The arguments are identical to CONFIDENCE.NORM, except that standard_dev is now the sample standard deviation, calculated using STDEV.S.
Worked example using the bolt diameter dataset (x̄ = 10.4, s = 0.6, n = 15, 95% confidence):
| Cell | Formula | Result |
|---|---|---|
| B1 | =AVERAGE(data_range) | 10.400 |
| B2 | =STDEV.S(data_range) | 0.600 |
| B3 | =CONFIDENCE.T(0.05, B2, 15) | 0.332 |
| B4 | =B1 - B3 | 10.068 |
| B5 | =B1 + B3 | 10.732 |
The 95% confidence interval is [10.07, 10.73] — again consistent with the manual result.
Building a Reusable CI Calculator in Excel
Rather than recalculating from scratch each time, the layout below organizes all inputs and outputs into a clean, self-contained calculator. Enter values in the yellow input cells and the interval updates automatically.
| Cell | Label | Formula or Value |
|---|---|---|
| B2 | Sample mean | =AVERAGE(data_range) or enter manually |
| B3 | Standard deviation | =STDEV.S(data_range) or enter manually |
| B4 | Sample size | Enter manually |
| B5 | Confidence level | Enter as decimal, e.g. 0.95 |
| B6 | Alpha | =1 - B5 |
| B7 | Margin of error | =CONFIDENCE.T(B6, B3, B4) |
| B8 | Lower bound | =B2 - B7 |
| B9 | Upper bound | =B2 + B7 |
Switching between the Z-based and t-based interval requires only changing the formula in B7 — replace CONFIDENCE.T with CONFIDENCE.NORM when the population standard deviation is known.
Calculating a Confidence Interval for a Proportion in Excel
Excel has no dedicated function for proportion confidence intervals, but the formula is straightforward to implement directly.
Worked example using the customer survey dataset (p̂ = 0.67, n = 200, 95% confidence):
| Cell | Label | Formula | Result |
|---|---|---|---|
| B1 | Sample proportion | =134/200 | 0.670 |
| B2 | Sample size | 200 | 200 |
| B3 | Z critical value | =NORM.S.INV(0.975) | 1.960 |
| B4 | Standard error | =SQRT(B1*(1-B1)/B2) | 0.0332 |
| B5 | Margin of error | =B3*B4 | 0.0651 |
| B6 | Lower bound | =B1-B5 | 0.6049 |
| B7 | Upper bound | =B1+B5 | 0.7351 |
The NORM.S.INV(0.975) function returns the Z critical value for a 95% two-sided interval by finding the value that cuts off 2.5% in the upper tail of the standard normal distribution. For other confidence levels, adjust the argument: use NORM.S.INV(0.95) for a 90% CI and NORM.S.INV(0.995) for a 99% CI.
Choosing the Right Function
| Scenario | Correct Excel Function |
|---|---|
| Population SD known, large sample | CONFIDENCE.NORM |
| Population SD unknown, any sample size | CONFIDENCE.T |
| Proportion data | NORM.S.INV + manual SE formula |
| Need exact Z or t critical value | NORM.S.INV or T.INV.2T |
Note that CONFIDENCE (without a suffix) is a legacy function retained for compatibility with older Excel versions. Microsoft recommends using CONFIDENCE.NORM or CONFIDENCE.T in all current workbooks, as the legacy version may be unavailable in future releases. Full documentation for each function is available on the Microsoft Excel support page.
Clinical trials rely on confidence intervals to assess the effectiveness and safety of treatments. When researchers test a new drug, they report not just whether the drug worked, but by how much — and with what degree of certainty. A confidence interval around a treatment effect tells clinicians whether the observed benefit is large enough to be clinically meaningful, not merely statistically detectable.
For example, a trial comparing a new blood pressure medication to a placebo might report a mean reduction of 9.4 mmHg with a 95% CI of [6.8, 12.0]. This tells physicians that the treatment effect is both statistically significant (the interval excludes zero) and clinically relevant (even the lower bound represents a meaningful reduction in cardiovascular risk).
Regulatory agencies including the U.S. Food and Drug Administration (FDA) and the European Medicines Agency (EMA) require confidence intervals in drug approval submissions. Point estimates alone are insufficient for regulatory decisions; the interval communicates the precision of the evidence and helps evaluators assess whether the benefit-risk profile holds across a plausible range of true effects.
Epidemiologists use confidence intervals to quantify the strength of association between exposures and health outcomes. Key measures — including relative risk, odds ratios, and hazard ratios — are routinely reported alongside confidence intervals to communicate both the direction and the reliability of an association.
A study finding that a particular occupational exposure is associated with an odds ratio of 2.3 for developing a respiratory condition carries very different implications depending on whether the 95% CI is [1.8, 2.9] or [0.9, 5.8]. The first interval is narrow and excludes one, indicating a precise and statistically significant finding. The second is wide and straddles one, suggesting high uncertainty and a non-significant result despite the large point estimate.
Confidence intervals also play a central role in disease surveillance. When public health agencies report disease incidence rates or vaccination coverage estimates from survey data, they accompany these figures with confidence intervals to convey sampling uncertainty — a critical consideration when making decisions about resource allocation and intervention targets.
Political polls and public opinion surveys are among the most familiar applications of confidence intervals in everyday life, though the term used in media reporting is typically margin of error rather than confidence interval.
When a poll reports that 52% of respondents support a candidate with a margin of error of ±3 percentage points, this is equivalent to reporting a 95% confidence interval of [49%, 55%]. The margin of error is calculated using the proportion confidence interval formula:
For a sample of 1,000 respondents with a 50/50 split — the worst-case scenario for variance — the margin of error at 95% confidence is approximately ±3.1 percentage points. This explains why national polls consistently use samples of around 1,000: it is the point at which the margin of error reaches a level precise enough to be informative without requiring prohibitively large samples.
Survey researchers also use confidence intervals to compare subgroup estimates, assess whether observed differences between demographic groups are statistically meaningful, and track changes over time with appropriate acknowledgment of sampling uncertainty.
In business analytics and economics, confidence intervals support evidence-based decision-making by quantifying the uncertainty around estimates derived from sample data.
A/B testing — the practice of comparing two versions of a product, webpage, or marketing message — produces confidence intervals around the difference in conversion rates, click-through rates, or revenue per user. A product team running an experiment on a new checkout flow needs to know not just whether the new version outperforms the original, but by how much and with what certainty. A 95% CI of [+1.2%, +4.8%] on conversion rate improvement provides a range that supports a business case; a CI of [−0.5%, +6.5%] that straddles zero warrants caution.
Economic forecasting uses confidence intervals — sometimes called prediction intervals in this context — to communicate uncertainty around projections of GDP growth, inflation, unemployment, and other macroeconomic indicators. Central banks and government statistical agencies routinely publish fan charts, which are visual representations of widening confidence bands around a central forecast as the time horizon extends.
Quality control in manufacturing uses confidence intervals to monitor whether production processes are operating within specification. A manufacturer sampling bolt diameters, circuit board tolerances, or fill weights uses confidence intervals to determine whether the true process mean is within the acceptable range — and whether an observed deviation from target is a genuine shift or the result of normal sampling variation.
The publication standards of the American Psychological Association (APA) now require confidence intervals to be reported alongside test statistics and p-values in research articles. This requirement reflects a broader movement in psychology toward effect size reporting and away from binary significance decisions based on p-values alone.
In practice, confidence intervals allow researchers to assess whether an observed effect — a difference in memory performance between two groups, a correlation between stress and productivity, a change in attitude scores following an intervention — is large enough to be theoretically meaningful, not merely present. A statistically significant effect with a very narrow CI centered near zero indicates a real but negligible relationship; a wide CI around a large point estimate flags an intriguing but uncertain finding that warrants replication.
In machine learning, confidence intervals appear in model evaluation, uncertainty quantification, and experiment tracking.
When evaluating classifier performance, data scientists report metrics such as accuracy, precision, recall, and AUC alongside confidence intervals computed via bootstrapping or binomial proportion methods. A model achieving 91.4% accuracy with a 95% CI of [89.8%, 93.0%] can be compared meaningfully to a competing model at 92.1% [90.2%, 94.0%] — whereas comparing point estimates alone would suggest a difference that may fall entirely within sampling noise.
Bootstrapped confidence intervals are particularly common in machine learning because many modern metrics do not follow a simple parametric distribution. Bootstrapping repeatedly resamples the test dataset with replacement, computes the metric on each resample, and uses the resulting distribution of estimates to construct an empirical confidence interval — no distributional assumptions required.
In online experimentation platforms used by technology companies, confidence intervals are displayed in real time as experiments accumulate data, allowing teams to monitor whether treatment effects are stabilizing and when sufficient evidence has been collected to reach a decision.