hypothesis testing book

Testing Statistical Hypotheses

• © 2022
• Latest edition
• E.L. Lehmann 0 ,
• Joseph P. Romano 1

(Deceased) Department of Statistics, University of California, Berkeley, USA

You can also search for this author in PubMed   Google Scholar

Department of Statistics and Economics, Stanford University, Stanford, USA

• Develops the foundations, principles, theory, and methods of hypothesis testing
• Offers new coverage of multiple hypothesis testing, high-dimensional testing, permutation, and more
• Features over 100 new problems, bringing the total to approximately 900 problems across both volumes.

Part of the book series: Springer Texts in Statistics (STS)

98k Accesses

42 Citations

9 Altmetric

This is a preview of subscription content, log in via an institution to check access.

Access this book

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

About this book

Testing Statistical Hypotheses, 4 th Edition updates and expands upon the classic graduate text, now a two-volume work. The first volume covers finite-sample theory, while the second volume discusses large-sample theory. A definitive resource for graduate students and researchers alike, this work grows to include new topics of current relevance. New additions include an expanded treatment of multiple hypothesis testing, a new section on extensions of the Central Limit Theorem, coverage of high-dimensional testing, expanded discussions of permutation and randomization tests, coverage of testing moment inequalities, and many new problems throughout the text.

Similar content being viewed by others

Large Sample Theory: The Basics

High-dimensional simultaneous inference with the bootstrap, large sample results for likelihood-based methods.

• Statistical Hypotheses
• Statistical Theory

Table of contents (18 chapters)

Front matter, conditional inference, the general decision problem.

• E. L. Lehmann, Joseph P. Romano

The Probability Background

Uniformly most powerful tests, unbiasedness: theory and first applications, unbiasedness: applications to normal distributions; confidence intervals, linear hypotheses, the minimax principle, multiple testing and simultaneous inference, asymptotic theory, basic large-sample theory, extensions of the clt to sums of dependent random variables, applications to inference, quadratic mean differentiable families, large-sample optimality, testing goodness of fit, permutation and randomization tests, authors and affiliations.

E.L. Lehmann

Joseph P. Romano

About the authors

E.L. Lehmann (1917 – 2009) was an American statistician and professor of statistics at the University of California, Berkeley. He made significant contributions to nonparametric hypothesis testing, and he is one of the eponyms of the Lehmann-Scheffé theorem and of the Hodges-Lehmann estimator. Dr. Lehmann was a member of the National Academy of Sciences and the American Academy of Arts and Sciences, and the recipient of honorary degrees from the University of Leiden, The Netherlands and the University of Chicago. He was the author of Elements of Large-Sample Theory (Springer 1999) and Theory of Point Estimation, Second Edition (Springer 1998, with George Casella).

Joseph P. Romano  has been on faculty in the Statistics Department at Stanford since 1986. Since 2007, he has held a joint professorship appointment in both Statistics and Economics. He is a coauthor of three books, as well as over 100 journal articles. Dr. Romano was named NOGLSTP's 2021 LGBTQ+ Scientist of the Year, has been a recipient of the Presidential Young Investigator Award and many other grants from the National Science Foundation, and is a Fellow of the Institute of Mathematical Statistics and of the International Association of Applied Econometrics. His research has focused on such topics as: bootstrap and resampling methods, subsampling, randomization methods, inference, optimality, large-sample theory, nonparametrics, multiple hypothesis testing, and econometrics. He has invented or co-invented a variety of new statistical methods, including subsampling and the stationary bootstrap, as well as methods for multiple hypothesis testing. These methods have been applied to such diverse fields as clinical trials, climate change, finance, and economics.

Bibliographic Information

Book Title : Testing Statistical Hypotheses

Authors : E.L. Lehmann, Joseph P. Romano

Series Title : Springer Texts in Statistics

DOI : https://doi.org/10.1007/978-3-030-70578-7

Publisher : Springer Cham

eBook Packages : Mathematics and Statistics , Mathematics and Statistics (R0)

Copyright Information : The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022

Hardcover ISBN : 978-3-030-70577-0 Published: 24 June 2022

Softcover ISBN : 978-3-030-70580-0 Published: 25 June 2023

eBook ISBN : 978-3-030-70578-7 Published: 22 June 2022

Series ISSN : 1431-875X

Series E-ISSN : 2197-4136

Edition Number : 4

Number of Pages : XV, 1012

Number of Illustrations : 6 b/w illustrations, 7 illustrations in colour

Topics : Statistical Theory and Methods , Probability Theory and Stochastic Processes , Statistics, general

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

New eBook Release! Hypothesis Testing: An Intuitive Guide

By Jim Frost 14 Comments

I’m thrilled to release my new book! Hypothesis Testing: An Intuitive Guide for Making Data Driven Decisions .

In today’s data-driven world, we hear about making decisions based on the data all the time. Hypothesis testing plays a crucial role in that process, whether you’re in academia, making business decisions, or in quality improvement. Without hypothesis tests, you risk drawing the wrong conclusions and making bad decisions. That can be costly, either in business dollars or for your reputation as an analyst or scientist.

Painlessly learn how to use these tests in this 367-page book! If you like the clear writing style I use on my website, you’ll love this book! Throughout this book, I use the same clear, concise language. I focus on helping you grasp key concepts, methodologies, and procedures while deemphasizing equations.

Hypothesis tests allow you to use sample data to draw conclusions about an entire population—not just the small sample with which you’re working. Consequently, these tests play a vital role in making discoveries in science, making decisions based on data, and making predictions. Additionally, given the growing importance of decisions and opinions based on data, your ability to critically assess the quality of analyses that others present to you is more crucial than ever.

By reading this book, you will build a solid foundation for understanding hypothesis tests and become confident that you know when to use each type of test, how to use them properly to obtain reliable results, and how to interpret the results correctly. I present a wide variety of tests that assess characteristics of different data types. Chances are high that you’ll need a working knowledge of hypothesis testing to produce new findings yourself and to understand the work of others. The world today produces more analyses designed to influence you than ever before. Are you ready for it?

To accomplish these goals, I teach you how these tests work using an intuitive approach, which helps you fully understand the results. At the end of this post, you’ll find the full table of contents.

Buy it on Amazon (US site)! ! Or go to my Web Store for other locations.

My Book Covers the Following Critical Hypothesis Testing Concepts, Methods, and Skills

This book enables you to build the skills and knowledge necessary for effective hypothesis testing, including the following:

• Why you need hypothesis tests and how they work.
• Using significance levels, p-values, confidence intervals.
• Select the correct type of hypothesis test to answer your question.
• Learn how to test means, medians, variances, proportions, distributions, counts, correlations for continuous and categorical data, and outliers.
• One-Way ANOVA, Two-Way ANOVA and interaction effects.
• Interpreting the results.
• Checking assumptions and obtaining reliable results.
• Manage the error rates for false positives and false negatives.
• Understand sampling distributions, central limit theorem, and statistical power.
• Know how t-tests, F-tests, chi-squared, and post hoc tests work.
• Learn about the differences between parametric, nonparametric, and bootstrapping methods.
• Examples of different types of hypothesis tests.
• Downloadable datasets so you can try it yourself.

For each hypothesis test I cover, you will learn what it tells you, understand its assumptions, know how to interpret the results, and work through examples with downloadable datasets.

Please consider buying my book and learn about hypothesis testing! I’m sure you’ll enjoy it and find it helpful! The full table of contents is below.

Share this:

Reader Interactions

April 26, 2024 at 2:44 pm

Thank you for the quick response! The thing that confuses me is the requirement for independent groups. For example, one or more people might be coincidentally selected in both polls.

April 26, 2024 at 3:12 pm

You bet, Travis!

Accidentally including the same people in both samples is a potential concern that depends on the size of your sample relative to the size of the population you’re sampling.

We do have to assume that you’re drawing two random samples from the same population to use the test. If a few people happen to be in both samples by chance, that’s technically an assumption violation but it won’t affect the results much. However, if there are many in both samples, it is a problem.

If you’re drawing two random samples of 100 out of a town of 200, you’d expect 25 people to both samples by chance. That’s too many. However, if you’re drawing two random samples of 100 out of 1000, you’d expect 1 person to be in both samples by chance. That should be OK. I don’t know of any guidelines surrounding that to provide but 1 person in a total of 200 in both samples can’t affect the results too much. But you really need to understand the likelihood of people being in both samples. Of course, if you know the identities of the respondents, you can remove those that are in both samples.

April 26, 2024 at 1:19 pm

This is an amazing book! However, I do a have question that I haven’t been able to answer. Imagine that I conduct a poll of the favorite pet of people in my town, with the following results:

Dog – 25% Cat – 25% Horse – 10% Other – 40% Margin of error: +/- 3%

After the poll, two things happen. First, several people that own horses move to town. Second, a nonprofit organization launches a campaign to adopt cats. So, I conduct another poll with these results:

Dog – 24% Cat – 30% Horse – 20% Other – 26% Margin of error: +/- 2.8%

Here’s my question: What would be the appropriate test to see if there is a statistically significant difference between “Cat” from the first poll to the second (i.e., 25% to 30%)? Please note: I’m not interested in the difference for “Dog,” “Horse,” or “Other” in this example – just “Cat.” Thank you in advance!

April 26, 2024 at 2:12 pm

I’m glad you enjoyed the book and that it was helpful!

You could perform a 2-sample proportions test to compare the proportion of “cat” responses in the first and second poll. If the results are statistically significant, then you can conclude that the preference for cats has changed between the two polls.

I go over the 2-proportions test in my Hypothesis Testing book, so look at that for more information!

July 23, 2021 at 1:46 am

Thanks a lot Jim. That was helpful !!

July 21, 2021 at 9:04 pm

Thanks a lot Jim. That was really helpful.

Being new to this kind of analysis, sometimes I face ambiguity with the statistical language. We draw two samples from the same population and then check for the difference in means and then we conclude that the difference is statistically significant at the population level. So indirectly aren’t we saying that the two samples are coming from two different populations? Seems a bit ambiguous to understand it intuitively.

July 21, 2021 at 11:02 pm

Hi Animesh,

In your case, if you’re using independent samples, you’d be drawing two random samples from the same population. So, you’d have two samples but they’re representing one population but at different points in time (in your case).

July 18, 2021 at 3:27 am

For a stock market index (lets say S&P 500), if I calculate returns for a particular period (lets say each month for the period 2012 to 2014 i.e. a sample of 36 months ) and then I calculate returns for another period (lets say each month for the period 2015 to 2017 i.e. another sample of 36 months). Now , I need to understand the difference between the mean monthly returns of these two samples of 36 months each. I need to understand whether the difference is statistically significant. Each period is a sample drawn from the larger population of monthly returns of S&P 500. So, for Hypothesis testing can I assume that each sample is drawn from the population of monthly returns of S&P 500? In other words , each sample is coming from the same population . Or do , I need to assume that each sample is coming from a different population ? First sample from population of monthly returns for the period 2012 to 2014 and second sample from the population of monthly returns for the period 2015 to 2017. Could you please clarify this point ? Thanks.

July 19, 2021 at 10:13 pm

I haven’t done stock market research like that so I don’t know if there’s a standard they use for that type of research. However, I’d imagine you can assume it’s the same population for the two time periods. If you’re using independent samples (different companies), you’d use a 2-sample t-test. Alternatively, you could draw one random sample at the first time period and then reassess the same group of companies during the 2nd time period. In that case, you’d use a paired t-test to determine whether the change in values was significantly different from zero (or other value you set).

I hope that helps!

April 29, 2021 at 8:47 pm

Hi Jim, I have purchased you boos intro to stats which I find very useful in plain English. I plan to do so for the other tow but I have a question about Hypothesis Testing. Some big statistician like Dr. Wheeler asserts that Hypothesis Testing can;t be use in real production or manufacturing environment as the assumption of normal data does not exist, Do yuo agree with his work, if you are familiar with his.

Thanks in advance

April 30, 2021 at 12:19 am

Hi Mohamed,

Thanks so much for buying my Introduction to Statistics book. I’m glad you found it to be helpful! 🙂

I disagree with the idea that you can’t perform hypothesis testing on non-normal data. There are various hypothesis testing methods available for non-normal data. For one thing, with a sample size of only 20-30 per group, parametric tests become robust to non-normal data. Additionally, you can use transformations to make the data normal. Alternatively, you can use nonparametric tests and bootstrapping methods to perform hypothesis tests with non-normal data. There are a variety of methods to analyze non-normal data. I cover those methods in my Hypothesis Testing book.

I have not read Dr. Wheeler’s work so I can’t say if I agree or disagree in general. But, I do say that you can use hypothesis testing with non-normal data!

April 7, 2021 at 7:04 pm

Do you plan to sell this book as a paperback as well?

April 7, 2021 at 10:12 pm

Yes! It’s currently available as a paperback. Go to My Web Store to get the Amazon links. It’s also available at other online retailers. Some physical bookstores can order it as well!

Thanks for asking!

September 9, 2020 at 12:31 pm

Thanks for writing this blog

Comments and Questions Cancel reply

• Comprehensive Learning Paths
• 150+ Hours of Videos
• Complete Access to Jupyter notebooks, Datasets, References.

Hypothesis Testing – A Deep Dive into Hypothesis Testing, The Backbone of Statistical Inference

• September 21, 2023

Explore the intricacies of hypothesis testing, a cornerstone of statistical analysis. Dive into methods, interpretations, and applications for making data-driven decisions.

In this Blog post we will learn:

• What is Hypothesis Testing?
• Steps in Hypothesis Testing 2.1. Set up Hypotheses: Null and Alternative 2.2. Choose a Significance Level (α) 2.3. Calculate a test statistic and P-Value 2.4. Make a Decision
• Example : Testing a new drug.
• Example in python

1. What is Hypothesis Testing?

In simple terms, hypothesis testing is a method used to make decisions or inferences about population parameters based on sample data. Imagine being handed a dice and asked if it’s biased. By rolling it a few times and analyzing the outcomes, you’d be engaging in the essence of hypothesis testing.

Think of hypothesis testing as the scientific method of the statistics world. Suppose you hear claims like “This new drug works wonders!” or “Our new website design boosts sales.” How do you know if these statements hold water? Enter hypothesis testing.

2. Steps in Hypothesis Testing

• Set up Hypotheses : Begin with a null hypothesis (H0) and an alternative hypothesis (Ha).
• Choose a Significance Level (α) : Typically 0.05, this is the probability of rejecting the null hypothesis when it’s actually true. Think of it as the chance of accusing an innocent person.
• Calculate Test statistic and P-Value : Gather evidence (data) and calculate a test statistic.
• p-value : This is the probability of observing the data, given that the null hypothesis is true. A small p-value (typically ≤ 0.05) suggests the data is inconsistent with the null hypothesis.
• Decision Rule : If the p-value is less than or equal to α, you reject the null hypothesis in favor of the alternative.

2.1. Set up Hypotheses: Null and Alternative

Before diving into testing, we must formulate hypotheses. The null hypothesis (H0) represents the default assumption, while the alternative hypothesis (H1) challenges it.

For instance, in drug testing, H0 : “The new drug is no better than the existing one,” H1 : “The new drug is superior .”

2.2. Choose a Significance Level (α)

When You collect and analyze data to test H0 and H1 hypotheses. Based on your analysis, you decide whether to reject the null hypothesis in favor of the alternative, or fail to reject / Accept the null hypothesis.

The significance level, often denoted by $α$, represents the probability of rejecting the null hypothesis when it is actually true.

In other words, it’s the risk you’re willing to take of making a Type I error (false positive).

Type I Error (False Positive) :

• Symbolized by the Greek letter alpha (α).
• Occurs when you incorrectly reject a true null hypothesis . In other words, you conclude that there is an effect or difference when, in reality, there isn’t.
• The probability of making a Type I error is denoted by the significance level of a test. Commonly, tests are conducted at the 0.05 significance level , which means there’s a 5% chance of making a Type I error .
• Commonly used significance levels are 0.01, 0.05, and 0.10, but the choice depends on the context of the study and the level of risk one is willing to accept.

Example : If a drug is not effective (truth), but a clinical trial incorrectly concludes that it is effective (based on the sample data), then a Type I error has occurred.

Type II Error (False Negative) :

• Symbolized by the Greek letter beta (β).
• Occurs when you accept a false null hypothesis . This means you conclude there is no effect or difference when, in reality, there is.
• The probability of making a Type II error is denoted by β. The power of a test (1 – β) represents the probability of correctly rejecting a false null hypothesis.

Example : If a drug is effective (truth), but a clinical trial incorrectly concludes that it is not effective (based on the sample data), then a Type II error has occurred.

Balancing the Errors :

In practice, there’s a trade-off between Type I and Type II errors. Reducing the risk of one typically increases the risk of the other. For example, if you want to decrease the probability of a Type I error (by setting a lower significance level), you might increase the probability of a Type II error unless you compensate by collecting more data or making other adjustments.

It’s essential to understand the consequences of both types of errors in any given context. In some situations, a Type I error might be more severe, while in others, a Type II error might be of greater concern. This understanding guides researchers in designing their experiments and choosing appropriate significance levels.

2.3. Calculate a test statistic and P-Value

Test statistic : A test statistic is a single number that helps us understand how far our sample data is from what we’d expect under a null hypothesis (a basic assumption we’re trying to test against). Generally, the larger the test statistic, the more evidence we have against our null hypothesis. It helps us decide whether the differences we observe in our data are due to random chance or if there’s an actual effect.

P-value : The P-value tells us how likely we would get our observed results (or something more extreme) if the null hypothesis were true. It’s a value between 0 and 1. – A smaller P-value (typically below 0.05) means that the observation is rare under the null hypothesis, so we might reject the null hypothesis. – A larger P-value suggests that what we observed could easily happen by random chance, so we might not reject the null hypothesis.

2.4. Make a Decision

Relationship between $α$ and P-Value

When conducting a hypothesis test:

We then calculate the p-value from our sample data and the test statistic.

Finally, we compare the p-value to our chosen $α$:

• If $p−value≤α$: We reject the null hypothesis in favor of the alternative hypothesis. The result is said to be statistically significant.
• If $p−value>α$: We fail to reject the null hypothesis. There isn’t enough statistical evidence to support the alternative hypothesis.

3. Example : Testing a new drug.

Imagine we are investigating whether a new drug is effective at treating headaches faster than drug B.

Setting Up the Experiment : You gather 100 people who suffer from headaches. Half of them (50 people) are given the new drug (let’s call this the ‘Drug Group’), and the other half are given a sugar pill, which doesn’t contain any medication.

• Set up Hypotheses : Before starting, you make a prediction:
• Null Hypothesis (H0): The new drug has no effect. Any difference in healing time between the two groups is just due to random chance.
• Alternative Hypothesis (H1): The new drug does have an effect. The difference in healing time between the two groups is significant and not just by chance.

Calculate Test statistic and P-Value : After the experiment, you analyze the data. The “test statistic” is a number that helps you understand the difference between the two groups in terms of standard units.

For instance, let’s say:

• The average healing time in the Drug Group is 2 hours.
• The average healing time in the Placebo Group is 3 hours.

The test statistic helps you understand how significant this 1-hour difference is. If the groups are large and the spread of healing times in each group is small, then this difference might be significant. But if there’s a huge variation in healing times, the 1-hour difference might not be so special.

Imagine the P-value as answering this question: “If the new drug had NO real effect, what’s the probability that I’d see a difference as extreme (or more extreme) as the one I found, just by random chance?”

For instance:

• P-value of 0.01 means there’s a 1% chance that the observed difference (or a more extreme difference) would occur if the drug had no effect. That’s pretty rare, so we might consider the drug effective.
• P-value of 0.5 means there’s a 50% chance you’d see this difference just by chance. That’s pretty high, so we might not be convinced the drug is doing much.
• If the P-value is less than ($α$) 0.05: the results are “statistically significant,” and they might reject the null hypothesis , believing the new drug has an effect.
• If the P-value is greater than ($α$) 0.05: the results are not statistically significant, and they don’t reject the null hypothesis , remaining unsure if the drug has a genuine effect.

4. Example in python

For simplicity, let’s say we’re using a t-test (common for comparing means). Let’s dive into Python:

Making a Decision : “The results are statistically significant! p-value < 0.05 , The drug seems to have an effect!” If not, we’d say, “Looks like the drug isn’t as miraculous as we thought.”

5. Conclusion

Hypothesis testing is an indispensable tool in data science, allowing us to make data-driven decisions with confidence. By understanding its principles, conducting tests properly, and considering real-world applications, you can harness the power of hypothesis testing to unlock valuable insights from your data.

More Articles

F statistic formula – explained, correlation – connecting the dots, the role of correlation in data analysis, sampling and sampling distributions – a comprehensive guide on sampling and sampling distributions, law of large numbers – a deep dive into the world of statistics, central limit theorem – a deep dive into central limit theorem and its significance in statistics, similar articles, complete introduction to linear regression in r, how to implement common statistical significance tests and find the p value, logistic regression – a complete tutorial with examples in r.

Subscribe to Machine Learning Plus for high value data science content

© Machinelearningplus. All rights reserved.

Machine Learning A-Z™: Hands-On Python & R In Data Science

Free sample videos:.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Hypothesis Testing | A Step-by-Step Guide with Easy Examples

Published on November 8, 2019 by Rebecca Bevans . Revised on June 22, 2023.

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics . It is most often used by scientists to test specific predictions, called hypotheses, that arise from theories.

There are 5 main steps in hypothesis testing:

• State your research hypothesis as a null hypothesis and alternate hypothesis (H o ) and (H a  or H 1 ).
• Collect data in a way designed to test the hypothesis.
• Perform an appropriate statistical test .
• Decide whether to reject or fail to reject your null hypothesis.
• Present the findings in your results and discussion section.

Though the specific details might vary, the procedure you will use when testing a hypothesis will always follow some version of these steps.

Table of contents

Step 1: state your null and alternate hypothesis, step 2: collect data, step 3: perform a statistical test, step 4: decide whether to reject or fail to reject your null hypothesis, step 5: present your findings, other interesting articles, frequently asked questions about hypothesis testing.

After developing your initial research hypothesis (the prediction that you want to investigate), it is important to restate it as a null (H o ) and alternate (H a ) hypothesis so that you can test it mathematically.

The alternate hypothesis is usually your initial hypothesis that predicts a relationship between variables. The null hypothesis is a prediction of no relationship between the variables you are interested in.

• H 0 : Men are, on average, not taller than women. H a : Men are, on average, taller than women.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

For a statistical test to be valid , it is important to perform sampling and collect data in a way that is designed to test your hypothesis. If your data are not representative, then you cannot make statistical inferences about the population you are interested in.

There are a variety of statistical tests available, but they are all based on the comparison of within-group variance (how spread out the data is within a category) versus between-group variance (how different the categories are from one another).

If the between-group variance is large enough that there is little or no overlap between groups, then your statistical test will reflect that by showing a low p -value . This means it is unlikely that the differences between these groups came about by chance.

Alternatively, if there is high within-group variance and low between-group variance, then your statistical test will reflect that with a high p -value. This means it is likely that any difference you measure between groups is due to chance.

Your choice of statistical test will be based on the type of variables and the level of measurement of your collected data .

• an estimate of the difference in average height between the two groups.
• a p -value showing how likely you are to see this difference if the null hypothesis of no difference is true.

Based on the outcome of your statistical test, you will have to decide whether to reject or fail to reject your null hypothesis.

In most cases you will use the p -value generated by your statistical test to guide your decision. And in most cases, your predetermined level of significance for rejecting the null hypothesis will be 0.05 – that is, when there is a less than 5% chance that you would see these results if the null hypothesis were true.

In some cases, researchers choose a more conservative level of significance, such as 0.01 (1%). This minimizes the risk of incorrectly rejecting the null hypothesis ( Type I error ).

The results of hypothesis testing will be presented in the results and discussion sections of your research paper , dissertation or thesis .

In the results section you should give a brief summary of the data and a summary of the results of your statistical test (for example, the estimated difference between group means and associated p -value). In the discussion , you can discuss whether your initial hypothesis was supported by your results or not.

In the formal language of hypothesis testing, we talk about rejecting or failing to reject the null hypothesis. You will probably be asked to do this in your statistics assignments.

However, when presenting research results in academic papers we rarely talk this way. Instead, we go back to our alternate hypothesis (in this case, the hypothesis that men are on average taller than women) and state whether the result of our test did or did not support the alternate hypothesis.

If your null hypothesis was rejected, this result is interpreted as “supported the alternate hypothesis.”

These are superficial differences; you can see that they mean the same thing.

You might notice that we don’t say that we reject or fail to reject the alternate hypothesis . This is because hypothesis testing is not designed to prove or disprove anything. It is only designed to test whether a pattern we measure could have arisen spuriously, or by chance.

If we reject the null hypothesis based on our research (i.e., we find that it is unlikely that the pattern arose by chance), then we can say our test lends support to our hypothesis . But if the pattern does not pass our decision rule, meaning that it could have arisen by chance, then we say the test is inconsistent with our hypothesis .

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Descriptive statistics
• Measures of central tendency
• Correlation coefficient

Methodology

• Cluster sampling
• Stratified sampling
• Types of interviews
• Cohort study
• Thematic analysis

Research bias

• Implicit bias
• Cognitive bias
• Survivorship bias
• Availability heuristic
• Nonresponse bias
• Regression to the mean

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics. It is used by scientists to test specific predictions, called hypotheses , by calculating how likely it is that a pattern or relationship between variables could have arisen by chance.

A hypothesis states your predictions about what your research will find. It is a tentative answer to your research question that has not yet been tested. For some research projects, you might have to write several hypotheses that address different aspects of your research question.

A hypothesis is not just a guess — it should be based on existing theories and knowledge. It also has to be testable, which means you can support or refute it through scientific research methods (such as experiments, observations and statistical analysis of data).

Null and alternative hypotheses are used in statistical hypothesis testing . The null hypothesis of a test always predicts no effect or no relationship between variables, while the alternative hypothesis states your research prediction of an effect or relationship.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bevans, R. (2023, June 22). Hypothesis Testing | A Step-by-Step Guide with Easy Examples. Scribbr. Retrieved August 26, 2024, from https://www.scribbr.com/statistics/hypothesis-testing/

Is this article helpful?

Rebecca Bevans

Other students also liked, choosing the right statistical test | types & examples, understanding p values | definition and examples, what is your plagiarism score.

• Higher Education Textbooks
• Science & Mathematics

Sorry, there was a problem.

Download the free Kindle app and start reading Kindle books instantly on your smartphone, tablet or computer – no Kindle device required .

Read instantly on your browser with Kindle for Web.

Using your mobile phone camera, scan the code below and download the Kindle app.

Image Unavailable

• To view this video download Flash Player

Follow the author

Hypothesis Testing: An Intuitive Guide for Making Data Driven Decisions Paperback – 17 September 2020

Save extra with 3 offers.

• Free Delivery

10 days Replacement

• Amazon Delivered
• Pay on Delivery
• Secure transaction

Replacement Instructions

Purchase options and add-ons

• ISBN-10 173543115X
• ISBN-13 978-1735431154
• Publication date 17 September 2020
• Language English
• Dimensions 15.24 x 2.18 x 22.86 cm
• Print length 382 pages
• See all details

Frequently bought together

Customers who viewed this item also viewed

Product description

About the author, product details.

• Publisher ‏ : ‎ Statistics by Jim Publishing (17 September 2020)
• Language ‏ : ‎ English
• Paperback ‏ : ‎ 382 pages
• ISBN-10 ‏ : ‎ 173543115X
• ISBN-13 ‏ : ‎ 978-1735431154
• Reading age ‏ : ‎ 13 - 18 years
• Item Weight ‏ : ‎ 508 g
• Dimensions ‏ : ‎ 15.24 x 2.18 x 22.86 cm
• Country of Origin ‏ : ‎ India
• #22 in Statistics
• #49 in Sociology (Books)
• #107 in Computer Science Books

About the author

Jim Frost has extensive experience using statistical analysis in academic research and consulting projects. He’s been performing statistical analysis on-the-job for over 20 years. For 10 of those years, he was at a statistical software company helping others make the most out of their data. Jim loves sharing the joy of statistics. In addition to writing books, he has a statistics website and writes a regular column for the American Society of Quality's Statistics Digest.

Customer reviews

• 5 star 4 star 3 star 2 star 1 star 5 star 73% 16% 4% 1% 5% 73%
• 5 star 4 star 3 star 2 star 1 star 4 star 73% 16% 4% 1% 5% 16%
• 5 star 4 star 3 star 2 star 1 star 3 star 73% 16% 4% 1% 5% 4%
• 5 star 4 star 3 star 2 star 1 star 2 star 73% 16% 4% 1% 5% 1%
• 5 star 4 star 3 star 2 star 1 star 1 star 73% 16% 4% 1% 5% 5%
• Sort reviews by Top reviews Most recent Top reviews

Buy for others

Buying and sending ebooks to others.

• Select quantity
• Buy and send eBooks
• Recipients can read on any device

These ebooks can only be redeemed by recipients in the US. Redemption links and eBooks cannot be resold.

Sorry, there was a problem.

Image unavailable.

• To view this video download Flash Player

Follow the author

Introduction to Hypothesis Testing (Easy Statistics) [Print Replica] Kindle Edition

This book covers most important and widely used four significant tests: Z-test, T-test, F-test and Chi- Square test. Under the above main topics below subtopics are also discussed in this book. • One sample Z test on mean and proportion • Two sample Z test on mean and proportion • One sample T test • Independent sample T test • Paired T test • One-way ANOVA • Two way ANOVA • Chi Square Test • Independence of two variables • Goodness of Fit A set of questions and answers for all topics are presented at the end of each topic

Anushabooks.com.

• Part of series Easy Statistics
• Language English
• Sticky notes Not Enabled
• Publication date October 11, 2020
• File size 6144 KB
• Page Flip Not Enabled
• Word Wise Not Enabled
• Enhanced typesetting Not Enabled
• See all details

Fire Tablets

• Fire HD 8 (8th Generation)
• Fire 7 (9th Generation)
• Fire HD 10 (9th Generation)
• Fire HD 8 (10th Generation)
• Fire HD 10 (11th Generation)
• Fire HD 10 Plus
• Fire 7 (12th Generation)
• Fire HD 8 (12th Generation)
• Fire HD 8 Plus

Free Kindle Reading Apps

• Kindle for Android Phones
• Kindle for Android Tablets
• Kindle for iPhone
• Kindle for iPad
• Kindle for Mac
• Kindle for PC
• Kindle for Web

Shop this series

• First 3 $10.77
• First 5 $17.95
• All 8 $26.33

This option includes 3 books.

This option includes 5 books., this option includes 8 books..

• Customers Also Enjoyed
• In This Series
• Probability & Statistics
• Mathematics

Customers who read this book also read

Product details

• ASIN ‏ : ‎ B08L3W4546
• Publisher ‏ : ‎ anushabooks.com (October 11, 2020)
• Publication date ‏ : ‎ October 11, 2020
• Language ‏ : ‎ English
• File size ‏ : ‎ 6144 KB
• Text-to-Speech ‏ : ‎ Not enabled
• Enhanced typesetting ‏ : ‎ Not Enabled
• X-Ray ‏ : ‎ Not Enabled
• Word Wise ‏ : ‎ Not Enabled
• Sticky notes ‏ : ‎ Not Enabled
• Format ‏ : ‎ Print Replica
• #157 in Science Methodology & Statistics
• #181 in Science Education Research
• #577 in Probability & Statistics (Kindle Store)

About the author

Anusha illukkumbura.

Introduction to Regression Analysis and Probability : Questions and Answers made it to the Best Statistics eBooks of All Time

BookAuthority Best Statistics eBooks of All TimeI'm happy to announce that my book, "Introduction to Regression Analysis (Easy Statistics)", made it to BookAuthority's Best Statistics eBooks of All Time:

https://bookauthority.org/books/best-statistics-ebooks?t=32ek7k&s=award&book=B089CLZMT4

BookAuthority collects and ranks the best books in the world, and it is a great honor to get this kind of recognition. Thank you for all your support!

Easy Statistics Series is a statistics book series ,written by Anusha Illukkumbura, with lucid explanations on theories and practical solutions. Many finds these books are concise and aid immensely in their studies and exams. They can be used as a self study material and also a handbook. The statistics books written by Anusha Illukkumbura, guide the reader to gain solid knowledge on the topic.

Anusha Illukkumbura is a Msc and B.A holder in Statistics. She has 9 years of experience in Statistical Data Analysis. Anusha involves in teaching statistics for private individuals , so that she knows where students miss to concentrate on. The books are written addressing the most essential points where the students can overlook or misunderstand.

visit my website http://www.anushabooks.com/

join Easy Statistics Facebook page: https://www.facebook.com/Easy-Statistics-105568354517034

Customer reviews

• 5 star 4 star 3 star 2 star 1 star 5 star 68% 12% 20% 0% 0% 68%
• 5 star 4 star 3 star 2 star 1 star 4 star 68% 12% 20% 0% 0% 12%
• 5 star 4 star 3 star 2 star 1 star 3 star 68% 12% 20% 0% 0% 20%
• 5 star 4 star 3 star 2 star 1 star 2 star 68% 12% 20% 0% 0% 0%
• 5 star 4 star 3 star 2 star 1 star 1 star 68% 12% 20% 0% 0% 0%

Customer Reviews, including Product Star Ratings help customers to learn more about the product and decide whether it is the right product for them.

To calculate the overall star rating and percentage breakdown by star, we don’t use a simple average. Instead, our system considers things like how recent a review is and if the reviewer bought the item on Amazon. It also analyzed reviews to verify trustworthiness.

• Sort reviews by Top reviews Most recent Top reviews

Top reviews from the United States

There was a problem filtering reviews right now. please try again later..

Report an issue

• About Amazon
• Investor Relations
• Amazon Devices
• Amazon Science
• Sell products on Amazon
• Sell on Amazon Business
• Sell apps on Amazon
• Become an Affiliate
• Advertise Your Products
• Self-Publish with Us
• Host an Amazon Hub
• › See More Make Money with Us
• Amazon Business Card
• Shop with Points
• Reload Your Balance
• Amazon Currency Converter
• Amazon and COVID-19
• Your Account
• Your Orders
• Shipping Rates & Policies
• Returns & Replacements
• Manage Your Content and Devices

• Conditions of Use
• Privacy Notice
• Consumer Health Data Privacy Disclosure
• Your Ads Privacy Choices

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2024 Jan-.

StatPearls [Internet].

Hypothesis testing, p values, confidence intervals, and significance.

Jacob Shreffler ; Martin R. Huecker .

Affiliations

Last Update: March 13, 2023 .

• Definition/Introduction

Medical providers often rely on evidence-based medicine to guide decision-making in practice. Often a research hypothesis is tested with results provided, typically with p values, confidence intervals, or both. Additionally, statistical or research significance is estimated or determined by the investigators. Unfortunately, healthcare providers may have different comfort levels in interpreting these findings, which may affect the adequate application of the data.

• Issues of Concern

Without a foundational understanding of hypothesis testing, p values, confidence intervals, and the difference between statistical and clinical significance, it may affect healthcare providers' ability to make clinical decisions without relying purely on the research investigators deemed level of significance. Therefore, an overview of these concepts is provided to allow medical professionals to use their expertise to determine if results are reported sufficiently and if the study outcomes are clinically appropriate to be applied in healthcare practice.

Hypothesis Testing

Investigators conducting studies need research questions and hypotheses to guide analyses. Starting with broad research questions (RQs), investigators then identify a gap in current clinical practice or research. Any research problem or statement is grounded in a better understanding of relationships between two or more variables. For this article, we will use the following research question example:

Research Question: Is Drug 23 an effective treatment for Disease A?

Research questions do not directly imply specific guesses or predictions; we must formulate research hypotheses. A hypothesis is a predetermined declaration regarding the research question in which the investigator(s) makes a precise, educated guess about a study outcome. This is sometimes called the alternative hypothesis and ultimately allows the researcher to take a stance based on experience or insight from medical literature. An example of a hypothesis is below.

Research Hypothesis: Drug 23 will significantly reduce symptoms associated with Disease A compared to Drug 22.

The null hypothesis states that there is no statistical difference between groups based on the stated research hypothesis.

Researchers should be aware of journal recommendations when considering how to report p values, and manuscripts should remain internally consistent.

Regarding p values, as the number of individuals enrolled in a study (the sample size) increases, the likelihood of finding a statistically significant effect increases. With very large sample sizes, the p-value can be very low significant differences in the reduction of symptoms for Disease A between Drug 23 and Drug 22. The null hypothesis is deemed true until a study presents significant data to support rejecting the null hypothesis. Based on the results, the investigators will either reject the null hypothesis (if they found significant differences or associations) or fail to reject the null hypothesis (they could not provide proof that there were significant differences or associations).

To test a hypothesis, researchers obtain data on a representative sample to determine whether to reject or fail to reject a null hypothesis. In most research studies, it is not feasible to obtain data for an entire population. Using a sampling procedure allows for statistical inference, though this involves a certain possibility of error. [1]  When determining whether to reject or fail to reject the null hypothesis, mistakes can be made: Type I and Type II errors. Though it is impossible to ensure that these errors have not occurred, researchers should limit the possibilities of these faults. [2]

Significance

Significance is a term to describe the substantive importance of medical research. Statistical significance is the likelihood of results due to chance. [3]  Healthcare providers should always delineate statistical significance from clinical significance, a common error when reviewing biomedical research. [4]  When conceptualizing findings reported as either significant or not significant, healthcare providers should not simply accept researchers' results or conclusions without considering the clinical significance. Healthcare professionals should consider the clinical importance of findings and understand both p values and confidence intervals so they do not have to rely on the researchers to determine the level of significance. [5]  One criterion often used to determine statistical significance is the utilization of p values.

P values are used in research to determine whether the sample estimate is significantly different from a hypothesized value. The p-value is the probability that the observed effect within the study would have occurred by chance if, in reality, there was no true effect. Conventionally, data yielding a p<0.05 or p<0.01 is considered statistically significant. While some have debated that the 0.05 level should be lowered, it is still universally practiced. [6]  Hypothesis testing allows us to determine the size of the effect.

An example of findings reported with p values are below:

Statement: Drug 23 reduced patients' symptoms compared to Drug 22. Patients who received Drug 23 (n=100) were 2.1 times less likely than patients who received Drug 22 (n = 100) to experience symptoms of Disease A, p<0.05.

Statement:Individuals who were prescribed Drug 23 experienced fewer symptoms (M = 1.3, SD = 0.7) compared to individuals who were prescribed Drug 22 (M = 5.3, SD = 1.9). This finding was statistically significant, p= 0.02.

For either statement, if the threshold had been set at 0.05, the null hypothesis (that there was no relationship) should be rejected, and we should conclude significant differences. Noticeably, as can be seen in the two statements above, some researchers will report findings with < or > and others will provide an exact p-value (0.000001) but never zero [6] . When examining research, readers should understand how p values are reported. The best practice is to report all p values for all variables within a study design, rather than only providing p values for variables with significant findings. [7]  The inclusion of all p values provides evidence for study validity and limits suspicion for selective reporting/data mining.  

While researchers have historically used p values, experts who find p values problematic encourage the use of confidence intervals. [8] . P-values alone do not allow us to understand the size or the extent of the differences or associations. [3]  In March 2016, the American Statistical Association (ASA) released a statement on p values, noting that scientific decision-making and conclusions should not be based on a fixed p-value threshold (e.g., 0.05). They recommend focusing on the significance of results in the context of study design, quality of measurements, and validity of data. Ultimately, the ASA statement noted that in isolation, a p-value does not provide strong evidence. [9]

When conceptualizing clinical work, healthcare professionals should consider p values with a concurrent appraisal study design validity. For example, a p-value from a double-blinded randomized clinical trial (designed to minimize bias) should be weighted higher than one from a retrospective observational study [7] . The p-value debate has smoldered since the 1950s [10] , and replacement with confidence intervals has been suggested since the 1980s. [11]

Confidence Intervals

A confidence interval provides a range of values within given confidence (e.g., 95%), including the accurate value of the statistical constraint within a targeted population. [12]  Most research uses a 95% CI, but investigators can set any level (e.g., 90% CI, 99% CI). [13]  A CI provides a range with the lower bound and upper bound limits of a difference or association that would be plausible for a population. [14]  Therefore, a CI of 95% indicates that if a study were to be carried out 100 times, the range would contain the true value in 95, [15]  confidence intervals provide more evidence regarding the precision of an estimate compared to p-values. [6]

In consideration of the similar research example provided above, one could make the following statement with 95% CI:

Statement: Individuals who were prescribed Drug 23 had no symptoms after three days, which was significantly faster than those prescribed Drug 22; there was a mean difference between the two groups of days to the recovery of 4.2 days (95% CI: 1.9 – 7.8).

It is important to note that the width of the CI is affected by the standard error and the sample size; reducing a study sample number will result in less precision of the CI (increase the width). [14]  A larger width indicates a smaller sample size or a larger variability. [16]  A researcher would want to increase the precision of the CI. For example, a 95% CI of 1.43 – 1.47 is much more precise than the one provided in the example above. In research and clinical practice, CIs provide valuable information on whether the interval includes or excludes any clinically significant values. [14]

Null values are sometimes used for differences with CI (zero for differential comparisons and 1 for ratios). However, CIs provide more information than that. [15]  Consider this example: A hospital implements a new protocol that reduced wait time for patients in the emergency department by an average of 25 minutes (95% CI: -2.5 – 41 minutes). Because the range crosses zero, implementing this protocol in different populations could result in longer wait times; however, the range is much higher on the positive side. Thus, while the p-value used to detect statistical significance for this may result in "not significant" findings, individuals should examine this range, consider the study design, and weigh whether or not it is still worth piloting in their workplace.

Similarly to p-values, 95% CIs cannot control for researchers' errors (e.g., study bias or improper data analysis). [14]  In consideration of whether to report p-values or CIs, researchers should examine journal preferences. When in doubt, reporting both may be beneficial. [13]  An example is below:

Reporting both: Individuals who were prescribed Drug 23 had no symptoms after three days, which was significantly faster than those prescribed Drug 22, p = 0.009. There was a mean difference between the two groups of days to the recovery of 4.2 days (95% CI: 1.9 – 7.8).

• Clinical Significance

Recall that clinical significance and statistical significance are two different concepts. Healthcare providers should remember that a study with statistically significant differences and large sample size may be of no interest to clinicians, whereas a study with smaller sample size and statistically non-significant results could impact clinical practice. [14]  Additionally, as previously mentioned, a non-significant finding may reflect the study design itself rather than relationships between variables.

Healthcare providers using evidence-based medicine to inform practice should use clinical judgment to determine the practical importance of studies through careful evaluation of the design, sample size, power, likelihood of type I and type II errors, data analysis, and reporting of statistical findings (p values, 95% CI or both). [4]  Interestingly, some experts have called for "statistically significant" or "not significant" to be excluded from work as statistical significance never has and will never be equivalent to clinical significance. [17]

The decision on what is clinically significant can be challenging, depending on the providers' experience and especially the severity of the disease. Providers should use their knowledge and experiences to determine the meaningfulness of study results and make inferences based not only on significant or insignificant results by researchers but through their understanding of study limitations and practical implications.

• Nursing, Allied Health, and Interprofessional Team Interventions

All physicians, nurses, pharmacists, and other healthcare professionals should strive to understand the concepts in this chapter. These individuals should maintain the ability to review and incorporate new literature for evidence-based and safe care. 

• Review Questions
• Access free multiple choice questions on this topic.
• Comment on this article.

Disclosure: Jacob Shreffler declares no relevant financial relationships with ineligible companies.

Disclosure: Martin Huecker declares no relevant financial relationships with ineligible companies.

This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) ( http://creativecommons.org/licenses/by-nc-nd/4.0/ ), which permits others to distribute the work, provided that the article is not altered or used commercially. You are not required to obtain permission to distribute this article, provided that you credit the author and journal.

• Cite this Page Shreffler J, Huecker MR. Hypothesis Testing, P Values, Confidence Intervals, and Significance. [Updated 2023 Mar 13]. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2024 Jan-.

In this Page

Bulk download.

• Bulk download StatPearls data from FTP

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Similar articles in PubMed

• The reporting of p values, confidence intervals and statistical significance in Preventive Veterinary Medicine (1997-2017). [PeerJ. 2021] The reporting of p values, confidence intervals and statistical significance in Preventive Veterinary Medicine (1997-2017). Messam LLM, Weng HY, Rosenberger NWY, Tan ZH, Payet SDM, Santbakshsing M. PeerJ. 2021; 9:e12453. Epub 2021 Nov 24.
• Review Clinical versus statistical significance: interpreting P values and confidence intervals related to measures of association to guide decision making. [J Pharm Pract. 2010] Review Clinical versus statistical significance: interpreting P values and confidence intervals related to measures of association to guide decision making. Ferrill MJ, Brown DA, Kyle JA. J Pharm Pract. 2010 Aug; 23(4):344-51. Epub 2010 Apr 13.
• Interpreting "statistical hypothesis testing" results in clinical research. [J Ayurveda Integr Med. 2012] Interpreting "statistical hypothesis testing" results in clinical research. Sarmukaddam SB. J Ayurveda Integr Med. 2012 Apr; 3(2):65-9.
• Confidence intervals in procedural dermatology: an intuitive approach to interpreting data. [Dermatol Surg. 2005] Confidence intervals in procedural dermatology: an intuitive approach to interpreting data. Alam M, Barzilai DA, Wrone DA. Dermatol Surg. 2005 Apr; 31(4):462-6.
• Review Is statistical significance testing useful in interpreting data? [Reprod Toxicol. 1993] Review Is statistical significance testing useful in interpreting data? Savitz DA. Reprod Toxicol. 1993; 7(2):95-100.

Recent Activity

• Hypothesis Testing, P Values, Confidence Intervals, and Significance - StatPearl... Hypothesis Testing, P Values, Confidence Intervals, and Significance - StatPearls

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers