• -- Navigate To -- CR Dashboard Connect for Researchers Connect for Participants
• Recent Press
• Papers Citing Connect
• Connect for Participants
• Connect for Researchers
• Connect AI Training
• Managed Research
• Prime Panels
• MTurk Toolkit
• Health & Medicine
• Conferences
• Knowledge Base
• A Researcher’s Guide To Statistical Significance And Sample Size Calculations

## What Does It Mean for Research to Be Statistically Significant?

What does it mean to be statistically significant, an example of null hypothesis significance testing, measuring statistical significance: understanding the p value (significance level), what factors affect the power of hypothesis test, 1. sample size, 2. significance level, 3. standard deviations, 4. effect size, why is statistical significance important for researchers, does your study need to be statistically significant, practical significance vs. statistical significance, part 1: how is statistical significance defined in research.

The world today is drowning in data.

That may sound like hyperbole but consider this. In 2018, humans around the world produced more than 2.5 quintillion bytes of data—each day. According to some estimates , every minute people conduct almost 4.5 million Google searches, post 511,200 tweets, watch 4.5 million YouTube videos, swipe 1.4 million times on Tinder, and order 8,683 meals from GrubHub. These numbers—and the world’s total data—are expected to continue growing exponentially in the coming years.

For behavioral researchers and businesses, this data represents a valuable opportunity. However, using data to learn about human behavior or make decisions about consumer behavior often requires an understanding of statistics and statistical significance.

Statistical significance is a measurement of how likely it is that the difference between two groups, models, or statistics occurred by chance or occurred because two variables are actually related to each other. This means that a “statistically significant” finding is one in which it is likely the finding is real, reliable, and not due to chance.

To evaluate whether a finding is statistically significant, researchers engage in a process known as null hypothesis significance testing . Null hypothesis significance testing is less of a mathematical formula and more of a logical process for thinking about the strength and legitimacy of a finding.

Imagine a Vice President of Marketing asks her team to test a new layout for the company website. The new layout streamlines the user experience by making it easier for people to place orders and suggesting additional items to go along with each customer’s purchase. After testing the new website, the VP finds that visitors to the site spend an average of $12.63. Under the old layout, visitors spent an average of$12.32, meaning the new layout increases average spending by $0.31 per person. The question the VP must answer is whether the difference of$0.31 per person is significant or something that likely occurred by chance.

To answer this question with statistical analysis, the VP begins by adopting a skeptical stance toward her data known as the null hypothesis . The null hypothesis assumes that whatever researchers are studying does not actually exist in the population of interest. So, in this case the VP assumes that the change in website layout does not influence how much people spend on purchases.

With the null hypothesis in mind, the manager asks how likely it is that she would obtain the results observed in her study—the average difference of $0.31 per visitor—if the change in website layout actually causes no difference in people’s spending (i.e., if the null hypothesis is true). If the probability of obtaining the observed results is low, the manager will reject the null hypothesis and conclude that her finding is statistically significant. Statistically significant findings indicate not only that the researchers’ results are unlikely the result of chance, but also that there is an effect or relationship between the variables being studied in the larger population. However, because researchers want to ensure they do not falsely conclude there is a meaningful difference between groups when in fact the difference is due to chance, they often set stringent criteria for their statistical tests. This criterion is known as the significance level . Within the social sciences, researchers often adopt a significance level of 5%. This means researchers are only willing to conclude that the results of their study are statistically significant if the probability of obtaining those results if the null hypothesis were true—known as the p value —is less than 5%. Five percent represents a stringent criterion, but there is nothing magical about it. In medical research, significance levels are often set at 1%. In cognitive neuroscience, researchers often adopt significance levels well below 1%. And, when astronomers seek to explain aspects of the universe or physicists study new particles like the Higgs Boson they set significance levels several orders of magnitude below .05. In other research contexts like business or industry, researchers may set more lenient significance levels depending on the aim of their research. However, in all research, the more stringently a researcher sets their significance level, the more confident they can be that their results are not due to chance. Determining whether a given set of results is statistically significant is only one half of the hypothesis testing equation. The other half is ensuring that the statistical tests a researcher conducts are powerful enough to detect an effect if one really exists. That is, when a researcher concludes their hypothesis was incorrect and there is no effect between the variables being studied, that conclusion is only meaningful if the study was powerful enough to detect an effect if one really existed. The power of a hypothesis test is influenced by several factors. Sample size—or, the number of participants the researcher collects data from—affects the power of a hypothesis test. Larger samples with more observations generally lead to higher-powered tests than smaller samples. In addition, large samples are more likely to produce replicable results because extreme scores that occur by chance are more likely to balance out in a large sample rather than in a small one. Although setting a low significance level helps researchers ensure their results are not due to chance, it also lowers their power to detect an effect because it makes rejecting the null hypothesis harder. In this respect, the significance level a researcher selects is often in competition with power. Standard deviations represent unexplained variability within data, also known as error. Generally speaking, the more unexplained variability within a dataset, the less power researchers have to detect an effect. Unexplained variability can be the result of measurement error, individual differences among participants, or situational noise. A final factor that influences power is the size of the effect a researcher is studying. As you might expect, big changes in behavior are easier to detect than small ones. Sometimes researchers do not know the strength of an effect before conducting a study. Even though this makes it harder to conduct a well powered study, it is important to keep in mind that phenomena that produce a large effect will lead to studies with more power than phenomena that produce only a small effect. Statistical significance is important because it allows researchers to hold a degree of confidence that their findings are real, reliable, and not due to chance. But statistical significance is not equally important to all researchers in all situations. The importance of obtaining statistically significant results depends on what a researcher studies and within what context. Within academic research, statistical significance is often critical because academic researchers study theoretical relationships between different variables and behavior. Furthermore, the goal of academic research is often to publish research reports in scientific journals. The threshold for publishing in academic journals is often a series of statistically significant results. Outside of academia, statistical significance is often less important. Researchers, managers, and decision makers in business may use statistical significance to understand how strongly the results of a study should inform the decisions they make. But, because statistical significance is simply a way of quantifying how much confidence to hold in a research finding, people in industry are often more interested in a finding’s practical significance than statistical significance. To demonstrate the difference between practical and statistical significance, imagine you’re a candidate for political office. Maybe you have decided to run for local or state-wide office, or, if you’re feeling bold, imagine you’re running for President. During your campaign, your team comes to you with data on messages intended to mobilize voters. These messages have been market tested and now you and your team must decide which ones to adopt. If you go with Message A, 41% of registered voters say they are likely to turn out at the polls and cast a ballot. If you go with Message B, this number drops to 37%. As a candidate, should you care whether this difference is statistically significant at a p value below .05? The answer is of course not. What you likely care about more than statistical significance is practical significance —the likelihood that the difference between groups is large enough to be meaningful in real life. You should ensure there is some rigor behind the difference in messages before you spend money on a marketing campaign, but when elections are sometimes decided by as little as one vote you should adopt the message that brings more people out to vote. Within business and industry, the practical significance of a research finding is often equally if not more important than the statistical significance. In addition, when findings have large practical significance, they are almost always statistically significant too. Conducting statistically significant research is a challenge, but it’s a challenge worth tackling. Flawed data and faulty analyses only lead to poor decisions. Start taking steps to ensure your surveys and experiments produce valid results by using CloudResearch. If you have the team to conduct your own studies, CloudResearch can help you find large samples of online participants quickly and easily. Regardless of your demographic criteria or sample size, we can help you get the participants you need. If your team doesn’t have the resources to run a study, we can run it for you. Our team of expert social scientists, computer scientists, and software engineers can design any study, collect the data, and analyze the results for you. Let us show you how conducting statistically significant research can improve your decision-making today. ## Continue Reading: A Researcher’s Guide to Statistical Significance and Sample Size Calculations ## Part 2: How to Calculate Statistical Significance ## Part 3: Determining Sample Size: How Many Survey Participants Do You Need? Related articles, what is data quality and why is it important. If you were a researcher studying human behavior 30 years ago, your options for identifying participants for your studies were limited. If you worked at a university, you might be... ## How to Identify and Handle Invalid Responses to Online Surveys As a researcher, you are aware that planning studies, designing materials and collecting data each take a lot of work. So when you get your hands on a new dataset,... ## SUBSCRIBE TO RECEIVE UPDATES 2024 grant application form, personal and institutional information. • Full Name * First Last • Position/Title * • Affiliated Academic Institution or Research Organization * ## Detailed Research Proposal Questions • Project Title * • Research Category * - Antisemitism Islamophobia Both • Objectives * • Methodology (including who the targeted participants are) * • Expected Outcomes * • Significance of the Study * ## Budget and Grant Tier Request • Requested Grant Tier * -$200 $500$1000 Applicants requesting larger grants may still be eligible for smaller awards if the full amount requested is not granted.
• Budget Justification *

## Research Timeline

• Projected Start Date * MM slash DD slash YYYY Preference will be given to projects that can commence soon, preferably before September 2024.
• Estimated Completion Date * MM slash DD slash YYYY Preference will be given to projects that aim to complete within a year.
• Project Timeline *
• Name This field is for validation purposes and should be left unchanged.

• Name * First Name Last Name
• I would like to request a demo of the Sentry platform
• Email This field is for validation purposes and should be left unchanged.
• Name * First name Last name
• Phone This field is for validation purposes and should be left unchanged.

• Name * First Last
• Name * First and Last
• Please select the best time to discuss your project goals/details to claim your free Sentry pilot for the next 60 days or to receive 10% off your first Managed Research study with Sentry.
• Comments This field is for validation purposes and should be left unchanged.

• Email * Enter Email Confirm Email
• Organization
• Job Title *

The Savvy Scientist

Experiences of a London PhD student and beyond

## What is the Significance of a Study? Examples and Guide

If you’re reading this post you’re probably wondering: what is the significance of a study?

No matter where you’re at with a piece of research, it is a good idea to think about the potential significance of your work. And sometimes you’ll have to explicitly write a statement of significance in your papers, it addition to it forming part of your thesis.

In this post I’ll cover what the significance of a study is, how to measure it, how to describe it with examples and add in some of my own experiences having now worked in research for over nine years.

If you’re reading this because you’re writing up your first paper, welcome! You may also like my how-to guide for all aspects of writing your first research paper .

Looking for guidance on writing the statement of significance for a paper or thesis? Click here to skip straight to that section.

## What is the Significance of a Study?

For research papers, theses or dissertations it’s common to explicitly write a section describing the significance of the study. We’ll come onto what to include in that section in just a moment.

However the significance of a study can actually refer to several different things.

Working our way from the most technical to the broadest, depending on the context, the significance of a study may refer to:

• Within your study: Statistical significance. Can we trust the findings?
• Wider research field: Research significance. How does your study progress the field?
• Commercial / economic significance: Could there be business opportunities for your findings?
• Societal significance: What impact could your study have on the wider society.
• And probably other domain-specific significance!

We’ll shortly cover each of them in turn, including how they’re measured and some examples for each type of study significance.

But first, let’s touch on why you should consider the significance of your research at an early stage.

## Why Care About the Significance of a Study?

No matter what is motivating you to carry out your research, it is sensible to think about the potential significance of your work. In the broadest sense this asks, how does the study contribute to the world?

After all, for many people research is only worth doing if it will result in some expected significance. For the vast majority of us our studies won’t be significant enough to reach the evening news, but most studies will help to enhance knowledge in a particular field and when research has at least some significance it makes for a far more fulfilling longterm pursuit.

Furthermore, a lot of us are carrying out research funded by the public. It therefore makes sense to keep an eye on what benefits the work could bring to the wider community.

Often in research you’ll come to a crossroads where you must decide which path of research to pursue. Thinking about the potential benefits of a strand of research can be useful for deciding how to spend your time, money and resources.

It’s worth noting though, that not all research activities have to work towards obvious significance. This is especially true while you’re a PhD student, where you’re figuring out what you enjoy and may simply be looking for an opportunity to learn a new skill.

However, if you’re trying to decide between two potential projects, it can be useful to weigh up the potential significance of each.

Let’s now dive into the different types of significance, starting with research significance.

## Research Significance

What is the research significance of a study.

Unless someone specifies which type of significance they’re referring to, it is fair to assume that they want to know about the research significance of your study.

Research significance describes how your work has contributed to the field, how it could inform future studies and progress research.

## Where should I write about my study’s significance in my thesis?

Typically you should write about your study’s significance in the Introduction and Conclusions sections of your thesis.

It’s important to mention it in the Introduction so that the relevance of your work and the potential impact and benefits it could have on the field are immediately apparent. Explaining why your work matters will help to engage readers (and examiners!) early on.

It’s also a good idea to detail the study’s significance in your Conclusions section. This adds weight to your findings and helps explain what your study contributes to the field.

On occasion you may also choose to include a brief description in your Abstract.

## What is expected when submitting an article to a journal

It is common for journals to request a statement of significance, although this can sometimes be called other things such as:

• Impact statement
• Significance statement

Here is one such example of what is expected:

Impact Statement:  An Impact Statement is required for all submissions.  Your impact statement will be evaluated by the Editor-in-Chief, Global Editors, and appropriate Associate Editor. For your manuscript to receive full review, the editors must be convinced that it is an important advance in for the field. The Impact Statement is not a restating of the abstract. It should address the following: Why is the work submitted important to the field? How does the work submitted advance the field? What new information does this work impart to the field? How does this new information impact the field? Experimental Biology and Medicine journal, author guidelines

Typically the impact statement will be shorter than the Abstract, around 150 words.

Defining the study’s significance is helpful not just for the impact statement (if the journal asks for one) but also for building a more compelling argument throughout your submission. For instance, usually you’ll start the Discussion section of a paper by highlighting the research significance of your work. You’ll also include a short description in your Abstract too.

## How to describe the research significance of a study, with examples

Whether you’re writing a thesis or a journal article, the approach to writing about the significance of a study are broadly the same.

I’d therefore suggest using the questions above as a starting point to base your statements on.

• Why is the work submitted important to the field?
• How does the work submitted advance the field?
• What new information does this work impart to the field?
• How does this new information impact the field?

Answer those questions and you’ll have a much clearer idea of the research significance of your work.

When describing it, try to clearly state what is novel about your study’s contribution to the literature. Then go on to discuss what impact it could have on progressing the field along with recommendations for future work.

## Potential sentence starters

If you’re not sure where to start, why not set a 10 minute timer and have a go at trying to finish a few of the following sentences. Not sure on what to put? Have a chat to your supervisor or lab mates and they may be able to suggest some ideas.

• This study is important to the field because…
• These findings advance the field by…
• Our results highlight the importance of…
• Our discoveries impact the field by…

Now you’ve had a go let’s have a look at some real life examples.

## Statement of significance examples

A statement of significance / impact:

Impact Statement This review highlights the historical development of the concept of “ideal protein” that began in the 1950s and 1980s for poultry and swine diets, respectively, and the major conceptual deficiencies of the long-standing concept of “ideal protein” in animal nutrition based on recent advances in amino acid (AA) metabolism and functions. Nutritionists should move beyond the “ideal protein” concept to consider optimum ratios and amounts of all proteinogenic AAs in animal foods and, in the case of carnivores, also taurine. This will help formulate effective low-protein diets for livestock, poultry, and fish, while sustaining global animal production. Because they are not only species of agricultural importance, but also useful models to study the biology and diseases of humans as well as companion (e.g. dogs and cats), zoo, and extinct animals in the world, our work applies to a more general readership than the nutritionists and producers of farm animals. Wu G, Li P. The “ideal protein” concept is not ideal in animal nutrition.  Experimental Biology and Medicine . 2022;247(13):1191-1201. doi: 10.1177/15353702221082658

And the same type of section but this time called “Advances in knowledge”:

Advances in knowledge: According to the MY-RADs criteria, size measurements of focal lesions in MRI are now of relevance for response assessment in patients with monoclonal plasma cell disorders. Size changes of 1 or 2 mm are frequently observed due to uncertainty of the measurement only, while the actual focal lesion has not undergone any biological change. Size changes of at least 6 mm or more in  T 1  weighted or  T 2  weighted short tau inversion recovery sequences occur in only 5% or less of cases when the focal lesion has not undergone any biological change. Wennmann M, Grözinger M, Weru V, et al. Test-retest, inter- and intra-rater reproducibility of size measurements of focal bone marrow lesions in MRI in patients with multiple myeloma [published online ahead of print, 2023 Apr 12].  Br J Radiol . 2023;20220745. doi: 10.1259/bjr.20220745

## Other examples of research significance

Moving beyond the formal statement of significance, here is how you can describe research significance more broadly within your paper.

Describing research impact in an Abstract of a paper:

Three-dimensional visualisation and quantification of the chondrocyte population within articular cartilage can be achieved across a field of view of several millimetres using laboratory-based micro-CT. The ability to map chondrocytes in 3D opens possibilities for research in fields from skeletal development through to medical device design and treatment of cartilage degeneration. Conclusions section of the abstract in my first paper .

In the Discussion section of a paper:

We report for the utility of a standard laboratory micro-CT scanner to visualise and quantify features of the chondrocyte population within intact articular cartilage in 3D. This study represents a complimentary addition to the growing body of evidence supporting the non-destructive imaging of the constituents of articular cartilage. This offers researchers the opportunity to image chondrocyte distributions in 3D without specialised synchrotron equipment, enabling investigations such as chondrocyte morphology across grades of cartilage damage, 3D strain mapping techniques such as digital volume correlation to evaluate mechanical properties  in situ , and models for 3D finite element analysis  in silico  simulations. This enables an objective quantification of chondrocyte distribution and morphology in three dimensions allowing greater insight for investigations into studies of cartilage development, degeneration and repair. One such application of our method, is as a means to provide a 3D pattern in the cartilage which, when combined with digital volume correlation, could determine 3D strain gradient measurements enabling potential treatment and repair of cartilage degeneration. Moreover, the method proposed here will allow evaluation of cartilage implanted with tissue engineered scaffolds designed to promote chondral repair, providing valuable insight into the induced regenerative process. The Discussion section of the paper is laced with references to research significance.

## How is longer term research significance measured?

Looking beyond writing impact statements within papers, sometimes you’ll want to quantify the long term research significance of your work. For instance when applying for jobs.

The most obvious measure of a study’s long term research significance is the number of citations it receives from future publications. The thinking is that a study which receives more citations will have had more research impact, and therefore significance , than a study which received less citations. Citations can give a broad indication of how useful the work is to other researchers but citations aren’t really a good measure of significance.

Bear in mind that us researchers can be lazy folks and sometimes are simply looking to cite the first paper which backs up one of our claims. You can find studies which receive a lot of citations simply for packaging up the obvious in a form which can be easily found and referenced, for instance by having a catchy or optimised title.

Likewise, research activity varies wildly between fields. Therefore a certain study may have had a big impact on a particular field but receive a modest number of citations, simply because not many other researchers are working in the field.

Nevertheless, citations are a standard measure of significance and for better or worse it remains impressive for someone to be the first author of a publication receiving lots of citations.

Other measures for the research significance of a study include:

• How much follow-on research the study creates. For instance, part of my PhD involved a novel material initially developed by another PhD student in the lab. That PhD student’s research had unlocked lots of potential new studies and now lots of people in the group were using the same material and developing it for different applications. The initial study may not receive a high number of citations yet long term it generated a lot of research activity.

That covers research significance, but you’ll often want to consider other types of significance for your study and we’ll cover those next.

## Statistical Significance

What is the statistical significance of a study.

Often as part of a study you’ll carry out statistical tests and then state the statistical significance of your findings: think p-values eg <0.05. It is useful to describe the outcome of these tests within your report or paper, to give a measure of statistical significance.

Effectively you are trying to show whether the performance of your innovation is actually better than a control or baseline and not just chance. Statistical significance deserves a whole other post so I won’t go into a huge amount of depth here.

Things that make publication in  The BMJ  impossible or unlikely Internal validity/robustness of the study • It had insufficient statistical power, making interpretation difficult; • Lack of statistical power; The British Medical Journal’s guide for authors

Calculating statistical significance isn’t always necessary (or valid) for a study, such as if you have a very small number of samples, but it is a very common requirement for scientific articles.

Writing a journal article? Check the journal’s guide for authors to see what they expect. Generally if you have approximately five or more samples or replicates it makes sense to start thinking about statistical tests. Speak to your supervisor and lab mates for advice, and look at other published articles in your field.

## How is statistical significance measured?

Statistical significance is quantified using p-values . Depending on your study design you’ll choose different statistical tests to compute the p-value.

A p-value of 0.05 is a common threshold value. The 0.05 means that there is a 1/20 chance that the difference in performance you’re reporting is just down to random chance.

• p-values above 0.05 mean that the result isn’t statistically significant enough to be trusted: it is too likely that the effect you’re showing is just luck.
• p-values less than or equal to 0.05 mean that the result is statistically significant. In other words: unlikely to just be chance, which is usually considered a good outcome.

Low p-values (eg p = 0.001) mean that it is highly unlikely to be random chance (1/1000 in the case of p = 0.001), therefore more statistically significant.

It is important to clarify that, although low p-values mean that your findings are statistically significant, it doesn’t automatically mean that the result is scientifically important. More on that in the next section on research significance.

## How to describe the statistical significance of your study, with examples

In the first paper from my PhD I ran some statistical tests to see if different staining techniques (basically dyes) increased how well you could see cells in cow tissue using micro-CT scanning (a 3D imaging technique).

In your methods section you should mention the statistical tests you conducted and then in the results you will have statements such as:

Between mediums for the two scan protocols C/N [contrast to noise ratio] was greater for EtOH than the PBS in both scanning methods (both  p  < 0.0001) with mean differences of 1.243 (95% CI [confidence interval] 0.709 to 1.778) for absorption contrast and 6.231 (95% CI 5.772 to 6.690) for propagation contrast. … Two repeat propagation scans were taken of samples from the PTA-stained groups. No difference in mean C/N was found with either medium: PBS had a mean difference of 0.058 ( p  = 0.852, 95% CI -0.560 to 0.676), EtOH had a mean difference of 1.183 ( p  = 0.112, 95% CI 0.281 to 2.648). From the Results section of my first paper, available here . Square brackets added for this post to aid clarity.

From this text the reader can infer from the first paragraph that there was a statistically significant difference in using EtOH compared to PBS (really small p-value of <0.0001). However, from the second paragraph, the difference between two repeat scans was statistically insignificant for both PBS (p = 0.852) and EtOH (p = 0.112).

By conducting these statistical tests you have then earned your right to make bold statements, such as these from the discussion section:

Propagation phase-contrast increases the contrast of individual chondrocytes [cartilage cells] compared to using absorption contrast. From the Discussion section from the same paper.

Without statistical tests you have no evidence that your results are not just down to random chance.

Beyond describing the statistical significance of a study in the main body text of your work, you can also show it in your figures.

In figures such as bar charts you’ll often see asterisks to represent statistical significance, and “n.s.” to show differences between groups which are not statistically significant. Here is one such figure, with some subplots, from the same paper:

In this example an asterisk (*) between two bars represents p < 0.05. Two asterisks (**) represents p < 0.001 and three asterisks (***) represents p < 0.0001. This should always be stated in the caption of your figure since the values that each asterisk refers to can vary.

Now that we know if a study is showing statistically and research significance, let’s zoom out a little and consider the potential for commercial significance.

## Commercial and Industrial Significance

What are commercial and industrial significance.

Moving beyond significance in relation to academia, your research may also have commercial or economic significance.

Simply put:

• Commercial significance: could the research be commercialised as a product or service? Perhaps the underlying technology described in your study could be licensed to a company or you could even start your own business using it.
• Industrial significance: more widely than just providing a product which could be sold, does your research provide insights which may affect a whole industry? Such as: revealing insights or issues with current practices, performance gains you don’t want to commercialise (e.g. solar power efficiency), providing suggested frameworks or improvements which could be employed industry-wide.

I’ve grouped these two together because there can certainly be overlap. For instance, perhaps your new technology could be commercialised whilst providing wider improvements for the whole industry.

Commercial and industrial significance are not relevant to most studies, so only write about it if you and your supervisor can think of reasonable routes to your work having an impact in these ways.

## How are commercial and industrial significance measured?

Unlike statistical and research significances, the measures of commercial and industrial significance can be much more broad.

Here are some potential measures of significance:

Commercial significance:

• How much value does your technology bring to potential customers or users?
• How big is the potential market and how much revenue could the product potentially generate?
• Is the intellectual property protectable? i.e. patentable, or if not could the novelty be protected with trade secrets: if so publish your method with caution!
• If commercialised, could the product bring employment to a geographical area?

Industrial significance:

What impact could it have on the industry? For instance if you’re revealing an issue with something, such as unintended negative consequences of a drug , what does that mean for the industry and the public? This could be:

• Better safety
• Faster production methods
• Improved scaleability

## How to describe the commercial and industrial significance of a study, with examples

Commercial significance.

If your technology could be commercially viable, and you’ve got an interest in commercialising it yourself, it is likely that you and your university may not want to immediately publish the study in a journal.

You’ll probably want to consider routes to exploiting the technology and your university may have a “technology transfer” team to help researchers navigate the various options.

However, if instead of publishing a paper you’re submitting a thesis or dissertation then it can be useful to highlight the commercial significance of your work. In this instance you could include statements of commercial significance such as:

The measurement technology described in this study provides state of the art performance and could enable the development of low cost devices for aerospace applications. An example of commercial significance I invented for this post

## Industrial significance

First, think about the industrial sectors who could benefit from the developments described in your study.

For example if you’re working to improve battery efficiency it is easy to think of how it could lead to performance gains for certain industries, like personal electronics or electric vehicles. In these instances you can describe the industrial significance relatively easily, based off your findings.

For example:

By utilising abundant materials in the described battery fabrication process we provide a framework for battery manufacturers to reduce dependence on rare earth components. Again, an invented example

For other technologies there may well be industrial applications but they are less immediately obvious and applicable. In these scenarios the best you can do is to simply reframe your research significance statement in terms of potential commercial applications in a broad way.

As a reminder: not all studies should address industrial significance, so don’t try to invent applications just for the sake of it!

## Societal Significance

What is the societal significance of a study.

The most broad category of significance is the societal impact which could stem from it.

If you’re working in an applied field it may be quite easy to see a route for your research to impact society. For others, the route to societal significance may be less immediate or clear.

Studies can help with big issues facing society such as:

• Medical applications : vaccines, surgical implants, drugs, improving patient safety. For instance this medical device and drug combination I worked on which has a very direct route to societal significance.
• Political significance : Your research may provide insights which could contribute towards potential changes in policy or better understanding of issues facing society.
• Public health : for instance COVID-19 transmission and related decisions.
• Climate change : mitigation such as more efficient solar panels and lower cost battery solutions, and studying required adaptation efforts and technologies. Also, better understanding around related societal issues, for instance this study on the effects of temperature on hate speech.

## How is societal significance measured?

Societal significance at a high level can be quantified by the size of its potential societal effect. Just like a lab risk assessment, you can think of it in terms of probability (or how many people it could help) and impact magnitude.

Societal impact = How many people it could help x the magnitude of the impact

Think about how widely applicable the findings are: for instance does it affect only certain people? Then think about the potential size of the impact: what kind of difference could it make to those people?

Between these two metrics you can get a pretty good overview of the potential societal significance of your research study.

## How to describe the societal significance of a study, with examples

Quite often the broad societal significance of your study is what you’re setting the scene for in your Introduction. In addition to describing the existing literature, it is common to for the study’s motivation to touch on its wider impact for society.

For those of us working in healthcare research it is usually pretty easy to see a path towards societal significance.

Our CLOUT model has state-of-the-art performance in mortality prediction, surpassing other competitive NN models and a logistic regression model … Our results show that the risk factors identified by the CLOUT model agree with physicians’ assessment, suggesting that CLOUT could be used in real-world clinicalsettings. Our results strongly support that CLOUT may be a useful tool to generate clinical prediction models, especially among hospitalized and critically ill patient populations. Learning Latent Space Representations to Predict Patient Outcomes: Model Development and Validation

In other domains the societal significance may either take longer or be more indirect, meaning that it can be more difficult to describe the societal impact.

Even so, here are some examples I’ve found from studies in non-healthcare domains:

We examined food waste as an initial investigation and test of this methodology, and there is clear potential for the examination of not only other policy texts related to food waste (e.g., liability protection, tax incentives, etc.; Broad Leib et al., 2020) but related to sustainable fishing (Worm et al., 2006) and energy use (Hawken, 2017). These other areas are of obvious relevance to climate change… AI-Based Text Analysis for Evaluating Food Waste Policies
The continued development of state-of-the art NLP tools tailored to climate policy will allow climate researchers and policy makers to extract meaningful information from this growing body of text, to monitor trends over time and administrative units, and to identify potential policy improvements. BERT Classification of Paris Agreement Climate Action Plans

## Top Tips For Identifying & Writing About the Significance of Your Study

• Writing a thesis? Describe the significance of your study in the Introduction and the Conclusion .
• Submitting a paper? Read the journal’s guidelines. If you’re writing a statement of significance for a journal, make sure you read any guidance they give for what they’re expecting.
• Take a step back from your research and consider your study’s main contributions.
• Read previously published studies in your field . Use this for inspiration and ideas on how to describe the significance of your own study
• Discuss the study with your supervisor and potential co-authors or collaborators and brainstorm potential types of significance for it.

Now you’ve finished reading up on the significance of a study you may also like my how-to guide for all aspects of writing your first research paper .

I hope that you’ve learned something useful from this article about the significance of a study. If you have any more research-related questions let me know, I’m here to help.

• Click to share on Facebook (Opens in new window)
• Click to share on LinkedIn (Opens in new window)
• Click to share on Twitter (Opens in new window)
• Click to share on Reddit (Opens in new window)

## How to Master Data Management in Research

25th April 2024 27th April 2024

## Thesis Title: Examples and Suggestions from a PhD Grad

23rd February 2024 23rd February 2024

## How to Stay Healthy as a Student

25th January 2024 25th January 2024

Notify me of follow-up comments by email.

This site uses Akismet to reduce spam. Learn how your comment data is processed .

## Privacy Overview

• SUGGESTED TOPICS
• The Magazine
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

## A Refresher on Statistical Significance

It’s too often misused and misunderstood.

When you run an experiment or analyze data, you want to know if your findings are “significant.” But business relevance (i.e., practical significance) isn’t always the same thing as confidence that a result isn’t due purely to chance (i.e., statistical significance). This is an important distinction; unfortunately, statistical significance is often misunderstood and misused in organizations today. And yet because more and more companies are relying on data to make critical business decisions, it’s an essential concept for managers to understand.

• Amy Gallo is a contributing editor at Harvard Business Review, cohost of the Women at Work podcast , and the author of two books: Getting Along: How to Work with Anyone (Even Difficult People) and the HBR Guide to Dealing with Conflict . She writes and speaks about workplace dynamics. Watch her TEDx talk on conflict and follow her on LinkedIn . amyegallo

## How To Write a Significance Statement for Your Research

A significance statement is an essential part of a research paper. It explains the importance and relevance of the study to the academic community and the world at large. To write a compelling significance statement, identify the research problem, explain why it is significant, provide evidence of its importance, and highlight its potential impact on future research, policy, or practice. A well-crafted significance statement should effectively communicate the value of the research to readers and help them understand why it matters.

Updated on May 4, 2023

A significance statement is a clearly stated, non-technical paragraph that explains why your research matters. It’s central in making the public aware of and gaining support for your research.

Write it in jargon-free language that a reader from any field can understand. Well-crafted, easily readable significance statements can improve your chances for citation and impact and make it easier for readers outside your field to find and understand your work.

Read on for more details on what a significance statement is, how it can enhance the impact of your research, and, of course, how to write one.

## What is a significance statement in research?

A significance statement answers the question: How will your research advance scientific knowledge and impact society at large (as well as specific populations)?

You might also see it called a “Significance of the study” statement. Some professional organizations in the STEM sciences and social sciences now recommended that journals in their disciplines make such statements a standard feature of each published article. Funding agencies also consider “significance” a key criterion for their awards.

Read some examples of significance statements from the Proceedings of the National Academy of Sciences (PNAS) here .

Depending upon the specific journal or funding agency’s requirements, your statement may be around 100 words and answer these questions:

1. What’s the purpose of this research?

2. What are its key findings?

3. Why do they matter?

4. Who benefits from the research results?

## Where to place the significance statement in your manuscript

Most journals ask you to place the significance statement before or after the abstract, so check with each journal’s guide.

This article is focused on the formal significance statement, even though you’ll naturally highlight your project’s significance elsewhere in your manuscript. (In the introduction, you’ll set out your research aims, and in the conclusion, you’ll explain the potential applications of your research and recommend areas for future research. You’re building an overall case for the value of your work.)

## Developing the significance statement

The main steps in planning and developing your statement are to assess the gaps to which your study contributes, and then define your work’s implications and impact.

## Identify what gaps your study fills and what it contributes

Your literature review was a big part of how you planned your study. To develop your research aims and objectives, you identified gaps or unanswered questions in the preceding research and designed your study to address them.

Go back to that lit review and look at those gaps again. Review your research proposal to refresh your memory. Ask:

• How have my research findings advanced knowledge or provided notable new insights?
• How has my research helped to prove (or disprove) a hypothesis or answer a research question?
• Why are those results important?

Consider your study’s potential impact at two levels:

• What contribution does my research make to my field?
• How does it specifically contribute to knowledge; that is, who will benefit the most from it?

## Define the implications and potential impact

As you make notes, keep the reasons in mind for why you are writing this statement. Whom will it impact, and why?

The first audience for your significance statement will be journal reviewers when you submit your article for publishing. Many journals require one for manuscript submissions. Study the author’s guide of your desired journal to see its criteria ( here’s an example ). Peer reviewers who can clearly understand the value of your research will be more likely to recommend publication.

Second, when you apply for funding, your significance statement will help justify why your research deserves a grant from a funding agency . The U.S. National Institutes of Health (NIH), for example, wants to see that a project will “exert a sustained, powerful influence on the research field(s) involved.” Clear, simple language is always valuable because not all reviewers will be specialists in your field.

You’re now ready to express the importance of your research clearly and concisely. Time to start writing.

## How to write a significance statement: Key elements

When drafting your statement, focus on both the content and writing style.

• In terms of content, emphasize the importance, timeliness, and relevance of your research results.
• Write the statement in plain, clear language rather than scientific or technical jargon. Your audience will include not just your fellow scientists but also non-specialists like journalists, funding reviewers, and members of the public.

Follow the process we outline below to build a solid, well-crafted, and informative statement.

## Get started

• The implications of this study are…
• Building upon previous contributions, our study moves the field forward because…
• Our study furthers previous understanding about…

## Include these components

Next, draft some sentences that include the following elements. A good example, which we’ll use here, is a significance statement by Rogers et al. (2022) published in the Journal of Climate .

1. Briefly situate your research study in its larger context . Start by introducing the topic, leading to a problem statement. Here’s an example:

‘Heatwaves pose a major threat to human health, ecosystems, and human systems.”

2. State the research problem.

“Simultaneous heatwaves affecting multiple regions can exacerbate such threats. For example, multiple food-producing regions simultaneously undergoing heat-related crop damage could drive global food shortages.”

“We assess recent changes in the occurrence of simultaneous large heatwaves.”

4. Provide brief but powerful evidence to support the claims your statement is making , Use quantifiable terms rather than vague ones (e.g., instead of “This phenomenon is happening now more than ever,” see below how Rogers et al. (2022) explained it). This evidence intensifies and illustrates the problem more vividly:

“Such simultaneous heatwaves are 7 times more likely now than 40 years ago. They are also hotter and affect a larger area. Their increasing occurrence is mainly driven by warming baseline temperatures due to global heating, but changes in weather patterns contribute to disproportionate increases over parts of Europe, the eastern United States, and Asia.

5. Relate your study’s impact to the broader context , starting with its general significance to society—then, when possible, move to the particular as you name specific applications of your research findings. (Our example lacks this second level of application.)

“Better understanding the drivers of weather pattern changes is therefore important for understanding future concurrent heatwave characteristics and their impacts.”

Don’t understate or overstate your findings – just make clear what your study contributes. When you have all the elements in place, review your draft to simplify and polish your language. Even better, get an expert AJE edit . Be sure to use “plain” language rather than academic jargon.

• Avoid acronyms, scientific jargon, and technical terms
• Use active verbs in your sentence structure rather than passive voice (e.g., instead of “It was found that...”, use “We found...”)
• Make sentence structures short, easy to understand – readable
• Try to address only one idea in each sentence and keep sentences within 25 words (15 words is even better)
• Eliminate nonessential words and phrases (“fluff” and wordiness)

## Enhance your significance statement’s impact

Always take time to review your draft multiple times. Make sure that you:

• Provide evidence to support your claims
• Relate the significance to the broader research context in your field

After revising your significance statement, request feedback from a reading mentor about how to make it even clearer. If you’re not a native English speaker, seek help from a native-English-speaking colleague or use an editing service like AJE to make sure your work is at a native level.

Understanding the significance of your study

Your readers may have much less interest than you do in the specific details of your research methods and measures. Many readers will scan your article to learn how your findings might apply to them and their own research.

## Different types of significance

Your findings may have different types of significance, relevant to different populations or fields of study for different reasons. You can emphasize your work’s statistical, clinical, or practical significance. Editors or reviewers in the social sciences might also evaluate your work’s social or political significance.

Statistical significance means that the results are unlikely to have occurred randomly. Instead, it implies a true cause-and-effect relationship.

Clinical significance means that your findings are applicable for treating patients and improving quality of life.

Practical significance is when your research outcomes are meaningful to society at large, in the “real world.” Practical significance is usually measured by the study’s  effect size . Similarly, evaluators may attribute social or political significance to research that addresses “real and immediate” social problems.

The AJE Team

Home » Research Findings – Types Examples and Writing Guide

## Research Findings

Definition:

Research findings refer to the results obtained from a study or investigation conducted through a systematic and scientific approach. These findings are the outcomes of the data analysis, interpretation, and evaluation carried out during the research process.

## Types of Research Findings

There are two main types of research findings:

## Qualitative Findings

Qualitative research is an exploratory research method used to understand the complexities of human behavior and experiences. Qualitative findings are non-numerical and descriptive data that describe the meaning and interpretation of the data collected. Examples of qualitative findings include quotes from participants, themes that emerge from the data, and descriptions of experiences and phenomena.

## Quantitative Findings

Quantitative research is a research method that uses numerical data and statistical analysis to measure and quantify a phenomenon or behavior. Quantitative findings include numerical data such as mean, median, and mode, as well as statistical analyses such as t-tests, ANOVA, and regression analysis. These findings are often presented in tables, graphs, or charts.

Both qualitative and quantitative findings are important in research and can provide different insights into a research question or problem. Combining both types of findings can provide a more comprehensive understanding of a phenomenon and improve the validity and reliability of research results.

## Parts of Research Findings

Research findings typically consist of several parts, including:

• Introduction: This section provides an overview of the research topic and the purpose of the study.
• Literature Review: This section summarizes previous research studies and findings that are relevant to the current study.
• Methodology : This section describes the research design, methods, and procedures used in the study, including details on the sample, data collection, and data analysis.
• Results : This section presents the findings of the study, including statistical analyses and data visualizations.
• Discussion : This section interprets the results and explains what they mean in relation to the research question(s) and hypotheses. It may also compare and contrast the current findings with previous research studies and explore any implications or limitations of the study.
• Conclusion : This section provides a summary of the key findings and the main conclusions of the study.
• Recommendations: This section suggests areas for further research and potential applications or implications of the study’s findings.

## How to Write Research Findings

Writing research findings requires careful planning and attention to detail. Here are some general steps to follow when writing research findings:

• Organize your findings: Before you begin writing, it’s essential to organize your findings logically. Consider creating an outline or a flowchart that outlines the main points you want to make and how they relate to one another.
• Use clear and concise language : When presenting your findings, be sure to use clear and concise language that is easy to understand. Avoid using jargon or technical terms unless they are necessary to convey your meaning.
• Use visual aids : Visual aids such as tables, charts, and graphs can be helpful in presenting your findings. Be sure to label and title your visual aids clearly, and make sure they are easy to read.
• Interpret your findings : When presenting your findings, it’s important to provide some interpretation of what the results mean. This can include discussing how your findings relate to the existing literature, identifying any limitations of your study, and suggesting areas for future research.
• Be precise and accurate : When presenting your findings, be sure to use precise and accurate language. Avoid making generalizations or overstatements and be careful not to misrepresent your data.
• Edit and revise: Once you have written your research findings, be sure to edit and revise them carefully. Check for grammar and spelling errors, make sure your formatting is consistent, and ensure that your writing is clear and concise.

## Research Findings Example

Following is a Research Findings Example sample for students:

Title: The Effects of Exercise on Mental Health

Sample : 500 participants, both men and women, between the ages of 18-45.

Methodology : Participants were divided into two groups. The first group engaged in 30 minutes of moderate intensity exercise five times a week for eight weeks. The second group did not exercise during the study period. Participants in both groups completed a questionnaire that assessed their mental health before and after the study period.

Findings : The group that engaged in regular exercise reported a significant improvement in mental health compared to the control group. Specifically, they reported lower levels of anxiety and depression, improved mood, and increased self-esteem.

Conclusion : Regular exercise can have a positive impact on mental health and may be an effective intervention for individuals experiencing symptoms of anxiety or depression.

## Applications of Research Findings

Research findings can be applied in various fields to improve processes, products, services, and outcomes. Here are some examples:

• Healthcare : Research findings in medicine and healthcare can be applied to improve patient outcomes, reduce morbidity and mortality rates, and develop new treatments for various diseases.
• Education : Research findings in education can be used to develop effective teaching methods, improve learning outcomes, and design new educational programs.
• Technology : Research findings in technology can be applied to develop new products, improve existing products, and enhance user experiences.
• Business : Research findings in business can be applied to develop new strategies, improve operations, and increase profitability.
• Public Policy: Research findings can be used to inform public policy decisions on issues such as environmental protection, social welfare, and economic development.
• Social Sciences: Research findings in social sciences can be used to improve understanding of human behavior and social phenomena, inform public policy decisions, and develop interventions to address social issues.
• Agriculture: Research findings in agriculture can be applied to improve crop yields, develop new farming techniques, and enhance food security.
• Sports : Research findings in sports can be applied to improve athlete performance, reduce injuries, and develop new training programs.

## When to use Research Findings

Research findings can be used in a variety of situations, depending on the context and the purpose. Here are some examples of when research findings may be useful:

• Decision-making : Research findings can be used to inform decisions in various fields, such as business, education, healthcare, and public policy. For example, a business may use market research findings to make decisions about new product development or marketing strategies.
• Problem-solving : Research findings can be used to solve problems or challenges in various fields, such as healthcare, engineering, and social sciences. For example, medical researchers may use findings from clinical trials to develop new treatments for diseases.
• Policy development : Research findings can be used to inform the development of policies in various fields, such as environmental protection, social welfare, and economic development. For example, policymakers may use research findings to develop policies aimed at reducing greenhouse gas emissions.
• Program evaluation: Research findings can be used to evaluate the effectiveness of programs or interventions in various fields, such as education, healthcare, and social services. For example, educational researchers may use findings from evaluations of educational programs to improve teaching and learning outcomes.
• Innovation: Research findings can be used to inspire or guide innovation in various fields, such as technology and engineering. For example, engineers may use research findings on materials science to develop new and innovative products.

## Purpose of Research Findings

The purpose of research findings is to contribute to the knowledge and understanding of a particular topic or issue. Research findings are the result of a systematic and rigorous investigation of a research question or hypothesis, using appropriate research methods and techniques.

The main purposes of research findings are:

• To generate new knowledge : Research findings contribute to the body of knowledge on a particular topic, by adding new information, insights, and understanding to the existing knowledge base.
• To test hypotheses or theories : Research findings can be used to test hypotheses or theories that have been proposed in a particular field or discipline. This helps to determine the validity and reliability of the hypotheses or theories, and to refine or develop new ones.
• To inform practice: Research findings can be used to inform practice in various fields, such as healthcare, education, and business. By identifying best practices and evidence-based interventions, research findings can help practitioners to make informed decisions and improve outcomes.
• To identify gaps in knowledge: Research findings can help to identify gaps in knowledge and understanding of a particular topic, which can then be addressed by further research.
• To contribute to policy development: Research findings can be used to inform policy development in various fields, such as environmental protection, social welfare, and economic development. By providing evidence-based recommendations, research findings can help policymakers to develop effective policies that address societal challenges.

## Characteristics of Research Findings

Research findings have several key characteristics that distinguish them from other types of information or knowledge. Here are some of the main characteristics of research findings:

• Objective : Research findings are based on a systematic and rigorous investigation of a research question or hypothesis, using appropriate research methods and techniques. As such, they are generally considered to be more objective and reliable than other types of information.
• Empirical : Research findings are based on empirical evidence, which means that they are derived from observations or measurements of the real world. This gives them a high degree of credibility and validity.
• Generalizable : Research findings are often intended to be generalizable to a larger population or context beyond the specific study. This means that the findings can be applied to other situations or populations with similar characteristics.
• Transparent : Research findings are typically reported in a transparent manner, with a clear description of the research methods and data analysis techniques used. This allows others to assess the credibility and reliability of the findings.
• Peer-reviewed: Research findings are often subject to a rigorous peer-review process, in which experts in the field review the research methods, data analysis, and conclusions of the study. This helps to ensure the validity and reliability of the findings.
• Reproducible : Research findings are often designed to be reproducible, meaning that other researchers can replicate the study using the same methods and obtain similar results. This helps to ensure the validity and reliability of the findings.

Research findings have many advantages, which make them valuable sources of knowledge and information. Here are some of the main advantages of research findings:

• Evidence-based: Research findings are based on empirical evidence, which means that they are grounded in data and observations from the real world. This makes them a reliable and credible source of information.
• Inform decision-making: Research findings can be used to inform decision-making in various fields, such as healthcare, education, and business. By identifying best practices and evidence-based interventions, research findings can help practitioners and policymakers to make informed decisions and improve outcomes.
• Identify gaps in knowledge: Research findings can help to identify gaps in knowledge and understanding of a particular topic, which can then be addressed by further research. This contributes to the ongoing development of knowledge in various fields.
• Improve outcomes : Research findings can be used to develop and implement evidence-based practices and interventions, which have been shown to improve outcomes in various fields, such as healthcare, education, and social services.
• Foster innovation: Research findings can inspire or guide innovation in various fields, such as technology and engineering. By providing new information and understanding of a particular topic, research findings can stimulate new ideas and approaches to problem-solving.
• Enhance credibility: Research findings are generally considered to be more credible and reliable than other types of information, as they are based on rigorous research methods and are subject to peer-review processes.

## Limitations of Research Findings

While research findings have many advantages, they also have some limitations. Here are some of the main limitations of research findings:

• Limited scope: Research findings are typically based on a particular study or set of studies, which may have a limited scope or focus. This means that they may not be applicable to other contexts or populations.
• Potential for bias : Research findings can be influenced by various sources of bias, such as researcher bias, selection bias, or measurement bias. This can affect the validity and reliability of the findings.
• Ethical considerations: Research findings can raise ethical considerations, particularly in studies involving human subjects. Researchers must ensure that their studies are conducted in an ethical and responsible manner, with appropriate measures to protect the welfare and privacy of participants.
• Time and resource constraints : Research studies can be time-consuming and require significant resources, which can limit the number and scope of studies that are conducted. This can lead to gaps in knowledge or a lack of research on certain topics.
• Complexity: Some research findings can be complex and difficult to interpret, particularly in fields such as science or medicine. This can make it challenging for practitioners and policymakers to apply the findings to their work.
• Lack of generalizability : While research findings are intended to be generalizable to larger populations or contexts, there may be factors that limit their generalizability. For example, cultural or environmental factors may influence how a particular intervention or treatment works in different populations or contexts.

## Significance of a Study: Revisiting the “So What” Question

• Open Access
• First Online: 03 December 2022

## Cite this chapter

• James Hiebert 6 ,
• Jinfa Cai 7 ,
• Stephen Hwang 7 ,
• Anne K Morris 6 &
• Charles Hohensee 6

Part of the book series: Research in Mathematics Education ((RME))

17k Accesses

Every researcher wants their study to matter—to make a positive difference for their professional communities. To ensure your study matters, you can formulate clear hypotheses and choose methods that will test them well, as described in Chaps. 1, 2, 3 and 4. You can go further, however, by considering some of the terms commonly used to describe the importance of studies, terms like significance, contributions, and implications. As you clarify for yourself the meanings of these terms, you learn that whether your study matters depends on how convincingly you can argue for its importance. Perhaps most surprising is that convincing others of its importance rests with the case you make before the data are ever gathered. The importance of your hypotheses should be apparent before you test them. Are your predictions about things the profession cares about? Can you make them with a striking degree of precision? Are the rationales that support them compelling? You are answering the “So what?” question as you formulate hypotheses and design tests of them. This means you can control the answer. You do not need to cross your fingers and hope as you collect data.

## Part I. Setting the Groundwork

One of the most common questions asked of researchers is “So what?” What difference does your study make? Why are the findings important? The “so what” question is one of the most basic questions, often perceived by novice researchers as the most difficult question to answer. Indeed, addressing the “so what” question continues to challenge even experienced researchers. It is not always easy to articulate a convincing argument for the importance of your work. It can be especially difficult to describe its importance without falling into the trap of making claims that reach beyond the data.

That this issue is a challenge for researchers is illustrated by our analysis of reviewer comments for JRME . About one-third of the reviews for manuscripts that were ultimately rejected included concerns about the importance of the study. Said another way, reviewers felt the “So what?” question had not been answered. To paraphrase one journal reviewer, “The manuscript left me unsure of what the contribution of this work to the field’s knowledge is, and therefore I doubt its significance.” We expect this is a frequent concern of reviewers for all research journals.

Our goal in this chapter is to help you navigate the pressing demands of journal reviewers, editors, and readers for demonstrating the importance of your work while staying within the bounds of acceptable claims based on your results. We will begin by reviewing what we have said about these issues in previous chapters. We will then clarify one of the confusing aspects of developing appropriate arguments—the absence of consensus definitions of key terms such as significance, contributions, and implications. Based on the definitions we propose, we will examine the critical role of alignment for realizing the potential significance of your study. Because the importance of your study is communicated through your evolving research paper, we will fold suggestions for writing your paper into the discussion of creating and executing your study.

We laid the groundwork in Chap. 1 for what we consider to be important research in education:

In our view, the ultimate goal of education is to offer all students the best possible learning opportunities. So, we believe the ultimate purpose of scientific inquiry in education is to support the improvement of learning opportunities for all students…. If there is no way to imagine a connection to improving learning opportunities for students, even a distant connection, we recommend you reconsider whether it is an important hypothesis within the education community.

Of course, you might prefer another “ultimate purpose” for research in education. That’s fine. The critical point is that the argument for the importance of the hypotheses you are testing should be connected to the value of a long-term goal you can describe. As long as most of the educational community agrees with this goal, and you can show how testing your hypotheses will move the field forward to achieving this goal, you will have developed a convincing argument for the importance of your work.

In Chap. 2 , we argued the importance of your hypotheses can and should be established before you collect data. Your theoretical framework should carry the weight of your argument because it should describe how your hypotheses will extend what is already known. Your methods should then show that you will test your hypotheses in an appropriate way—in a way that will allow you to detect how the results did, and did not, confirm the hypotheses. This will, in turn, allow you to formulate revised hypotheses. We described establishing the importance of your study by saying, “The importance can come from the fact that, based on the results, you will be able to offer revised hypotheses that help the field better understand an issue relevant for improving all students’ learning opportunities.”

The ideas from Chaps. 1 , 2 , and 3 go a long way toward setting the parameters for what counts as an important study and how its importance can be determined. Chapter 4 focused on ensuring that the importance of a study can be realized. The next section fills in the details by proposing definitions for the most common terms used to claim importance: significance, contributions, and implications.

You might notice that we do not have a chapter dedicated to discussing the presentation of the findings—that is, a “results” chapter. We do not mean to imply that presenting results is trivial. However, we believe that if you follow our recommendations for writing your evolving research paper, presenting the results will be quite straightforward. The key is to present your results so they can be most easily compared with your predictions. This means, among other things, organizing your presentation of results according to your earlier presentation of hypotheses.

## Part II. Clarifying Importance by Revisiting the Definitions of Key Terms

What does it mean to say your findings are significant? Statistical significance is clear. There are widely accepted standards for determining the statistical significance of findings. But what about educational significance? Is this the same as claiming that your study makes an important contribution? Or, that your study has important implications? Different researchers might answer these questions in different ways. When key terms like these are overused, their definitions gradually broaden or shift, and they can lose their meaning. That is unfortunate, because it creates confusion about how to develop claims for the importance of a study.

By clarifying the definitions, we hope to clarify what is required to claim that a study is significant , that it makes a contribution , and that it has important implications . Not everyone defines the terms as we do. Our definitions are probably a bit narrower or more targeted than those you may encounter elsewhere. Depending on where you want to publish your study, you may want to adapt your use of these terms to match more closely the expectations of a particular journal. But the way we define and address these terms is not antithetical to common uses. And we believe ridding the terms of unnecessary overlap allows us to discriminate among different key concepts with respect to claims for the importance of research studies. It is not necessary to define the terms exactly as we have, but it is critical that the ideas embedded in our definitions be distinguished and that all of them be taken into account when examining the importance of a study.

We will use the following definitions:

Significance: The importance of the problem, questions, and/or hypotheses for improving the learning opportunities for all students (you can substitute a different long-term goal if its value is widely shared). Significance can be determined before data are gathered. Significance is an attribute of the research problem , not the research findings .

Contributions : The value of the findings for revising the hypotheses, making clear what has been learned, what is now better understood.

Implications : Deductions about what can be concluded from the findings that are not already included in “contributions.” The most common deductions in educational research are for improving educational practice. Deductions for research practice that are not already defined as contributions are often suggestions about research methods that are especially useful or methods to avoid.

## Significance

The significance of a study is built by formulating research questions and hypotheses you connect through a careful argument to a long-term goal of widely shared value (e.g., improving learning opportunities for all students). Significance applies both to the domain in which your study is located and to your individual study. The significance of the domain is established by choosing a goal of widely shared value and then identifying a domain you can show is connected to achieving the goal. For example, if the goal to which your study contributes is improving the learning opportunities for all students, your study might aim to understand more fully how things work in a domain such as teaching for conceptual understanding, or preparing teachers to attend to all students, or designing curricula to support all learners, or connecting learning opportunities to particular learning outcomes.

The significance of your individual study is something you build ; it is not predetermined or self-evident. Significance of a study is established by making a case for it, not by simply choosing hypotheses everyone already thinks are important. Although you might believe the significance of your study is obvious, readers will need to be convinced.

Significance is something you develop in your evolving research paper. The theoretical framework you present connects your study to what has been investigated previously. Your argument for significance of the domain comes from the significance of the line of research of which your study is a part. The significance of your study is developed by showing, through the presentation of your framework, how your study advances this line of research. This means the lion’s share of your answer to the “So what?” question will be developed as part of your theoretical framework.

Although defining significance as located in your paper prior to presenting results is not a definition universally shared among educational researchers, it is becoming an increasingly common view. In fact, there is movement toward evaluating the significance of a study based only on the first sections of a research paper—the sections prior to the results (Makel et al., 2021 ).

In addition to addressing the “So what?” question, your theoretical framework can address another common concern often voiced by readers: “What is so interesting? I could have predicted those results.” Predictions do not need to be surprising to be interesting and significant. The significance comes from the rationales that show how the predictions extend what is currently known. It is irrelevant how many researchers could have made the predictions. What makes a study significant is that the theoretical framework and the predictions make clear how the study will increase the field’s understanding toward achieving a goal of shared value.

An important consequence of interpreting significance as a carefully developed argument for the importance of your research study within a larger domain is that it reveals the advantage of conducting a series of connected studies rather than single, disconnected studies. Building the significance of a research study requires time and effort. Once you have established significance for a particular study, you can build on this same argument for related studies. This saves time, allows you to continue to refine your argument across studies, and increases the likelihood your studies will contribute to the field.

## Contributions

As we have noted, in fields as complicated as education, it is unlikely that your predictions will be entirely accurate. If the problem you are investigating is significant, the hypotheses will be formulated in such a way that they extend a line of research to understand more deeply phenomena related to students’ learning opportunities or another goal of shared value. Often, this means investigating the conditions under which phenomena occur. This gets complicated very quickly, so the data you gather will likely differ from your predictions in a variety of ways. The contributions your study makes will depend on how you interpret these results in light of the original hypotheses.

## Contributions Emerge from Revisions to your Hypotheses

We view interpreting results as a process of comparing the data with the predictions and then examining the way in which hypotheses should be revised to more fully account for the results. Revising will almost always be warranted because, as we noted, predictions are unlikely to be entirely accurate. For example, if researchers expect Outcome A to occur under specified conditions but find that it does not occur to the extent predicted or actually does occur but without all the conditions, they must ask what changes to the hypotheses are needed to predict more accurately the conditions under which Outcome A occurred. Are there, for example, essential conditions that were not anticipated and that should be included in the revised hypotheses?

Consider an example from a recently published study (Wang et al., 2021 ). A team of researchers investigated the following research question: “How are students’ perceptions of their parents’ expectations related to students’ mathematics-related beliefs and their perceived mathematics achievement?” The researchers predicted that students’ perceptions of their parents’ expectations would be highly related to students’ mathematics-related beliefs and their perceived mathematics achievement. The rationale was based largely on prior research that had consistently found parents’ general educational expectations to be highly correlated with students’ achievement.

The findings showed that Chinese high school students’ perceptions of their parents’ educational expectations were positively related to these students’ mathematics-related beliefs. In other words, students who believed their parents expected them to attain higher levels of education had more desirable mathematics-related beliefs.

However, students’ perceptions of their parents’ expectations about mathematics achievement were not related to students’ mathematics-related beliefs in the same way as the more general parental educational expectations. Students who reported that their parents had no specific expectations possessed more desirable mathematics-related beliefs than all other subgroups. In addition, these students tended to perceive their mathematics achievement rank in their class to be higher on average than students who reported that their parents expressed some level of expectation for mathematics achievement.

Because this finding was not predicted, the researchers revised the original hypothesis. Their new prediction was that students who believe their parents have no specific mathematics achievement expectations possess more positive mathematics-related beliefs and higher perceived mathematics achievement than students who believe their parents do have specific expectations. They developed a revised rationale that drew on research on parental pressure and mathematics anxiety, positing that parents’ specific mathematics achievement expectations might increase their children’s sense of pressure and anxiety, thus fostering less positive mathematics-related beliefs. The team then conducted a follow-up study. Their findings aligned more closely with the new predictions and affirmed the better explanatory power of the revised rationale. The contributions of the study are found in this increased explanatory power—in the new understandings of this phenomenon contained in the revisions to the rationale.

Interpreting findings in order to revise hypotheses is not a straightforward task. Usually, the rationales blend multiple constructs or variables and predict multiple outcomes, with different outcomes connected to different research questions and addressed by different sets of data. Nevertheless, the contributions of your study depend on specifying the differences between your original hypotheses and your revised hypotheses. What can you explain now that you could not explain before?

We believe that revising hypotheses is an optimal response to any question of contributions because a researcher’s initial hypotheses plus the revisions suggested by the data are the most productive way to tie a study into the larger chain of research of which it is a part. Revised hypotheses represent growth in knowledge. Building on other researchers’ revised hypotheses and revising them further by more explicitly and precisely describing the conditions that are expected to influence the outcomes in the next study accumulates knowledge in a form that can be recorded, shared, built upon, and improved.

The significance of your study is presented in the opening sections of your evolving research paper whereas the contributions are presented in the final section, after the results. In fact, the central focus in this “Discussion” section should be a specification of the contributions (note, though, that this guidance may not fully align with the requirements of some journals).

## Contributions Answer the Question of Generalizability

A common and often contentious, confusing issue that can befuddle novice and experienced researchers alike is the generalizability of results. All researchers prefer to believe the results they report apply to more than the sample of participants in their study. How important would a study be if the results applied only to, say, two fourth-grade classrooms in one school, or to the exact same tasks used as measures? How do you decide to which larger population (of students or tasks) your results could generalize? How can you state your claims so they are precisely those justified by the data?

To illustrate the challenge faced by researchers in answering these questions, we return to the JRME reviewers. We found that 30% of the reviews expressed concerns about the match between the results and the claims. For manuscripts that ultimately received a decision of Reject, the majority of reviewers said the authors had overreached—the claims were not supported by the data. In other words, authors generalized their claims beyond those that could be justified.

## The Connection Between Contributions and Generalizability

In our view, claims about contributions can be examined productively alongside considerations of generalizability. To make the case for this view, we need to back up a bit. Recall that the purpose of research is to understand a phenomenon. To understand a phenomenon, you need to determine the conditions under which it occurs. Consequently, productive hypotheses specify the conditions under which the predictions hold and explain why and how these conditions make a difference. And the conditions set the parameters on generalizability. They identify when, where, and for whom the effect or situation will occur. So, hypotheses describe the extent of expected generalizability, and revised hypotheses that contain the contributions recalibrate generalizability and offer new predictions within these parameters.

## An Example That Illustrates the Connection

In Chap. 4 , we introduced an example with a research question asking whether second graders improve their understanding of place value after a specially designed instructional intervention. We suggested asking a few second and third graders to complete your tasks to see if they generated the expected variation in performance. Suppose you completed this pilot study and now have satisfactory tasks. What conditions might influence the effect of the intervention? After careful study, you developed rationales that supported three conditions: the entry level of students’ understanding, the way in which the intervention is implemented, and the classroom norms that set expectations for students’ participation.

Suppose your original hypotheses predicted the desired effect of the intervention only if the students possessed an understanding of several concepts on which place value is built, only if the intervention was implemented with fidelity to the detailed instructional guidelines, and only if classroom norms encouraged students to participate in small-group work and whole-class discussions. Your claims of generalizability will apply to second-grade settings with these characteristics.

Now suppose you designed the study so the intervention occurred in five second-grade classrooms that agreed to participate. The pre-intervention assessment showed all students with the minimal level of entry understanding. The same well-trained teacher was employed to teach the intervention in all five classrooms, none of which included her own students. And you learned from prior observations and reports of the classroom teachers that three of the classrooms operated with the desired classroom norms, but two did not. Because of these conditions, your study is now designed to test one of your hypotheses—the desired effect will occur only if classroom norms encouraged students to participate in small-group work and whole-class discussions. This is the only condition that will vary; the other two (prior level of understanding and fidelity of implementation) are the same across classrooms so you will not learn how these affect the results.

Suppose the classrooms performed equally well on the post-intervention assessments. You expected lower performance in the two classrooms with less student participation, so you need to revise your hypotheses. The challenge is to explain the higher-than-expected performance of these students. Because you were interested in understanding the effects of this condition, you observed several lessons in all the classrooms during the intervention. You can now use this information to explain why the intervention worked equally well in classrooms with different norms.

Your revised hypothesis captures this part of your study’s contribution. You can now say more about the ways in which the intervention can help students improve their understanding of place value because you have different information about the role of classroom norms. This, in turn, allows you to specify more precisely the nature and extent of the generalizability of your findings. You now can generalize your findings to classrooms with different norms. However, because you did not learn more about the impact of students’ entry level understandings or of different kinds of implementation, the generalizability along these dimensions remains as limited as before.

This example is simplified. In many studies, the findings will be more complicated, and more conditions will likely be identified, some of which were anticipated and some of which emerged while conducting the study and analyzing the data. Nevertheless, the point is that generalizability should be tied to the conditions that are expected to affect the results. Further, unanticipated conditions almost always appear, so generalizations should be conservative and made with caution and humility. They are likely to change after testing the new predictions.

## Contributions Are Assured When Hypotheses Are Significant and Methods Are Appropriate and Aligned

We have argued that the contributions of your study are produced by the revised hypotheses you can formulate based on your results. Will these revisions always represent contributions to the field? What if the revisions are minor? What if your results do not inform revisions to your hypotheses?

We will answer these questions briefly now and then develop them further in Part IV of this chapter. The answer to the primary question is “yes,” your revisions will always be a contribution to the field if (1) your hypotheses are significant and (2) you crafted appropriate methods to test the hypotheses. This is true even if your revisions are minor or if your data are not as informative as you expected. However, this is true only if you meet the two conditions in the earlier sentence. The first condition (significant hypotheses) can be satisfied by following the suggestions in the earlier section on significance. The second condition (appropriate methods) is addressed further in Part III in this chapter.

## Implications

Before examining the role of methods in connecting significance with important contributions, we elaborate briefly our definition of “implications.” We reserve implications for the conclusions you can logically deduce from your findings that are not already presented as contributions. This means that, like contributions, implications are presented in the Discussion section of your research paper.

Many educational researchers present two types of implications: implications for future research and implications for practice. Although we are aware of this common usage, we believe our definition of “contributions” cover these implications. Clarifying why we call these “contributions” will explain why we largely reserve the word “implications” for recommendations regarding methods.

## Implications for Future Research

Implications for future research often include (1) recommendations for empirical studies that would extend the findings of this study, (2) inferences about the usefulness of theoretical constructs, and (3) conclusions about the advisability of using particular kinds of methods. Given our earlier definitions, we prefer to label the first two types of implications as contributions.

Consider recommendations for empirical studies. After analyzing the data and presenting the results, we have suggested you compare the results with those predicted, revise the rationales for the original predictions to account for the results, and make new predictions based on the revised rationales. It is precisely these new predictions that can form the basis for recommending future research. Testing these new predictions is what would most productively extend this line of research. It can sometimes sound as if researchers are recommending future studies based on hunches about what research might yield useful findings. But researchers can do better than this. It would be more productive to base recommendations on a careful analysis of how the predictions of the original study could be sharpened and improved.

Now consider inferences about the usefulness of theoretical constructs. Our argument for labeling these inferences as contributions is similar. Rationales for predictions are where the relevant theoretical constructs are located. Revisions to these rationales based on the differences between the results and the predictions reveal the theoretical constructs that were affirmed to support accurate predictions and those that must be revised. In our view, usefulness is determined through this revision process.

Implications that do not come under our meaning of contributions are in the third type of implications, namely the appropriateness of methods for generating rich contributions. These kinds of implications are produced by your evaluation of your methods: research design, sampling procedures, tasks, data collection procedures, and data analyses. Although not always included in the discussion of findings, we believe it would be helpful for researchers to identify particular methods that were useful for conducting their study and those that should be modified or avoided. We believe these are appropriately called implications.

## Implications for Practice

If the purpose of research is to better understand how to improve learning opportunities for all students, then it is appropriate to consider whether implications for improving educational practice can be drawn from the results of a study. How are these implications formulated? This is an important question because, in our view, these claims often come across as an afterthought, “Oh, I need to add some implications for practice.” But here is the sobering reality facing researchers: By any measure, the history of educational research shows that identifying these implications has had little positive effect on practice.

Perhaps the most challenging task for researchers who attempt to draw implications for practice is to interpret their findings for appropriate settings. A researcher who studied the instructional intervention for second graders on place value and found that average performance in the intervention classrooms improved more than in the textbook classrooms might be tempted to draw implications for practice. What should the researcher say? That second-grade teachers should adopt the intervention? Such an implication would be an overreach because, as we noted earlier, the findings cannot be generalized to all second-grade classrooms. Moreover, an improvement in average performance does not mean the intervention was better for all students.

The challenge is to identify the conditions under which the intervention would improve the learning opportunities for all students. Some of these conditions will be identified as the theoretical framework is built because the predictions need to account for these conditions. But some will be unforeseen, and some that are identified will not be informed by the findings. Recall that, in the study described earlier, a condition of entry level of understanding was hypothesized but the design of the study did not allow the researcher to draw any conclusions about its effect.

What can researchers say about implications for practice given the complexities involved in generalizing findings to other settings? We offer two recommendations. First, because it is difficult to specify all the conditions under which a phenomenon occurs, it is rarely appropriate to prescribe an educational practice. Researchers cannot anticipate the conditions under which individual teachers operate, conditions that often require adaptation of a suggested practice rather than implementation of a practice as prescribed.

Our second recommendation comes from returning to the purpose for educational research—to understand more fully how to improve learning opportunities for all students (or to achieve another goal of widely shared value). As we have described, understanding comes primarily from building and reevaluating rationales for your predictions. If you reach a new understanding related to improving learning opportunities, an understanding that could have practical implications, we recommend you share this understanding as an implication for practice.

For example, suppose the researcher who found better average performance of second graders after the intervention on place value had also studied several conditions under which performance improved. And suppose the researcher found that most students who did not improve their performance misunderstood a concept that appeared early in the intervention (e.g., the multiplicative relationship between positional values of a numeral). An implication for practice the researcher might share would be to describe the potential importance of understanding this concept early in the sequence of activities if teachers try out this intervention.

If you use our definitions, these implications for practice would be presented as contributions because they emerge directly from reevaluating and revising your rationales. We believe it is appropriate to use “Contributions” as the heading for this section in the Discussion section of your research paper. However, if editors prefer “Implications” we recommend following their suggestion.

We want to be clear that the terms you use for the different ways your study is important is not critical. We chose to define the terms significance, contributions, and implications in very specific and not universally shared ways to distinguish all the meanings of importance you should consider. Some of these can be established through your theoretical framework, some by the revisions of your hypotheses, and some by reflecting on the value of particular methods. The important thing, from our point of view, is that the ideas we defined for each of these terms are distinguished and recognized as specific ways of determining the importance of your study.

## Part III. The Role of Methods in Determining Contributions

We have argued that every part of the study (and of the evolving research paper) should be aligned. All parts should be connected through a coherent chain of reasoning. In this chapter, we argue that the chain of reasoning is not complete until the methods are presented and the results are interpreted and discussed. The methods of the study create a bridge that connects the introductory material (research questions, theoretical framework, literature review, hypotheses) with the results and interpretations.

The role that methods play in scientific inquiry is to ensure that your hypotheses will be tested appropriately so the significance of your study will yield its potential contributions. To do this, the methods must do more than follow the standard guidelines and be technically correct (see Chap. 4 ). They must also fit with the surrounding parts of the study. We call this coherence.

## Coherence Across the Phases of Scientific Inquiry

Coherence means the parts of a whole are fully aligned. When doing scientific inquiry, the early parts or phases should motivate the later phases. The methods you use should be motivated or explained by the earlier phases (e.g., research questions, theoretical framework, hypotheses). Your methods, in turn, should produce results that can be interpreted by comparing them with your predictions. Methods are aligned with earlier phases when you can use the rationales contained in your hypotheses to decide what kinds of data are needed to test your predictions, how best to gather these kinds of data, and what analyses should be performed (see Chap. 4 and Cai et al., 2019a ).

For a visual representation of this coherence, see Fig. 5.1 . Each box identifies an aspect of scientific inquiry. Hypotheses (shown in Box 1) include the rationales and predictions. Because the rationales encompass the theoretical framework and the literature review, Box 1 establishes the significance of the study. Box 2 represents the methods, which we defined in Chap. 4 as the entire set of procedures you will use, including the basic design, measures for collecting data, and analytic approaches. In Fig. 5.1 , the hypothesis in Box 1 points you to the methods you will use. That is, you will choose methods that provide data for analyses that will generate results or findings (Box 3) that allow you to make comparisons against your predictions. Based on those comparisons, you will revise your hypotheses and derive the contributions and implications of your study (Box 4).

The Chain of Coherence That Runs Through All Parts of a Research Study

We intend Fig. 5.1 to carry several messages. One is that coherence of a study and the associated research paper require all aspects of the study to flow from one into the other. Each set of prior entries must motivate and justify the next one. For example, the data and analyses you intend to gather and use in Box 2 (Methods) must be those that are motivated and explained by the research question and hypothesis (prediction and rationale) in Box 1.

A second message in the figure is that coherence includes Box 4, “Discussion.” Aligned with the first three boxes, the fourth box flows from these boxes but is also constrained by them. The contributions and implications authors describe in the Discussion section of the paper cannot go beyond what is allowed by the original hypotheses and the revisions to these hypotheses indicated by the findings.

## Methods Enable Significance to Yield Contributions

We begin this section by identifying a third message conveyed in Fig. 5.1 . The methods of the study, represented by Box 2, provide a bridge that connects the significance of the study (Box 1) with the contributions of the study (Box 4). The results (Box 3) indicate the nature of the contributions by determining the revisions to the original hypotheses.

In our view, the connecting role played by the methods is often underappreciated. Crafting appropriate methods aligned with the significance of the study, on one hand, and the interpretations, on the other, can determine whether a study is judged to make a contribution.

If the hypotheses are established as significant, and if appropriate methods are used to test the predictions, the study will make important contributions even if the data are not statistically significant. We can say this another way. When researchers establish the significance of the hypotheses (i.e., convince readers they are of interest to the field) and use methods that provide a sound test of these hypotheses, the data they present will be of interest regardless of how they turn out. This is why Makel et al. ( 2021 ) endorse a review process for publication that emphasizes the significance of the study as presented in the first sections of a research paper.

Treating the methods as connecting the introductory arguments to the interpretations of data prevent researchers from making a common mistake: When writing the research paper, some researchers lose track of the research questions and/or the predictions. In other words, results are presented but are not interpreted as answers to the research questions or compared with the predictions. It is as if the introductory material of the paper begins one story, and the interpretations of results ends a different story. Lack of alignment makes it impossible to tell one coherent story.

A final point is that the alignment of a study cannot be evaluated and appreciated if the methods are not fully described. Methods must be described clearly and completely in the research paper so readers can see how they flow from the earlier phases of the study and how they yield the data presented. We suggested in Chap. 4 a rule of thumb for deciding whether the methods have been fully described: “Readers should be able to replicate the study if they wish.”

## Part IV. Special Considerations that Affect a Study’s Contributions

We conclude Chap. 5 by addressing two additional issues that can affect how researchers interpret the results and make claims about the contributions of a study. Usually, researchers deal with these issues in the Discussion section of their research paper, but we believe it is useful to consider them as you plan and conduct your study. The issues can be posed as questions: How should I treat the limitations of my study? How should I deal with findings that are completely unexpected?

## Limitations of a Study

We can identify two kinds of limitations: (1) limitations that constrain your ability to interpret your results because of unfortunate choices you made, and (2) limitations that constrain your ability to generalize your results because of missing variables you could not fit into the scope of your study or did not anticipate. We recommend different ways of dealing with these.

## Limitations Due to Unfortunate Choices

All researchers make unfortunate choices. These are mistakes that could have been prevented. Often, they are choices in how a study was designed and/or executed. Maybe the sample did not have the characteristics assumed, or a task did not assess what was expected, or the intervention was not implemented as planned. Although many unfortunate choices can be prevented by thinking through the consequences of every decision or by conducting a well-designed pilot study or two, some will occur anyway. How should you deal with them?

The consequence of unfortunate choices is that the data do not test the hypotheses as precisely or completely as hoped. When this happens, the data must be interpreted with these constraints in mind. Almost always, this limits the researcher to making fewer or narrower claims than desired about differences and similarities between the results and the predictions. Usually this means conclusions about the ways in which the rationales must be revised require extra qualifications. In other words, claims about contributions of the study must be made with extra caution.

Research papers frequently include a subsection in the Discussion called “Limitations of the Study.” Researchers often use this subsection to identify the study’s limitations by describing the unfortunate choices, but they do not always spell out how these limitations should affect the contributions of the paper. Sometimes, it appears that researchers are simply checking off a requirement to identify the limitations by saying something like “The results should be interpreted with caution.” But this does not help readers understand exactly what cautions should be applied and it does not hold researchers accountable for the limitations.

We recommend something different. We suggest you do the hard work of figuring out how the data should be interpreted in light of the limitations and share these details with the readers. You might do this when the results are presented or when you interpret them. Rather than presenting your claims about the contributions of the study and then saying readers should interpret these with “caution” because of the study’s limitations, we suggest presenting only those interpretations and claims of contributions that can be made with the limitations in mind.

One way to think about the constraints you will likely need to impose on your interpretations is in terms of generalizability. Recall that earlier in this chapter, we described the close relationship between contributions and generalizability. When generalizability is restricted, so are contributions.

## Limitations Due to Missing Variables

Because of the complexity of problems, questions, and hypotheses explored in educational research, researchers are unlikely to anticipate in their studies all the variables that affect the data and results. In addition, tradeoffs often must be made. Researchers cannot study everything at once, so decisions must be made about which variables to study carefully and which to either control or ignore.

In the earlier example of studying whether second graders improve their understanding of place value after a specially designed instructional intervention, the researcher identified three variables that were expected to influence the effect of the intervention: students’ entry level of understanding, implementation of the intervention, and norms of the classrooms in which the intervention was implemented. The researcher decided to control the implementation variable by hiring one experienced teacher to implement the intervention in all the classrooms. This meant the variable of individual teacher differences was not included in the study and the researcher could not generalize to classrooms with these differences.

Some researchers might see controlling the implementation of the intervention as a limitation. We do not. As a factor that is not allowed to vary, it constrains the generalizations a researcher can make, but we believe these kinds of controlled variables are better treated as opportunities for future research. Perhaps the researcher’s observations in the classroom provided information that could be used to make some predictions about which elements of the intervention are essential and which are optional—about which aspects of the intervention must be implemented as written and which can vary with different teachers. When revising the rationales to show what was learned in this study, the researcher could include rationales for new, tentative predictions about the effects of the intervention in classrooms where implementation differed in specified ways. These predictions create a genuine contribution of the study. If you use our definitions, these new predictions, often presented under “implications for future research,” would be presented as “contributions.”

Notice that if you follow our advice, you would not need to include a separate section in the Discussion of your paper labeled “Limitations.” We acknowledge, however, that some journal editors recommend such a subsection. In this case, we suggest you include this subsection along with treating the two different kinds of limitations as we recommend. You can do both.

## Dealing with Unexpected Findings

Researchers are often faced with unexpected and perhaps surprising results, even when they have developed a convincing theoretical framework, posed research questions tightly connected to this framework, presented predictions about expected outcomes, and selected methods that appropriately test these predictions. Indeed, the unexpected findings can be the most interesting and valuable products of the study. They can range from mildly surprising to “Wow. I didn’t expect that.” How should researchers treat such findings? Our answer is based on two principles.

The first principle is that the value of research does not lie in whether the predictions are completely accurate but in helping the field learn more about the explanatory power of theoretical frameworks. That is, the value lies in the increased understanding of phenomena generated by examining the ability of theoretical frameworks (or rationales) to predict outcomes and explain results. The second principle, a corollary to the first, is to treat unexpected findings in a way that is most educative for the reader.

Based on our arguments to this point, you could guess we will say there will always be unexpected findings. Predicted answers to significant research questions in education will rarely, if ever, be entirely accurate. So, you can count on dealing with unexpected findings.

Consistent with the two principles above, your goal should be to use unexpected findings to understand more fully the phenomenon under investigation. We recommend one of three different paths. The choice of which path to take depends on what you decide after reflecting again on the decisions you made at each phase of the study.

The first path is appropriate when researchers reexamine their theoretical framework in light of the unexpected findings and decide that it is still a compelling framework based on previous work. They reason that readers are likely to have been convinced by this framework and would likely have made similar predictions. In this case, we believe that it is educative for researchers to (a) summarize their initial framework, (b) present the findings and distinguish those that were aligned with the predictions from those that were not, and (c) explain why the theoretical framework was inadequate and propose changes to the framework that would have created more alignment with the unexpected findings.

Revisions to initial hypotheses are especially useful if they include explanations for why a researcher might have been wrong (and researchers who ask significant questions in domains as complex as education are almost always wrong in some way). Depending on the ways in which the revised framework differs from the original, the authors have two options. If the revised framework is an expansion of the original, it would be appropriate for the authors to propose directions for future research that would extend this study. Alternatively, if the revised framework is still largely within the scope of the original study and consists of revisions to the original hypotheses, the revisions could guide a second study to check the adequacy of the revisions. This second study could be conducted by the same researchers (perhaps before the final manuscript is written and presented as two parts of the same report) or it could be proposed in the Discussion as a specific study that could be conducted by other researchers.

The second path is appropriate when researchers reexamine their theoretical framework in light of the unexpected findings and recognize serious flaws in the framework. The flaws could result from a number of factors, including defining elements of the framework in too general a way to formulate well-grounded hypotheses, failing to include a variable, or not accounting carefully enough for the previous work in this domain, both theoretical and empirical. In many of these cases, readers would not be well served by reading a poorly developed framework and then learning that the framework, which had not been convincing, did not accurately predict the results. Before scrapping the study and starting over, we suggest stepping back and reexamining the framework. Is it possible to develop a more coherent, complete, and convincing framework? Would this framework predict the results more accurately? If the findings remain unexpected based on the predictions generated by this revised, more compelling framework, then the first path applies.

It is likely that the new framework will better predict the findings. After all, the researchers now know the findings they will report. However, it is unlikely that the framework will accurately predict all the findings. This is because the framework is not built around the findings of this study of which authors are now aware (but have not yet been presented). Frameworks are built on research and theory already published. This means the redesigned framework is built from exactly the same empirical findings and theoretical arguments available before the study was conducted. The redesigned framework also is constrained by needing to justify exactly those methods used in the study. The redesigned framework cannot justify different methods or even slightly altered methods. The task for researchers is to show how the new theoretical framework necessarily generates, using the same methods, the predictions they present in the research paper. Just as before, it is unlikely this framework can account for all the findings. Just as before, after presenting the results the researchers should explain why they believe particular hypotheses were confirmed and why others should be revised, even in small ways, based on the findings reported. Researchers can now use these findings to revise the hypotheses presented in the paper. The point we are making is that we believe it is acceptable to reconstruct frameworks before writing research reports if doing so would be more educative for the reader.

Finally, the third path becomes appropriate when researchers, in reexamining their theoretical framework, trace the problem to a misalignment between the methods they used and the theoretical framework or the research questions. Perhaps the researchers recognize that the tasks they used did not yield data that could test the predictions and address the research questions. Or perhaps the researchers realize that the sample they selected would likely have been heavily influenced by a factor they failed to take into account. In other words, the researchers decide that the unexpected findings were due to a problem with the methods they used, not with the framework or the accompanying predictions. In this case, we recommend that the researchers correct the methodological problems and conduct the study again.

## Part V. A Few Suggestions for Structuring Your Discussion Section

Writing the Discussion section of your research paper can be overwhelming given all our suggestions about what to include in this section. Here are a few tips that might help you create a simple template for this section.

We recommend the Discussion begin with a brief summary of the main results, especially those you will interpret in this section. This summary should not contain new data or results not previously presented in the paper.

The Discussion could then move to presenting the contributions in the ways we have described. To do this you could point out the ways in which the results differed from the predictions and suggest revisions to your rationales that would have better predicted the results. Doing this will show how the contributions of your study extend what is known beyond the research you drew on to build your original rationale. You can then propose how to extend your contributions to research by proposing future research studies that would test your new predictions. If you believe the revisions you make to your rationales produce new insights or understandings that could be helpful for educational practitioners, you can identify these contributions to practice as well. This comprises the bulk of the Discussion section.

If you have embedded the limitations in earlier sections of the paper, you will have presented your results and interpreted your findings constrained by these limitations. If you choose (or are asked) to describe limitations in the Discussion, you could identify the limitations and then point to the ways they affected your interpretations of the findings. Finally, the Discussion could conclude with the implications of the study for methodological choices that could improve research in the domain in which your study is located or how future studies could overcome the limitations you identified.

Because we are providing guidance on writing your research paper for publication, we will reiterate here that you should investigate the expectations and conventions of the journal to which you will submit your paper. Usually, it will be acceptable to use the terms “significance,” “contributions,” and “implications” as we have defined them. However, if the editors expect you to use the terms differently, follow the editors’ expectations. Our definitions in this chapter are meant to help you think clearly about the different ways you can make a case for the importance of your research. What matters is that you have carefully built and described a coherent chain of scientific inquiry that allows your study to translate the significance of your research problem into contributions to the field.

We began the chapter with the “So what?” question. The question looks simple and straightforward but is challenging and complicated. Its simple appearance can lead researchers to believe it should have a simple answer. But it almost never does. In this chapter, we tried to address the many complications that arise when answering the question. We hope you now have some new insights and new tools for answering the question in your next study.

Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., Cirillo, M., Kramer, S. L., & Hiebert, J. (2019a). Choosing and justifying robust methods for educational research. Journal for Research in Mathematics Education, 50 (4), 342–348. https://doi.org/10.5951/jresematheduc.50.2.0114

Makel, M. C., Hodges, J., Cook, G. B., & Plucker, J. A. (2021). Both questionable and open research practices are prevalent in education research. Educational Researcher, 50 (8), 493–504.

Wang, G., Zhang, S., & Cai, J. (2021). How are parental expectations related to students’ beliefs and their perceived achievement? Educational Studies in Mathematics., 108 , 429–450. https://doi.org/10.1007/s10649-021-10073-w

## Author information

Authors and affiliations.

School of Education, University of Delaware, Newark, DE, USA

James Hiebert, Anne K Morris & Charles Hohensee

Department of Mathematical Sciences, University of Delaware, Newark, DE, USA

Jinfa Cai & Stephen Hwang

You can also search for this author in PubMed   Google Scholar

## Rights and permissions

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Hiebert, J., Cai, J., Hwang, S., Morris, A.K., Hohensee, C. (2023). Significance of a Study: Revisiting the “So What” Question. In: Doing Research: A New Researcher’s Guide. Research in Mathematics Education. Springer, Cham. https://doi.org/10.1007/978-3-031-19078-0_5

DOI : https://doi.org/10.1007/978-3-031-19078-0_5

Published : 03 December 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-19077-3

Online ISBN : 978-3-031-19078-0

eBook Packages : Education Education (R0)

Anyone you share the following link with will be able to read this content:

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal

## What it means when scientists say their results are ‘significant’

Post-Doctoral Researcher, South Australian Health & Medical Research Institute

## Disclosure statement

Yazad Irani does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

View all partners

This article is part of the series This is research , where we ask academics to share and discuss open access articles that reveal important aspects of science. Today’s piece looks at statistics, and how to interpret them for meaning in the real world.

Let’s face it, scientific papers aren’t exactly page turners. They are written by scientists, for scientists, and often in a language that seems to only vaguely resemble English.

And perhaps one of the most daunting aspects of a scientific paper is the statistics (“stats”) section.

But what do stats really mean in the real world? Here’s an example from leukaemia research to help you break it down.

Read more: Our survey found 'questionable research practices' by ecologists and biologists – here's what that means

## Strength of results

Stats are key to good research – they help researchers determine whether the results observed are strong enough to be due to an important scientific phenomenon.

As a research student I would always look for the magic number which indicates statistically significant differences in my experiments: most people agree this number to be 0.05 (you may see this in a paper written as p < 0.05).

When comparing two groups in a scientific study, statistical significance indicated by a p-value of less than 0.05 means that, in the case where there was no real difference between groups, there’s less than a 5% chance of the observed result arising.

But the focus on looking for statistically significant differences can blind us to the bigger picture. As I advanced through my scientific training I learnt to look for biologically significant differences.

Read more: The curious case of the missing workplace teaspoons

## Biological significance

Biological significance addresses the question of whether the statistical difference actually means anything in terms of a real outcome, like a disease. Can the result explain how the disease is caused? Does it provide a new avenue to treat the disease? Basically, is it relevant?

A recent paper published in the journal Leukaemia will help explain my point. The paper looked at why some people are able to stop their treatment for chronic myeloid leukaemia without the cancer coming back, while in others the cancer relapsed.

The key finding of this study was that patients who did not relapse had a higher proportion of natural killer cells compared to patients that did relapse.

Natural killer cells are a type of immune cell that controls viral infections and tumours. So, the more cells there are to kill the cancer, the less likely the cancer was to relapse – makes sense!

This finding has the potential to guide doctors in seeing which patients are likely to remain cancer-free after stopping treatment. This is definitely biologically significant.

Read more: My cancer is in remission – does this mean I'm cured?

## Not so relevant

Another result from the same paper (Figure 3a if you want to click through to the data) shows a statistically significant difference in a sub-type of natural killer cells (called adaptive natural killer cells). But is this difference biologically relevant?

At this stage there is little evidence of a role for adaptive natural killer cells in the context of leukaemia. Also, the difference between the groups is relatively small, with a large variation within the groups (there are large error bars on the graph).

These factors make it more likely that the differences may be due to the mathematics involved in the statistical test rather than a biological effect. As with any new finding, time and further studies will be vital in working out whether this result actually means anything.

## Act like an expert

So how do you pick if the statistical differences have biological value? Being a highly trained expert in the field certainly helps.

Another way to determine if the findings in a paper have biological relevance is to look for other papers that show similar results. If a result is “real” it should be found by other scientists who will build on it and publish more papers.

This means there will be lots of papers for you to read and apply your new-found passion for statistics.

The open access research paper for this analysis is Increased proportion of mature NK cells is associated with successful imatinib discontinuation in chronic myeloid leukemia .

The definition of statistical significance has been edited since this article was first published.

• This is research

## User Preferences

Content preview.

Arcu felis bibendum ut tristique et egestas quis:

• Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
• Duis aute irure dolor in reprehenderit in voluptate
• Excepteur sint occaecat cupidatat non proident

## Keyboard Shortcuts

6.4 - practical significance.

In the last lesson, you learned how to identify statistically significant differences using hypothesis testing methods. If the p value is less than the $$\alpha$$ level (typically 0.05), then the results are  statistically significant . Results are said to be statistically significant when the difference between the hypothesized population parameter and observed sample statistic is large enough to conclude that it is unlikely to have occurred by chance.

Practical significance  refers to the magnitude of the difference, which is known as the  effect size . Results are practically significant when the difference is large enough to be meaningful in real life. What is meaningful may be subjective and may depend on the context.

Note that statistical significance is directly impacted by sample size. Recall that there is an inverse relationship between sample size and the standard error (i.e., standard deviation of the sampling distribution). Very small differences will be statistically significant with a very large sample size. Thus, when results are statistically significant it is important to also examine practical significance. Practical significance is not directly influenced by sample size.

## Example: Weight-Loss Program Section

Researchers are studying a new weight-loss program. Using a large sample they construct a 95% confidence interval for the mean amount of weight loss after six months on the program to be [0.12, 0.20]. All measurements were taken in pounds. Note that this confidence interval does not contain 0, so we know that their results were statistically significant at a 0.05 alpha level. However, most people would say that the results are not practically significant because a six month weight-loss program should yield a mean weight loss much greater than the one observed in this study.

## Effect Size Section

For some tests there are commonly used measures of effect size. For example, when comparing the difference in two means we often compute Cohen's $$d$$ which is the difference between the two observed sample means in standard deviation units:

$d=\frac{\overline x_1 - \overline x_2}{s_p}$

Where $$s_p$$ is the pooled standard deviation

$s_p= \sqrt{\frac{(n_1-1)s_1^2 + (n_2 -1)s_2^2}{n_1+n_2-2}}$

Below are commonly used standards when interpreting Cohen's $$d$$:

For a single mean, you can compute the difference between the observed mean and hypothesized mean in standard deviation units: $d=\frac{\overline x - \mu_0}{s}$

For correlation and regression we can compute $$r^2$$ which is known as the coefficient of determination. This is the proportion of shared variation. We will learn more about $$r^2$$ when we study simple linear regression and correlation at the end of this course.

## Example: SAT-Math Scores Section

Research question :  Are SAT-Math scores at one college greater than the known population mean of 500?

$$H_0\colon \mu = 500$$

$$H_a\colon \mu >500$$

Data are collected from a random sample of 1,200 students at that college. In that sample, $$\overline{x}=506$$ and the sample standard deviation was 100. A one-sample mean test was performed and the resulting p-value was 0.0188. Because $$p \leq \alpha$$, the null hypothesis should be rejected. These results are statistically significant. There is evidence that the population mean is greater than 500.

But, let's also consider practical significance. The difference between an SAT-Math score 500 and an SAT-Math score of 506 is very small. With a standard deviation of 100, this difference is only $$\frac{506-500}{100}=0.06$$ standard deviations. In most cases, this would not be considered practically significant.

## Example: Commute Times Section

Research question:  Are the mean commute times different in Atlanta and St. Louis?

Using the dataset built in to StatKey , a two-tailed randomization test was conducted resulting in a p value < 0.001. Because the null hypothesis was rejected, the results are said to be statistically significant.

Practical significance can be examined by computing Cohen's d. We'll use the equations from above:

First, we compute the pooled standard deviation:

$s_p= \sqrt{\frac{(500-1)20.718^2 + (500-1)14.232^2}{500+500-2}}$

$s_p= \sqrt{\frac{(499)(429.236)+ (499)(202.550)}{998}}$

$s_p= \sqrt{\frac{214188.527+ 101072.362}{998}}$

$s_p= \sqrt{\frac{315260.853}{998}}$

$s_p= \sqrt{315.893}$

$s_p= 17.773$

Note: The pooled standard deviation should always be between the two sample standard deviations.

Next, we can compute Cohen's d:

$d=\frac{29.110-21.970}{17.773}$

$d=\frac{7.14}{17.773}$

$d= 0.402$

The mean commute time in Atlanta was 0.402 standard deviations greater than the mean commute time in St. Louis. Using the guidelines for interpreting Cohen's d in the table above, this is a small effect size.

• USC Libraries
• Research Guides

## Organizing Your Social Sciences Research Paper

• 8. The Discussion
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Narrowing a Topic Idea
• Extending the Timeliness of a Topic Idea
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The purpose of the discussion section is to interpret and describe the significance of your findings in relation to what was already known about the research problem being investigated and to explain any new understanding or insights that emerged as a result of your research. The discussion will always connect to the introduction by way of the research questions or hypotheses you posed and the literature you reviewed, but the discussion does not simply repeat or rearrange the first parts of your paper; the discussion clearly explains how your study advanced the reader's understanding of the research problem from where you left them at the end of your review of prior research.

Annesley, Thomas M. “The Discussion Section: Your Closing Argument.” Clinical Chemistry 56 (November 2010): 1671-1674; Peacock, Matthew. “Communicative Moves in the Discussion Section of Research Articles.” System 30 (December 2002): 479-497.

## Importance of a Good Discussion

The discussion section is often considered the most important part of your research paper because it:

• Most effectively demonstrates your ability as a researcher to think critically about an issue, to develop creative solutions to problems based upon a logical synthesis of the findings, and to formulate a deeper, more profound understanding of the research problem under investigation;
• Presents the underlying meaning of your research, notes possible implications in other areas of study, and explores possible improvements that can be made in order to further develop the concerns of your research;
• Highlights the importance of your study and how it can contribute to understanding the research problem within the field of study;
• Presents how the findings from your study revealed and helped fill gaps in the literature that had not been previously exposed or adequately described; and,
• Engages the reader in thinking critically about issues based on an evidence-based interpretation of findings; it is not governed strictly by objective reporting of information.

Annesley Thomas M. “The Discussion Section: Your Closing Argument.” Clinical Chemistry 56 (November 2010): 1671-1674; Bitchener, John and Helen Basturkmen. “Perceptions of the Difficulties of Postgraduate L2 Thesis Students Writing the Discussion Section.” Journal of English for Academic Purposes 5 (January 2006): 4-18; Kretchmer, Paul. Fourteen Steps to Writing an Effective Discussion Section. San Francisco Edit, 2003-2008.

## Structure and Writing Style

I.  General Rules

These are the general rules you should adopt when composing your discussion of the results :

• Do not be verbose or repetitive; be concise and make your points clearly
• Avoid the use of jargon or undefined technical language
• Follow a logical stream of thought; in general, interpret and discuss the significance of your findings in the same sequence you described them in your results section [a notable exception is to begin by highlighting an unexpected result or a finding that can grab the reader's attention]
• Use the present verb tense, especially for established facts; however, refer to specific works or prior studies in the past tense
• If needed, use subheadings to help organize your discussion or to categorize your interpretations into themes

II.  The Content

The content of the discussion section of your paper most often includes :

• Explanation of results : Comment on whether or not the results were expected for each set of findings; go into greater depth to explain findings that were unexpected or especially profound. If appropriate, note any unusual or unanticipated patterns or trends that emerged from your results and explain their meaning in relation to the research problem.
• References to previous research : Either compare your results with the findings from other studies or use the studies to support a claim. This can include re-visiting key sources already cited in your literature review section, or, save them to cite later in the discussion section if they are more important to compare with your results instead of being a part of the general literature review of prior research used to provide context and background information. Note that you can make this decision to highlight specific studies after you have begun writing the discussion section.
• Deduction : A claim for how the results can be applied more generally. For example, describing lessons learned, proposing recommendations that can help improve a situation, or highlighting best practices.
• Hypothesis : A more general claim or possible conclusion arising from the results [which may be proved or disproved in subsequent research]. This can be framed as new research questions that emerged as a consequence of your analysis.

III.  Organization and Structure

Keep the following sequential points in mind as you organize and write the discussion section of your paper:

• Think of your discussion as an inverted pyramid. Organize the discussion from the general to the specific, linking your findings to the literature, then to theory, then to practice [if appropriate].
• Use the same key terms, narrative style, and verb tense [present] that you used when describing the research problem in your introduction.
• Begin by briefly re-stating the research problem you were investigating and answer all of the research questions underpinning the problem that you posed in the introduction.
• Describe the patterns, principles, and relationships shown by each major findings and place them in proper perspective. The sequence of this information is important; first state the answer, then the relevant results, then cite the work of others. If appropriate, refer the reader to a figure or table to help enhance the interpretation of the data [either within the text or as an appendix].
• Regardless of where it's mentioned, a good discussion section includes analysis of any unexpected findings. This part of the discussion should begin with a description of the unanticipated finding, followed by a brief interpretation as to why you believe it appeared and, if necessary, its possible significance in relation to the overall study. If more than one unexpected finding emerged during the study, describe each of them in the order they appeared as you gathered or analyzed the data. As noted, the exception to discussing findings in the same order you described them in the results section would be to begin by highlighting the implications of a particularly unexpected or significant finding that emerged from the study, followed by a discussion of the remaining findings.
• Before concluding the discussion, identify potential limitations and weaknesses if you do not plan to do so in the conclusion of the paper. Comment on their relative importance in relation to your overall interpretation of the results and, if necessary, note how they may affect the validity of your findings. Avoid using an apologetic tone; however, be honest and self-critical [e.g., in retrospect, had you included a particular question in a survey instrument, additional data could have been revealed].
• The discussion section should end with a concise summary of the principal implications of the findings regardless of their significance. Give a brief explanation about why you believe the findings and conclusions of your study are important and how they support broader knowledge or understanding of the research problem. This can be followed by any recommendations for further research. However, do not offer recommendations which could have been easily addressed within the study. This would demonstrate to the reader that you have inadequately examined and interpreted the data.

IV.  Overall Objectives

The objectives of your discussion section should include the following: I.  Reiterate the Research Problem/State the Major Findings

Briefly reiterate the research problem or problems you are investigating and the methods you used to investigate them, then move quickly to describe the major findings of the study. You should write a direct, declarative, and succinct proclamation of the study results, usually in one paragraph.

II.  Explain the Meaning of the Findings and Why They are Important

No one has thought as long and hard about your study as you have. Systematically explain the underlying meaning of your findings and state why you believe they are significant. After reading the discussion section, you want the reader to think critically about the results and why they are important. You don’t want to force the reader to go through the paper multiple times to figure out what it all means. If applicable, begin this part of the section by repeating what you consider to be your most significant or unanticipated finding first, then systematically review each finding. Otherwise, follow the general order you reported the findings presented in the results section.

III.  Relate the Findings to Similar Studies

No study in the social sciences is so novel or possesses such a restricted focus that it has absolutely no relation to previously published research. The discussion section should relate your results to those found in other studies, particularly if questions raised from prior studies served as the motivation for your research. This is important because comparing and contrasting the findings of other studies helps to support the overall importance of your results and it highlights how and in what ways your study differs from other research about the topic. Note that any significant or unanticipated finding is often because there was no prior research to indicate the finding could occur. If there is prior research to indicate this, you need to explain why it was significant or unanticipated. IV.  Consider Alternative Explanations of the Findings

It is important to remember that the purpose of research in the social sciences is to discover and not to prove . When writing the discussion section, you should carefully consider all possible explanations for the study results, rather than just those that fit your hypothesis or prior assumptions and biases. This is especially important when describing the discovery of significant or unanticipated findings.

V.  Acknowledge the Study’s Limitations

It is far better for you to identify and acknowledge your study’s limitations than to have them pointed out by your professor! Note any unanswered questions or issues your study could not address and describe the generalizability of your results to other situations. If a limitation is applicable to the method chosen to gather information, then describe in detail the problems you encountered and why. VI.  Make Suggestions for Further Research

You may choose to conclude the discussion section by making suggestions for further research [as opposed to offering suggestions in the conclusion of your paper]. Although your study can offer important insights about the research problem, this is where you can address other questions related to the problem that remain unanswered or highlight hidden issues that were revealed as a result of conducting your research. You should frame your suggestions by linking the need for further research to the limitations of your study [e.g., in future studies, the survey instrument should include more questions that ask..."] or linking to critical issues revealed from the data that were not considered initially in your research.

NOTE: Besides the literature review section, the preponderance of references to sources is usually found in the discussion section . A few historical references may be helpful for perspective, but most of the references should be relatively recent and included to aid in the interpretation of your results, to support the significance of a finding, and/or to place a finding within a particular context. If a study that you cited does not support your findings, don't ignore it--clearly explain why your research findings differ from theirs.

V.  Problems to Avoid

• Do not waste time restating your results . Should you need to remind the reader of a finding to be discussed, use "bridge sentences" that relate the result to the interpretation. An example would be: “In the case of determining available housing to single women with children in rural areas of Texas, the findings suggest that access to good schools is important...," then move on to further explaining this finding and its implications.
• As noted, recommendations for further research can be included in either the discussion or conclusion of your paper, but do not repeat your recommendations in the both sections. Think about the overall narrative flow of your paper to determine where best to locate this information. However, if your findings raise a lot of new questions or issues, consider including suggestions for further research in the discussion section.
• Do not introduce new results in the discussion section. Be wary of mistaking the reiteration of a specific finding for an interpretation because it may confuse the reader. The description of findings [results section] and the interpretation of their significance [discussion section] should be distinct parts of your paper. If you choose to combine the results section and the discussion section into a single narrative, you must be clear in how you report the information discovered and your own interpretation of each finding. This approach is not recommended if you lack experience writing college-level research papers.
• Use of the first person pronoun is generally acceptable. Using first person singular pronouns can help emphasize a point or illustrate a contrasting finding. However, keep in mind that too much use of the first person can actually distract the reader from the main points [i.e., I know you're telling me this--just tell me!].

Analyzing vs. Summarizing. Department of English Writing Guide. George Mason University; Discussion. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College; Hess, Dean R. "How to Write an Effective Discussion." Respiratory Care 49 (October 2004); Kretchmer, Paul. Fourteen Steps to Writing to Writing an Effective Discussion Section. San Francisco Edit, 2003-2008; The Lab Report. University College Writing Centre. University of Toronto; Sauaia, A. et al. "The Anatomy of an Article: The Discussion Section: "How Does the Article I Read Today Change What I Will Recommend to my Patients Tomorrow?” The Journal of Trauma and Acute Care Surgery 74 (June 2013): 1599-1602; Research Limitations & Future Research . Lund Research Ltd., 2012; Summary: Using it Wisely. The Writing Center. University of North Carolina; Schafer, Mickey S. Writing the Discussion. Writing in Psychology course syllabus. University of Florida; Yellin, Linda L. A Sociology Writer's Guide . Boston, MA: Allyn and Bacon, 2009.

## Writing Tip

Don’t Over-Interpret the Results!

Interpretation is a subjective exercise. As such, you should always approach the selection and interpretation of your findings introspectively and to think critically about the possibility of judgmental biases unintentionally entering into discussions about the significance of your work. With this in mind, be careful that you do not read more into the findings than can be supported by the evidence you have gathered. Remember that the data are the data: nothing more, nothing less.

MacCoun, Robert J. "Biases in the Interpretation and Use of Research Results." Annual Review of Psychology 49 (February 1998): 259-287; Ward, Paulet al, editors. The Oxford Handbook of Expertise . Oxford, UK: Oxford University Press, 2018.

## Another Writing Tip

Don't Write Two Results Sections!

One of the most common mistakes that you can make when discussing the results of your study is to present a superficial interpretation of the findings that more or less re-states the results section of your paper. Obviously, you must refer to your results when discussing them, but focus on the interpretation of those results and their significance in relation to the research problem, not the data itself.

Azar, Beth. "Discussing Your Findings."  American Psychological Association gradPSYCH Magazine (January 2006).

## Yet Another Writing Tip

Avoid Unwarranted Speculation!

The discussion section should remain focused on the findings of your study. For example, if the purpose of your research was to measure the impact of foreign aid on increasing access to education among disadvantaged children in Bangladesh, it would not be appropriate to speculate about how your findings might apply to populations in other countries without drawing from existing studies to support your claim or if analysis of other countries was not a part of your original research design. If you feel compelled to speculate, do so in the form of describing possible implications or explaining possible impacts. Be certain that you clearly identify your comments as speculation or as a suggestion for where further research is needed. Sometimes your professor will encourage you to expand your discussion of the results in this way, while others don’t care what your opinion is beyond your effort to interpret the data in relation to the research problem.

• << Previous: Using Non-Textual Elements
• Next: Limitations of the Study >>
• Last Updated: May 20, 2024 9:47 AM
• URL: https://libguides.usc.edu/writingguide
• Search Search Please fill out this field.
• Corporate Finance
• Financial Analysis

## What Is Statistical Significance?

Statistical significance is a determination made by an analyst that the results in the data are not explainable by chance alone. Statistical  hypothesis testing  is the method by which the analyst makes this determination. This test provides a p-value , which is the probability of observing results as extreme as those in the data, assuming the results are truly due to chance alone. A p-value of 5% or lower is often considered to be statistically significant.

## Key Takeaways

• Statistical significance is a determination that a relationship between two or more variables is caused by something other than chance.
• Statistical significance is used to provide evidence concerning the plausibility of the null hypothesis, which hypothesizes that there is nothing more than random chance at work in the data.
• Statistical hypothesis testing is used to determine whether the result of a data set is statistically significant.
• Generally, a p-value of 5% or lower is considered statistically significant.

Investopedia / Paige McLaughlin

## Understanding Statistical Significance

Statistical significance is a determination of the  null hypothesis , which suggests that the results are due to chance alone. A data set provides statistical significance when the p-value is sufficiently small.

When the p-value is large, then the results in the data are explainable by chance alone , and the data are deemed consistent with (while not proving) the null hypothesis.

When the p-value is sufficiently small (typically 5% or less), the results are not easily explained by chance alone, and the data are deemed inconsistent with the null hypothesis. In this case, the null hypothesis of chance alone as an explanation of the data is rejected in favor of a more systematic explanation.

Statistical significance is often used for new pharmaceutical drug trials, to test vaccines, and in the study of pathology for effectiveness testing and to inform investors on how successful the company is at releasing new products.

## Examples of Statistical Significance

Suppose Alex, a financial analyst, is curious as to whether some investors had advance knowledge of a company's sudden failure. Alex decides to compare the average of daily market returns prior to the company's failure with those after to see if there is a statistically significant difference between the two averages.

The study's p-value was 28% (>5%), indicating that a difference as large as the observed (-0.0033 to +0.0007) is not unusual under the chance-only explanation. Thus, the data did not provide compelling evidence of advance knowledge of the failure. On the other hand, if the p-value were 0.01% (much less than 5%), then the observed difference would be very unusual under the chance-only explanation. In this case, Alex may decide to reject the null hypothesis and to investigate further whether some traders had advance knowledge.

Statistical significance is also used to test new medical products, including drugs, devices, and vaccines. Publicly available reports of statistical significance also inform investors on how successful the company is at releasing new products.

Say, for example, a pharmaceutical leader in diabetes medication reported that there was a statistically significant reduction in type 1 diabetes when it tested its new insulin. The test consisted of 26 weeks of randomized therapy among diabetes patients, and the data gave a p-value of 4%. This signifies to investors and regulatory agencies that the data show a statistically significant reduction in type 1 diabetes.

Stock prices of pharmaceutical companies are often affected by announcements of the statistical significance of their new products.

## How Is Statistical Significance Determined?

Statistical hypothesis testing is used to determine whether the data is statistically significant. In other words, whether or not the phenomenon can be explained as a byproduct of chance alone. Statistical significance is a determination about the null hypothesis, which posits that the results are due to chance alone. The rejection of the null hypothesis is needed for the data to be deemed statistically significant.

## What Is P-Value?

A p-value is a measure of the probability that an observed difference could have occurred just by random chance. When the p-value is sufficiently small (e.g., 5% or less), then the results are not easily explained by chance alone and the null hypothesis can be rejected. When the p-value is large, then the results in the data are explainable by chance alone, and the data is deemed consistent with (while proving) the null hypothesis.

## How Is Statistical Significance Used?

Statistical significance is often used to test the effectiveness of new medical products, including drugs, devices, and vaccines. Publicly available reports of statistical significance also inform investors on how successful the company is at releasing new products. Stock prices of pharmaceutical companies are often affected strongly by announcements of the statistical significance of their new products.

Steven Tenny and Ibrahim Abdelgawad. " Statistical Significance. " StatPearls Publishing, 2022.

American Diabetes Association. " Efficacy and Safety of Fast-Acting Aspart Compared With Insulin Aspart, Both in Combination With Insulin Degludec, in Children and Adolescents With Type 1 Diabetes: The Onset 7 Trial ."

Thomas J. Hwang. " Stock Market Returns and Clinical Trial Results of Investigational Compounds: An Event Study Analysis of Large Biopharmaceutical Companies. " PLOS ONE, 2013.

Rothenstein, Jeffrey et al. " Company Stock Prices Before and After Public Announcements Related to Oncology Drugs. " Journal of the National Cancer Institute, vol. 103, no. 20, October 2011, pp. 1507-1512.

• Editorial Policy
• Open access
• Published: 16 May 2024

## Predictors for interest to participate in digital mental health therapy: a cross-sectional survey of individuals with anxiety and depression

• Tharidu Gunawardena 1 ,
• Marilyn M. Bartholmae 1 , 2 ,
• Matvey V. Karpov 1 ,
• Rohan Dod 1 ,
• Kripa Ahuja 1 ,
• Aishwarya Rajendran 1 ,
• Mayuri Kathrotia 1 &
• Sunita Dodani 1 , 3

BMC Digital Health volume  2 , Article number:  21 ( 2024 ) Cite this article

80 Accesses

Metrics details

Due to a multitude of factors, the onset of the COVID-19 pandemic resulted in a significant increase in mental health issues within society, including depression and anxiety. Due to the increased trend of mental health disorders in society, digital mental health therapies are more useful than ever. With the emergence of programs utilizing Internet Cognitive Behavioral Therapy (iCBT), mental health resources are easily accessible and can be widely implemented to those in need.

The aim of this study was to identify predictors for interest to participate in SilverCloud digital mental health therapy among individuals with mild to severe anxiety and/or depression based on preliminary findings from the COVIDsmart study.

COVIDSmart participants who had moderate to severe anxiety and/or depression based on the PHQ-9 and GAD-7 scores, and who consented to be contacted for future studies, were invited to complete a needs assessment survey via Research Electronic Data Capture (REDCap). This assessment used self-reported measures including medical diagnoses, mental health services received, reasons for anxiety and/or depression, the use of coping strategies, suicidal ideology using the Ask Suicide Questions tool, and whether they would be interested in receiving free digital mental therapy. Descriptive statistics were used to report participants’ demographics and a logistic regression was used to find predictors for interest in participation in SilverCloud. SAS 9.4 was used and p values < 0.05 were considered significant.

Out of the original 782 COVIDsmart participants, 634 consented to be contacted for future studies, 280 were subsequently invited to complete the SilverCloud needs assessment, and 120 individuals completed it. The largest demographic among these participants were females (70.83%) who identified as White (80.83%). The mean age was 48.74 years ( SD  = 14.66). Results revealed that having a mental health comorbidity significantly predicted the likelihood of interest in participating in the SilverCloud digital mental health program ( p  = 0.027).

## Conclusions

In this study, mental illness comorbidities predicted the interest to participate in digital mental therapy. Fragmented healthcare and perceptions of unmet care needs are likely contributor factors. Further research with a diverse sample of participants is necessary for generalizability. Findings may have important implications for healthcare best practices.

Peer Review reports

## Introduction

Repeatedly, it has been evident that periods of widespread human catastrophe or disaster often result in large-scale detrimental impacts on mental health. During the COVID-19 pandemic, there were not only new cases of mental health disorders but also an exacerbation of existing mental health conditions [ 1 ]. According to a briefing released by the World Health Organization in March 2022, during the first year of the COVID-19 pandemic, the prevalence of major depressive disorders and anxiety disorders had increased significantly [ 2 ]. There were multiple factors leading to a significant spike in mental health disorders. Common stressors were social isolationism, perpetual fear of COVID-19 infection, loss of employment, death of a loved one or friend and uncertainty about the future [ 3 ]. Despite many of the COVID-19 restrictions and lockdowns being relaxed since March of 2020, the damaging adverse mental health effects of the pandemic seem to have persisted throughout the United States (U.S). According to a CDC report, the prevalence of anxiety and depressive disorders has been three and four times higher, respectively, compared to pre-pandemic levels within the U.S.[ 4 ]. In 2021, mental health screenings were taken by over 5.4 million people, indicating a near 500% increase since 2019 and a 103% increase since 2020 [ 5 ].

## COVID-19 impact

The primary outcome for this analysis is COVID-19 impact. COVID-19 impact is assessed across 8 different domains: physical health, day-to-day life, sexual behaviors, substance use, HIV care, mental health, financial status, and having conflict with a partner, using a brief survey that we developed and have used to describe COVID-19 impact in other populations (Appendix 1 ) [ 22 , 23 , 24 ]. Participants are asked, “How much has COVID-19 directly affected…” for each domain, responding on a Likert scale of 1 (not at all) to 5 (extremely). We also ask, “How has COVID-19 affected you in terms of…” for each domain where participants can select all that apply from a list of options, including “other” with the option of a free-text response.

## IPV exposure

All enrolled participants have experienced IPV in their lifetimes. The primary explanatory variable of interest is ongoing IPV exposure. Type of current IPV exposure is only assessed for women who were partnered in a relationship in the 30 days prior to study enrollment or the most recent HIV clinic visit.

Physical IPV is measured with the Revised Conflict Tactics Scale-2 (CTS-2) across 12 items in subscales for physical assault (α = 0.86), injury (α = 0.96), and negotiation (α = 0.86) in an intimate relationship [ 25 ]. Injury is defined as “partner-inflicted physical injury” causing bone or tissue damage, warranting medical attention, or causing pain for a day or more; and negotiation is the “action to settle a disagreement through discussion.” The cognitive subscales of negotiation are “examples of such discussions” whereas the emotion subscale measures “the extent to which positive affect is communicated.” [ 25 ] Response options referring to the 30 day period are: 0 = never; 1 = once; 2 = twice; 3 = 3–5 times; 4 = 6–10 times; 5 = 11–20 times; 6 = more than 20 times; 7 = not in those 30 days, but it happened before in our relationship. Responses are recoded to the midpoint of the range of scores and are transformed using standardized syntax and categorized into type of physical IPV victimization, frequency and severity [ 26 , 27 ]. Sexual IPV is measured by the Sexual Experiences Survey (SES), using the 10 items that classify and measure degrees of sexual victimization (α = .74 for women), using the same response options as for the CTS-2 as above [ 28 , 29 ]. Variables are transformed using standardized syntax to calculate any sexual violence exposure in one’s lifetime [ 29 ]. Psychological IPV is measured using the Psychological Maltreatment of Women Inventory Short Version (PMWI-S), a 14-item instrument designed to assess the level of psychological abuse of women by their intimate male partners including subscales for dominance/isolation (α = .88) and emotional/verbal (α = .92) abuse [ 30 ]. Participants are asked how frequently they have experienced these things in the past 30 days; response options are: 1 = never; 2 = rarely; 3 = occasionally; 4 = frequently; 5 = very frequently. Total PMWI score ranges 14–70 and each of the type sub-scores range 7–35, with higher scores indicating higher severity of psychological abuse.

We use the Past Abusive Behavior Inventory (also known as Past Abusive Relationships; PAR) to measure the number of past adult relationships in which women experienced psychological, physical, or sexual IPV [ 31 , 32 ]. In addition, participants who are currently in a relationship or were in a relationship in the 30 days prior to the baseline interview are asked if they experienced minor physical, severe physical, sexual, psychological, or monitoring violence with that partner, using the same response options as for the PMWI-S as above. Current IPV exposure is defined as > 1 on any of these 5 items; no current IPV exposure (i.e., past lifetime IPV exposure only) is defined as 1 on all 5 items or not currently partnered.

## Sociodemographic and health characteristics

We assess participant age, gender identity, ethnicity/race, sexual orientation, education level, housing, employment, income level, relationship status, basic sociodemographic characteristics of their current partners, health insurance status, years since HIV diagnosis, and usual frequency of visiting an HIV care provider.

## Mental health

We assess for depression symptoms using the Center for Epidemiological Studies-Depression Scale (CES-D; α = 0.85) [ 33 ]. Scores range 0–60, with higher scores indicating greater severity of depressive symptoms, and are dichotomized at < 16 vs. ≥16, with the latter indicating clinically significant depressive symptoms or probable depression. We assess for anxiety using the Generalized Anxiety Disorder instrument (GAD-7; α = 0.85), which consists of 7 items [ 34 ]. Each of the 7 items is scored 0–3, and the total score ranges 0–21; scores are dichotomized at < 10 vs. ≥10, with the latter indicating probable generalized anxiety disorder. We assess PTSD symptom severity using the Posttraumatic Diagnostic Scale for DSM-5 (PDS-5; α = 0.95) [ 35 ], across 4 domains of PTSD: (1) re-experiencing; (2) avoidance; (3) negative alterations in cognition and mood; and (4) hyper-arousal. Each of the 20 items is scored 0–4, where 0 = not at all; 1 = once a week or less/a little; 2 = 2 to 3 times a week/somewhat; 3 = 4 to 5 times a week/very much; 4 = 6 or more times a week/severe , and the PDS-5 total score ranges 0–80. Scores of 28–80 reflect probable diagnosis of PTSD.

## Substance use

We assess alcohol use with the Alcohol Use Disorders Identification Test (AUDIT; α = 0.93) [ 36 ]. Total scores are dichotomized at < 8 vs. ≥8, with the latter indicating hazardous and harmful alcohol use [ 37 ]. We use the NIDA-Modified Alcohol, Smoking, and Substance Involvement Screening Test (NM-ASSIST) to assess use of criminalized drugs or prescription drugs for “non-medical reasons”, including cannabis (α = 0.85), cocaine (α = 0.91), prescription stimulants (k = 0.74), methamphetamine, inhalants, sedatives (α = 0.87), hallucinogens, “street opioids”, and prescription opioids (α = 0.85) [ 38 , 39 ]. “Non-medical reasons” for substance use are defined as taking medications for reasons or in doses other than prescribed to you . For each substance, participants are asked about past 3-month use frequency and substance use disorder criteria; substance involvement scores are summed to reflect current substance-specific severity, or an estimate of an individual’s risk for problems associated with that substance [ 38 ]. Scores 0–3 are categorized as lower severity; 4–26 as moderate severity, and scores ≥ 27 suggest high severity drug use; levels are used to identify appropriate interventions. Participants are also asked about the use of medications for the treatment of opioid use disorder.

## Statistical analysis

We used descriptive statistics to characterize the study sample. Continuous measures are presented as means with standard deviations or medians with interquartile ranges (IQR) if not normally distributed, and categorical measures as frequencies with proportions. To evaluate how the COVID-19 pandemic impacted women differently based on whether they experienced ongoing IPV, we compared participants experiencing current/ongoing IPV to participants not experiencing current/ongoing IPV (past IPV only) in terms of sociodemographic, mental health, and substance use characteristics, using independent t-test or Fisher’s exact test for continuous variables, and Pearson’s chi-squared test for categorical variables. Next, we explored associations between ongoing IPV exposure and different types of COVID-19 impact (physical health, day-to-day life, sexual/relationship behaviors, HIV care engagement, mental health, and having conflict with a partner). We did not build separate models for substance use or financial/economic COVID-19 impact domains because most women reported no impact of COVID-19 in terms of these factors, so there was insufficient variability to allow for generation of meaningful models. Otherwise, we conducted multiple linear regression analyses for each COVID-19 impact domain. The primary explanatory variable of interest was ongoing IPV; other included explanatory variables were presence of mental health problems and substance use. We also included sociodemographic variables (age, race, ethnicity, years of education, and employment status) as potential covariates. We developed full models of COVID-19 impact that included IPV exposure, age, race, ethnicity, years of education, employment status, PTSD, anxiety, depression, and substance use severity (for alcohol, cannabis, cocaine, opioids). Only variables with p -value < 0.2 in the full model are included in the reduced model. If co-linearity was plausible and supported by cross-tabulation of the data, only the variable that was more strongly associated with COVID-19 impact was retained in multivariable models. For the final, reduced model, statistical significance was defined as a p -value < 0.05. All analyses were performed using SAS (SAS 9.4, SAS Institute, Inc., Cary, NC).

## Baseline characteristics by IPV exposure recency

Eighty-four women (including 79 cis- and 5 trans-gender women) were enrolled and included in this analysis. As shown in Table  1 , participants ranged from 23 to 75 years of age, with a mean age of 53.6 (SD = 9.9) years. The sample was racially/ethnically diverse with more than two-thirds (69.1%) identifying as Black/African American and 22.6% identifying as Hispanic/Latina. Most participants had a high school education (with 11.8 mean years of formal education; SD = 1.9) and experienced unemployment (84.5%). Participants had been living with diagnosed HIV for a mean of 23.3 years (SD = 10).

Mental health problems were highly prevalent: 31.0% ( n  = 26) met the threshold for probable PTSD diagnosis, 27.4% ( n  = 23) screened positive for generalized anxiety, and over half (52.4%; n  = 44) had clinically significant depression symptoms. Substance use was common, including 14% ( n  = 12) meeting criteria for hazardous drinking. Most (73.8%) participants reported cannabis use and half (50%) had moderately severe cannabis use. Among the participants who used cocaine ( n  = 58), over half had moderate or high severity cocaine use. Of the participants who used street opioids ( n  = 24), nearly 80% had moderately severe opioid use. Additionally, 16 participants reported the use of prescription opioids and over half of them (56.3%) had moderately severe use of prescription opioids.

Given study inclusion criteria, all WLWH in the sample experienced some form of lifetime IPV, and of the 49 (58.3%) women currently in a relationship, 39 (79.5%) reported ongoing IPV exposure. As shown in Table  1 , there were no significant differences between those experiencing ongoing IPV ( n  = 39) and those not experiencing ongoing IPV or not partnered ( n  = 10 and n  = 35, respectively) in terms of any sociodemographic characteristics, mental health problems, or substance use.

## Current IPV exposure type and severity

Among the 49 currently partnered participants, almost half (44.9%) experienced physical assault, including minor physical assault (42.9%) and severe physical assault (28.6%). Fourteen (28.6%) were injured during conflicts with partners, 26.5% ( n  = 13) of whom experienced a minor injury, and 14.3% ( n  = 7) experienced a severe injury. Most currently partnered participants (95.9%) used negotiation strategies at some point in the last 30 days to deal with conflicts, including emotional negotiation (95.9%) and cognitive negotiation (93.9%).

## Correlates of COVID-19 impact

As shown in Fig.  1 , COVID-19 had the greatest impact on women’s mental health and the least impact on sexual behaviors. Across all domains, mean COVID-19 impact scales (that ranged from 1 to 5) were higher in those experiencing ongoing IPV than among those who were not experiencing ongoing IPV, though the differences were not statistically significant.

Mean COVID-19 impact scales (ongoing IPV vs. no ongoing IPV)

Next, we turned to developing separate multivariate linear regression models for each of six COVID-19 impact domains. Contrary to our original hypothesis, after controlling for other key demographic, mental health problems and substance use in multivariable models, we found that ongoing IPV exposure was not significantly associated with any of the COVID-19 impact domains.

Participants reported a range of COVID-19-related physical health impacts, including experiencing symptoms of COVID-19 but not testing ( n  = 4); testing negative ( n  = 63); testing positive ( n  = 11); being exposed ( n  = 21); and being hospitalized for COVID-19 ( n  = 3). As shown in Appendix Table  1 , the mean physical health impact score for women of Hispanic ethnicity was 0.927 points higher than for women who were not Hispanic ( p  = 0.007). WLWH who met the threshold for generalized anxiety were 0.537 points lower on the health impact scale than those who did not meet criteria for generalized anxiety, though this difference was not statistically significant ( p  = 0.144). WLWH with clinically significant depression symptoms scored 1.14 points higher on COVID-19 physical health impact than those without clinically significant depression ( p  = 0.0006).

Appendix Table  2 shows full and reduced models of COVID-19-related impact on day-to-day life. Women who identified as white reported lower COVID-impact on day-to-day-life compared to women who identified as Black or African American ( p  = 0.046). Compared to women with less severe cocaine use, WLWH with moderate/high severity cocaine use reported lower (though not statistically significant) COVID-19-related impact on day-to-day life ( p  = 0.076). Women with moderately/high severity use of street opioids and prescription opioids experienced a greater COVID-related impact on their day-to-day life than women with less severe opioid use.

Participants reported that COVID-19 impacted sexual/relationship behaviors in terms of reduced close contact ( n  = 13) or in-person dating ( n  = 5) and more frequently using barrier protection, like condoms or dental dams, ( n  = 4). Of the 7 participants who reported “other”, 2 specified: partner hasn’t felt comfortable with having sex as often ; and being more careful with other partners. In the full model of COVID-impact on sexual behavior, only ethnicity was significant in that women who were Hispanic had a 0.734 (SE 0.487) point higher score compared to women who were not Hispanic ( p  = 0.137).

Women reported that COVID-19 affected how they engaged in HIV care, for example they reported: requesting a 90-day supply of medications ( n  = 8) or initiating home delivery of medications ( n  = 7), restarting HIV medications ( n  = 1), cancelling appointments ( n  = 12) and requesting telehealth visits ( n  = 14) during COVID-19. Additionally, some women reported difficulty accessing their pharmacy ( n  = 8), lab testing ( n  = 11) or other testing services ( n  = 14), and appointment scheduling ( n  = 22) during COVID-19. Two participants specified other impacts that included: using drive thru, made sure to take HIV meds every day, doctor made telehealth appointments, contacted doctor about the COVID-19 vaccine. As shown in Appendix Table  3 , women who met the threshold for anxiety reported a greater impact of COVID-19 on their HIV care than women who did not ( p  = 0.030). Among WLWH using street opioids, those with moderate/high severity use reported greater impact of COVID-19 HIV care than those with lower severity opioid use (0 = 0.044). Associations between other mental health and substance use factors and COVID-19 impact on HIV care were not statistically significant in reduced models.

Participants reported that COVID-19 impacted their mental health in negative ways, including increased frustration or boredom ( n  = 41), greater anxiety ( n  = 50) or depression symptoms ( n  = 40), disruptions to sleep ( n  = 44), increased loneliness ( n  = 38), and increased trauma symptoms ( n  = 15). Five participants specified “other” difficulties/challenges that included: couldn’t go to church; homelessness; took visits away while incarcerated and at halfway house, so social support is difficult; mandated to wear masks again; going out to places less; not being able to physically get around yourself. Some women also reported that COVID-19 impacted their mental health in positive ways, including receiving social support from family, friends, partners, or counselors ( n  = 35) or people in the community or local agencies ( n  = 22). Four participants specified other benefits that included: spend more time with friends/family; isolation (I don’t like to be around a lot of people); learn my husband a little more; having my own transportation. There were no explanatory variables in the full model of COVID-impact on mental health that met criteria for inclusion in reduced models.

Appendix Table  4 depicts findings from multivariate models of COVID-19-related conflict with partners. Employment status, probable diagnosis of PTSD, cocaine use, and street opioid use were included in reduced models of COVID-19-related conflict with partners. WLWH who were employed experienced less impact of COVID-19 on conflict with their partners than unemployed women ( p  = 0.061). Compared to women who did not meet PTSD criteria, women who met the threshold for PTSD experienced higher impact of COVID-19 on having conflict with their partners ( p  = 0.003). Among women who were using cocaine, compared to those with lower risk use, those who used at moderately/high risk levels experienced greater COVID-19 impact on having conflict with partners ( p  = 0.231). In contrast, compared to those with lower risk use of street opioids, women with moderately/high risk street opioid use experienced lower overall impact of COVID-19 on having conflict with their partners ( p  = 0.153). In the reduced model, only PTSD remained a statistically significant correlate of COVID-19 related conflict with partners.

Some women qualitatively identified positive impacts of the COVID-19 pandemic period on their lives, such as: stimulus checks, started going back to the gym; had less access to drugs; able to spend more time with family; we look at each other differently now because of it; we talk more than we’ve ever talked; it’s not a good thing to talk about, but it changed our relationship to draw us closer.

To our knowledge, this study is the first to systematically assess the broad impact of COVID-19 and the experiences of IPV during COVID-19 among WLWH. We did so among a sample of WLWH (mean 53.6 years of age) who had been living with diagnosed HIV for many years (mean 23.3 years), and all of whom had experienced IPV in their lifetimes.

In national surveys, lifetime experience of IPV is relatively common among all U.S. women [ 1 , 40 ], and even more frequent among WLWH, 55% of whom report IPV exposure [ 12 ]. In our cohort of 84 WLWH, all of whom had lifetime IPV exposure, we found higher than expected rates of ongoing IPV. Among 49 women who were currently partnered, 79.6% ( n  = 39) were experiencing ongoing IPV, including physical assault and sexual violence. Findings have important implications for engagement in care, as experiences of IPV among WLWH have been associated with lower levels of treatment adherence and a reduced likelihood of achieving viral suppression [ 41 ]. We did not report on HIV treatment outcomes here, which is a primary focus of the ongoing parent study, and these findings will be reported once data collection is complete. Focused research is needed to disentangle the ways in which IPV affects women’s day-to-day experiences of living with HIV, considering potential targets for interventions that support both healthy relationships and engagement in care.

Our findings highlight the many ways in which the COVID-19 pandemic emergency period impacted WLWH who are IPV survivors. Although we do not have a pre-pandemic sample for comparison, the observed high rates of ongoing IPV may reflect increased IPV exposure to WLWH related to “stay at home” regulations [ 18 ]. At the height of pandemic-related restrictions in many U.S. settings, HIV care, research participation, and workplace settings transitioned to virtual or telephone-based methods [ 17 ]. Participants in this study frequently reported that COVID-19 affected how they received HIV care. Telehealth was critical for the continuous delivery of HIV care during pandemic restrictions, allowing clinics to provide care to highly vulnerable members of the community without compromising health or safety of patients or staff. Yet one inadvertent downside to telehealth was that it may have fostered social isolation, wherein WLWH may have been less able to connect to the community’s critical social and protective networks during the pandemic [ 17 ]. Telehealth was also not accessible for many WLWH because of limited health or digital literacy, or limited access to needed technology. Some news reports suggested that telehealth may reduce women’s ability to openly discuss their experiences of IPV (if they choose to do so with their healthcare provider) because of concerns about privacy and risk of their abuser overhearing [ 5 ]. We did not address IPV disclosure to healthcare providers via telehealth in this study, though this may be an interesting area for future study.

Though we expected to find that current IPV exposure was associated with more significant COVID-19 impact on various aspects of women’s daily lives, we found no statistically significant differences between those experiencing ongoing IPV and those not experiencing ongoing IPV in terms of demographic characteristics, substance use, mental health, or COVID-19 impact in any domain. We did not find any association between ongoing IPV exposure and COVID-19-related conflict with partners, perhaps because the latter construct was relatively broad and may include relationship stressors that do not meet behavioral threshold of IPV. Alternatively, the impacts of IPV may endure, so that even if these women hadn’t experienced IPV recently, they are still experiencing the impacts of prior experiences, which would make it difficult to see differences between recent and past IPV. Women who had PTSD, however, did experience a greater impact of COVID-19 on having conflict with a partner as compared to women without PTSD.

When we disarticulated the different types of COVID-19 impact domains, we did find additional important correlates of COVID-19 impact. In multivariate linear regression models, we found that COVID-19 impact on physical health was significantly associated with Hispanic ethnicity. Compared to women who were not Hispanic, WLWH who identified as Hispanic reported that COVID-19 had a greater direct effect on their physical health. This finding is consistent with CDC data showing that people who are Hispanic or Latino are 1.5 times more likely to acquire COVID-19, 1.9 times more likely to be hospitalized from COVID-19, and 1.7 times more likely to die from COVID-19 than their non-Hispanic white counterparts [ 42 ]. These disparate health outcomes are not because of any biological factor, but rather because of socioeconomic disparities experienced by minoritized communities.

Employment status also was associated with COVID-19 impact on partner conflict, though this was not statistically significant in reduced models. Women who were employed may have experienced changes in their mental health due to shifts to working from home, for those whose jobs allowed them to do so. A previous study showed that working from home was associated with decreased overall mental health due to fewer face-to-face interactions with coworkers, distraction while working, adjusted work hours, and less social support [ 43 ]. Working from home may have been particularly stressful for women responsible for parenting with remote schooling, though most of our participants do not have any children under 18 years old living with them. WLWH who were employed were less affected by COVID-19 in terms of conflicts with partners, as compared to unemployed WLWH. This is also consistent with a previous study, in which participants who reported increased conflicts with partners were more likely to be unemployed and less conflict was associated with working part-time [ 44 ]. It is unclear whether these findings are related to women’s financial dependency on partners or other factors that may generate stress and conflict within a relationship.

The COVID-19 pandemic brought unique challenges for people with substance use [ 45 ]. We found that more severe use of opioids was significantly associated with higher COVID-19 impact on day-to-day life, HIV care, and lower impact on having conflict with partners, whereas more severe use of cocaine had the opposite effects on each domain. These findings are consistent with previous studies showing a rise in substance use and fatal overdoses in the U.S. during the COVID-19 pandemic [ 46 ]. People with substance disorders also had increased risk for poor COVID-19 outcomes if they acquired COVID-19 [ 46 ]. From an HIV care perspective, untreated substance use disorders are associated with more rapid HIV disease progression, impaired adherence to antiretroviral therapy, and worse overall HIV treatment outcomes [ 47 ]. Findings illustrate the importance of addressing and treating substance use disorders to improve substance use outcomes and, secondarily, HIV outcomes. Intervention was particularly important during the pandemic period, when substance use in isolation was associated with high rates of fatal and non-fatal overdose [ 48 ].

In our sample, mental health problems were associated with a range of COVID-19 impacts. In a recent study of the general U.S. population, nearly half of those surveyed reported recent symptoms of an anxiety or depressive disorder [ 49 ]. According to the World Health Organization (WHO), the global prevalence of anxiety and depression has increased 25% since the beginning of the COVID-19 pandemic [ 50 ]. Anxiety and depression may be particularly high among WLWH who are IPV survivors. In our analysis, mental health problems, including generalized anxiety, PTSD, and clinically significant depression symptoms were associated with COVID-19 impacts on physical health, mental health, HIV care, and having conflicts with partners, though directionality of this association is not clear. According to WHO, people with pre-existing psychiatric disorders are more likely to experience hospitalization, severe illness and death when they contract COVID-19 than people who do not have psychiatric disorders [ 50 ]. People living with HIV (PLWH) report a higher baseline prevalence of psychiatric disorders compared with general population [ 51 ]. Especially for WLWH who are IPV survivors, COVID-19 could exacerbate underlying depression, anxiety, and PTSD symptoms, leading to worse mental health outcomes. From an HIV care perspective, we found that WLWH with greater anxiety symptoms experienced a higher impact of COVID-19 on HIV care compared with those who did not have anxiety. This finding is consistent with that from a study of people living with HIV (PLWH) in China, who experienced prolonged lockdowns and isolation [ 52 ]. Mental health problems impact HIV care and can worsen health outcomes among PLWH, including decreased medication adherence and viral suppression, and increasing onward HIV transmission risk. Recognition of depression, anxiety, and PTSD symptoms is thus an important priority for WLWH, particularly women who have experienced or are experiencing IPV.

Though novel in its scope and approach, this study is limited by several important factors. First, the results are based on a secondary data analysis from a single point in time. As such, causation cannot be inferred, and any delayed impact of the COVID-19 pandemic and IPV on WLWH was not captured. Second, all measures were self-reported, which can be subject to retrospective and social desirability biases. Third, the sample size was relatively small and geographically confined to a highly resourced setting in New England, and may not reflect the experiences of other WLWH in the U.S. Finally, the sample of WLWH here had all experienced lifetime IPV, so it may have been difficult to tease out associations between types or timing of IPV exposure and COVID-19 impact in this otherwise somewhat homogenous sample.

This study is the first assessment of IPV exposure and COVID-19 impact among WLWH who are IPV-survivors. In this sample, we found high rates of ongoing IPV, and the COVID-19 pandemic affected WLWH in broad and varied ways. This study can inform future strategies to support WLWH who are IPV survivors, which is particularly crucial during emergencies and public health crises.

## Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Leemis R, Friar N, Khatiwada S, Chen M, Kresnow M, Smith S et al. The national intimate partner and sexual violence survey: 2016/2017 report on intimate partner violence. Atlanta, GA: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention; 2022.

Peitzmeier SM, Fedina L, Ashwell L, Herrenkohl TI, Tolman R. Increases in intimate partner violence during COVID-19: prevalence and correlates. J Interpers Violence. 2022;37(21–22):NP20482–512.

Zhang W. Domestic violence cases surge during COVID-19 epidemic: Sixth Tone; 2020 [ https://www.sixthtone.com/news/1005253/domestic-violence-cases-surge-during-covid19-epidemic

Taub AA. New Covid-19 crisis: domestic abuse rises worldwide: The New York Times; 2020, April 6 [ https://www.nytimes.com/2020/04/06/world/coronavirus-domestic-violence.html

Mineo L. ‘Shadow pandemic’ of domestic violence: the Harvard gazette; 2022 [ https://news.harvard.edu/gazette/story/2022/06/shadow-pandemic-of-domestic-violence/

Peterman A, Potts A, O’Donnell M, Thompson K, Shah N, Oertelt-Prigione S et al. Pandemics and violence against women and children: center for global development; 2020 [ https://www.cgdev.org/publication/pandemics-and-violence-against-women-and-children

Perez-Vincent SMC, Enrique; Gibbons MaríaA, Murphy TE, Rossi Martín. COVID-19 Lockdowns and domestic violence: evidence from two studies in Argentina: Inter-American Development Bank; 2020 [ https://publications.iadb.org/en/covid-19-lockdowns-and-domestic-violence-evidence-from-two-studies-in-argentina

Boxall H, Morgan A, Brown R. The prevalence of domestic violence among women during the COVID-19 pandemic. 2020. Contract No.: Statistical Bulletin 28.

Beland LP, Brodeur A, Haddad J, Mikola D. Determinants of family stress and domestic violence: lessons from the COVID-19 outbreak. Can Public Policy. 2021;47(3):439–59.

Article   PubMed   PubMed Central   Google Scholar

Arenas-Arroyo E, Fernandez-Kranz D, Nollenberger N. Intimate partner violence under forced cohabitation and economic stress: evidence from the COVID-19 pandemic. J Public Econ. 2021;194:104350.

Davis M, Gilbar O, Padilla-Medina DM. Intimate partner violence victimization and perpetration among U.S. adults during the earliest stage of the COVID-19 pandemic. Violence Vict. 2021;36(5):583–603.

Dawson L, Kates J, Ramaswamy A, HIV, Intimate Partner Violence (IPV), and women: an emerging policy landscape: Kaiser Family Foundation; 2019 [ https://www.kff.org/hivaids/issue-brief/hiv-intimate-partner-violence-ipv-and-women-an-emerging-policy-landscape/

Hatcher AM, Smout EM, Turan JM, Christofides N, Stöckl H. Intimate partner violence and engagement in HIV care and treatment among women: a systematic review and meta-analysis. AIDS. 2015;29(16):2183–94.

Stockman JK, Hayashi H, Campbell JC. Intimate partner violence and its health impact on ethnic minority women [corrected]. J Womens Health (Larchmt). 2015;24(1):62–79.

Machtinger EL, Haberer JE, Wilson TC, Weiss DS. Recent trauma is associated with antiretroviral failure and HIV transmission risk behavior among HIV-positive women and female-identified transgenders. AIDS Behav. 2012;16(8):2160–70.

Lopez EJ, Jones DL, Villar-Loubet OM, Arheart KL, Weiss SM. Violence, coping, and consistent medication adherence in HIV-positive couples. AIDS Educ Prev. 2010;22(1):61–8.

Roesch E, Amin A, Gupta J, Garcia-Moreno C. Violence against women during covid-19 pandemic restrictions. BMJ. 2020;369:m1712.

Closson K, Lee M, Gibbs A, Kaida A. When home is not a safe place: impacts of social distancing directives on women living with HIV. AIDS Behav. 2020;24(11):3017–9.

Trull TJ, Ebner-Priemer UW. Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: introduction to the special section. Psychol Assess. 2009;21(4):457–62.

Meyer JP, Price CR, Ye Y, Qin Y, Tracey D, Demidont AC et al. A PrEP demonstration project using ehealth and community outreach to justice-involved cisgender women and their risk networks. AIDS and Behavior. 2022.

Harris P, Taylor R, Minor B, Elliott V, Fernandez M, O’Neal L et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95.

Mistler CB, Sullivan MC, Copenhaver MM, Meyer JP, Roth AM, Shenoi SV, et al. Differential impacts of COVID-19 across racial-ethnic identities in persons with opioid use disorder. J Subst Abuse Treat. 2021;129:108387.

Article   CAS   PubMed   PubMed Central   Google Scholar

Shrestha R, Meyer JP, Shenoi S, Khati A, Altice FL, Mistler C et al. COVID-19 vaccine hesitancy and associated factors among people with HIV in the United States: findings from a national survey. Vaccines (Basel). 2022;10(3).

Wickersham JA, Meyer JP, Shenoi S, Altice FL, Barakat LA, Virata M, et al. Willingness to be vaccinated against COVID-19 among people with HIV in the United States: results from a national survey. Front Med (Lausanne). 2022;9:886936.

Straus MA, Hamby SL, Boney-McCoy S, Sugarman DB. The revised conflict tactics scales (CTS2) development and preliminary psychometric data. J Fam Issues. 1996;17(3):283–316.

Straus MA, et al. The conflict tactics scales handbook. Los Angeles: Western Psychological Services.; 2003.

Weiss NH, Duke AA, Sullivan TP. Probable posttraumatic stress disorder and women’s use of aggression in intimate relationships: the moderating role of alcohol dependence. J Trauma Stress. 2014;27(5):550–7.

Sullivan TP, Holt LJ. PTSD symptom clusters are differentially related to substance use among community women exposed to intimate partner violence. J Trauma Stress. 2008;21(2):173–80.

Koss MP, Gidycz CA, Wisniewski N. The scope of rape: incidence and prevalence of sexual aggression and victimization in a national sample of higher education students. J Consult Clin Psychol. 1987;55(2):162–70.

Tolman RM. The validation of the psychological maltreatment of women inventory. Violence Vict. 1999;14(1):25–37.

Swan S, Sullivan T. Past abusive behavior inventory. New Haven, CT; 2004.

Jaquier V, Sullivan TP. Fear of past abusive partner(s) impacts current posttraumatic stress among women experiencing partner violence. Violence against Women. 2014;20(2):208–27.

Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. 1977;1(3):385–401.

Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7.

Foa EB, McLean CP, Zang Y, Zhong J, Powers MB, Kauffman BY, et al. Psychometric properties of the posttraumatic diagnostic scale for DSM-5 (PDS-5). Psychol Assess. 2016;28(10):1166–71.

Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption–II. Addiction. 1993;88(6):791–804.

Conigrave K, Hall W, Saunders J. The AUDIT questionnaire: choosing a cut-off score. Addiction. 2006;90:1349–56.

WHO Assist Working Group. The alcohol, smoking and substance involvement screening test (ASSIST): development, reliability and feasibility. Addiction. 2002;97(9):1183–94.

World Health Organization. The NIDA Modified ASSIST [ https://www.drugabuse.gov/sites/default/files/pdf/nmassist.pdf

Centers for Disease Control and Prevention. Fast Facts: Preventing Intimate Partner Violence |Violence Prevention|Injury Center|CDC: Centers for Disease Control and Prevention; 2022 [ https://www.cdc.gov/violenceprevention/intimatepartnerviolence/fastfact.html

Erickson M, Shannon K, Ranville F, Magagula P, Braschel M, Ratzlaff A, et al. Interpersonal violence and other social-structural barriers associated with needing HIV treatment support for women living with HIV. J Interpers Violence. 2022;37(11–12):NP9926–52.

Centers for Disease Control and Prevention. Risk for COVID-19 infection, hospitalization, and death by race/ethnicity: Centers for Disease Control and Prevention; 2022 [ https://www.cdc.gov/coronavirus/2019-ncov/covid-data/investigations-discovery/hospitalization-death-by-race-ethnicity.html

Xiao Y, Becerik-Gerber B, Lucas G, Roll SC. Impacts of working from home during COVID-19 pandemic on physical and mental well-being of office workstation users. J Occup Environ Med. 2021;63(3):181–90.

Langhinrichsen-Rohling J, Schroeder GE, Langhinrichsen-Rohling RA, Mennicke A, Harris YJ, Sullivan S et al. Couple conflict and intimate partner violence during the early lockdown of the pandemic: the good, the bad, or is it just the same in a North Carolina, low-resource population? Int J Environ Res Public Health. 2022;19(5).

National Institutes of Health. COVID-19 & Substance use: National Institutes of Health. 2023 [ https://nida.nih.gov/research-topics/comorbidity/covid-19-substance-use

U.S. Department of health and human services. Covid-19 & Substance use: National Institutes of Health. 2022 [ https://nida.nih.gov/research-topics/comorbidity/covid-19-substance-use

Centers for Disease Control and Prevention. HIV and substance use 2021 [ https://www.cdc.gov/hiv/basics/hiv-transmission/substance-use.html

Eaton EF, Tamhane A, Turner W, Raper JL, Saag MS, Cropsey KL. Safer in care: a pandemic-tested model of integrated HIV/OUD care. Drug Alcohol Depend. 2022;231:109241.

COVID-19 Mental Health Information and Resources. National Institutes of Health; [ https://covid19.nih.gov/covid-19-topics/mental-health

World Health Organization. COVID-19 pandemic triggers 25% increase in prevalence of anxiety and depression worldwide: World Health Organization. 2022 [ https://www.who.int/news/item/02-03-2022-covid-19-pandemic-triggers-25-increase-in-prevalence-of-anxiety-and-depression-worldwide

Parisi CE, Varma DS, Wang Y, Vaddiparti K, Ibanez GE, Cruz L, et al. Changes in mental health among people with HIV during the COVID-19 pandemic: qualitative and quantitative perspectives. AIDS Behav. 2022;26(6):1980–91.

Sun S, Hou J, Chen Y, Lu Y, Brown L, Operario D. Challenges to HIV care and psychological health during the COVID-19 pandemic among people living with HIV in China. AIDS Behav. 2020;24(10):2764–5.

## Acknowledgements

We wish to thank the study participants, community partners, and study staff who make this work possible.

Support for the parent project provided by the National Institute on Mental Health (R01 MH121991) to TPS and JPM. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health. Funding sources played no role in data analysis, interpretation of results, or decision to submit the manuscript for publication.

## Author information

Authors and affiliations.

Chronic Disease Epidemiology, Yale School of Public Health, New Haven, CT, USA

Xinyi Zhang & Jaimie P. Meyer

Section of Infectious Diseases, Yale School of Medicine, 135 College Street, Suite 323, New Haven, CT, 06510, USA

Carolina R. Price & Jaimie P. Meyer

Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA

Alexandrya S. Pope & Tami P. Sullivan

You can also search for this author in PubMed   Google Scholar

## Contributions

JPM and TPS acquired funding for the parent study. Acquisition, analysis, and interpretation of data by XZ, with input from CRP, ASP, and JPM. Study conceptualization and design by XZ and JPM. XZ and JPM initially drafted the manuscript, with critical input and revisions by CRP, ASP, and TPS. All authors meet ICMJE criteria for authorship and have seen and approved the final manuscript version prior to submission. The analysis was conducted as part of a Master of Public Health Thesis (XZ) and subsequently adapted for publication.

## Corresponding author

Correspondence to Jaimie P. Meyer .

## Ethics declarations

Ethics approval and consent to participate.

Individuals who meet all eligibility criteria are offered enrollment and undergo written informed consent procedures. All procedures were approved by the Yale University Human Investigations Committee (IRB). The authors confirm that all methods were carried out in accordance with relevant guidelines and regulations.

Not applicable.

## Competing interests

The authors declare no competing interests.

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Electronic supplementary material

Below is the link to the electronic supplementary material.

## Supplementary Material 1

Supplementary material 2, rights and permissions.

Reprints and permissions

Zhang, X., Price, C.R., Pope, A.S. et al. Impact of COVID-19 on women living with HIV who are survivors of intimate partner violence. BMC Public Health 24 , 1352 (2024). https://doi.org/10.1186/s12889-024-18862-7

Accepted : 15 May 2024

Published : 20 May 2024

DOI : https://doi.org/10.1186/s12889-024-18862-7

Anyone you share the following link with will be able to read this content:

Provided by the Springer Nature SharedIt content-sharing initiative

• Intimate partner violence

## BMC Public Health

ISSN: 1471-2458

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Journal List
• Braz J Phys Ther
• v.22(3); May-Jun 2018

## The importance of determining the clinical significance of research results in physical therapy clinical research

Susan armijo-olivo.

a University of Alberta, Faculty of Rehabilitation Medicine, Faculty of Medicine and Dentistry, Edmonton, Canada

b Institute of Health Economics, Edmonton, Canada

Clinical research in physical therapy is fundamental to generate new knowledge and validate our therapies. The evaluation of research findings is crucial to help clinical decision making and to comply with the principles of evidence based-practice. Statistical significance testing has dominated the way researchers typically report their results and evaluate their significance. 1 , 2 This approach has been commonly used to determine the importance and generalizability of research results and to demonstrate the effect of an intervention in health research. However, this approach has limited use for clinicians and decision makers trying to offer the best possible treatment to patients.

Statistical significance is based on hypothesis testing (i.e. null hypothesis vs. alternative hypothesis). 3 The decision to accept or reject the null hypothesis is based on predetermined levels of probability (i.e. p  < 0.05 or 0.01) used to test the strength of the evidence against the null hypothesis. 4 The dichotomous choice that emerges from the hypothesis testing procedure (i.e. null hypothesis rejected or accepted) does not provide any insights whether the results of the study are important for different stakeholders such as patients, clinicians, and decision makers. 5 , 6

Statistical significance does not assure that the results are clinically relevant. Indeed, the use of null hypothesis significance testing rarely determines the practical importance or clinical relevance of findings. 1 , 7 In addition, statistical significance can also provide misleading results to the clinical community since a statistical difference between groups could be found if the sample size was large and/or if the inter-subject variability was low, even though the difference between groups was small to be considered clinically important by the patients or clinicians. 5 This has been exemplified in one of our studies, where a difference between subjects with TMD and healthy controls on head and cervical posture was statistically significant but that difference was only 3.3°, [95%CI 0.15, 6.41], which according to any clinician working on the field, would not be clinically relevant since clinicians generally use clinical observation or in some cases, photographs to evaluate posture and it is unlikely that this assessment would allow to consistently detect such a small difference.

Given the limitations of statistical significance, it is relevant for physical therapy practice that results of clinical research are analyzed having in mind the clinical relevance of the results. The question whether a patient has improved in a meaningful way is fundamental to improve clinical decision making regarding treatment management. Since clinicians are interested in whether or not the intervention had an impact on clinical outcomes and also in the magnitude of such impact, relying solely on statistical significance to conclude about relevance of results seems to be limited and insufficient.

Clinical relevance (also known as clinical significance) indicates whether the results of a study are meaningful or not for several stakeholders. 7 A clinically relevant intervention is the one whose effects are large enough to make the associated costs, inconveniences, and harms worthwhile. 8 Clinical relevance facilitates the understanding and interpretation of results for clinicians. In physical therapy, the assessment of this approach has become a popular method to assist the transfer of knowledge into clinical practice. 1 , 7 , 9

Diverse methodologies have been developed in the attempt to determine the clinical significance of an intervention. The most common methods are the “distribution-based methods” and the “anchor-based methods”. The calculation of the effect size (ES), the minimum detectable change (MDC)/difference (MDD), 10 and the standard error of measurement (SEM) are examples of the distribution-based methods. 1 , 7 Anchor-based methods involve the client's perspective using an anchor, commonly the use of the Global Rating Scale of Change (GRSC) 2 to define the minimal important difference (MID). Researchers and clinicians interested in these methods are encouraged to see Jaeschke et al., 2 Armijo-Olivo et al., 7 Musselman, 1 as well as De Vet et al., 10 for a complete description.

Researchers conducting clinical trials in the field of physical therapy have the obligation to report the clinical relevance of results to the clinical community to adhere to the principles of evidence based practice. This will help disseminate evidence in a useful and understandable way for end-users such as patients, health care clinicians, and policy/decision-makers. The information of “ p ” values is insufficient to achieve these requirements and because it provides insufficient and limited information, clinical researchers needed to present the clinical relevance of their results to help busy clinicians with interpretation and easy uptake of research results in clinical practice.

• Open access
• Published: 15 May 2024

## Statins did not reduce the frequency of exacerbations in individuals with COPD and cardiovascular comorbidities in the COSYCONET cohort

• N. Frantzi 1 ,
• X. P. Nguyen 1 ,
• C. Herr 2 ,
• P. Alter 3 ,
• S. Söhler 3 ,
• D. Soriano 1 ,
• H. Watz 4 , 5 ,
• B. Waschki 4 , 6 , 7 ,
• F. Trinkmann 8 ,
• M. Eichenlaub 9 ,
• F. C. Trudzinski 8 ,
• J. D. Michels-Zetsche 8 ,
• A. Omlor 2 ,
• F. Seiler 2 ,
• I. Moneke 10 ,
• F. Biertz 11 ,
• G. Rohde 12 ,
• D. Stolz 1 ,
• T. Welte 13 ,
• H. U. Kauczor 14 ,
• K. Kahnert 15 ,
• R. A. Jörres 16 ,
• C. F. Vogelmeier 3 ,
• R. Bals 2 , 17 &
• S. Fähndrich 1

## on behalf of the German COSYCONET Cohort

Respiratory Research volume  25 , Article number:  207 ( 2024 ) Cite this article

125 Accesses

Metrics details

The evidence regarding effects of statins on exacerbation risk in COPD remains controversial. Previous studies often excluded patients with cardiovascular comorbidities despite their high prevalence in COPD and role for exacerbations. Based on the cardioprotective properties of statins, we hypothesised that statins may reduce the risk of exacerbations especially in patients with cardiovascular comorbidities.

One thousand eight hundred eighty seven patients of the German COPD cohort COSYCONET (COPD and Systemic Consequences Comorbidities Network) of GOLD grades 1–4 (37.8% female, mean age 64.78 ± 8.3) were examined at baseline and over a period of 4.5 years for the occurrence of at least one exacerbation or severe exacerbation per year in cross-sectional and longitudinal analyses adjusted for age, gender, BMI, GOLD grade and pack-years. Due to their collinearity, various cardiovascular diseases were tested in separate analyses, whereby the potential effect of statins in the presence of a specific comorbidity was tested as interaction between statins and comorbidity. We also identified patients who never took statins, always took statins, or initiated statin intake during the follow-up.

One thousand three hundred six patients never took statins, 31.6% were statin user, and 12.9% initiated statins during the follow-up. Most cardiovascular diseases were significantly ( p  < 0.05)may associated with an increased risk of COPD exacerbations, but in none of them the intake of statins was a significant attenuating factor, neither overall nor in modulating the increased risk linked to the specific comorbidities. The results of the cross-sectional and longitudinal analyses were consistent with each other, also those regarding at least 1 exacerbation or at least 1 severe exacerbation per year.

These findings complement the existing literature and may suggest that even in patients with COPD, cardiovascular comorbidities and a statin therapy that targets these comorbidities, the effects of statins on exacerbation risk are either negligible or more subtle than a reduction in exacerbation frequency.

## Trial registration

Trial registration ClinicalTrials.gov, Identifier: NCT01245933.

Other Study ID (BMBF grant): 01GI0881, registered 18 November 2010, study start 2010–11, primary completion 2013–12, study completion 2023–09.

https://clinicaltrials.gov/study/NCT01245933?cond=COPD&term=COSYCONET&rank=3

## Introduction

Chronic obstructive pulmonary disease (COPD) is a heterogeneous disease with chronic airway inflammation and multi-organ involvement caused by the interplay of inhaled pollutants, genetic predisposition and socioeconomic factors [ 1 ]. Exacerbations are known to be associated with disease progression and mortality [ 2 ], thus a major treatment goal is the reduction of the frequency of exacerbations, including the possibility of drug repurposing. In the past, statins (HMG-CoA reductase inhibitors), commonly used in patients with increased cardiovascular risk, have been considered regarding this effect [ 3 ]. The reason for this is that, in addition to its specific lipid-lowering effects, this class of drugs also has pleiotropic effects, including anti-inflammatory effects, improving endothelial function, increasing the stability of atherosclerotic plaques and reducing oxidative stress [ 4 ]. Even in a population without marked hyperlipidemia (total cholesterol level < 200 mg/dL), statin intake resulted in the reduction of cardiovascular disease events [ 5 ]. Several cardiovascular conditions, such as heart failure, ischemic heart disease and atrial fibrillation, are associated with an increased risk of exacerbations and mortality in COPD [ 6 , 7 , 8 , 9 ], while statins are known to have positive effects on cardiovascular disease [ 5 ]. Thus, the hypothesis of an effect of statins on COPD exacerbations is well founded.

Consequently, several prospective placebo-controlled studies and cohort studies have examined the potential effect of statins on the exacerbation rate in individuals with COPD, the results, however, were equivocal [ 10 , 11 , 12 , 13 ]. In patients with high risk for exacerbations, for example, a prospective randomised placebo-controlled trial found no effect of 40 mg simvastatin daily on exacerbation rate or time to exacerbation [ 11 ]; however, patients taking or requiring statins due to their cardiovascular risk profile were excluded. In contrast, another study reported that simvastatin significantly prolonged the time to first COPD exacerbation and reduced the exacerbation rate [ 13 ]. Based on such findings, it was stated [ 14 ] that at present there was no evidence of a positive effect of statins on exacerbations rate and mortality but that further studies were needed.

Due to the considerations mentioned above, it seems a reasonable hypothesis that the effect of statins on exacerbation risk in COPD might only occur in the presence of cardiovascular comorbidities or cardiovascular risk factors that are targeted by this medication, and that this dependence might explain the heterogeneous data. To address this, the comprehensive set of data from patients with stable COPD of the COSYCONET (COPD and Systemic Consequences–Comorbidities Network) cohort was used.

## Study population

The longitudinal, observational cohort COSYCONET addresses the association between chronic lung disease and comorbidities [ 15 ]. Between 2010 and 2013, 2741 individuals with stable COPD were recruited. Major inclusion criteria were age ≥ 40 years and a physician-based diagnosis of COPD; further details can be found elsewhere [ 15 ]. Assessments took part 6, 18, 36, and 54 months (Visits 2–5) after the baseline visit (Visit 1). Approval by the local ethical committees of each study center was achieved, and all patients gave their written informed consent. The study was performed according to the declaration of Helsinki, and the identifier on Clinical-Trials.gov is NCT01245933.

## Functional and clinical assessments

Details of clinical and functional assessments in COSYCONET have been described previously [ 15 ]. The measured variables included age, body mass index (BMI), smoking status, packyears, forced expiratory volume in 1 s (FEV 1 ) and forced vital capacity (FVC) determined according to ATS/ERS recommendations [ 16 ], diffusing capacity for carbon monoxide (TLCO), 6-min walk distance (6-MWD) [ 17 ], the St George’s Respiratory Questionnaire (SGRQ) [ 18 ], the number of exacerbations and of severe exacerbations in the previous year; from these variables the BODE index was computed [ 19 ]. Predicted values for lung function were taken from GLI [ 20 , 21 ]. In addition, the ankle-brachial (ABI) was assessed [ 15 ] and categorised according to values ≤ 0.90. Routine laboratory parameters included C-reactive protein (CRP), HbA1c [ 15 ], total cholesterol, triglycerides, and HDL- and LDL-cholesterol.

Patient-reported, physician-diagnosed comorbidities comprised diabetes, history of stroke, cardiovascular disorders including heart failure, coronary artery disease and history of myocardial infarction. All patients were categorised into GOLD groups A/B/E based on their symptoms assessed via the mMRC [ 22 ] and their exacerbation history following the criteria proposed by GOLD [ 2 ]. The documentation of the exacerbation history occurred at each visit based on the definition of acute worsening of a lung disease, such as increased shortness of breath, increased or purulent sputum. The definition of exacerbations was according to GOLD: Acute respiratory deterioration over several days and the need for specific measures (mild exacerbation: handled by the patient itself, moderate exacerbation: visit to the primary care physician, severe exacerbation: resulted in a hospital admission) [ 23 ]. Patients were asked to bring all of their medication to each study visit, which was then categorised according to ATC codes as described previously [ 24 ].

## Definition of groups and outcome

We included only patients with a ratio FEV 1 /FVC < 0.7 who could be categorised into GOLD grades 1–4 according to FEV 1 [ 2 ]. The follow-up included visits 1, 2, 3, 4 and 5, covering a period of 4.5 years (Fig.  1 ). For analysis, we defined the three groups of patients who were either taking statins already at baseline (statin users, SU) or statin-naïve (SN) or initiated the intake of statins during follow-up (incident-statins, IS). The endpoint of the study was the occurrence of at least one exacerbation or at least one severe exacerbation in the last year, based on the clinical history documented at each visit.

Flow chart showing the selection process of patients for analysis

## Statistical analysis

We present categorical data as absolute counts and percentages, and continuous data as mean ± standard deviation (SD). Three types of comparison were performed. First, baseline data were compared between groups SN and SU, as well as SN and IS, using unpaired t-tests for metric variables and chi-square statistics for categorical variables. Second, we utilised logistic regression models to assess the impact of comorbidities and statins on the occurrence of at least one exacerbation per year in the baseline data, while adjusting for age, sex, BMI, GOLD grade and pack-years. Third, we employed mixed models (Generalised Estimating Equations GEE, binomial distribution, logit link) for the longitudinal analysis of statin prescription and its effect on exacerbations over visits V1 to V5 equivalent to 4.5 years of follow-up, again adjusting for age, sex, BMI, GOLD grade and pack-years; the outcomes were either the occurrence of at least one exacerbation or of at least one severe exacerbation per year. All analyses were performed separately including different cardiovascular comorbidities as predictor, due to their significant correlation with each other. Thus, besides age, sex, BMI, GOLD grade and pack-years the predictors, the logistic and the longitudinal analyses included the presence of the specific comorbidity, that of statins and an interaction term between these two factors. All patients with all their visits were kept in the longitudinal analyses but additionally a sensitivity analysis was performed including only those patients that remained in the study until visit 5. A two-sided p -value < 0.05 was considered as statistically significant. For data analysis, the software package SPSS (Version: 29.0.0.0., IBM, Armonk, NJ, USA) was used.

## Baseline characteristics

Figure  1 illustrates the inclusion and follow-up criteria of the patients. 450 patients were excluded. Patients with FEV1/FVC ≥ 0.7 ( n  = 430) dropped out. 20 patients had no available information on their GOLD class. 234 patients were not considered due to missing information on their pack-years. 127 patients were excluded due to missing information on LDL cholesterol. The remaining exclusions were attributed to missing information in various parameters: BMI: 2 patients, Exacerbations: 3 patients, FEV1: 16 patients, TLCO: 160 patients, CRP: 41 patients, HbA1c: 71 patients. Out of the 413 patients in the SU group, 154 (37.3%) completed visit 5. Similarly, among the 1306 patients in the SN group, 476 (36.4%) completed visit 5. Table 1 shows the baseline characteristics of the three groups.

## Statin users (SU) versus statin-naïve (SN)

There were significant differences between the two groups. More men (74.6% vs. 57.8%) and more patients with diabetes (25.4 vs. 9.1%) and stroke (8.7 vs. 3.0%) took statins from the beginning until visit 5, compared to the individuals who were statin-naïve. Furthermore, subjects in the SU group were older (67.5 ± 7.3 vs. 63.8 ± 8.6 years), had higher BMI (27.5 ± 5.0 vs. 26.4 ± 5.4 kg/m 2 ), and exhibited significantly lower LDL cholesterol levels (2.73 ± 1.01 vs. 3.39 ± 0.97), lower HDL cholesterol levels (1.61 ± 0.6 vs. 1.78 ± 0.6), lower total cholesterol levels (4.87 ± 1.21 vs. 5.67 ± 1.14), higher triglyceride levels (1.70 ± 1.11 vs. 1.50 ± 0.96), and higher HbA1c levels (6.14 ± 0.83 vs. 5.84 ± 0.62). The two groups did not show significant differences with regard to FEV 1 % predicted, FVC % predicted, SQRQ, the distribution of GOLD grades, BODE index, and CRP levels.

## Incident-statins (IS) versus statin-naïve (SN)

Compared to the SN group, statins were newly prescribed during the follow-up (IS) in older subjects (66.2 ± 7.0 vs. 63.8 ± 8.6) and individuals with more packyears of smoking (54.5 ± 40.3 vs. 47.2 ± 34.2). These subjects also had a better lung function in terms of FEV 1 % predicted (59.6 ± 18.3 vs. 52.2 ± 18.3) and higher FVC % predicted (84.6 ± 18.9 vs. 78.7 ± 18.7). At baseline, total cholesterol was lower (4.78 ± 1.16 vs. 5.67 ± 1.14), LDL cholesterol was lower (2.60 ± 1.0 vs. 3.39 ± 0.97), and HDL cholesterol was lower (1.67 ± 0.52 vs. 1.78 ± 0.60). BMI, prevalence of diabetes, history of stroke, SQRQ, BODE index, HbA1c and CRP did not significantly differ between the two groups.

## Association between statin intake, comorbidities and exacerbation frequency

Overall, 7777 acute COPD exacerbations were registered during the 4.5-year follow-up. Table 2 displays the distribution of exacerbations across visits. In the unpaired t-test there were no significant differences in exacerbation rates per person year between the SU and the SN groups, with values of 1.2 ± 1.87 vs. 1.22 ± 1.87, respectively ( p  = 0.87).

## Cardiovascular diseases and exacerbation frequency at baseline

We investigated whether cardiovascular comorbidities were associated with an elevated risk of exacerbations, using baseline data and logistic regression models. All five cardiovascular entities that had been included were significantly associated with the exacerbation risk, with Odds Ratios as shown in Table  3 and illustrated in Fig.  2 A. The presence of statins did not show any significant associations with exacerbations (Table  3 , Fig.  2 B), and the same was true for the interaction term between statins and comorbidity (Table  3 ), suggesting the absence of differential effects in patients with specific cardiovascular comorbidities.

A Forest plot displaying the adjusted odds ratios and their 95% confidence intervals for the association between each cardiovascular comorbidity (and a summary value for at least one disease and a summary value for the combination of two comorbidities) and the occurrence of at least one exacerbation per year at baseline. The numerical data can be found in Table  3 and were obtained by logistic regression analysis adjusted for sex, age, BMI, GOLD grade and pack-years. B Forest plot displaying the adjusted odds ratios and their 95% confidence intervals for the association between the medication with statins and the occurrence of at least one exacerbation per year at baseline for each of the cardiovascular comorbidities (and a summary value for at least one disease and a summary value for the combination of two comorbidities) shown in Fig. 2A. The numerical data can be found in Table  3 and were obtained by logistic regression analysis adjusted for sex, age, BMI, GOLD grade and pack-years

## Cardiovascular diseases and exacerbation frequency in the follow-up

In the longitudinal analysis over 4.5 years, the occurrence of at least one exacerbation per year, heart failure, cardiac arrhythmias, CAD and hypertension showed significant associations with the occurrence of exacerbations, while PAD did not (Table  4 , Fig.  3 A). Statins did not have significant effects in any of the cardiovascular conditions (Table  3 , Fig.  3 B), and the same applied to the interaction terms (Table  4 ).

A Forest plot displaying the adjusted odds ratios and their 95% confidence intervals for the association between each cardiovascular comorbidity (and a summary value for at least one disease and a summary value for the combination of two comorbidities) and the occurrence of at least one exacerbation per year between the investigated cardiovascular diseases and exacerbation occurrence during the 4.5 years of follow up. The numerical data can be found in Table  4 and were obtained by a GEE analysis adjusted for sex, age, BMI, GOLD grade and pack-years. B Forest plot displaying the adjusted odds ratios and their 95% confidence intervals for the association between the medication with statins and the occurrence of at least one exacerbation per year at baseline for each of the cardiovascular comorbidities (and a summary value for at least one disease and a summary value for the combination of two comorbidities) shown in Fig. 3A during the 4.5 years of follow up. The numerical data can be found in Table  4 and were obtained by a GEE analysis adjusted for sex, age, BMI, GOLD grade and pack-years

When repeating the analysis with the outcome of at least one severe exacerbation per year, qualitatively similar results were obtained as for exacerbations in general (Table  5 , Figs.  4 A and B), with the exception that hypertension was no more significantly associated with an increased exacerbation risk. It also became apparent that only the presence of at least 2 cardiovascular comorbidities was significantly associated with increased risk of severe exacerbation.

A Forest plot displaying the adjusted odds ratios and their 95% confidence intervals for the association between each cardiovascular comorbidity and the occurrence of at least one severe exacerbation per year between the investigated cardiovascular diseases and exacerbation occurrence during the 4.5 years of follow up (and a summary value for at least one disease and a summary value for the combination of two comorbidities). The numerical data can be found in Table  5 and were obtained by a GEE analysis adjusted for sex, age, BMI, GOLD grade and pack-years. B Forest plot displaying the adjusted odds ratios and their 95% confidence intervals for the association between the medication with statins and the occurrence of at least one severe exacerbation per year at baseline for each of the cardiovascular comorbidities shown in Fig. 4A during the 4.5 years of follow up (and a summary value for at least one disease and a summary value for the combination of two comorbidities). The numerical data can be found in Table  5 and were obtained by an GEE analysis adjusted for sex, age, BMI, GOLD grade and pack-years

In order to reveal whether the result critically depended on patients’ loss to follow-up, we performed a sensitivity analysis. For this purpose, the longitudinal analyses were repeated while including only those patients who participated until visit 5. The results of this sensitivity analysis for at least one exacerbation are given in the Supplemental Table S1 and those for at least one severe exacerbation in the Supplemental Table S2. It can be seen that only heart failure and cardiac arrhythmias were significantly related to the occurrence of at least one exacerbation or one severe exacerbation and that in case of severe exacerbations the presence of at least 2 comorbidities was required for a significant association. Statins still did not show significant associations.

In this study, we examined potential effects of statins on exacerbation risk in patients with COPD, focusing on the potential role of cardiovascular comorbidities for such effects. Given the high prevalence of cardiovascular comorbidities in COPD and the fact that a number of previous studies did exclude patients with cardiovascular disease [ 11 , 13 ], this seems of both scientific and clinical interest. The present study confirmed the results of previous cross-sectional analyses in the COSYCONET cohort that cardiovascular comorbidities were associated with an increased risk of exacerbations [ 25 , 26 ]. In line with this, it also demonstrated in a longitudinal analysis over a 4.5-year period that the presence of heart failure, cardiac arrhythmias, coronary artery disease and hypertension was associated with exacerbation risk. Patients with COPD often show cardiovascular comorbidities that can be the cause of an acute deterioration independent of the lung disease [ 25 , 26 ]. This association could be based on several factors, a fact that was covered by the rather general (until 2022) definition of exacerbations as “acute worsening of respiratory symptoms that results in additional therapy” which did not address specific causes [ 27 ]. Statins have their place in the prevention of cardiovascular diseases [ 28 , 29 , 30 ]. Thus, it appears reasonable to expect effects on COPD exacerbation risk in subjects with at least one cardiovascular comorbidity. Contrary to our hypothesis, however, there was no reduction in the risk of exacerbations including severe exacerbations. Even in patients with at least two cardiovascular comorbidities, statins did not have a significant effect in the longitudinal follow-up compared to patients not taking statins. This also applied to patients who got their first prescribed statins during the observation period of 4.5 years. We used cross-sectional and longitudinal statistical standard approaches including main factors and interaction terms, but neither the main factors of statin intake nor their interactions with cardiovascular comorbidities indicated an effect.

Our data are in line with those of the multi-centre placebo-controlled study by Criner et al., in which simvastatin at a daily dose of 40 mg had no effect on the rate of exacerbations and the time to the next exacerbation [ 11 ]. This study was performed in 885 patients with COPD treated from 12 to 36 months, who were at high risk for exacerbations but without cardiovascular diseases, diabetes and without requiring statins. These findings and our own findings are, however, not in line with those of the recently published prospective, randomised, double-blinded, placebo-controlled study by Schenk et al. investigating the impact of simvastatin on exacerbation rate and time to next exacerbation [ 13 ]. It was found that 40 mg simvastatin daily prolonged the time to first exacerbation and reduced the exacerbation rate. As the authors excluded patients with cardiovascular diseases, this seems remarkable, if it is argued that effects were to be expected primarily in these patients. There are, however, differences in sample size and follow-up period that might be relevant. While in COSYCONET subjects were observed over a follow-up period of 4.5 years, the scope of the prospective randomised study was limited to 1 year. During this time, 66 subjects with placebo and 72 subjects on simvastatin completed the study. On the other hand, a clear strength was that Schenk et al. used diaries that enabled a precise recording and reporting of exacerbations.

Although the available data therefore remain conflicting, our study adds to the current knowledge as it particularly addressed the hypothesis that the effects of statins would be apparent in those patients with COPD who had cardiovascular diseases and thus a risk factor for exacerbations that was targeted by the statins. The observed lack of effect suggests that the effects of statins on acute cardiovascular events that are related to exacerbations were secondary, despite the well-known long-term protective effect of statins on the progression of cardiovascular disease. Nevertheless, in the future, a randomized controlled study would be beneficial which specifically examines the effect of statins on cardiovascular events and exacerbations in patients with COPD and cardiovascular diseases.

## Limitations and strengths of the study

COSYCONET is not a placebo-controlled, randomised study, and its observational character naturally limits the scope of conclusions that can be drawn. Moreover, although the definition of exacerbations followed established criteria and practice, it certainly would have benefitted from a more precise recording, for example via an electronic diary. On the other hand, at least the occurrence of severe exacerbations can be assumed to have been recorded reliably. We also cannot provide data on the association between statin intake and the precise time to the next exacerbation (time to event). Moreover, we analysed statins as a drug class without distinguishing between different drugs of this class, whereas the randomised studies examined specific drugs, in particular simvastatin, and it cannot be excluded that effects differ between different statins. Furthermore, we did not have information about the duration of intake until inclusion in the cohort study, and the different dosages and types of preparations could not be taken into account. It should also be noted that we cannot exclude a healthy survivor bias due to loss to follow-up, although the comparison of all patients with those remaining in the study until the end of follow-up yielded consistent results. In the longitudinal analyses reported here, we included visit 2 data despite their overlap with visit 1 data, as sensitivity analyses omitting visit 2 data did not yield different results. Moreover, in the study population visits were performed throughout the year, thus potential inhomogeneity over time due to the fact that exacerbations were reported for the previous 12 months and study visits were mostly separated by 18 months, are unlikely. The strengths of our study are that we examined a large number of subjects over a long period of time and covered a whole range of cardiovascular comorbidities that allowed for a detailed analysis of potential effects of statins in specific cardiovascular conditions.

The present study addressed the hypothesis that in patients with COPD the presence of cardiovascular comorbidities might be a relevant factor determining the effect of statins on exacerbation risk. Using a large data set, we found in cross-sectional and longitudinal analysis that most cardiovascular comorbidities were associated with increased exacerbation risk but that statins were neither linked to an overall reduction of this risk nor to specific effects in patients with different cardiovascular comorbidities. These data complement the existing literature and may suggest that even in patients with a therapy that specifically targets such comorbidities, potential effects of statins are either negligible or more subtle than a reduction of exacerbation frequency. Additional research is required to expand upon this hypothesis in future studies.

## Availability of data and materials

No datasets were generated or analysed during the current study.

Stolz D, Mkorombindo T, Schumann DM, Agusti A, Ash SY, Bafadhel M, et al. Towards the elimination of chronic obstructive pulmonary disease: a lancet commission. Lancet (London, England). 2022;400(10356):921–72.

Vogelmeier CF, Criner GJ, Martinez FJ, Anzueto A, Barnes PJ, Bourbeau J, et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive lung disease 2017 report GOLD executive summary. Am J Respir Crit Care Med. 2017;195(5):557–82.

Mannino DM, Thorn D, Swensen A, Holguin F. Prevalence and outcomes of diabetes, hypertension and cardiovascular disease in COPD. Eur Respir J. 2008;32(4):962–9.

Liao JK, Laufs U. Pleiotropic effects of statins. Annu Rev Pharmacol Toxicol. 2005;45:89–118.

Article   CAS   PubMed   PubMed Central   Google Scholar

Chou R, Dana T, Blazina I, Daeges M, Jeanne TL. Statins for prevention of cardiovascular disease in adults: evidence report and systematic review for the US preventive services task force. JAMA. 2016;316(19):2008–24.

Westerik JA, Metting EI, van Boven JF, Tiersma W, Kocks JW, Schermer TR. Associations between chronic comorbidity and exacerbation risk in primary care patients with COPD. Respir Res. 2017;18(1):31.

Article   PubMed   PubMed Central   Google Scholar

Divo M, Cote C, de Torres JP, Casanova C, Marin JM, Pinto-Plata V, et al. Comorbidities and risk of mortality in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2012;186(2):155–61.

Miller J, Edwards LD, Agustí A, Bakke P, Calverley PM, Celli B, et al. Comorbidity, systemic inflammation and outcomes in the ECLIPSE cohort. Respir Med. 2013;107(9):1376–84.

Lahousse L, Tiemeier H, Ikram MA, Brusselle GG. Chronic obstructive pulmonary disease and cerebrovascular disease: a comprehensive review. Respir Med. 2015;109(11):1371–80.

Obeidat M, Fishbane N, Nie Y, Chen V, Hollander Z, Tebbutt SJ, et al. The effect of statins on blood gene expression in COPD. Plos One. 2015;10(10): e0140022.

Criner GJ, Connett JE, Aaron SD, Albert RK, Bailey WC, Casaburi R, et al. Simvastatin for the prevention of exacerbations in moderate-to-severe COPD. N Engl J Med. 2014;370(23):2201–10.

Søyseth V, Brekke PH, Smith P, Omland T. Statin use is associated with reduced mortality in COPD. Eur Respir J. 2007;29(2):279–83.

Schenk P, Spiel AO, Hüttinger F, Gmeiner M, Fugger J, Pichler M, et al. Can simvastatin reduce COPD exacerbations? A randomised double-blind controlled study. Eur Respir J. 2021;58(1):2001798.

Walsh A, Perrem L, Khashan AS, Henry MT, Ni Chroinin M. Statins versus placebo for people with chronic obstructive pulmonary disease. Cochrane Database System Rev. 2019;7(7):Cd011959.

Karch A, Vogelmeier C, Welte T, Bals R, Kauczor HU, Biederer J, et al. The German COPD cohort COSYCONET: Aims, methods and descriptive analysis of the study population at baseline. Respir Med. 2016;114:27–37.

Brusasco V, Crapo R, Viegi G. Coming together: the ATS/ERS consensus on clinical pulmonary function testing. Eur Respir J. 2005;26(1):1–2.

Holland AE, Hill CJ, Rasekaba T, Lee A, Naughton MT, McDonald CF. Updating the minimal important difference for six-minute walk distance in patients with chronic obstructive pulmonary disease. Arch Phys Med Rehabil. 2010;91(2):221–5.

Jones PW, Quirk FH, Baveystock CM, Littlejohns P. A self-complete measure of health status for chronic airflow limitation. The St. George’s respiratory questionnaire. Am Rev Respir Dis. 1992;145(6):1321–7.

Celli BR, Cote CG, Marin JM, Casanova C, de Oca Montes M, Mendez RA, et al. The body-mass index, airflow obstruction, dyspnea, and exercise capacity index in chronic obstructive pulmonary disease. N Engl J Med. 2004;350(10):1005–12.

Quanjer PH, Stanojevic S, Cole TJ, Baur X, Hall GL, Culver BH, et al. Multi-ethnic reference values for spirometry for the 3–95-yr age range: the global lung function 2012 equations. Eur Respir J. 2012;40(6):1324–43.

Stanojevic S, Graham BL, Cooper BG, Thompson BR, Carter KW, Francis RW, et al. Official ERS technical standards: Global lung function initiative reference values for the carbon monoxide transfer factor for Caucasians. Eur Respir J. 2017;50(3):1700010.

Hajiro T, Nishimura K, Tsukino M, Ikeda A, Koyama H, Izumi T. Analysis of clinical methods used to evaluate dyspnea in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 1998;158(4):1185–9.

Byng D, Lutter JI, Wacker ME, Jörres RA, Liu X, Karrasch S, et al. Determinants of healthcare utilization and costs in COPD patients: first longitudinal results from the German COPD cohort COSYCONET. Int J Chron Obstruct Pulmon Dis. 2019;14:1423–39.

Lucke T, Herrera R, Wacker M, Holle R, Biertz F, Nowak D, et al. Systematic analysis of self-reported comorbidities in large cohort studies - a novel stepwise approach by evaluation of medication. Plos One. 2016;11(10): e0163408.

Alter P, Lucke T, Watz H, Andreas S, Kahnert K, Trudzinski FC, et al. Cardiovascular predictors of mortality and exacerbations in patients with COPD. Sci Rep. 2022;12(1):21882.

Kahnert K, Alter P, Young D, Lucke T, Heinrich J, Huber RM, et al. The revised GOLD 2017 COPD categorization in relation to comorbidities. Respir Med. 2018;134:79–85.

GOLD. GIfCOLDA. Global Initiative for Chronic Obstructive Lung Disease. Available from: www.goldcopdorg 2022.

Treasure CB, Klein JL, Weintraub WS, Talley JD, Stillabower ME, Kosinski AS, et al. Beneficial effects of cholesterol-lowering therapy on the coronary endothelium in patients with coronary artery disease. N Engl J Med. 1995;332(8):481–7.

Sacks FM, Pasternak RC, Gibson CM, Rosner B, Stone PH. Effect on coronary atherosclerosis of decrease in plasma cholesterol concentrations in normocholesterolaemic patients. Harvard Atherosclerosis Reversibility Project (HARP) Group. Lancet (London, England). 1994;344(8931):1182–6.

Collins R, Armitage J, Parish S, Sleight P, Peto R. Effects of cholesterol-lowering with simvastatin on stroke and other major vascular events in 20536 people with cerebrovascular disease or other high-risk conditions. Lancet (London, England). 2004;363(9411):757–67.

## Acknowledgements

The authors thank all patients of COSYCONET for their kind cooperation and all study centers for their excellent work. We are grateful to the Scientific Advisory Board of COSYCONET for continuing support und helpful recommendations.

Open Access funding enabled and organized by Projekt DEAL. COSYCONET is supported by the German Federal Ministry of Education and Research (BMBF) Competence Network Asthma and COPD (ASCONET) and performed in collaboration with the German Center for Lung Research (DZL). The project is funded by the BMBF with grant number 01 GI 0881, and is supported by unrestricted grants from AstraZeneca GmbH, Bayer Schering Pharma AG, Boehringer Ingelheim Pharma GmbH & Co. KG, Chiesi GmbH, GlaxoSmithKline, Grifols Deutschland GmbH, MSD Sharp & Dohme GmbH, Mundipharma GmbH, Novartis Deutschland GmbH, Pfizer Pharma GmbH, Takeda Pharma Vertrieb GmbH & Co. KG, Teva GmbH for patient investigations and laboratory measurements. For the present study, an additional grant for data management was given by Novartis Pharma GmbH. The funding body had no involvement in the design of the study, or the collection, analysis or interpretation of the data.

## Author information

Authors and affiliations.

Department of Pneumology, Faculty of Medicine, Medical Center, University of Freiburg, Freiburg, Germany

N. Frantzi, X. P. Nguyen, D. Soriano, D. Stolz & S. Fähndrich

Department of Internal Medicine V - Pulmonology, Allergology, Critical Care Care Medicine, Saarland University Hospital, Homburg, Germany

C. Herr, A. Omlor, F. Seiler & R. Bals

Department of Medicine, Pulmonary, Critical Care and Sleep Medicine, German Center for Lung Research (DZL), Philipps University of Marburg (UMR), Marburg, Germany

P. Alter, S. Söhler & C. F. Vogelmeier

Airway Research Center North (ARCN), Pulmonary Research Institute at LungenClinic Grosshansdorf, Grosshansdorf, DZ, Germany

H. Watz & B. Waschki

LungenClinic Grosshansdorf, Airway Research Center North (ARCN), Member of the German Center for Lung Research (DZL), Grosshansdorf, Germany

Hospital Itzehoe, Pneumology, Itzehoe, Germany

Department of Cardiology, University Heart & Vascular Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany

Department of Pneumology and Critical Care, Member of the German Center for Lung Research (DZL), Translational Lung Research Center Heidelberg (TLRC-H), Thoraxklinik Heidelberg gGmbH, Heidelberg, Germany

F. Trinkmann, F. C. Trudzinski & J. D. Michels-Zetsche

Department of Cardiology and Angiology, Medical Center, University of Freiburg, Freiburg, Germany

M. Eichenlaub

Department of Thoracic Surgery, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany

Hannover Medical School, CAPNETZ STIFTUNG, Hannover, Germany

Department of Respiratory Medicine, Goethe University Frankfurt, University Hospital, Medical Clinic I, Frankfurt/Main, Germany

Department of Respiratory Medicine, (BREATH), Member of the German Center for Lung Research (DZL), Research in Endstage and Obstructive Lung Disease Hannover, Hannover, Germany

Diagnostic and Interventional Radiology, Member of the German Center of Lung Research, University Hospital Heidelberg, Heidelberg, Germany

H. U. Kauczor

Department of Internal Medicine V, LMU University Hospital, LMU Munich, Comprehensive Pneumology Center, Member of the German Center for Lung Research (DZL), Ludwig-Maximilians-University Munich (LMU), Munich, Germany

Institute and Outpatient Clinic for Occupational, Social and Environmental Medicine, Comprehensive Pneumology Center, Member of the German Center for Lung Research (DZL), LMU University Hospital, Ludwig-Maximilians-University Munich (LMU), Munich, Germany

R. A. Jörres

Helmholtz Centre for Infection Research (HZI), Helmholtz Institute for Pharmaceutical Research Saarland (HIPS), Saarland University Campus, Saarbrücken, Germany

You can also search for this author in PubMed   Google Scholar

• Stefan Andreas
• , Jürgen Behr
• , K. Kahnert
• , Thomas Bahmer
• , Burkhard Bewig
• , Ralf Ewert
• , Beate Stubbe
• , HJoachim Ficker
• , Christian Grohé
• , Matthias Held
• , Markus Henke
• , Felix Herth
• , Anne-Marie Kirsten
• , Rembert Koczulla
• , Juliane Kronsbein
• , Cornelia Kropf-Sanchen
• , Christian Herzmann
• , Michael Pfeifer
• , JWinfried Randerath
• , Werner Seeger
• , Michael Studnicka
• , Christian Taube
• , Hartmut Timmermann
• , Bernd Schmeck
• , C. F. Vogelmeier
•  & Hubert Wirtz

## Contributions

N. Frantzi was involved in the conceptualization of the study, interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. X. P. Nguyen was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. C. Herr was involved in the laboratory determination of the biomarkers and approved the final submitted version, and agreed to be accountable for all aspects of the work. P. Alter was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. S. Söhler was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. D. Soriano was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. H. Watz was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. B. Waschki was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. F Trinkmann was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. M. Eichenlaub was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. F.C. Trudzinski was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. J.D. Michels-Zetsche was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. A. Omlor was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. F. Seiler was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. I. Moneke was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. F. Biertz was involved in the data analysis, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. G. Rohde was involved in the data analysis, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. D. Stolz was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. T. Welte was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. H.U. Kauczor was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. K. Kahnert was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. R.A. Jörres was involved in the data analysis, interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. C. Vogelmeier was involved in the interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. R. Bals was involved in the conceptualization of the study, interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work. S. Fähndrich was involved in the conceptualization of the study, interpretation of the data, drafting and finalisation of the manuscript, approved the final submitted version, and agreed to be accountable for all aspects of the work.

## Corresponding author

Correspondence to S. Fähndrich .

## Ethics declarations

Ethics approval and consent to participate.

The study protocol was approved by the central ethical committee in Marburg (Ethikkommission FB Medizin Marburg) and the respective local ethical committees: Bad Reichenhall (Ethikkommission Bayerische Landesärztekammer); Berlin (Ethikkommission Ärztekammer Berlin); Bochum (Ethikkommission Medizinische Fakultät der RUB); Borstel (Ethikkommission Universität Lübeck); Coswig (Ethikkommission TU Dresden); Donaustauf (Ethikkommission Universitätsklinikum Regensburg); Essen (Ethikkommission Medizinische Fakultät Duisburg-Essen); Gießen (Ethikkommission Fachbereich Medizin); Greifswald (Ethikkommission Universitätsmedizin Greifswald); Großhansdorf (Ethikkommission Ärztekammer Schleswig–Holstein); Hamburg (Ethikkommission Ärztekammer Hamburg); MHH Hannover/Coppenbrügge (MHH Ethikkommission); Heidelberg Thorax/Uniklinik (Ethikkommission Universität Heidelberg); Homburg (Ethikkommission Saarbrücken); Immenhausen (Ethikkommission Landesärztekammer Hessen); Kiel (Ethikkommission Christian-Albrechts-Universität zu Kiel); Leipzig (Ethikkommission Universität Leipzig); Löwenstein (Ethikkommission Landesärztekammer Baden-Württemberg); Mainz (Ethikkommission Landesärztekammer Rheinland-Pfalz); München LMU/Gauting (Ethikkommission Klinikum Universität München); Nürnberg (Ethikkommission Friedrich-Alexander-Universität Erlangen Nürnberg); Rostock (Ethikkommission Universität Rostock); Berchtesgadener Land (Ethikkommission Land Salzburg); Schmallenberg (Ethikkommission Ärztekammer Westfalen-Lippe); Solingen (Ethikkommission Universität Witten-Herdecke); Ulm (Ethikkommission Universität Ulm); Würzburg (Ethikkommission Universität Würzburg). The study was performed in accordance with the declaration of Helsinki, and all participants gave their written informed consent.

Not applicable.

## Competing interests

Robert Bals is a member of the Editorial Board at Respiratory Research.

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Supplementary Information

12931_2024_2822_moesm1_esm.docx.

Additional file 1: Supplemental Table S1: Results of (logistic) GEE analyses on the occurrence of at least one exacerbation using data over the course of 4.5 years (visits V1-V5). Only patients who maintained in the study until visit V5 were included. As predictors each analysis comprised one of the comorbidities, the presence of statins and an interaction term between these two factors, as well as the covariates age, sex, BMI, GOLD grade and pack-years. The table shows the odds ratios (OR) and their 95% confidence intervals (95% CI) regarding comorbidities (left column), statins (middle column) and the interaction (right column) for each of the analyses. Supplemental Table S2: Results of (logistic) GEE analyses on the occurrence of at least one severe exacerbation using data over the course of 4.5 years (visits V1-V5). Only patients who maintained in the study until visit V5 were included. As predictors each analysis comprised one of the comorbidities, the presence of statins and an interaction term between these two factors, as well as the covariates age, sex, BMI, GOLD grade and pack-years. The table shows the odds ratios (OR) and their 95% confidence intervals (95% CI) regarding comorbidities (left column), statins (middle column) and the interaction (right column) for each of the analyses.

## Rights and permissions

Reprints and permissions

Frantzi, N., Nguyen, X.P., Herr, C. et al. Statins did not reduce the frequency of exacerbations in individuals with COPD and cardiovascular comorbidities in the COSYCONET cohort. Respir Res 25 , 207 (2024). https://doi.org/10.1186/s12931-024-02822-1

Accepted : 25 April 2024

Published : 15 May 2024

DOI : https://doi.org/10.1186/s12931-024-02822-1

Anyone you share the following link with will be able to read this content:

Provided by the Springer Nature SharedIt content-sharing initiative

• Exacerbations
• Cardiovascular comorbidities

## Respiratory Research

ISSN: 1465-993X

#### IMAGES

1. What Is Significance Of Study In Research Proposal

2. Statistical Significance Definition, Types, and How It's Calculated

3. 10 Easy Steps: How to Write Findings in Research

4. Key Findings of the Research

5. SOLUTION: How to write research hypothesis

6. What are research insights? What makes them different from findings?

#### VIDEO

1. Research Meaning

2. How to Find a Research Gap Quickly (Step-by-Step Tutorial in Sinhala)

3. Research variables #researchmethodology

4. What AI (Artificial Intelligence) Is All Around Us? w/Neil deGrasse Tyson #shorts #facts #trending

5. Sample Design

6. Optimism to Pessimism #podcast #motivation #buildingrelationships #positivechoices

1. An Easy Introduction to Statistical Significance (With Examples)

Practical significance shows you whether the research outcome is important enough to be meaningful in the real world. It's indicated by the effect size of the study. Practical significance To report practical significance, you calculate the effect size of your statistically significant finding of higher happiness ratings in the experimental ...

2. Statistical Significance

In research, statistical significance measures the probability of the null hypothesis being true compared to the acceptable level of uncertainty regarding the true answer. We can better understand statistical significance if we break apart a study design.[1][2][3][4][5][6][7]

3. What Does It Mean for Research to Be Statistically Significant?

Statistical significance is a measurement of how likely it is that the difference between two groups, models, or statistics occurred by chance or occurred because two variables are actually related to each other. This means that a "statistically significant" finding is one in which it is likely the finding is real, reliable, and not due to ...

4. Significance of the Study

Definition: Significance of the study in research refers to the potential importance, relevance, or impact of the research findings. It outlines how the research contributes to the existing body of knowledge, what gaps it fills, or what new understanding it brings to a particular field of study. In general, the significance of a study can be ...

5. What is the Significance of a Study? Examples and Guide

The most obvious measure of a study's long term research significance is the number of citations it receives from future publications. The thinking is that a study which receives more citations will have had more research impact, and therefore significance, than a study which received less citations.

6. Statistical Significance: Definition & Meaning

The definition of statistically significant is that the sample effect is unlikely to be caused by chance (i.e., sampling error). In other words, what we see in the sample likely reflects an effect or relationship that exists in the population. Using a more statistically correct technical definition, statistical significance relates to the ...

7. A Refresher on Statistical Significance

A Refresher on Statistical Significance. It's too often misused and misunderstood. When you run an experiment or analyze data, you want to know if your findings are "significant.". But ...

8. Statistical Significance: Definition, Examples

Statistical significance is a measure of whether your research findings are meaningful. ... The meaning of statistical significance is limited in another way as well. Many people think, at first, that statistical significance tells them that the results must have meaningful real-world implications, that the results are practically significant. ...

9. PDF Analyzing and Interpreting Findings

Taking time to reflect on your findings and what these might possibly mean requires some serious mind work—so do not try and rush this phase. Spend a few days away from your research, giving careful thought to the findings, trying to put them in perspective, and trying to gain some deeper insights. To begin facilitating the kind of thinking ...

10. How To Write a Significance Statement for Your Research

To write a compelling significance statement, identify the research problem, explain why it is significant, provide evidence of its importance, and highlight its potential impact on future research, policy, or practice. A well-crafted significance statement should effectively communicate the value of the research to readers and help them ...

11. Research Findings

Definition: Research findings refer to the results obtained from a study or investigation conducted through a systematic and scientific approach. These findings are the outcomes of the data analysis, interpretation, and evaluation carried out during the research process. ... Research studies can be time-consuming and require significant ...

12. Statistical Significance: Definition, Types, and How It's Calculated

Statistical significance means that a result from testing or experimenting is not likely to occur randomly or by chance, but is instead likely to be attributable to a specific cause. Statistical ...

13. Significance of a Study: Revisiting the "So What" Question

Significance: The importance of the problem, questions, and/or hypotheses for improving the learning opportunities for all students (you can substitute a different long-term goal if its value is widely shared). Significance can be determined before data are gathered. Significance is an attribute of the research problem, not the research findings.

14. What it means when scientists say their results are 'significant'

Strength of results. Stats are key to good research - they help researchers determine whether the results observed are strong enough to be due to an important scientific phenomenon. As a ...

15. What is the significance of a study and how is it stated in a research

Answer: In simple terms, the significance of the study is basically the importance of your research. The significance of a study must be stated in the Introduction section of your research paper. While stating the significance, you must highlight how your research will be beneficial to the development of science and the society in general.

16. PDF Results/Findings Sections for Empirical Research Papers

The Results (also sometimes called Findings) section in an empirical research paper describes what the researcher(s) found when they analyzed their data. Its primary purpose is to use the data collected to answer the research question(s) posed in the introduction, even if the findings challenge the hypothesis.

17. 6.4

Practical significance refers to the magnitude of the difference, which is known as the effect size. Results are practically significant when the difference is large enough to be meaningful in real life. What is meaningful may be subjective and may depend on the context. Note that statistical significance is directly impacted by sample size.

18. Organizing Your Social Sciences Research Paper

The discussion section is often considered the most important part of your research paper because it: Most effectively demonstrates your ability as a researcher to think critically about an issue, to develop creative solutions to problems based upon a logical synthesis of the findings, and to formulate a deeper, more profound understanding of the research problem under investigation;

19. Statistical Significance: What It Is, How It Works, With Examples

Statistically significant is the likelihood that a relationship between two or more variables is caused by something other than random chance. Statistical hypothesis testing is used to determine ...

20. Statistical significance or clinical significance? A researcher's

Keywords: Bias, biostatistics, clinical significance, research design, sample size, statistical significance Introduction Currently, in the publish or perish era where most of the researches are judged based on their statistically significant findings, it is often difficult for young researchers to interpret the correct findings of the research.

21. A Practical Guide to Writing Quantitative and Qualitative Research

INTRODUCTION. Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses.1,2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results.3,4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the ...

22. Common misconceptions about the nature of science and scientific research

Finally, scientific research involves interpreting and trying to make sense of those data — going beyond the data themselves to draw conclusions about the underlying phenomenon being studied. Driving the whole research endeavor are one or more research problems or questions that the researcher is trying to address and potentially solve.

23. Predictors for interest to participate in digital mental health therapy

Due to a multitude of factors, the onset of the COVID-19 pandemic resulted in a significant increase in mental health issues within society, including depression and anxiety. Due to the increased trend of mental health disorders in society, digital mental health therapies are more useful than ever. With the emergence of programs utilizing Internet Cognitive Behavioral Therapy (iCBT), mental ...

24. Impact of COVID-19 on women living with HIV who are survivors of

Background Women living with HIV (WLWH) experience higher rates of intimate partner violence (IPV) compared to women without HIV, but there has been minimal research to date on the impact of the COVID-19 pandemic on the lived experiences of WLWH who are IPV survivors. Methods This is a secondary analysis of COVID-19 impact using baseline data from an ongoing, prospective, micro-longitudinal ...

25. The importance of determining the clinical significance of research

Indeed, the use of null hypothesis significance testing rarely determines the practical importance or clinical relevance of findings.1, 7 In addition, statistical significance can also provide misleading results to the clinical community since a statistical difference between groups could be found if the sample size was large and/or if the ...

26. Cancers

Similar findings were reported for Canadian research that included 549 patients with histologically confirmed EC, identifying CRP, but not IL-6 or TNF-α, as a risk factor for type I EC. Additionally, that study identified a statistically significant interaction between BMI and CRP levels in influencing EC risk.

27. Statins did not reduce the frequency of exacerbations in individuals

Chronic obstructive pulmonary disease (COPD) is a heterogeneous disease with chronic airway inflammation and multi-organ involvement caused by the interplay of inhaled pollutants, genetic predisposition and socioeconomic factors [].Exacerbations are known to be associated with disease progression and mortality [], thus a major treatment goal is the reduction of the frequency of exacerbations ...