## blog @ precision

Presenting your qualitative analysis findings: tables to include in chapter 4.

The earliest stages of developing a doctoral dissertation—most specifically the topic development  and literature review  stages—require that you immerse yourself in a ton of existing research related to your potential topic. If you have begun writing your dissertation proposal, you have undoubtedly reviewed countless results and findings sections of studies in order to help gain an understanding of what is currently known about your topic.

In this process, we’re guessing that you observed a distinct pattern: Results sections are full of tables. Indeed, the results chapter for your own dissertation will need to be similarly packed with tables. So, if you’re preparing to write up the results of your statistical analysis or qualitative analysis, it will probably help to review your APA editing  manual to brush up on your table formatting skills. But, aside from formatting, how should you develop the tables in your results chapter?

In quantitative studies, tables are a handy way of presenting the variety of statistical analysis results in a form that readers can easily process. You’ve probably noticed that quantitative studies present descriptive results like mean, mode, range, standard deviation, etc., as well the inferential results that indicate whether significant relationships or differences were found through the statistical analysis . These are pretty standard tables that you probably learned about in your pre-dissertation statistics courses.

But, what if you are conducting qualitative analysis? What tables are appropriate for this type of study? This is a question we hear often from our dissertation assistance  clients, and with good reason. University guidelines for results chapters often contain vague instructions that guide you to include “appropriate tables” without specifying what exactly those are. To help clarify on this point, we asked our qualitative analysis experts to share their recommendations for tables to include in your Chapter 4.

Demographics Tables

As with studies using quantitative methods , presenting an overview of your sample demographics is useful in studies that use qualitative research methods. The standard demographics table in a quantitative study provides aggregate information for what are often large samples. In other words, such tables present totals and percentages for demographic categories within the sample that are relevant to the study (e.g., age, gender, job title).

If conducting qualitative research  for your dissertation, however, you will use a smaller sample and obtain richer data from each participant than in quantitative studies. To enhance thick description—a dimension of trustworthiness—it will help to present sample demographics in a table that includes information on each participant. Remember that ethical standards of research require that all participant information be deidentified, so use participant identification numbers or pseudonyms for each participant, and do not present any personal information that would allow others to identify the participant (Blignault & Ritchie, 2009). Table 1 provides participant demographics for a hypothetical qualitative research study exploring the perspectives of persons who were formerly homeless regarding their experiences of transitioning into stable housing and obtaining employment.

Participant Demographics

Tables to Illustrate Initial Codes

Most of our dissertation consulting clients who are conducting qualitative research choose a form of thematic analysis . Qualitative analysis to identify themes in the data typically involves a progression from (a) identifying surface-level codes to (b) developing themes by combining codes based on shared similarities. As this process is inherently subjective, it is important that readers be able to evaluate the correspondence between the data and your findings (Anfara et al., 2002). This supports confirmability, another dimension of trustworthiness .

A great way to illustrate the trustworthiness of your qualitative analysis is to create a table that displays quotes from the data that exemplify each of your initial codes. Providing a sample quote for each of your codes can help the reader to assess whether your coding was faithful to the meanings in the data, and it can also help to create clarity about each code’s meaning and bring the voices of your participants into your work (Blignault & Ritchie, 2009).

Table 2 is an example of how you might present information regarding initial codes. Depending on your preference or your dissertation committee’s preference, you might also present percentages of the sample that expressed each code. Another common piece of information to include is which actual participants expressed each code. Note that if your qualitative analysis yields a high volume of codes, it may be appropriate to present the table as an appendix.

Initial Codes

Tables to Present the Groups of Codes That Form Each Theme

As noted previously, most of our dissertation assistance clients use a thematic analysis approach, which involves multiple phases of qualitative analysis  that eventually result in themes that answer the dissertation’s research questions. After initial coding is completed, the analysis process involves (a) examining what different codes have in common and then (b) grouping similar codes together in ways that are meaningful given your research questions. In other words, the common threads that you identify across multiple codes become the theme that holds them all together—and that theme answers one of your research questions.

As with initial coding, grouping codes together into themes involves your own subjective interpretations, even when aided by qualitative analysis software such as NVivo  or MAXQDA. In fact, our dissertation assistance clients are often surprised to learn that qualitative analysis software does not complete the analysis in the same ways that statistical analysis software such as SPSS does. While statistical analysis software completes the computations for you, qualitative analysis software does not have such analysis capabilities. Software such as NVivo provides a set of organizational tools that make the qualitative analysis far more convenient, but the analysis itself is still a very human process (Burnard et al., 2008).

Because of the subjective nature of qualitative analysis, it is important to show the underlying logic behind your thematic analysis in tables—such tables help readers to assess the trustworthiness of your analysis. Table 3 provides an example of how to present the codes that were grouped together to create themes, and you can modify the specifics of the table based on your preferences or your dissertation committee’s requirements. For example, this type of table might be presented to illustrate the codes associated with themes that answer each research question.

Grouping of Initial Codes to Form Themes

Tables to Illustrate the Themes That Answer Each Research Question

Creating alignment throughout your dissertation is an important objective, and to maintain alignment in your results chapter, the themes you present must clearly answer your research questions. Conducting qualitative analysis is an in-depth process of immersion in the data, and many of our dissertation consulting  clients have shared that it’s easy to lose your direction during the process. So, it is important to stay focused on your research questions during the qualitative analysis and also to show the reader exactly which themes—and subthemes, as applicable—answered each of the research questions.

Below, Table 4 provides an example of how to display the thematic findings of your study in table form. Depending on your dissertation committee’s preference or your own, you might present all research questions and all themes and subthemes in a single table. Or, you might provide separate tables to introduce the themes for each research question as you progress through your presentation of the findings in the chapter.

Emergent Themes and Research Questions

Bonus Tip! Figures to Spice Up Your Results

Although dissertation committees most often wish to see tables such as the above in qualitative results chapters, some also like to see figures that illustrate the data. Qualitative software packages such as NVivo offer many options for visualizing your data, such as mind maps, concept maps, charts, and cluster diagrams. A common choice for this type of figure among our dissertation assistance clients is a tree diagram, which shows the connections between specified words and the words or phrases that participants shared most often in the same context. Another common choice of figure is the word cloud, as depicted in Figure 1. The word cloud simply reflects frequencies of words in the data, which may provide an indication of the importance of related concepts for the participants.

As you move forward with your qualitative analysis and development of your results chapter, we hope that this brief overview of useful tables and figures helps you to decide on an ideal presentation to showcase the trustworthiness your findings. Completing a rigorous qualitative analysis for your dissertation requires many hours of careful interpretation of your data, and your end product should be a rich and detailed results presentation that you can be proud of. Reach out if we can help  in any way, as our dissertation coaches would be thrilled to assist as you move through this exciting stage of your dissertation journey!

Anfara Jr., V. A., Brown, K. M., & Mangione, T. L. (2002). Qualitative analysis on stage: Making the research process more public.  Educational Researcher ,  31 (7), 28-38. https://doi.org/10.3102/0013189X031007028

Blignault, I., & Ritchie, J. (2009). Revealing the wood and the trees: Reporting qualitative research.  Health Promotion Journal of Australia ,  20 (2), 140-145. https://doi.org/10.1071/HE09140

Burnard, P., Gill, P., Stewart, K., Treasure, E., & Chadwick, B. (2008). Analysing and presenting qualitative data.  British Dental Journal ,  204 (8), 429-432. https://doi.org/10.1038/sj.bdj.2008.292

## How To Write The Results/Findings Chapter

For qualitative studies (dissertations & theses).

By: Jenna Crossley (PhD). Expert Reviewed By: Dr. Eunice Rautenbach | August 2021

So, you’ve collected and analysed your qualitative data, and it’s time to write up your results chapter. But where do you start? In this post, we’ll guide you through the qualitative results chapter (also called the findings chapter), step by step.

## Overview: Qualitative Results Chapter

• What (exactly) the qualitative results chapter is
• What to include in your results chapter
• How to write up your results chapter
• A few tips and tricks to help you along the way
• Free results chapter template

## What exactly is the results chapter?

The results chapter in a dissertation or thesis (or any formal academic research piece) is where you objectively and neutrally present the findings of your qualitative analysis (or analyses if you used multiple qualitative analysis methods ). This chapter can sometimes be combined with the discussion chapter (where you interpret the data and discuss its meaning), depending on your university’s preference.  We’ll treat the two chapters as separate, as that’s the most common approach.

In contrast to a quantitative results chapter that presents numbers and statistics, a qualitative results chapter presents data primarily in the form of words . But this doesn’t mean that a qualitative study can’t have quantitative elements – you could, for example, present the number of times a theme or topic pops up in your data, depending on the analysis method(s) you adopt.

Adding a quantitative element to your study can add some rigour, which strengthens your results by providing more evidence for your claims. This is particularly common when using qualitative content analysis. Keep in mind though that qualitative research aims to achieve depth, richness and identify nuances , so don’t get tunnel vision by focusing on the numbers. They’re just cream on top in a qualitative analysis.

So, to recap, the results chapter is where you objectively present the findings of your analysis, without interpreting them (you’ll save that for the discussion chapter). With that out the way, let’s take a look at what you should include in your results chapter.

## What should you include in the results chapter?

As we’ve mentioned, your qualitative results chapter should purely present and describe your results , not interpret them in relation to the existing literature or your research questions . Any speculations or discussion about the implications of your findings should be reserved for your discussion chapter.

In your results chapter, you’ll want to talk about your analysis findings and whether or not they support your hypotheses (if you have any). Naturally, the exact contents of your results chapter will depend on which qualitative analysis method (or methods) you use. For example, if you were to use thematic analysis, you’d detail the themes identified in your analysis, using extracts from the transcripts or text to support your claims.

While you do need to present your analysis findings in some detail, you should avoid dumping large amounts of raw data in this chapter. Instead, focus on presenting the key findings and using a handful of select quotes or text extracts to support each finding . The reams of data and analysis can be relegated to your appendices.

While it’s tempting to include every last detail you found in your qualitative analysis, it is important to make sure that you report only that which is relevant to your research aims, objectives and research questions .  Always keep these three components, as well as your hypotheses (if you have any) front of mind when writing the chapter and use them as a filter to decide what’s relevant and what’s not.

## How do I write the results chapter?

Now that we’ve covered the basics, it’s time to look at how to structure your chapter. Broadly speaking, the results chapter needs to contain three core components – the introduction, the body and the concluding summary. Let’s take a look at each of these.

## Section 1: Introduction

The first step is to craft a brief introduction to the chapter. This intro is vital as it provides some context for your findings. In your introduction, you should begin by reiterating your problem statement and research questions and highlight the purpose of your research . Make sure that you spell this out for the reader so that the rest of your chapter is well contextualised.

The next step is to briefly outline the structure of your results chapter. In other words, explain what’s included in the chapter and what the reader can expect. In the results chapter, you want to tell a story that is coherent, flows logically, and is easy to follow , so make sure that you plan your structure out well and convey that structure (at a high level), so that your reader is well oriented.

The introduction section shouldn’t be lengthy. Two or three short paragraphs should be more than adequate. It is merely an introduction and overview, not a summary of the chapter.

## Section 2: Body

Before we get started on what to include in the body of your chapter, it’s vital to remember that a results section should be completely objective and descriptive, not interpretive . So, be careful not to use words such as, “suggests” or “implies”, as these usually accompany some form of interpretation – that’s reserved for your discussion chapter.

The structure of your body section is very important , so make sure that you plan it out well. When planning out your qualitative results chapter, create sections and subsections so that you can maintain the flow of the story you’re trying to tell. Be sure to systematically and consistently describe each portion of results. Try to adopt a standardised structure for each portion so that you achieve a high level of consistency throughout the chapter.

For qualitative studies, results chapters tend to be structured according to themes , which makes it easier for readers to follow. However, keep in mind that not all results chapters have to be structured in this manner. For example, if you’re conducting a longitudinal study, you may want to structure your chapter chronologically. Similarly, you might structure this chapter based on your theoretical framework . The exact structure of your chapter will depend on the nature of your study , especially your research questions.

As you work through the body of your chapter, make sure that you use quotes to substantiate every one of your claims . You can present these quotes in italics to differentiate them from your own words. A general rule of thumb is to use at least two pieces of evidence per claim, and these should be linked directly to your data. Also, remember that you need to include all relevant results , not just the ones that support your assumptions or initial leanings.

In addition to including quotes, you can also link your claims to the data by using appendices , which you should reference throughout your text. When you reference, make sure that you include both the name/number of the appendix , as well as the line(s) from which you drew your data.

As referencing styles can vary greatly, be sure to look up the appendix referencing conventions of your university’s prescribed style (e.g. APA , Harvard, etc) and keep this consistent throughout your chapter.

## Section 3: Concluding summary

The concluding summary is very important because it summarises your key findings and lays the foundation for the discussion chapter . Keep in mind that some readers may skip directly to this section (from the introduction section), so make sure that it can be read and understood well in isolation.

In this section, you need to remind the reader of the key findings. That is, the results that directly relate to your research questions and that you will build upon in your discussion chapter. Remember, your reader has digested a lot of information in this chapter, so you need to use this section to remind them of the most important takeaways.

Importantly, the concluding summary should not present any new information and should only describe what you’ve already presented in your chapter. Keep it concise – you’re not summarising the whole chapter, just the essentials.

## Tips for writing an A-grade results chapter

Now that you’ve got a clear picture of what the qualitative results chapter is all about, here are some quick tips and reminders to help you craft a high-quality chapter:

• Your results chapter should be written in the past tense . You’ve done the work already, so you want to tell the reader what you found , not what you are currently finding .
• Make sure that you review your work multiple times and check that every claim is adequately backed up by evidence . Aim for at least two examples per claim, and make use of an appendix to reference these.
• When writing up your results, make sure that you stick to only what is relevant . Don’t waste time on data that are not relevant to your research objectives and research questions.
• Use headings and subheadings to create an intuitive, easy to follow piece of writing. Make use of Microsoft Word’s “heading styles” and be sure to use them consistently.
• When referring to numerical data, tables and figures can provide a useful visual aid. When using these, make sure that they can be read and understood independent of your body text (i.e. that they can stand-alone). To this end, use clear, concise labels for each of your tables or figures and make use of colours to code indicate differences or hierarchy.
• Similarly, when you’re writing up your chapter, it can be useful to highlight topics and themes in different colours . This can help you to differentiate between your data if you get a bit overwhelmed and will also help you to ensure that your results flow logically and coherently.

If you have any questions, leave a comment below and we’ll do our best to help. If you’d like 1-on-1 help with your results chapter (or any chapter of your dissertation or thesis), check out our private dissertation coaching service here or book a free initial consultation to discuss how we can help you.

## Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

## You Might Also Like:

This was extremely helpful. Thanks a lot guys

Hi, thanks for the great research support platform created by the gradcoach team!

I wanted to ask- While “suggests” or “implies” are interpretive terms, what terms could we use for the results chapter? Could you share some examples of descriptive terms?

I think that instead of saying, ‘The data suggested, or The data implied,’ you can say, ‘The Data showed or revealed, or illustrated or outlined’…If interview data, you may say Jane Doe illuminated or elaborated, or Jane Doe described… or Jane Doe expressed or stated.

I found this article very useful. Thank you very much for the outstanding work you are doing.

What if i have 3 different interviewees answering the same interview questions? Should i then present the results in form of the table with the division on the 3 perspectives or rather give a results in form of the text and highlight who said what?

I think this tabular representation of results is a great idea. I am doing it too along with the text. Thanks

That was helpful was struggling to separate the discussion from the findings

this was very useful, Thank you.

Very helpful, I am confident to write my results chapter now.

It is so helpful! It is a good job. Thank you very much!

Very useful, well explained. Many thanks.

Hello, I appreciate the way you provided a supportive comments about qualitative results presenting tips

I loved this! It explains everything needed, and it has helped me better organize my thoughts. What words should I not use while writing my results section, other than subjective ones.

Thanks a lot, it is really helpful

Thank you so much for this! I was wondering if anyone could help with how to prproperly integrate quotations (Excerpts) from interviews in the finding chapter in a qualitative research. Please GradCoach, address this issue and provide examples.

what if I’m not doing any interviews myself and all the information is coming from case studies that have already done the research.

This was very helpful as I was wondering how to structure this part of my dissertation, to include the quotes… Thanks for this explanation

This is very helpful, thanks! I am required to write up my results chapters with the discussion in each of them – any tips and tricks for this strategy?

## Submit a Comment Cancel reply

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

## Chapter 4. Finding a Research Question and Approaches to Qualitative Research

We’ve discussed the research design process in general and ways of knowing favored by qualitative researchers.  In chapter 2, I asked you to think about what interests you in terms of a focus of study, including your motivations and research purpose.  It might be helpful to start this chapter with those short paragraphs you wrote about motivations and purpose in front of you.  We are now going to try to develop those interests into actual research questions (first part of this chapter) and then choose among various “traditions of inquiry” that will be best suited to answering those questions.  You’ve already been introduced to some of this (in chapter 1), but we will go further here.

## Developing a Research Question

Research questions are different from general questions people have about the social world.  They are narrowly tailored to fit a very specific issue, complete with context and time boundaries.  Because we are engaged in empirical science and thus use “data” to answer our questions, the questions we ask must be answerable by data.  A question is not the same as stating a problem.  The point of the entire research project is to answer a particular question or set of questions.  The question(s) should be interesting, relevant, practical, and ethical.  Let’s say I am generally interested in the problem of student loan debt.  That’s a good place to start, but we can’t simply ask,

General question: Is student loan debt really a problem today?

How could we possibly answer that question? What data could we use? Isn’t this really an axiological (values-based) question? There are no clues in the question as to what data would be appropriate here to help us get started. Students often begin with these large unanswerable questions. They are not research questions. Instead, we could ask,

Poor research question: How many people have debt?

This is still not a very good research question. Why not? It is answerable, although we would probably want to clarify the context. We could add some context to improve it so that the question now reads,

Mediocre research question: How many people in the US have debt today? And does this amount vary by age and location?

Fair research question: What impact does student debt have on the lives of debtors?

Good start, but we still need some context to help guide the project. It is not nearly specific enough.

Better research question: What impact does student debt have on young adults (ages twenty-five to thirty-five) living in the US today?

Now we’ve added context, but we can still do a little bit better in narrowing our research question so that it is both clear and doable; in other words, we want to frame it in a way that provides a very clear research program:

Optimal research question: How do young adults (ages twenty-five to thirty-five) living in the US today who have taken on \$30,000 or more in student debt describe the impact of their debt on their lives in terms of finding/choosing a job, buying a house, getting married, and other major life events?

Now you have a research question that can be answered and a clear plan of how to answer it. You will talk to young adults living in the US today who have high debt loads and ask them to describe the impacts of debt on their lives. That is all now in the research question. Note how different this very specific question is from where we started with the “problem” of student debt.

Take some time practicing turning the following general questions into research questions:

• What can be done about the excessive use of force by police officers?
• Why haven’t societies taken firmer steps to address climate change?
• How do communities react to / deal with the opioid epidemic?
• Who has been the most adversely affected by COVID?
• When did political polarization get so bad?

Hint: Step back from each of the questions and try to articulate a possible underlying motivation, then formulate a research question that is specific and answerable.

It is important to take the time to come up with a research question, even if this research question changes a bit as you conduct your research (yes, research questions can change!). If you don’t have a clear question to start your research, you are likely to get very confused when designing your study because you will not be able to make coherent decisions about things like samples, sites, methods of data collection, and so on. Your research question is your anchor: “If we don’t have a question, we risk the possibility of going out into the field thinking we know what we’ll find and looking only for proof of what we expect to be there. That’s not empirical research (it’s not systematic)” ( Rubin 2021:37 ).

Researcher Note

How do you come up with ideas for what to study?

I study what surprises me. Usually, I come across a statistic that suggests something is common that I thought was rare. I tend to think it’s rare because the theories I read suggest it should be, and there’s not a lot of work in that area that helps me understand how the statistic came to be. So, for example, I learned that it’s common for Americans to marry partners who grew up in a different class than them and that about half of White kids born into the upper-middle class are downwardly mobile. I was so shocked by these facts that they naturally led to research questions. How do people come to marry someone who grew up in a different class? How do White kids born near the top of the class structure fall?

—Jessi Streib, author of The Power of the Past and Privilege Lost

What if you have literally no idea what the research question should be? How do you find a research question? Even if you have an interest in a topic before you get started, you see the problem now: topics and issues are not research questions! A research question doesn’t easily emerge; it takes a lot of time to hone one, as the practice above should demonstrate. In some research designs, the research question doesn’t even get clearly articulated until the end of data collection . More on that later. But you must start somewhere, of course. Start with your chosen discipline. This might seem obvious, but it is often overlooked. There is a reason it is called a discipline. We tend to think of “sociology,” “public health,” and “physics” as so many clusters of courses that are linked together by subject matter, but they are also disciplines in the sense that the study of each focuses the mind in a particular way and for particular ends. For example, in my own field, sociology, there is a loosely shared commitment to social justice and a general “sociological imagination” that enables its practitioners to connect personal experiences to society at large and to historical forces. It is helpful to think of issues and questions that are germane to your discipline. Within that overall field, there may be a particular course or unit of study you found most interesting. Within that course or unit of study, there may be an issue that intrigued you. And finally, within that issue, there may be an aspect or topic that you want to know more about.

When I was pursuing my dissertation research, I was asked often, “Why did you choose to study intimate partner violence among Native American women?” This question is necessary, and each time I answered, it helped shape me into a better researcher. I was interested in intimate partner violence because I am a survivor. I didn’t have intentions to work with a particular population or demographic—that came from my own deep introspection on my role as a researcher. I always questioned my positionality: What privileges do I hold as an academic? How has public health extracted information from institutionally marginalized populations? How can I build bridges between communities using my position, knowledge, and power? Public health as a field would not exist without the contributions of Indigenous people. So I started hanging out with them at community events, making friends, and engaging in self-education. Through these organic relationships built with Native women in the community, I saw that intimate partner violence was a huge issue. This led me to partner with Indigenous organizations to pursue a better understanding of how Native survivors of intimate partner violence seek support.

—Susanna Y. Park, PhD, mixed-methods researcher in public health and author of “How Native Women Seek Support as Survivors of Intimate Partner Violence: A Mixed-Methods Study”

Once you think you have an issue or topic that is worth exploring, you will need to (eventually) turn that into a good research question. A good research question is specific, clear, and feasible .

Specific . How specific a research question needs to be is somewhat related to the disciplinary conventions and whether the study is conceived inductively or deductively. In deductive research, one begins with a specific research question developed from the literature. You then collect data to test the theory or hypotheses accompanying your research question. In inductive research, however, one begins with data collection and analysis and builds theory from there. So naturally, the research question is a bit vaguer. In general, the more closely aligned to the natural sciences (and thus the deductive approach), the more a very tight and specific research question (along with specific, focused hypotheses) is required. This includes disciplines like psychology, geography, public health, environmental science, and marine resources management. The more one moves toward the humanities pole (and the inductive approach), the more looseness is permitted, as there is a general belief that we go into the field to find what is there, not necessarily what we imagine we are looking for (see figure 4.2). Disciplines such as sociology, anthropology, and gender and sexuality studies and some subdisciplines of public policy/public administration are closer to the humanities pole in this sense.

Regardless of discipline and approach, however, it is a good idea for beginning researchers to create a research question as specific as possible, as this will serve as your guide throughout the process. You can tweak it later if needed, but start with something specific enough that you know what it is you are doing and why. It is more difficult to deal with ambiguity when you are starting out than later in your career, when you have a better handle on what you are doing. Being under a time constraint means the more specific the question, the better. Questions should always specify contexts, geographical locations, and time frames. Go back to your practice research questions and make sure that these are included.

Clear . A clear research question doesn’t only need to be intelligible to any reader (which, of course, it should); it needs to clarify any meanings of particular words or concepts (e.g., What is excessive force?). Check all your concepts to see if there are ways you can clarify them further—for example, note that we shifted from impact of debt to impact of high debt load and specified this as beginning at \$30,000. Ideally, we would use the literature to help us clarify what a high debt load is or how to define “excessive” force.

Feasible . In order to know if your question is feasible, you are going to have to think a little bit about your entire research design. For example, a question that asks about the real-time impact of COVID restrictions on learning outcomes would require a time machine. You could tweak the question to ask instead about the long-term impacts of COVID restrictions, as measured two years after their end. Or let’s say you are interested in assessing the damage of opioid abuse on small-town communities across the United States. Is it feasible to cover the entire US? You might need a team of researchers to do this if you are planning on on-the-ground observations. Perhaps a case study of one particular community might be best. Then your research question needs to be changed accordingly.

Here are some things to consider in terms of feasibility:

• Is the question too general for what you actually intend to do or examine? (Are you specifying the world when you only have time to explore a sliver of that world?)
• Is the question suitable for the time you have available? (You will need different research questions for a study that can be completed in a term than one where you have one to two years, as in a master’s program, or even three to eight years, as in a doctoral program.)
• Is the focus specific enough that you know where and how to begin?
• What are the costs involved in doing this study, including time? Will you need to travel somewhere, and if so, how will you pay for it?
• Will there be problems with “access”? (More on this in later chapters, but for now, consider how you might actually find people to interview or places to observe and whether gatekeepers exist who might keep you out.)
• Will you need to submit an application proposal for your university’s IRB (institutional review board)? If you are doing any research with live human subjects, you probably need to factor in the time and potential hassle of an IRB review (see chapter 8). If you are under severe time constraints, you might need to consider developing a research question that can be addressed with secondary sources, online content, or historical archives (see chapters 16 and 17).

In addition to these practicalities, you will also want to consider the research question in terms of what is best for you now. Are you engaged in research because you are required to be—jumping a hurdle for a course or for your degree? If so, you really do want to think about your project as training and develop a question that will allow you to practice whatever data collection and analysis techniques you want to develop. For example, if you are a grad student in a public health program who is interested in eventually doing work that requires conducting interviews with patients, develop a research question and research design that is interview based. Focus on the practicality (and practice) of the study more than the theoretical impact or academic contribution, in other words. On the other hand, if you are a PhD candidate who is seeking an academic position in the future, your research question should be pitched in a way to build theoretical knowledge as well (the phrasing is typically “original contribution to scholarship”).

On the other hand, if you are an undergraduate and you really have very little time, some of the best advice I have heard is to find a study you really like and adapt it to a new context. Perhaps you read a study about how students select majors and how this differs by class ( Hurst 2019 ). You can try to replicate the study on a small scale among your classmates. Use the same research question, but revise for your context. You can probably even find the exact questions I  used and ask them in the new sample. Then when you get to the analysis and write-up, you have a comparison study to guide you, and you can say interesting things about the new context and whether the original findings were confirmed (similar) or not. You can even propose reasons why you might have found differences between one and the other.

Another way of thinking about research questions is to explicitly tie them to the type of purpose of your study. Of course, this means being very clear about what your ultimate purpose is! Marshall and Rossman ( 2016 ) break down the purpose of a study into four categories: exploratory, explanatory, descriptive, and emancipatory ( 78 ). Exploratory purpose types include wanting to investigate little-understood phenomena, or identifying or discovering important new categories of meaning, or generating hypotheses for further research. For these, research questions might be fairly loose: What is going on here? How are people interacting on this site? What do people talk about when you ask them about the state of the world? You are almost (but never entirely) starting from scratch. Be careful though—just because a topic is new to you does not mean it is really new. Someone else (or many other someones) may already have done this exploratory research. Part of your job is to find this out (more on this in “What Is a ‘Literature Review’?” in chapter 9). Descriptive purposes (documenting and describing a phenomenon) are similar to exploratory purposes but with a much clearer goal (description). A good research question for a descriptive study would specify the actions, events, beliefs, attitudes, structures, and/or processes that will be described.

Most researchers find that their topic has already been explored and described, so they move to trying to explain a relationship or phenomenon. For these, you will want research questions that capture the relationships of interest. For example, how does gender influence one’s understanding of police brutality (because we already know from the literature that it does, so now we are interested in understanding how and why)? Or what is the relationship between education and climate change denialism? If you find that prior research has already provided a lot of evidence about those relationships as well as explanations for how they work, and you want to move the needle past explanation into action, you might find yourself trying to conduct an emancipatory study. You want to be even more clear in acknowledging past research if you find yourself here. Then create a research question that will allow you to “create opportunities and the will to engage in social action” ( Marshall and Rossman 2016:78 ). Research questions might ask, “How do participants problematize their circumstances and take positive social action?” If we know that some students have come together to fight against student debt, how are they doing this, and with what success? Your purpose would be to help evaluate possibilities for social change and to use your research to make recommendations for more successful emancipatory actions.

Recap: Be specific. Be clear. Be practical. And do what you love.

## Choosing an Approach or Tradition

Qualitative researchers may be defined as those who are working with data that is not in numerical form, but there are actually multiple traditions or approaches that fall under this broad category. I find it useful to know a little bit about the history and development of qualitative research to better understand the differences in these approaches. The following chart provides an overview of the six phases of development identified by Denzin and Lincoln ( 2005 ):

Table 4.1. Six Phases of Development

There are other ways one could present the history as well. Feminist theory and methodologies came to the fore in the 1970s and 1980s and had a lot to do with the internal critique of more positivist approaches. Feminists were quite aware that standpoint matters—that the identity of the researcher plays a role in the research, and they were ardent supporters of dismantling unjust power systems and using qualitative methods to help advance this mission. You might note, too, that many of the internal disputes were basically epistemological disputes about how we know what we know and whether one’s social location/position delimits that knowledge. Today, we are in a bountiful world of qualitative research, one that embraces multiple forms of knowing and knowledge. This is good, but it means that you, the student, have more choice when it comes to situating your study and framing your research question, and some will expect you to signal the choices you have made in any research protocols you write or publications and presentations.

Creswell’s ( 1998 ) definition of qualitative research includes the notion of distinct traditions of inquiry: “Qualitative research is an inquiry process of understanding based on distinct methodological traditions of inquiry that explore a social or human problem. The research builds complex,   holistic pictures, analyzes words, reports detailed views of informants , and conducted the study in a natural setting” (15; emphases added). I usually caution my students against taking shelter under one of these approaches, as, practically speaking, there is a lot of mixing of traditions among researchers. And yet it is useful to know something about the various histories and approaches, particularly as you are first starting out. Each tradition tends to favor a particular epistemological perspective (see chapter 3), a way of reasoning (see “ Advanced: Inductive versus Deductive Reasoning ”), and a data-collection technique.

There are anywhere from ten to twenty “traditions of inquiry,” depending on how one draws the boundaries. In my accounting, there are twelve, but three approaches tend to dominate the field.

## Ethnography

Ethnography was developed from the discipline of anthropology, as the study of (other) culture(s). From a relatively positivist/objective approach to writing down the “truth” of what is observed during the colonial era (where this “truth” was then often used to help colonial administrators maintain order and exploit people and extract resources more effectively), ethnography was adopted by all kinds of social science researchers to get a better understanding of how groups of people (various subcultures and cultures) live their lives. Today, ethnographers are more likely to be seeking to dismantle power relations than to support them. They often study groups of people that are overlooked and marginalized, and sometimes they do the obverse by demonstrating how truly strange the familiar practices of the dominant group are. Ethnography is also central to organizational studies (e.g., How does this institution actually work?) and studies of education (e.g., What is it like to be a student during the COVID era?).

Ethnographers use methods of participant observation and intensive fieldwork in their studies, often living or working among the group under study for months at a time (and, in some cases, years). I’ve called this “deep ethnography,” and it is the subject of chapter 14. The data ethnographers analyze are copious “field notes” written while in the field, often supplemented by in-depth interviews and many more casual conversations. The final product of ethnographers is a “thick” description of the culture. This makes reading ethnographies enjoyable, as the goal is to write in such a way that the reader feels immersed in the culture.

There are variations on the ethnography, such as the autoethnography , where the researcher uses a systematic and rigorous study of themselves to better understand the culture in which they find themselves. Autoethnography is a relatively new approach, even though it is derived from one of the oldest approaches. One can say that it takes to heart the feminist directive to “make the personal political,” to underscore the connections between personal experiences and larger social and political structures. Introspection becomes the primary data source.

## Grounded Theory

Grounded Theory holds a special place in qualitative research for a few reasons, not least of which is that nonqualitative researchers often mistakenly believe that Grounded Theory is the only qualitative research methodology . Sometimes, it is easier for students to explain what they are doing as “Grounded Theory” because it sounds “more scientific” than the alternative descriptions of qualitative research. This is definitely part of its appeal. Grounded Theory is the name given to the systematic inductive approach first developed by Glaser and Strauss in 1967, The Discovery of Grounded Theory: Strategies for Qualitative Research . Too few people actually read Glaser and Strauss’s book. It is both groundbreaking and fairly unremarkable at the same time. As a historical intervention into research methods generally, it is both a sharp critique of positivist methods in the social sciences (theory testing) and a rejection of purely descriptive accounts-building qualitative research. Glaser and Strauss argued for an approach whose goal was to construct (middle-level) theories from recursive data analysis of nonnumerical data (interviews and observations). They advocated a “constant comparative method” in which coding and analysis take place simultaneously and recursively. The demands are fairly strenuous. If done correctly, the result is the development of a new theory about the social world.

So why do I call this “fairly unremarkable”? To some extent, all qualitative research already does what Glaser and Strauss ( 1967 ) recommend, albeit without denoting the processes quite so specifically. As will be seen throughout the rest of this textbook, all qualitative research employs some “constant comparisons” through recursive data analyses. Where Grounded Theory sets itself apart from a significant number of qualitative research projects, however, is in its dedication to inductively building theory. Personally, I think it is important to understand that Glaser and Strauss were rejecting deductive theory testing in sociology when they first wrote their book. They were part of a rising cohort who rejected the positivist mathematical approaches that were taking over sociology journals in the 1950s and 1960s. Here are some of the comments and points they make against this kind of work:

Accurate description and verification are not so crucial when one’s purpose is to generate theory. ( 28 ; further arguing that sampling strategies are different when one is not trying to test a theory or generalize results)

Illuminating perspectives are too often suppressed when the main emphasis is verifying theory. ( 40 )

Testing for statistical significance can obscure from theoretical relevance. ( 201 )

Instead, they argued, sociologists should be building theories about the social world. They are not physicists who spend time testing and refining theories. And they are not journalists who report descriptions. What makes sociologists better than journalists and other professionals is that they develop theory from their work “In their driving efforts to get the facts [research sociologists] tend to forget that the distinctive offering of sociology to our society is sociological theory, not research description” ( 30–31 ).

Grounded Theory’s inductive approach can be off-putting to students who have a general research question in mind and a working hypothesis. The true Grounded Theory approach is often used in exploratory studies where there are no extant theories. After all, the promise of this approach is theory generation, not theory testing. Flying totally free at the start can be terrifying. It can also be a little disingenuous, as there are very few things under the sun that have not been considered before. Barbour ( 2008:197 ) laments that this approach is sometimes used because the researcher is too lazy to read the relevant literature.

To summarize, Glaser and Strauss justified the qualitative research project in a way that gave it standing among the social sciences, especially vis-à-vis quantitative researchers. By distinguishing the constant comparative method from journalism, Glaser and Strauss enabled qualitative research to gain legitimacy.

So what is it exactly, and how does one do it? The following stages provide a succinct and basic overview, differentiating the portions that are similar to/in accordance with qualitative research methods generally and those that are distinct from the Grounded Theory approach:

Step 1. Select a case, sample, and setting (similar—unless you begin with a theory to test!).

Step 2. Begin data collection (similar).

Step 3. Engage data analysis (similar in general but specificity of details somewhat unique to Grounded Theory): (1) emergent coding (initial followed by focused), (2) axial (a priori) coding , (3) theoretical coding , (4) creation of theoretical categories; analysis ends when “theoretical saturation ” has been achieved.

Grounded Theory’s prescriptive (i.e., it has a set of rules) framework can appeal to beginning students, but it is unnecessary to adopt the entire approach in order to make use of some of its suggestions. And if one does not exactly follow the Grounded Theory rulebook, it can mislead others if you tend to call what you are doing Grounded Theory when you are not:

Grounded theory continues to be a misunderstood method, although many researchers purport to use it. Qualitative researchers often claim to conduct grounded theory studies without fully understanding or adopting its distinctive guidelines. They may employ one or two of the strategies or mistake qualitative analysis for grounded theory. Conversely, other researchers employ grounded theory methods in reductionist, mechanistic ways. Neither approach embodies the flexible yet systematic mode of inquiry, directed but open-ended analysis, and imaginative theorizing from empirical data that grounded theory methods can foster. Subsequently, the potential of grounded theory methods for generating middle-range theory has not been fully realized ( Charmaz 2014 ).

## Phenomenology

Where Grounded Theory sets itself apart for its inductive systematic approach to data analysis, phenomenologies are distinct for their focus on what is studied—in this case, the meanings of “lived experiences” of a group of persons sharing a particular event or circumstance. There are phenomenologies of being working class ( Charlesworth 2000 ), of the tourist experience ( Cohen 1979 ), of Whiteness ( Ahmed 2007 ). The phenomenon of interest may also be an emotion or circumstance. One can study the phenomenon of “White rage,” for example, or the phenomenon of arranged marriage.

The roots of phenomenology lie in philosophy (Husserl, Heidegger, Merleau-Ponty, Sartre) but have been adapted by sociologists in particular. Phenomenologists explore “how human beings make sense of experience and transform experience into consciousness, both individually and as shared meaning” ( Patton 2002:104 ).

One of the most important aspects of conducting a good phenomenological study is getting the sample exactly right so that each person can speak to the phenomenon in question. Because the researcher is interested in the meanings of an experience, in-depth interviews are the preferred method of data collection. Observations are not nearly as helpful here because people may do a great number of things without meaning to or without being conscious of their implications. This is important to note because phenomenologists are studying not “the reality” of what happens at all but an articulated understanding of a lived experience. When reading a phenomenological study, it is important to keep this straight—too often I have heard students critique a study because the interviewer didn’t actually see how people’s behavior might conflict with what they say (which is, at heart, an epistemological issue!).

In addition to the “big three,” there are many other approaches; some are variations, and some are distinct approaches in their own right. Case studies focus explicitly on context and dynamic interactions over time and can be accomplished with quantitative or qualitative methods or a mixture of both (for this reason, I am not considering it as one of the big three qualitative methods, even though it is a very common approach). Whatever methods are used, a contextualized deep understanding of the case (or cases) is central.

Critical inquiry is a loose collection of techniques held together by a core argument that understanding issues of power should be the focus of much social science research or, to put this another way, that it is impossible to understand society (its people and institutions) without paying attention to the ways that power relations and power dynamics inform and deform those people and institutions. This attention to power dynamics includes how research is conducted too. All research fundamentally involves issues of power. For this reason, many critical inquiry traditions include a place for collaboration between researcher and researched. Examples include (1) critical narrative analysis, which seeks to describe the meaning of experience for marginalized or oppressed persons or groups through storytelling; (2) participatory action research, which requires collaboration between the researcher and the research subjects or community of interest; and (3) critical race analysis, a methodological application of Critical Race Theory (CRT), which posits that racial oppression is endemic (if not always throughout time and place, at least now and here).

Shawn Wilson’s book, Research Is Ceremony: Indigenous Research Methods , is my holy grail. It really flipped my understanding of research and relationships. Rather than thinking linearly and approaching research in a more canonical sense, Wilson shook my world view by drawing me into a pattern of inquiry that emphasized transparency and relational accountability. The Indigenous research paradigm is applicable in all research settings, and I follow it because it pushes me to constantly evaluate my position as a knowledge seeker and knowledge sharer.

Autoethnography takes the researcher as the subject. This is one approach that is difficult to explain to more quantitatively minded researchers, as it seems to violate many of the norms of “scientific research” as understood by them. First, the sample size is quite small—the n is 1, the researcher. Two, the researcher is not a neutral observer—indeed, the subjectivity of the researcher is the main strength of this approach. Autoethnographies can be extremely powerful for their depth of understanding and reflexivity, but they need to be conducted in their own version of rigor to stand up to scrutiny by skeptics. If you are skeptical, read one of the excellent published examples out there—I bet you will be impressed with what you take away. As they say, the proof is in the pudding on this approach.

## Advanced: Inductive versus Deductive Reasoning

There has been a great deal of ink shed in the discussion of inductive versus deductive approaches, not all of it very instructive. Although there is a huge conceptual difference between them, in practical terms, most researchers cycle between the two, even within the same research project. The simplest way to explain the difference between the two is that we are using deductive reasoning when we test an existing theory (move from general to particular), and we are using inductive reasoning when we are generating theory (move from particular to general). Figure 4.2 provides a schematic of the deductive approach. From the literature, we select a theory about the impact of student loan debt: student loan debt will delay homeownership among young adults. We then formulate a hypothesis based on this theory: adults in their thirties with high debt loads will be less likely to own homes than their peers who do not have high debt loads. We then collect data to test the hypothesis and analyze the results. We find that homeownership is substantially lower among persons of color and those who were the first in their families to graduate from college. Notably, high debt loads did not affect homeownership among White adults whose parents held college degrees. We thus refine the theory to match the new findings: student debt loads delay homeownership among some young adults, thereby increasing inequalities in this generation. We have now contributed new knowledge to our collective corpus.

The inductive approach is contrasted in figure 4.3. Here, we did not begin with a preexisting theory or previous literature but instead began with an observation. Perhaps we were conducting interviews with young adults who held high amounts of debt and stumbled across this observation, struck by how many were renting apartments or small houses. We then noted a pattern—not all the young adults we were talking to were renting; race and class seemed to play a role here. We would then probably expand our study in a way to be able to further test this developing theory, ensuring that we were not seeing anomalous patterns. Once we were confident about our observations and analyses, we would then develop a theory, coming to the same place as our deductive approach, but in reverse.

A third form of reasoning, abductive (sometimes referred to as probabilistic reasoning) was developed in the late nineteenth century by American philosopher Charles Sanders Peirce. I have included some articles for further reading for those interested.

Among social scientists, the deductive approach is often relaxed so that a research question is set based on the existing literature rather than creating a hypothesis or set of hypotheses to test. Some journals still require researchers to articulate hypotheses, however. If you have in mind a publication, it is probably a good idea to take a look at how most articles are organized and whether specific hypotheses statements are included.

Table 4.2. Twelve Approaches. Adapted from Patton 2002:132-133.

The following readings have been examples of various approaches or traditions of inquiry:

Ahmed, Sara. 2007. “A Phenomenology of Whiteness.” Feminist Theory 8(2):149–168.

Charlesworth, Simon. 2000. A Phenomenology of Working-Class Experience . Cambridge: Cambridge University Press.*

Clandinin, D. Jean, and F. Michael Connelly. 2000. Narrative Inquiry: Experience and Story in Qualitative Research . San Francisco: Jossey-Bass.

Cohen, E. 1979. “A Phenomenology of Tourist Experiences.” Sociology 13(2):179–201.

Cooke, Bill, and Uma Kothari, eds. 2001. Participation: The New Tyranny? London: Zed Books. A critique of participatory action.

Corbin, Juliet, and Anselm Strauss. 2008. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory . 3rd ed. Thousand Oaks, CA: SAGE.

Crabtree, B. F., and W. L. Miller, eds. 1999. Doing Qualitative Research: Multiple Strategies . Thousand Oaks, CA: SAGE.

Creswell, John W. 1997. Qualitative Inquiry and Research Design: Choosing among Five Approaches. Thousand Oaks, CA: SAGE.

Glaser, Barney G., and Anselm Strauss. 1967. The Discovery of Grounded Theory: Strategies for Qualitative Research . New York: Aldine.

Gobo, Giampetro, and Andrea Molle. 2008. Doing Ethnography . Thousand Oaks, CA: SAGE.

Hancock, Dawson B., and Bob Algozzine. 2016. Doing Case Study Research: A Practical Guide for Beginning Research . 3rd ed. New York: Teachers College Press.

Harding, Sandra. 1987. Feminism and Methodology . Bloomington: Indiana University Press.

Husserl, Edmund. (1913) 2017. Ideas: Introduction to Pure Phenomenology . Eastford, CT: Martino Fine Books.

Rose, Gillian. 2012. Visual Methodologies . 3rd ed. London: SAGE.

Van der Riet, M. 2009. “Participatory Research and the Philosophy of Social Science: Beyond the Moral Imperative.” Qualitative Inquiry 14(4):546–565.

Van Manen, Max. 1990. Researching Lived Experience: Human Science for an Action Sensitive Pedagogy . Albany: State University of New York.

Wortham, Stanton. 2001. Narratives in Action: A Strategy for Research and Analysis . New York: Teachers College Press.

## Inductive, Deductive, and Abductive Reasoning and Nomothetic Science in General

Aliseda, Atocha. 2003. “Mathematical Reasoning vs. Abductive Reasoning: A Structural Approach.” Synthese 134(1/2):25–44.

Bonk, Thomas. 1997. “Newtonian Gravity, Quantum Discontinuity and the Determination of Theory by Evidence.” Synthese 112(1):53–73. A (natural) scientific discussion of inductive reasoning.

Bonnell, Victoria E. 1980. “The Uses of Theory, Concepts and Comparison in Historical Sociology.” C omparative Studies in Society and History 22(2):156–173.

Crane, Mark, and Michael C. Newman. 1996. “Scientific Method in Environmental Toxicology.” Environmental Reviews 4(2):112–122.

Huang, Philip C. C., and Yuan Gao. 2015. “Should Social Science and Jurisprudence Imitate Natural Science?” Modern China 41(2):131–167.

Mingers, J. 2012. “Abduction: The Missing Link between Deduction and Induction. A Comment on Ormerod’s ‘Rational Inference: Deductive, Inductive and Probabilistic Thinking.’” Journal of the Operational Research Society 63(6):860–861.

Ormerod, Richard J. 2010. “Rational Inference: Deductive, Inductive and Probabilistic Thinking.” Journal of the Operational Research Society 61(8):1207–1223.

Perry, Charner P. 1927. “Inductive vs. Deductive Method in Social Science Research.” Southwestern Political and Social Science Quarterly 8(1):66–74.

Plutynski, Anya. 2011. “Four Problems of Abduction: A Brief History.” HOPOS: The Journal of the International Society for the History of Philosophy of Science 1(2):227–248.

Thompson, Bruce, and Gloria M. Borrello. 1992. “Different Views of Love: Deductive and Inductive Lines of Inquiry.” Current Directions in Psychological Science 1(5):154–156.

Tracy, Sarah J. 2012. “The Toxic and Mythical Combination of a Deductive Writing Logic for Inductive Qualitative Research.” Qualitative Communication Research 1(1):109–141.

A place or collection containing records, documents, or other materials of historical interest; most universities have an archive of material related to the university’s history, as well as other “special collections” that may be of interest to members of the community.

A person who introduces the researcher to a field site’s culture and population.  Also referred to as guides.  Used in ethnography .

A form of research and a methodological tradition of inquiry in which the researcher uses self-reflection and writing to explore personal experiences and connect this autobiographical story to wider cultural, political, and social meanings and understandings.  “Autoethnography is a research method that uses a researcher's personal experience to describe and critique cultural beliefs, practices, and experiences” ( Adams, Jones, and Ellis 2015 ).

The philosophical framework in which research is conducted; the approach to “research” (what practices this entails, etc.).  Inevitably, one’s epistemological perspective will also guide one’s methodological choices, as in the case of a constructivist who employs a Grounded Theory approach to observations and interviews, or an objectivist who surveys key figures in an organization to find out how that organization is run.  One of the key methodological distinctions in social science research is that between quantitative and qualitative research.

The process of labeling and organizing qualitative data to identify different themes and the relationships between them; a way of simplifying data to allow better management and retrieval of key themes and illustrative passages.  See coding frame and  codebook.

A later stage coding process used in Grounded Theory in which data is reassembled around a category, or axis.

A later stage-coding process used in Grounded Theory in which key words or key phrases capture the emergent theory.

The point at which you can conclude data collection because every person you are interviewing, the interaction you are observing, or content you are analyzing merely confirms what you have already noted.  Achieving saturation is often used as the justification for the final sample size.

A methodological tradition of inquiry that focuses on the meanings held by individuals and/or groups about a particular phenomenon (e.g., a “phenomenology of whiteness” or a “phenomenology of first-generation college students”).  Sometimes this is referred to as understanding “the lived experience” of a particular group or culture.  Interviews form the primary tool of data collection for phenomenological studies.  Derived from the German philosophy of phenomenology (Husserl 1913; 2017).

The number of individuals (or units) included in your sample

A form of reasoning which employs a “top-down” approach to drawing conclusions: it begins with a premise or hypothesis and seeks to verify it (or disconfirm it) with newly collected data.  Inferences are made based on widely accepted facts or premises.  Deduction is idea-first, followed by observations and a conclusion.  This form of reasoning is often used in quantitative research and less often in qualitative research.  Compare to inductive reasoning .  See also abductive reasoning .

A form of reasoning that employs a “bottom-up” approach to drawing conclusions: it begins with the collection of data relevant to a particular question and then seeks to build an argument or theory based on an analysis of that data.  Induction is observation first, followed by an idea that could explain what has been observed.  This form of reasoning is often used in qualitative research and seldom used in qualitative research.  Compare to deductive reasoning .  See also abductive reasoning .

An “interpretivist” form of reasoning in which “most likely” conclusions are drawn, based on inference.  This approach is often used by qualitative researchers who stress the recursive nature of qualitative data analysis.  Compare with deductive reasoning and inductive reasoning .

A form of social science research that generally follows the scientific method as established in the natural sciences.  In contrast to idiographic research , the nomothetic researcher looks for general patterns and “laws” of human behavior and social relationships.  Once discovered, these patterns and laws will be expected to be widely applicable.  Quantitative social science research is nomothetic because it seeks to generalize findings from samples to larger populations.  Most qualitative social science research is also nomothetic, although generalizability is here understood to be theoretical in nature rather than statistical .  Some qualitative researchers, however, espouse the idiographic research paradigm instead.

Academia.edu no longer supports Internet Explorer.

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

## Related Papers

Physiotherapy

Corinne Dalelio , Christina Selby

This chapter makes an argument for using qualitative methods for assessing students’ knowledge at the program level in order to observe their abilities and application of knowledge as demonstrated in an actual learning environment. A model for using focus groups to evaluate achievement of higher level learning outcomes is presented, drawing on a case study based on assessment of students in the communication major at Coastal Carolina University. Specifically, six focus groups with 8-12 students in each were conducted. Guided discussion, following presentations of an electronic and a print-based message led to a high level of student engagement. Case study findings showed that although students demonstrated proficiency in understanding basic communication principles and immediate implications of each message, students were less able to demonstrate higher-level critical evaluation by identifying the messages’ longer-term societal implications. This model provides an opportunity for assessors in different academic contexts to identify specific weaknesses in their students’ learning processes and to adjust curricula accordingly.

Sıdıka Gizir

English for Specific Purposes World Journal, Issue 2(18) Vol. 7, pp. 1-14, 2008.

Ourania Katsara

This article reports on the findings of a survey on students’ views of the ESP course in University of Ioannina. Four focus group interviews were conducted, each consisting of 5 students. These interviews were carried out in order to further explore in depth data gathered in the two previous stages of the research. The finding indicated that there is a consistency over time on certain issues identified in previous interpretation of in-depth interviews and content analysis of essays. The evidence shows that there are problems regarding policies of foreign language teaching in the state sector and culture specific attitudes concerning English language learning. It is apparent that more adequate and descriptive information on the ESP course is required both to students and to academic staff of university departments. Co-operation among all appears to facilitate an effort to make the ESP course useful and worth attending in University. Keywords Focus group Interviews, Qualitative analysis, English for Specific Purposes, Greek students, needs analysis, motivation

Peter Gibbings

Benefits of collecting student feedback on courses, programs, learning experiences, and their perceptions of teaching quality, are well documented in literature. In the higher education system, this feedback is generally collected via student evaluation surveys. During the past few years, the Faculty of Engineering and Surveying (FoES), one of five faculties at USQ, has identified problems of small response rates, possible bias, general questions over data validity, relevance and wording of survey questions, and systems problems such as timeliness of official reports. Due to these problems, the data may not be reliable and the views of those who respond may not be representative of the overall student cohort. It is therefore risky to base management decisions on student satisfaction, teaching quality, and course quality on these data. Clearly, a more robust and reliable method of gathering feedback from students was needed that would provide more confidence in the data. For various ...

Monica Stitt-Bergh

Journal of Southeast Asian American Education and Advancement

Hanum Tyagita

Wayne Wright , Sovicheth Boun

Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author (s) and the Journal of Southeast Asian American Education & Advancement, it is distributed for noncommercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons. org/licenses/by-nc-nd/3.0/. All other uses must be approved by the author (s) or JSAAEA.

## RELATED PAPERS

Ecancermedicalscience

Gustavo Sarria

William Fenstemacher

Evert van Leeuwen

Javier Ignacio Montoya Maya

Michael Mares

Metal Indonesia

Agus Pramono

Macromolecular Symposia

Renata Drozdzak

antje wemhoener

BioMed Research International

Abinaya Manivannan

hassan Khan

Zeitschrift für Württembergische Landesgeschichte

Armin Panter

CLEAN - Soil, Air, Water

Emine Özbaş

Suresha Bheemappa

Julie Rursch

International Journal of Older People Nursing

Jekaterina Demidenko Shteinmiller

Engineering, Construction and Architectural Management

Ikechukwu Diugwu

Ngo Sy Thang QP2365

Pediatric Infectious Disease Journal

JURNAL ILMU FAAL OLAHRAGA INDONESIA

Informática na educação: teoria & prática

Lucicleide Araujo de Sousa Alves

Journal of Applied Physics

Masakazu Ichikawa

## RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Chapter 4: Home
• Qualtrics Survey Tool
• Statistics Help This link opens in a new window
• Statistics and APA Format This link opens in a new window
• Analysis and Coding Example- Qualitative Data

## Trustworthiness of Qualitative Data

• Hypothesis Testing This link opens in a new window

As noted in the dissertation template for qualitative studies, the section directly following the Chapter 4 introduction is to be labeled Trustworthiness of the Data, and in this section, qualitative researchers are required to articulate evidence of four primary criteria to ensure trustworthiness of the final study data set:

## Credibility (e.g., triangulation, member checks)

Credibility of qualitative data can be assured through multiple perspectives throughout data collection to ensure data are appropriate. This may be done through data, investigator, or theoretical triangulation; participant validation or member checks; or the rigorous techniques used to gather the data.

## Transferability (e.g., the extent to which the findings are generalizable to other situations)

Generalizability is not expected in qualitative research, so transferability of qualitative data assures the study findings are applicable to similar settings or individuals. Transferability can be demonstrated by clear assumptions and contextual inferences of the research setting and participants.

## Dependability (e.g., an in-depth description of the methodology and design to allow the study to be repeated)

Dependability of the qualitative data is demonstrated through assurances that the findings were established despite any changes within the research setting or participants during data collection. Again, rigorous data collection techniques and procedures can assure dependability of the final data set.

## Confirmability (e.g., the steps to ensure that the data and findings are not due to participant and/or researcher bias)

Confirmability of qualitative data is assured when data are checked and rechecked throughout data collection and analysis to ensure results would likely be repeatable by others. This can be documented by a clear coding schema that identifies the codes and patterns identified in analyses. Finally, a data audit prior to analysis can also ensure dependability.

For more information on these criteria, visit the Sage Research Methods database in the NU Library: https://resources.nu.edu/sagerm

• << Previous: Analysis and Coding Example- Qualitative Data
• Next: Hypothesis Testing >>
• Last Updated: Apr 19, 2024 3:09 PM
• URL: https://resources.nu.edu/c.php?g=1007180

• Open access
• Published: 13 May 2024

## What are the strengths and limitations to utilising creative methods in public and patient involvement in health and social care research? A qualitative systematic review

• Olivia R. Phillips 1 , 2   na1 ,
• Cerian Harries 2 , 3   na1 ,
• Jo Leonardi-Bee 1 , 2 , 4   na1 ,
• Holly Knight 1 , 2 ,
• Lauren B. Sherar 2 , 3 ,
• Veronica Varela-Mato 2 , 3 &
• Joanne R. Morling 1 , 2 , 5

Research Involvement and Engagement volume  10 , Article number:  48 ( 2024 ) Cite this article

1 Altmetric

Metrics details

There is increasing interest in using patient and public involvement (PPI) in research to improve the quality of healthcare. Ordinarily, traditional methods have been used such as interviews or focus groups. However, these methods tend to engage a similar demographic of people. Thus, creative methods are being developed to involve patients for whom traditional methods are inaccessible or non-engaging.

To determine the strengths and limitations to using creative PPI methods in health and social care research.

Electronic searches were conducted over five databases on 14th April 2023 (Web of Science, PubMed, ASSIA, CINAHL, Cochrane Library). Studies that involved traditional, non-creative PPI methods were excluded. Creative PPI methods were used to engage with people as research advisors, rather than study participants. Only primary data published in English from 2009 were accepted. Title, abstract and full text screening was undertaken by two independent reviewers before inductive thematic analysis was used to generate themes.

Twelve papers met the inclusion criteria. The creative methods used included songs, poems, drawings, photograph elicitation, drama performance, visualisations, social media, photography, prototype development, cultural animation, card sorting and persona development. Analysis identified four limitations and five strengths to the creative approaches. Limitations included the time and resource intensive nature of creative PPI, the lack of generalisation to wider populations and ethical issues. External factors, such as the lack of infrastructure to support creative PPI, also affected their implementation. Strengths included the disruption of power hierarchies and the creation of a safe space for people to express mundane or “taboo” topics. Creative methods are also engaging, inclusive of people who struggle to participate in traditional PPI and can also be cost and time efficient.

‘Creative PPI’ is an umbrella term encapsulating many different methods of engagement and there are strengths and limitations to each. The choice of which should be determined by the aims and requirements of the research, as well as the characteristics of the PPI group and practical limitations. Creative PPI can be advantageous over more traditional methods, however a hybrid approach could be considered to reap the benefits of both. Creative PPI methods are not widely used; however, this could change over time as PPI becomes embedded even more into research.

## Plain English Summary

It is important that patients and public are included in the research process from initial brainstorming, through design to delivery. This is known as public and patient involvement (PPI). Their input means that research closely aligns with their wants and needs. Traditionally to get this input, interviews and group discussions are held, but this can exclude people who find these activities non-engaging or inaccessible, for example those with language challenges, learning disabilities or memory issues. Creative methods of PPI can overcome this. This is a broad term describing different (non-traditional) ways of engaging patients and public in research, such as through the use or art, animation or performance. This review investigated the reasons why creative approaches to PPI could be difficult (limitations) or helpful (strengths) in health and social care research. After searching 5 online databases, 12 studies were included in the review. PPI groups included adults, children and people with language and memory impairments. Creative methods included songs, poems, drawings, the use of photos and drama, visualisations, Facebook, creating prototypes, personas and card sorting. Limitations included the time, cost and effort associated with creative methods, the lack of application to other populations, ethical issues and buy-in from the wider research community. Strengths included the feeling of equality between academics and the public, creation of a safe space for people to express themselves, inclusivity, and that creative PPI can be cost and time efficient. Overall, this review suggests that creative PPI is worthwhile, however each method has its own strengths and limitations and the choice of which will depend on the research project, PPI group characteristics and other practical limitations, such as time and financial constraints.

Peer Review reports

## Introduction

Patient and public involvement (PPI) is the term used to describe the partnership between patients (including caregivers, potential patients, healthcare users etc.) or the public (a community member with no known interest in the topic) with researchers. It describes research that is done “‘with’ or ‘by’ the public, rather than ‘to,’ ‘about’ or ‘for’ them” [ 1 ]. In 2009, it became a legislative requirement for certain health and social care organisations to include patients, families, carers and communities in not only the planning of health and social care services, but the commissioning, delivery and evaluation of them too [ 2 ]. For example, funding applications for the National Institute of Health and Care Research (NIHR), a UK funding body, mandates a demonstration of how researchers plan to include patients/service users, the public and carers at each stage of the project [ 3 ]. However, this should not simply be a tokenistic, tick-box exercise. PPI should help formulate initial ideas and should be an instrumental, continuous part of the research process. Input from PPI can provide unique insights not yet considered and can ensure that research and health services are closely aligned to the needs and requirements of service users PPI also generally makes research more relevant with clearer outcomes and impacts [ 4 ]. Although this review refers to both patients and the public using the umbrella term ‘PPI’, it is important to acknowledge that these are two different groups with different motivations, needs and interests when it comes to health research and service delivery [ 5 ].

We aimed to conduct a qualitative systematic review to highlight the strengths of creative PPI in health and social care research, as well as the limitations, which might act as a barrier to their implementation. A qualitative systematic review “brings together research on a topic, systematically searching for research evidence from primary qualitative studies and drawing the findings together” [ 8 ]. This review can then advise researchers of the best practices when designing PPI.

## Public involvement

The PHIRST-LIGHT Public Advisory Group (PAG) consists of a team of experienced public contributors with a diverse range of characteristics from across the UK. The PAG was involved in the initial question setting and study design for this review.

## Search strategy

For the purpose of this review, the JBI approach for conducting qualitative systematic reviews was followed [ 9 ]. The search terms were (“creativ*” OR “innovat*” OR “authentic” OR “original” OR “inclu*”) AND (“public and patient involvement” OR “patient and public involvement” OR “public and patient involvement and engagement” OR “patient and public involvement and engagement” OR “PPI” OR “PPIE” OR “co-produc*” OR “co-creat*” OR “co-design*” OR “cooperat*” OR “co-operat*”). This search string was modified according to the requirements of each database. Papers were filtered by title, abstract and keywords (see Additional file 1 for search strings). The databases searched included Web of Science (WoS), PubMed, ASSIA and CINAHL. The Cochrane Library was also searched to identify relevant reviews which could lead to the identification of primary research. The search was conducted on 14/04/23. As our aim was to report on the use of creative PPI in research, rather than more generic public engagement, we used electronic databases of scholarly peer-reviewed literature, which represent a wide range of recognised databases. These identified studies published in general international journals (WoS, PubMed), those in social sciences journals (ASSIA), those in nursing and allied health journals (CINAHL), and trials of interventions (Cochrane Library).

## Inclusion criteria

Only full-text, English language, primary research papers from 2009 to 2023 were included. This was the chosen timeframe as in 2009 the Health and Social Reform Act made it mandatory for certain Health and Social Care organisations to involve the public and patients in planning, delivering, and evaluating services [ 2 ]. Only creative methods of PPI were accepted, rather than traditional methods, such as interviews or focus groups. For the purposes of this paper, creative PPI included creative art or arts-based approaches (e.g., e.g. stories, songs, drama, drawing, painting, poetry, photography) to enhance engagement. Titles were related to health and social care and the creative PPI was used to engage with people as research advisors, not as study participants. Meta-analyses, conference abstracts, book chapters, commentaries and reviews were excluded. There were no limits concerning study location or the demographic characteristics of the PPI groups. Only qualitative data were accepted.

## Quality appraisal

Quality appraisal using the Critical Appraisal Skills Programme (CASP) checklist [ 10 ] was conducted by the primary authors (ORP and CH). This was done independently, and discrepancies were discussed and resolved. If a consensus could not be reached, a third independent reviewer was consulted (JRM). The full list of quality appraisal questions can be found in Additional file 2 .

## Data extraction

ORP extracted the study characteristics and a subset of these were checked by CH. Discrepancies were discussed and amendments made. Extracted data included author, title, location, year of publication, year study was carried out, research question/aim, creative methods used, number of participants, mean age, gender, ethnicity of participants, setting, limitations and strengths of creative PPI and main findings.

## Data analysis

The included studies were analysed using inductive thematic analysis [ 11 ], where themes were determined by the data. The familiarisation stage took place during full-text reading of the included articles. Anything identified as a strength or limitation to creative PPI methods was extracted verbatim as an initial code and inputted into the data extraction Excel sheet. Similar codes were sorted into broader themes, either under ‘strengths’ or ‘limitations’ and reviewed. Themes were then assigned a name according to the codes.

The search yielded 9978 titles across the 5 databases: Web of Science (1480 results), PubMed (94 results), ASSIA (2454 results), CINAHL (5948 results) and Cochrane Library (2 results), resulting in 8553 different studies after deduplication. ORP and CH independently screened their titles and abstracts, excluding those that did not meet the criteria. After assessment, 12 studies were included (see Fig.  1 ).

PRISMA flowchart of the study selection process

## Study characteristics

The included studies were published between 2018 and 2022. Seven were conducted in the UK [ 12 , 14 , 15 , 17 , 18 , 19 , 23 ], two in Canada [ 21 , 22 ], one in Australia [ 13 ], one in Norway [ 16 ] and one in Ireland [ 20 ]. The PPI activities occurred across various settings, including a school [ 12 ], social club [ 12 ], hospital [ 17 ], university [ 22 ], theatre [ 19 ], hotel [ 20 ], or online [ 15 , 21 ], however this information was omitted in 5 studies [ 13 , 14 , 16 , 18 , 23 ]. The number of people attending the PPI sessions varied, ranging from 6 to 289, however the majority (ten studies) had less than 70 participants [ 13 , 14 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. Seven studies did not provide information on the age or gender of the PPI groups. Of those that did, ages ranged from 8 to 76 and were mostly female. The ethnicities of the PPI group members were also rarely recorded (see Additional file 3 for data extraction table).

## Types of creative methods

The type of creative methods used to engage the PPI groups were varied. These included songs, poems, drawings, photograph elicitation, drama performance, visualisations, Facebook, photography, prototype development, cultural animation, card sorting and creating personas (see Table  1 ). These were sometimes accompanied by traditional methods of PPI such as interviews and focus group discussions.

The 12 included studies were all deemed to be of good methodological quality, with scores ranging from 6/10 to 10/10 with the CASP critical appraisal tool [ 10 ] (Table  2 ).

## Thematic analysis

Analysis identified four limitations and five strengths to creative PPI (see Fig.  2 ). Limitations included the time and resource intensity of creative PPI methods, its lack of generalisation, ethical issues and external factors. Strengths included the disruption of power hierarchies, the engaging and inclusive nature of the methods and their long-term cost and time efficiency. Creative PPI methods also allowed mundane and “taboo” topics to be discussed within a safe space.

Theme map of strengths and limitations

## Limitations of creative PPI

Creative ppi methods are time and resource intensive.

The time and resource intensive nature of creative PPI methods is a limitation, most notably for the persona-scenario methodology. Valaitis et al. [ 22 ] used 14 persona-scenario workshops with 70 participants to co-design a healthcare intervention, which aimed to promote optimal aging in Canada. Using the persona method, pairs composed of patients, healthcare providers, community service providers and volunteers developed a fictional character which they believed represented an ‘end-user’ of the healthcare intervention. Due to the depth and richness of the data produced the authors reported that it was time consuming to analyse. Further, they commented that the amount of information was difficult to disseminate to scientific leads and present at team meetings. Additionally, to ensure the production of high-quality data, to probe for details and lead group discussion there was a need for highly skilled facilitators. The resource intensive nature of the creative co-production was also noted in a study using the persona scenario and creative worksheets to develop a prototype decision support tool for individuals with malignant pleural effusion [ 17 ]. With approximately 50 people, this was also likely to yield a high volume of data to consider.

To prepare materials for populations who cannot engage in traditional methods of PPI was also timely. Kearns et al. [ 18 ] developed a feedback questionnaire for people with aphasia to evaluate ICT-delivered rehabilitation. To ensure people could participate effectively, the resources used during the workshops, such as PowerPoints, online images and photographs, had to be aphasia-accessible, which was labour and time intensive. The author warned that this time commitment should not be underestimated.

There are further practical limitations to implementing creative PPI, such as the costs of materials for activities as well as hiring a space for workshops. For example, the included studies in this review utilised pens, paper, worksheets, laptops, arts and craft supplies and magazines and took place in venues such as universities, a social club, and a hotel. Further, although not limited to creative PPI methods exclusively but rather most studies involving the public, a financial incentive was often offered for participation, as well as food, parking, transport and accommodation [ 21 , 22 ].

## Creative PPI lacks generalisation

Another barrier to the use of creative PPI methods in health and social care research was the individual nature of its output. Those who participate, usually small in number, produce unique creative outputs specific to their own experiences, opinions and location. Craven et al. [ 13 ], used arts-based visualisations to develop a toolbox for adults with mental health difficulties. They commented, “such an approach might still not be worthwhile”, as the visualisations were individualised and highly personal. This indicates that the output may fail to meet the needs of its end-users. Further, these creative PPI groups were based in certain geographical regions such as Stoke-on-Trent [ 19 ] Sheffield [ 23 ], South Wales [ 12 ] or Ireland [ 20 ], which limits the extent the findings can be applied to wider populations, even within the same area due to individual nuances. Further, the study by Galler et al. [ 16 ], is specific to the Norwegian context and even then, maybe only a sub-group of the Norwegian population as the sample used was of higher socioeconomic status.

However, Grindell et al. [ 17 ], who used persona scenarios, creative worksheets and prototype development, pointed out that the purpose of this type of research is to improve a certain place, rather than apply findings across other populations and locations. Individualised output may, therefore, only be a limitation to research wanting to conduct PPI on a large scale.

If, however, greater generalisation within PPI is deemed necessary, then social media may offer a resolution. Fedorowicz et al. [ 15 ], used Facebook to gain feedback from the public on the use of video-recording methodology for an upcoming project. This had the benefit of including a more diverse range of people (289 people joined the closed group), who were spread geographically around the UK, as well as seven people from overseas.

## Creative PPI has ethical issues

As with other research, ethical issues must be taken into consideration. Due to the nature of creative approaches, as well as the personal effort put into them, people often want to be recognised for their work. However, this compromises principles so heavily instilled in research such as anonymity and confidentiality. With the aim of exploring issues related to health and well-being in a town in South Wales, Byrne et al. [ 12 ], asked year 4/5 and year 10 pupils to create poems, songs, drawings and photographs. Community members also created a performance, mainly of monologues, to explore how poverty and inequalities are dealt with. Byrne noted the risks of these arts-based approaches, that being the possibility of over-disclosure and consequent emotional distress, as well as people’s desire to be named for their work. On one hand, the anonymity reduces the sense of ownership of the output as it does not portray a particular individual’s lived experience anymore. On the other hand, however, it could promote a more honest account of lived experience. Supporting this, Webber et al. [ 23 ], who used the persona method to co-design a back pain educational resource prototype, claimed that the anonymity provided by this creative technique allowed individuals to externalise and anonymise their own personal experience, thus creating a more authentic and genuine resource for future users. This implies that anonymity can be both a limitation and strength here.

## The use of creative PPI methods is impeded by external factors

Despite the above limitations influencing the implementation of creative PPI techniques, perhaps the most influential is that creative methodologies are simply not mainstream [ 19 ]. This could be linked to the issues above, like time and resource intensity, generalisation and ethical issues but it is also likely to involve more systemic factors within the research community. Micsinszki et al. [ 21 ], who co-designed a hub for the health and well-being of vulnerable populations, commented that there is insufficient infrastructure to conduct meaningful co-design as well as a dominant medical model. Through a more holistic lens, there are “sociopolitical environments that privilege individualism over collectivism, self-sufficiency over collaboration, and scientific expertise over other ways of knowing based on lived experience” [ 21 ]. This, it could be suggested, renders creative co-design methodologies, which are based on the foundations of collectivism, collaboration and imagination an invalid technique in the research field, which is heavily dominated by more scientific methods offering reproducibility, objectivity and reliability.

Although we acknowledge that creative PPI techniques are not always appropriate, it may be that their main limitation is the lack of awareness of these methods or lack of willingness to use them. Further, there is always the risk that PPI, despite being a mandatory part of research, is used in a tokenistic or tick-box fashion [ 20 ], without considering the contribution that meaningful PPI could make to enhancing the research. It may be that PPI, let alone creative PPI, is not at the forefront of researchers’ minds when planning research.

## Strengths of creative PPI

Creative ppi disrupts power hierarchies.

One of the main strengths of creative PPI techniques, cited most frequently in the included literature, was that they disrupt traditional power hierarchies [ 12 , 13 , 17 , 19 , 23 ]. For example, the use of theatre performance blurred the lines between professional and lay roles between the community and policy makers [ 12 ]. Individuals created a monologue to portray how poverty and inequality impact daily life and presented this to representatives of the National Assembly of Wales, Welsh Government, the Local Authority, Arts Council and Westminster. Byrne et al. [ 12 ], states how this medium allowed the community to engage with the people who make decisions about their lives in an environment of respect and understanding, where the hierarchies are not as visible as in other settings, e.g., political surgeries. Creative PPI methods have also removed traditional power hierarchies between researchers and adolescents. Cook et al. [ 13 ], used arts-based approaches to explore adolescents’ ideas about the “perfect” condom. They utilised the “Life Happens” resource, where adolescents drew and then decorated a person with their thoughts about sexual relationships, not too dissimilar from the persona-scenario method. This was then combined with hypothetical scenarios about sexuality. A condom-mapping exercise was then implemented, where groups shared the characteristics that make a condom “perfect” on large pieces of paper. Cook et al. [ 13 ], noted that usually power imbalances make it difficult to elicit information from adolescents, however these power imbalances were reduced due to the use of creative co-design techniques.

The same reduction in power hierarchies was noted by Grindell et al. [ 17 ], who used the person-scenario method and creative worksheets with individuals with malignant pleural effusion. This was with the aim of developing a prototype of a decision support tool for patients to help with treatment options. Although this process involved a variety of stakeholders, such as patients, carers and healthcare professionals, creative co-design was cited as a mechanism that worked to reduce power imbalances – a limitation of more traditional methods of research. Creative co-design blurred boundaries between end-users and clinical staff and enabled the sharing of ideas from multiple, valuable perspectives, meaning the prototype was able to suit user needs whilst addressing clinical problems.

Similarly, a specific creative method named cultural animation was also cited to dissolve hierarchies and encourage equal contributions from participants. Within this arts-based approach, Keleman et al. [ 19 ], explored the concept of “good health” with individuals from Stoke-on Trent. Members of the group created art installations using ribbons, buttons, cardboard and straws to depict their idea of a “healthy community”, which was accompanied by a poem. They also created a 3D Facebook page and produced another poem or song addressing the government to communicate their version of a “picture of health”. Public participants said that they found the process empowering, honest, democratic, valuable and practical.

This dissolving of hierarchies and levelling of power is beneficial as it increases the sense of ownership experienced by the creators/producers of the output [ 12 , 17 , 23 ]. This is advantageous as it has been suggested to improve its quality [ 23 ].

## Creative PPI allows the unsayable to be said

Creative PPI fosters a safe space for mundane or taboo topics to be shared, which may be difficult to communicate using traditional methods of PPI. For example, the hypothetical nature of condom mapping and persona-scenarios meant that adolescents could discuss a personal topic without fear of discrimination, judgement or personal disclosure [ 13 ]. The safe space allowed a greater volume of ideas to be generated amongst peers where they might not have otherwise. Similarly, Webber et al. [ 23 ], , who used the persona method to co-design the prototype back pain educational resource, also noted how this method creates anonymity whilst allowing people the opportunity to externalise personal experiences, thoughts and feelings. Other creative methods were also used, such as drawing, collaging, role play and creating mood boards. A cardboard cube (labelled a “magic box”) was used to symbolise a physical representation of their final prototype. These creative methods levelled the playing field and made personal experiences accessible in a safe, open environment that fostered trust, as well as understanding from the researchers.

It is not only sensitive subjects that were made easier to articulate through creative PPI. The communication of mundane everyday experiences were also facilitated, which were deemed typically ‘unsayable’. This was specifically given in the context of describing intangible aspects of everyday health and wellbeing [ 11 ]. Graphic designers can also be used to visually represent the outputs of creative PPI. These captured the movement and fluidity of people and well as the relationships between them - things that cannot be spoken but can be depicted [ 21 ].

## Creative PPI methods are inclusive

Another strength of creative PPI was that it is inclusive and accessible [ 17 , 19 , 21 ]. The safe space it fosters, as well as the dismantling of hierarchies, welcomed people from a diverse range of backgrounds and provided equal opportunities [ 21 ], especially for those with communication and memory difficulties who might be otherwise excluded from PPI. Kelemen et al. [ 19 ], who used creative methods to explore health and well-being in Stoke-on-Trent, discussed how people from different backgrounds came together and connected, discussed and reached a consensus over a topic which evoked strong emotions, that they all have in common. Individuals said that the techniques used “sets people to open up as they are not overwhelmed by words”. Similarly, creative activities, such as the persona method, have been stated to allow people to express themselves in an inclusive environment using a common language. Kearns et al. [ 18 ], who used aphasia-accessible material to develop a questionnaire with aphasic individuals, described how they felt comfortable in contributing to workshops (although this material was time-consuming to make, see ‘Limitations of creative PPI’ ).

Despite the general inclusivity of creative PPI, it can also be exclusive, particularly if online mediums are used. Fedorowicz et al. [ 15 ], used Facebook to create a PPI group, and although this may rectify previous drawbacks about lack of generalisation of creative methods (as Facebook can reach a greater number of people, globally), it excluded those who are not digitally active or have limited internet access or knowledge of technology. Online methods have other issues too. Maintaining the online group was cited as challenging and the volume of responses required researchers to interact outside of their working hours. Despite this, online methods like Facebook are very accessible for people who are physically disabled.

## Creative PPI methods are engaging

The process of creative PPI is typically more engaging and produces more colourful data than traditional methods [ 13 ]. Individuals are permitted and encouraged to explore a creative self [ 19 ], which can lead to the exploration of new ideas and an overall increased enjoyment of the process. This increased engagement is particularly beneficial for younger PPI groups. For example, to involve children in the development of health food products, Galler et al. [ 16 ] asked 9-12-year-olds to take photos of their food and present it to other children in a “show and tell” fashion. They then created a newspaper article describing a new healthy snack. In this creative focus group, children were given lab coats to further their identity as inventors. Galler et al. [ 16 ], notes that the methods were highly engaging and facilitated teamwork and group learning. This collaborative nature of problem-solving was also observed in adults who used personas and creative worksheets to develop the resource for lower back pain [ 23 ]. Dementia patients too have been reported to enjoy the creative and informal approach to idea generation [ 20 ].

The use of cultural animation allowed people to connect with each other in a way that traditional methods do not [ 19 , 21 ]. These connections were held in place by boundary objects, such as ribbons, buttons, fabric and picture frames, which symbolised a shared meaning between people and an exchange of knowledge and emotion. Asking groups to create an art installation using these objects further fostered teamwork and collaboration, both at an individual and collective level. The exploration of a creative self increased energy levels and encouraged productive discussions and problem-solving [ 19 ]. Objects also encouraged a solution-focused approach and permitted people to think beyond their usual everyday scope [ 17 ]. They also allowed facilitators to probe deeper about the greater meanings carried by the object, which acted as a metaphor [ 21 ].

From the researcher’s point of view, co-creative methods gave rise to ideas they might not have initially considered. Valaitis et al. [ 22 ], found that over 40% of the creative outputs were novel ideas brought to light by patients, healthcare providers/community care providers, community service providers and volunteers. One researcher commented, “It [the creative methods] took me on a journey, in a way that when we do other pieces of research it can feel disconnected” [ 23 ]. Another researcher also stated they could not return to the way they used to do research, as they have learnt so much about their own health and community and how they are perceived [ 19 ]. This demonstrates that creative processes not only benefit the project outcomes and the PPI group, but also facilitators and researchers. However, although engaging, creative methods have been criticised for not demonstrating academic rigour [ 17 ]. Moreover, creative PPI may also be exclusive to people who do not like or enjoy creative activities.

## Creative PPI methods are cost and time efficient

Creative PPI workshops can often produce output that is visible and tangible. This can save time and money in the long run as the output is either ready to be implemented in a healthcare setting or a first iteration has already been developed. This may also offset the time and costs it takes to implement creative PPI. For example, the prototype of the decision support tool for people with malignant pleural effusion was developed using personas and creative worksheets. The end result was two tangible prototypes to drive the initial idea forward as something to be used in practice [ 17 ]. The use of creative co-design in this case saved clinician time as well as the time it would take to develop this product without the help of its end-users. In the development of this particular prototype, analysis was iterative and informed the next stage of development, which again saved time. The same applies for the feedback questionnaire for the assessment of ICT delivered aphasia rehabilitation. The co-created questionnaire, designed with people with aphasia, was ready to be used in practice [ 18 ]. This suggests that to overcome time and resource barriers to creative PPI, researchers should aim for it to be engaging whilst also producing output.

That useable products are generated during creative workshops signals to participating patients and public members that they have been listened to and their thoughts and opinions acted upon [ 23 ]. For example, the development of the back pain resource based on patient experiences implies that their suggestions were valid and valuable. Further, those who participated in the cultural animation workshop reported that the process visualises change, and that it already feels as though the process of change has started [ 19 ].

The most cost and time efficient method of creative PPI in this review is most likely the use of Facebook to gather feedback on project methodology [ 15 ]. Although there were drawbacks to this, researchers could involve more people from a range of geographical areas at little to no cost. Feedback was instantaneous and no training was required. From the perspective of the PPI group, they could interact however much or little they wish with no time commitment.

This systematic review identified four limitations and five strengths to the use of creative PPI in health and social care research. Creative PPI is time and resource intensive, can raise ethical issues and lacks generalisability. It is also not accepted by the mainstream. These factors may act as barriers to the implementation of creative PPI. However, creative PPI disrupts traditional power hierarchies and creates a safe space for taboo or mundane topics. It is also engaging, inclusive and can be time and cost efficient in the long term.

Something that became apparent during data analysis was that these are not blanket strengths and limitations of creative PPI as a whole. The umbrella term ‘creative PPI’ is broad and encapsulates a wide range of activities, ranging from music and poems to prototype development and persona-scenarios, to more simplistic things like the use of sticky notes and ordering cards. Many different activities can be deemed ‘creative’ and the strengths and limitations of one does not necessarily apply to another. For example, cultural animation takes greater effort to prepare than the use of sticky notes and sorting cards, and the use of Facebook is cheaper and wider reaching than persona development. Researchers should use their discretion and weigh up the benefits and drawbacks of each method to decide on a technique which suits the project. What might be a limitation to creative PPI in one project may not be in another. In some cases, creative PPI may not be suitable at all.

Furthermore, the choice of creative PPI method also depends on the needs and characteristics of the PPI group. Children, adults and people living with dementia or language difficulties all have different engagement needs and capabilities. This indicates that creative PPI is not one size fits all and that the most appropriate method will change depending on the composition of the group. The choice of method will also be determined by the constraints of the research project, namely time, money and the research aim. For example, if there are time constraints, then a method which yields a lot of data and requires a lot of preparation may not be appropriate. If generalisation is important, then an online method is more suitable. Together this indicates that the choice of creative PPI method is highly individualised and dependent on multiple factors.

Although the limitations discussed in this review apply to creative PPI, they are not exclusive to creative PPI. Ethical issues are a consideration within general PPI research, especially when working with more vulnerable populations, such as children or adults living with a disability. It can also be the case that traditional PPI methods lack generalisability, as people who volunteer to be part of such a group are more likely be older, middle class and retired [ 24 ]. Most research is vulnerable to this type of bias, however, it is worth noting that generalisation is not always a goal and research remains valid and meaningful in its absence. Although online methods may somewhat combat issues related to generalisability, these methods still exclude people who do not have access to the internet/technology or who choose not to use it, implying that online PPI methods may not be wholly representative of the general population. Saying this, however, the accessibility of creative PPI techniques differs from person to person, and for some, online mediums may be more accessible (for example for those with a physical disability), and for others, this might be face-to-face. To combat this, a range of methods should be implemented. Planning multiple focus group and interviews for traditional PPI is also time and resource intensive, however the extra resources required to make this creative may be even greater. Although, the rich data provided may be worth the preparation and analysis time, which is also likely to depend on the number of participants and workshop sessions required. PPI, not just creative PPI, often requires the provision of a financial incentive, refreshments, parking and accommodation, which increase costs. These, however, are imperative and non-negotiable, as they increase the accessibility of research, especially to minority and lower-income groups less likely to participate. Adequate funding is also important for co-design studies where repeated engagement is required. One barrier to implementation, which appears to be exclusive to creative methods, however, is that creative methods are not mainstream. This cannot be said for traditional PPI as this is often a mandatory part of research applications.

Regarding the strengths of creative PPI, it could be argued that most appear to be exclusive to creative methodologies. These are inclusive by nature as multiple approaches can be taken to evoke ideas from different populations - approaches that do not necessarily rely on verbal or written communication like interviews and focus groups do. Given the anonymity provided by some creative methods, such as personas, people may be more likely to discuss their personal experiences under the guise of a general end-user, which might be more difficult to maintain when an interviewer is asking an individual questions directly. Additionally, creative methods are by nature more engaging and interactive than traditional methods, although this is a blanket statement and there may be people who find the question-and-answer/group discussion format more engaging. Creative methods have also been cited to eliminate power imbalances which exist in traditional research [ 12 , 13 , 17 , 19 , 23 ]. These imbalances exist between researchers and policy makers and adolescents, adults and the community. Lastly, although this may occur to a greater extent in creative methods like prototype development, it could be suggested that PPI in general – regardless of whether it is creative - is more time and cost efficient in the long-term than not using any PPI to guide or refine the research process. It must be noted that these are observations based on the literature. To be certain these differences exist between creative and traditional methods of PPI, direct empirical evaluation of both should be conducted.

To the best of our knowledge, this is the first review to identify the strengths and limitations to creative PPI, however, similar literature has identified barriers and facilitators to PPI in general. In the context of clinical trials, recruitment difficulties were cited as a barrier, as well as finding public contributors who were free during work/school hours. Trial managers reported finding group dynamics difficult to manage and the academic environment also made some public contributors feel nervous and lacking confidence to speak. Facilitators, however, included the shared ownership of the research – something that has been identified in the current review too. In addition, planning and the provision of knowledge, information and communication were also identified as facilitators [ 25 ]. Other research on the barriers to meaningful PPI in trial oversight committees included trialist confusion or scepticism over the PPI role and the difficulties in finding PPI members who had a basic understanding of research [ 26 ]. However, it could be argued that this is not representative of the average patient or public member. The formality of oversight meetings and the technical language used also acted as a barrier, which may imply that the informal nature of creative methods and its lack of dependency on literacy skills could overcome this. Further, a review of 42 reviews on PPI in health and social care identified financial compensation, resources, training and general support as necessary to conduct PPI, much like in the current review where the resource intensiveness of creative PPI was identified as a limitation. However, others were identified too, such as recruitment and representativeness of public contributors [ 27 ]. Like in the current review, power imbalances were also noted, however this was included as both a barrier and facilitator. Collaboration seemed to diminish hierarchies but not always, as sometimes these imbalances remained between public contributors and healthcare staff, described as a ‘them and us’ culture [ 27 ]. Although these studies compliment the findings of the current review, a direct comparison cannot be made as they do not concern creative methods. However, it does suggest that some strengths and weaknesses are shared between creative and traditional methods of PPI.

## Strengths and limitations of this review

Although a general definition of creative PPI exists, it was up to our discretion to decide exactly which activities were deemed as such for this review. For example, we included sorting cards, the use of interactive whiteboards and sticky notes. Other researchers may have a more or less stringent criteria. However, two reviewers were involved in this decision which aids the reliability of the included articles. Further, it may be that some of the strengths and limitations cannot fully be attributed to the creative nature of the PPI process, but rather their co-created nature, however this is hard to disentangle as the included papers involved both these aspects.

During screening, it was difficult to decide whether the article was utilising creative qualitative methodology or creative PPI , as it was often not explicitly labelled as such. Regardless, both approaches involved the public/patients refining a healthcare product/service. This implies that if this review were to be replicated, others may do it differently. This may call for greater standardisation in the reporting of the public’s involvement in research. For example, the NIHR outlines different approaches to PPI, namely “consultation”, “collaboration”, “co-production” and “user-controlled”, which each signify an increased level of public power and influence [ 28 ]. Papers with elements of PPI could use these labels to clarify the extent of public involvement, or even explicitly state that there was no PPI. Further, given our decision to include only scholarly peer-reviewed literature, it is possible that data were missed within the grey literature. Similarly, the literature search will not have identified all papers relating to different types of accessible inclusion. However, the intent of the review was to focus solely on those within the definition of creative.

This review fills a gap in the literature and helps circulate and promote the concept of creative PPI. Each stage of this review, namely screening and quality appraisal, was conducted by two independent reviewers. However, four full texts could not be accessed during the full text reading stage, meaning there are missing data that could have altered or contributed to the findings of this review.

## Research recommendations

Given that creative PPI can require effort to prepare, perform and analyse, sufficient time and funding should be allocated in the research protocol to enable meaningful and continuous PPI. This is worthwhile as PPI can significantly change the research output so that it aligns closely with the needs of the group it is to benefit. Researchers should also consider prototype development as a creative PPI activity as this might reduce future time/resource constraints. Shifting from a top-down approach within research to a bottom-up can be advantageous to all stakeholders and can help move creative PPI towards the mainstream. This, however, is the collective responsibility of funding bodies, universities and researchers, as well as committees who approve research bids.

A few of the included studies used creative techniques alongside traditional methods, such as interviews, which could also be used as a hybrid method of PPI, perhaps by researchers who are unfamiliar with creative techniques or to those who wish to reap the benefits of both. Often the characteristics of the PPI group were not included, including age, gender and ethnicity. It would be useful to include such information to assess how representative the PPI group is of the population of interest.

Creative PPI is a relatively novel approach of engaging the public and patients in research and it has both advantages and disadvantages compared to more traditional methods. There are many approaches to implementing creative PPI and the choice of technique will be unique to each piece of research and is reliant on several factors. These include the age and ability of the PPI group as well as the resource limitations of the project. Each method has benefits and drawbacks, which should be considered at the protocol-writing stage. However, given adequate funding, time and planning, creative PPI is a worthwhile and engaging method of generating ideas with end-users of research – ideas which may not be otherwise generated using traditional methods.

## Data availability

No datasets were generated or analysed during the current study.

## Abbreviations

Critical Appraisal Skills Programme

The Joanna Briggs Institute

National Institute of Health and Care Research

Public and Patient Involvement

Web of Science

National Institute for Health and Care Research. What Is Patient and Public Involvement and Public Engagement? https://www.spcr.nihr.ac.uk/PPI/what-is-patient-and-public-involvement-and-engagement Accessed 01 Sept 2023.

Department of Health. Personal and Public Involvement (PPI) https://www.health-ni.gov.uk/topics/safety-and-quality-standards/personal-and-public-involvement-ppi#:~:text=The Health and Social Care Reform Act (NI) 2009 placed,delivery and evaluation of services . Accessed 01 Sept 2023.

National Institute for Health and Care Research. Policy Research Programme – Guidance for Stage 1 Applications https://www.nihr.ac.uk/documents/policy-research-programme-guidance-for-stage-1-applications-updated/26398 Accessed 01 Sept 2023.

Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, Chant A. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect. 2019. https://doi.org/10.1111/hex.12888

Article   PubMed   PubMed Central   Google Scholar

Street JM, Stafinski T, Lopes E, Menon D. Defining the role of the public in health technology assessment (HTA) and HTA-informed decision-making processes. Int J Technol Assess Health Care. 2020. https://doi.org/10.1017/S0266462320000094

Morrison C, Dearden A. Beyond tokenistic participation: using representational artefacts to enable meaningful public participation in health service design. Health Policy. 2013. https://doi.org/10.1016/j.healthpol.2013.05.008

Leavy P. Method meets art: arts-Based Research Practice. New York: Guilford; 2020.

Seers K. Qualitative systematic reviews: their importance for our understanding of research relevant to pain. Br J Pain. 2015. https://doi.org/10.1177/2049463714549777

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis JBI. 2020. https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03

CASP. CASP Checklists https://casp-uk.net/images/checklist/documents/CASP-Qualitative-Studies-Checklist/CASP-Qualitative-Checklist-2018_fillable_form.pdf (2022).

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006. https://doi.org/10.1191/1478088706qp063oa

Byrne E, Elliott E, Saltus R, Angharad J. The creative turn in evidence for public health: community and arts-based methodologies. J Public Health. 2018. https://doi.org/10.1093/pubmed/fdx151

Cook S, Grozdanovski L, Renda G, Santoso D, Gorkin R, Senior K. Can you design the perfect condom? Engaging young people to inform safe sexual health practice and innovation. Sex Educ. 2022. https://doi.org/10.1080/14681811.2021.1891040

Craven MP, Goodwin R, Rawsthorne M, Butler D, Waddingham P, Brown S, Jamieson M. Try to see it my way: exploring the co-design of visual presentations of wellbeing through a workshop process. Perspect Public Health. 2019. https://doi.org/10.1177/1757913919835231

Fedorowicz S, Riley V, Cowap L, Ellis NJ, Chambers R, Grogan S, Crone D, Cottrell E, Clark-Carter D, Roberts L, Gidlow CJ. Using social media for patient and public involvement and engagement in health research: the process and impact of a closed Facebook group. Health Expect. 2022. https://doi.org/10.1111/hex.13515

Galler M, Myhrer K, Ares G, Varela P. Listening to children voices in early stages of new product development through co-creation – creative focus group and online platform. Food Res Int. 2022. https://doi.org/10.1016/j.foodres.2022.111000

Grindell C, Tod A, Bec R, Wolstenholme D, Bhatnagar R, Sivakumar P, Morley A, Holme J, Lyons J, Ahmed M, Jackson S, Wallace D, Noorzad F, Kamalanathan M, Ahmed L, Evison M. Using creative co-design to develop a decision support tool for people with malignant pleural effusion. BMC Med Inf Decis Mak. 2020. https://doi.org/10.1186/s12911-020-01200-3

Kearns Á, Kelly H, Pitt I. Rating experience of ICT-delivered aphasia rehabilitation: co-design of a feedback questionnaire. Aphasiology. 2020. https://doi.org/10.1080/02687038.2019.1649913

Kelemen M, Surman E, Dikomitis L. Cultural animation in health research: an innovative methodology for patient and public involvement and engagement. Health Expect. 2018. https://doi.org/10.1111/hex.12677

Keogh F, Carney P, O’Shea E. Innovative methods for involving people with dementia and carers in the policymaking process. Health Expect. 2021. https://doi.org/10.1111/hex.13213

Micsinszki SK, Buettgen A, Mulvale G, Moll S, Wyndham-West M, Bruce E, Rogerson K, Murray-Leung L, Fleisig R, Park S, Phoenix M. Creative processes in co-designing a co-design hub: towards system change in health and social services in collaboration with structurally vulnerable populations. Evid Policy. 2022. https://doi.org/10.1332/174426421X16366319768599

Valaitis R, Longaphy J, Ploeg J, Agarwal G, Oliver D, Nair K, Kastner M, Avilla E, Dolovich L. Health TAPESTRY: co-designing interprofessional primary care programs for older adults using the persona-scenario method. BMC Fam Pract. 2019. https://doi.org/10.1186/s12875-019-1013-9

Webber R, Partridge R, Grindell C. The creative co-design of low back pain education resources. Evid Policy. 2022. https://doi.org/10.1332/174426421X16437342906266

National Institute for Health and Care Research. A Researcher’s Guide to Patient and Public Involvement. https://oxfordbrc.nihr.ac.uk/wp-content/uploads/2017/03/A-Researchers-Guide-to-PPI.pdf Accessed 01 Nov 2023.

Selman L, Clement C, Douglas M, Douglas K, Taylor J, Metcalfe C, Lane J, Horwood J. Patient and public involvement in randomised clinical trials: a mixed-methods study of a clinical trials unit to identify good practice, barriers and facilitators. Trials. 2021 https://doi.org/10.1186/s13063-021-05701-y

Coulman K, Nicholson A, Shaw A, Daykin A, Selman L, Macefield R, Shorter G, Cramer H, Sydes M, Gamble C, Pick M, Taylor G, Lane J. Understanding and optimising patient and public involvement in trial oversight: an ethnographic study of eight clinical trials. Trials. 2020. https://doi.org/10.1186/s13063-020-04495-9

Ocloo J, Garfield S, Franklin B, Dawson S. Exploring the theory, barriers and enablers for patient and public involvement across health, social care and patient safety: a systematic review of reviews. Health Res Policy Sys. 2021. https://doi.org/10.1186/s12961-020-00644-3

National Institute for Health and Care Research. Briefing notes for researchers - public involvement in NHS, health and social care research. https://www.nihr.ac.uk/documents/briefing-notes-for-researchers-public-involvement-in-nhs-health-and-social-care-research/27371 Accessed 01 Nov 2023.

## Acknowledgements

With thanks to the PHIRST-LIGHT public advisory group and consortium for their thoughts and contributions to the design of this work.

The research team is supported by a National Institute for Health and Care Research grant (PHIRST-LIGHT Reference NIHR 135190).

## Author information

Olivia R. Phillips and Cerian Harries share joint first authorship.

## Authors and Affiliations

Nottingham Centre for Public Health and Epidemiology, Lifespan and Population Health, School of Medicine, University of Nottingham, Clinical Sciences Building, City Hospital Campus, Hucknall Road, Nottingham, NG5 1PB, UK

Olivia R. Phillips, Jo Leonardi-Bee, Holly Knight & Joanne R. Morling

National Institute for Health and Care Research (NIHR) PHIRST-LIGHT, Nottingham, UK

Olivia R. Phillips, Cerian Harries, Jo Leonardi-Bee, Holly Knight, Lauren B. Sherar, Veronica Varela-Mato & Joanne R. Morling

School of Sport, Exercise and Health Sciences, Loughborough University, Epinal Way, Loughborough, Leicestershire, LE11 3TU, UK

Cerian Harries, Lauren B. Sherar & Veronica Varela-Mato

Nottingham Centre for Evidence Based Healthcare, School of Medicine, University of Nottingham, Nottingham, UK

Jo Leonardi-Bee

NIHR Nottingham Biomedical Research Centre (BRC), Nottingham University Hospitals NHS Trust, University of Nottingham, Nottingham, NG7 2UH, UK

Joanne R. Morling

You can also search for this author in PubMed   Google Scholar

## Contributions

Author contributions: study design: ORP, CH, JRM, JLB, HK, LBS, VVM, literature searching and screening: ORP, CH, JRM, data curation: ORP, CH, analysis: ORP, CH, JRM, manuscript draft: ORP, CH, JRM, Plain English Summary: ORP, manuscript critical review and editing: ORP, CH, JRM, JLB, HK, LBS, VVM.

## Corresponding author

Correspondence to Olivia R. Phillips .

## Ethics declarations

Ethics approval and consent to participate.

The Ethics Committee of the Faculty of Medicine and Health Sciences, University of Nottingham advised that approval from the ethics committee and consent to participate was not required for systematic review studies.

Not applicable.

## Competing interests

The authors declare no competing interests.

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Electronic supplementary material

Below is the link to the electronic supplementary material.

## 40900_2024_580_MOESM1_ESM.docx

Additional file 1: Search strings: Description of data: the search strings and filters used in each of the 5 databases in this review

## Additional file 2: Quality appraisal questions: Description of data: CASP quality appraisal questions

40900_2024_580_moesm3_esm.docx.

Additional file 3: Table 1: Description of data: elements of the data extraction table that are not in the main manuscript

## Rights and permissions

Reprints and permissions

Phillips, O.R., Harries, C., Leonardi-Bee, J. et al. What are the strengths and limitations to utilising creative methods in public and patient involvement in health and social care research? A qualitative systematic review. Res Involv Engagem 10 , 48 (2024). https://doi.org/10.1186/s40900-024-00580-4

Accepted : 25 April 2024

Published : 13 May 2024

DOI : https://doi.org/10.1186/s40900-024-00580-4

Anyone you share the following link with will be able to read this content:

Provided by the Springer Nature SharedIt content-sharing initiative

• Public and patient involvement
• Creative PPI
• Qualitative systematic review

## Research Involvement and Engagement

ISSN: 2056-7529

1. PDF Chapter 4: Analysis and Interpretation of Results

4.1 INTRODUCTION To complete this study properly, it is necessary to analyse the data collected in order to test the hypothesis and answer the research questions. As already indicated in the preceding chapter, data is interpreted in a descriptive form. This chapter comprises the analysis, presentation and interpretation of the findings resulting

2. PDF CHAPTER 4 QUALITATIVE DATA ANALYSIS

4.1 INTRODUCTION. In this chapter, I describe the qualitative analysis of the data, including the practical steps involved in the analysis. A quantitative analysis of the data follows in Chapter 5. In the qualitative phase, I analyzed the data into generative themes, which will be described individually. I describe how the themes overlap.

3. PDF CHAPTER FOUR Qualitative Research

Research. methods that delve deeply into experiences, social processes, and subcultures are referred to as qualitative research. As a group, qualitative research methods: Recognize that every individual is situated in an unfolding life context, that is, a set of circumstances, values, and influences. Respect the meanings each individual assigns ...

4. Presenting Your Qualitative Analysis Findings: Tables to Include in

To help clarify on this point, we asked our qualitative analysis experts to share their recommendations for tables to include in your Chapter 4. Demographics Tables. As with studies using quantitative methods, presenting an overview of your sample demographics is useful in studies that use qualitative research methods. The standard demographics ...

5. Writing Chapter 4 : Analysis & Results for Qualitative Research

Chapter 4 for Qualitative Research carries different titles such as 'Analysis of Data', 'Results of Study', 'Analysis and Results'

6. Dissertation Results & Findings Chapter (Qualitative)

The results chapter in a dissertation or thesis (or any formal academic research piece) is where you objectively and neutrally present the findings of your qualitative analysis (or analyses if you used multiple qualitative analysis methods ). This chapter can sometimes be combined with the discussion chapter (where you interpret the data and ...

7. PDF Writing a Dissertation's Chapter 4 and 5 1 By Dr. Kimberly Blum Rita

In general, the length of a qualitative chapter four is 25-35 pages (Simon, 2006), depending on how many themes chapter four discovered. ... the research study gave the same or very close responses. For instance, if the researcher ... Writing a Dissertation's Chapter 4 and 5 6 example, if four out of 20 participants in a research study about ...

8. The Elements of Chapter 4

Chapter 4. What needs to be included in the chapter? The topics below are typically included in this chapter, and often in this order (check with your Chair): Introduction. Remind the reader what your research questions were. In a qualitative study you will restate the research questions. In a quantitative study you will present the hypotheses.

9. PDF Essentials of Descriptive-Interpretive Qualitative Research: A Generic

Therefore, we talk about "generic" or "descriptive-interpretive" approaches to qualitative research that share in common an effort to describe, summarize, and classify what is present in the data, which always, as we explain in Chapter 4, involves a degree of interpretation. 3.

10. Chapter 4: Home

Chapter 4 presents the study findings. It is an overview of the purpose of the research study. This chapter conveys the trustworthiness/validity and reliability of data. It includes the factors impacting the interpretation of data collection or analysis. Students conducting qualitative studies can use NVivo software to analyze data, and SPSS is ...

11. Analysis and Coding Example- Qualitative Data

Step 1: Open Coding. Codes for the qualitative data are created through a line by line analysis of the comments. Codes would be based on the research questions, literature review, and theoretical perspective articulated. Numbering the lines is helpful so that the researcher can make notes regarding which comments they might like to quote in ...

12. Sample-Chapter-4- Qualitative

CHAPTER 4 null. Results and Discussions. Presented in this chapter is the result of the data analysi s. Discussions are also provided to give a comprehensive explanation of the themes that were generated in response to the objectives set in this study. First subheading, based on first Research Objectives Effect of Poor Internet Connection

13. PDF Writing Chapters 4 & 5 of the Research Study

Present Demographics. Present the descriptive data: explaining the age, gender, or relevant related information on the population (describe the sample). Summarize the demographics of the sample, and present in a table format after the narration (Simon, 2006). Otherwise, the table is included as an Appendix and referred to in the narrative of ...

14. PDF Chapter 4 Key Findings and Discussion

Chapter 4 Key Findings and Discussion This chapter presents principal findings from the primary research. The findings can be divided into two groups: qualitative and quantitative results. Figure 4.1 illustrates how these two types of results are integrated. According to this figure, the qualitative results,

15. PDF Dissertation Chapter 4 Sample

older represented 10% of the sample, 35% were between 51 and 60, 20% were between the. ages of 41-50. The 31-40 age group was also 20% of the sample and 15% of the participants. declined to answer. Graphic displays of demographics on company size, work status, age, and industry sector are provided in Appendix F.

16. Chapter 4. Finding a Research Question and Approaches to Qualitative

Chapter 4. Finding a Research Question and Approaches to Qualitative Research ... In order to know if your question is feasible, you are going to have to think a little bit about your entire research design. For example, a question that asks about the real-time impact of COVID restrictions on learning outcomes would require a time machine ...

17. PDF CHAPTER 4 Research design and methodology

Chapter 4 - 13 5 - 4.1.2 Case study as a qualitative research The qualitative approach to research is typically used to answer questions about the nature of phenomena with the purpose of describing and understanding them ... Kos's (1991) research provides an example of an explanatory case study, and an evaluative case study is illustrated by ...

18. CHAPTER 4 FINDINGS 4.1 Qualitative Data Source

4.9 Summary This chapter has obtained satisfactory key findings of the qualitative data analysis collected throughout Phase 1, Phase 2 and Phase 3. The research framework and research questions guided the patterns of the findings, which represented the most relevant results to the research's objectives.

19. LibGuides: Chapter 4: Trustworthiness of Qualitative Data

Trustworthiness of Qualitative Data. As noted in the dissertation template for qualitative studies, the section directly following the Chapter 4 introduction is to be labeled Trustworthiness of the Data, and in this section, qualitative researchers are required to articulate evidence of four primary criteria to ensure trustworthiness of the ...

20. PDF CHAPTER 4 QUALITATIVE STUDY

4.1 THE SAMPLE With reference to the research methodology presented in chapter 1, step 1 of the qualitative study, namely the selection of the sample, is discussed below. The sample that was selected for this research is the senior self-managed group of the Commercial Department of a large bank in the Western Cape. The Commercial

21. (Pdf) Chapter 4 Research Design and Methodology

CHAPTER 4 RESEARCH DESIGN AND METHODOLOGY. September 2021; ... research approach, research sample, data collection methods and analysis of the collected data. ... 4.4.2.1 Qualitative research ...

22. What are the strengths and limitations to utilising creative methods in

There is increasing interest in using patient and public involvement (PPI) in research to improve the quality of healthcare. Ordinarily, traditional methods have been used such as interviews or focus groups. However, these methods tend to engage a similar demographic of people. Thus, creative methods are being developed to involve patients for whom traditional methods are inaccessible or non ...