Approach

From the NIH Guidelines:

"Are the conceptual or clinical framework, design, methods, and analyses adequately developed, well integrated, well reasoned, and appropriate to the aims of the project?  Does the applicant acknowledge potential problem areas and consider alternative tactics?"

Reviewers of qualitative grant proposal should expect:

• the qualitative method the researcher proposes to use fits the research question or problem being addressed - the research question dictates method choices. Some general guidelines to consider:

• Research questions designed to understand the beliefs, feelings, perceptions of a group of people generally require investigators to ask members of a group questions via an interview or focus group. 

• Research questions designed to understand the behaviors of a group of people generally require some type of observational method.  

• Research questions designed to understand the culture of a group may require a combination of observational and interviewing methods.  Researchers will also want to consider collecting material artifacts produced and used by members of a culture.

There are variety of approaches for interviewing and observing people. 

The goals of the project will often shape the type of interviewing and/or observational method chosen.  There are, however, other important considerations (budget, time and access) that influence methods decisions.

• A rationale for method choices should be provided, such that it is clear the best method has been selected to provide the information needed

The issue of bias

The grant proposal should describe the setting, participants, and or situations that will be studied.  These choices should be explained.  Often, the goal in qualitative research is to maximize the opportunity to observe the phenomenon of interest

• it is important for reviewers to understand that in many qualitative studies it is best to maximize 'bias' because this allows researchers to gather the best examples of the phenomenon of interest

Morse (2003) suggests asking the following questions:

• will the setting optimally allow for examination of the phenomenon of interest

• will the participants selected be the best examples

• will the events observed be the most representative of the phenomenon of interest

• Reviewers should NOT necessarily expect that the events selected will be representative of the population

Sampling Issues

Reviewers should expect the grant to give approximate values regarding the size of the sample required to identify clear themes, and/or develop theory. 

• The sample selected must be one that will allow the researcher to address the research question posed.

• Theoretical or purposeful sampling is generally the most highly regarded sampling method in qualitative research.  However, there are a range of methods for generating a qualitative sample.  For more information click here. 

• Sample size considerations in qualitative research focus on achieving saturation.  A good study design will build in the flexibility needed to evaluate when saturation is reached.  Typically, this involves moving between data colletion, data analysis and reflection. For more on iterative sampling and saturation, click here.

• While qualitative researchers cannot predict how large the sample size will need to be to reach saturation, they can:

• Use sample sizes from prior studies to approximate what kind of sample size will be needed in the proposed study

• Dicuss the scope of the study, use the literature to discuss the expected variation in the setting or phenonmeon and provide a rationale for sampling expectations

Proposed design 

Are the proposed methods and strategies well justified?  Will they answer the proposed research question?

Evaluators should expect:

A detailed description of how the data will be collected and analyzed that includes, for example:

• details about the type of interviews that will be conducted; how interview guides will be created, tested and refined; when interviews will be conducted; by whom they will be conducted; how the interviews will be recorded; how interviews will be transcribed...

• details of how focus group participants will be selected; who will lead them; will they be heterogeneous or homogenous focus groups; plans for how data will be analyzed...

• if observing, how will observations be done (by a field worker, by audio or video); how will the person be positioned; what will be selected for observation; how will field notes (if using) be prepared; how will audio or video be recorded, saved, and transcribed...

Different methods pose different types of considerations.  Above, we identify just a few of the issues that will need description for studies using interview, focus group or observational methods. Reviewers should expect a detailed account of the proposed steps for managing and collecting data.

• For more on the types of methodological issues that need to be considered, click here.

The validity issue

There is fairly strong agreement among qualitative researchers in the health care field that good qualitative research should be evaluated in terms of its completeness, adequacy and trustworthiness.  In other words, another researcher should be able to understand and appreciate the account and feel it is trustworthy. 

• For more on the evaluation of qualitative research please click here.

• There are several techniques for verifying qualitative accounts.  Many scholars cited by in health care researcher suggest that one or more of these techniques should be intergrated into the design of a qualitative study. These techniques are:

• triangulation
• prolonged engagement
• persistent observation
• referential adequacy
• peer debriefings
• thick description
• member checking
• external audits
• Reflexivity
• searching for confirming or disconfirming or deviant or negative cases
• examination of rival explanations

Data Analysis

As Morse (2003) suggests, reviewers should be able to evalute whether the proposed study will:

• answer the research question

• produce the necessary results

• produce the degree of certainty or depth necessary for answering the question 

One of the common pitfalls in analyzing qualitative data is that the researchers fail to develop an in depth analysis of the data and interpret the meaning of the rich data they have collected. 

For more on data analysis please click here.

In general it is recommended that that the analysis strategy proposed describe a process that:

• Allows for sample generation and analysis to co-occur or take place iteratively and continue until saturation is reached

Click here for more on iterative sampling

• Allows adequate time for researchers to immerse themselves in analyzing the details of the data, intermittently emerging from this process in order to identify and refine emergent themes

• Identifies a multidiscipinary team that is committed to take the time to analyze data

• It is feasible to propose that day-to-day analyses occur through telephone conferencing.  However, the research team should plan on a number of long, face-to-face team meetings in order to accomplish data synthesis

• Researchers should propose to take steps to identify and analyze cases that disconfirm emergent understandings.  This lends strength to one's argument and can also lead to a more refined and developed understanding of a phenomenon.

Click here for more on analyzing disconfirming cases. 

• Researchers should describe, in adequate detail, the analytical steps to be taken in a study.  This account should limit the use of jargon and explain the steps and the rationale in plain language.  This should include a plan for how data will be managed, coded and analyzed. These steps should be congruent with the proposed methods.

From the NIH guidelines: "Does the application acknowledge potential problem areas and consider alternative tactics?" 

We would recommend rephrasing this question for qualitative work as follows: 

Does the application recognize that potential problems are likely to arise, and uncovering unantipated understandings is the purpose of qualitative research?  To address this, has the researcher:

• built in the time and process for reflection such that new information and understandings as well as problems can be recognized when they arise? 

• Is there flexibility in the study's design and timeline in order to change course as warranted? 

One thing that the qualitative researcher can count on when conducting a study is that something unanticipated and unknowable at the study's outset will arise.  This is the nature of qualitative research.

As such, reviewers need to understand this. 

Features of flexible study design

The following are a few features of flexible study designs.  These are study design features that grant reviewers may look for when evaluating a proposal:

• An ongoing evaluative process that involves preliminary analysis of data and periodic assessment of data collection processes.  Ideally, this would be accomplished semi-independently of those collecting data.  The purpose is to provide the research team with the time to be reflective.  This can provide the insight needed to identify new understandings and new directions as well as to identify potential problems.

• Does the study identify a number of 'outsiders' who will provide a fresh perspective when the internal team is entrenched in the research process?  Are there adequate opportunities to meet with these outsiders and gain their insight?  Are these opportunities timed in ways that, if a problem should be identified, a change in the study protocol is still possible? 

• Does data collection and analysis iterate?  Early analysis of data can allow the research team the opportunity to make real time changes that refine and improve:

• Interview guides

• Observation strategies

• Sampling choices

• Refine training

• Reposition personnel

Click here to return to Guidelines page