Maureen Hannley, PhD, is currently Chief of the Research Division of the Department of Otolaryngology and Communication Sciences at Medical College of Wisconsin and Research Consultant for the Triological Society. She formerly served as the Chief Research Officer of the American Academy of Otolaryngology- Head and Neck Surgery Foundation and has held positions at the National Institutes of Health, Stanford University Medical School, and Arizona State University.
Explore This IssueApril 2008
Disease-specific outcomes measures in otolaryngology-head and neck surgery can be completed by your patients before and after treatment, enabling tracking of these important outcomes with a minimum of disruption to the normal practice routine (see ENT Today, January 2008, p. 8).
The first step in setting up such an outcomes assessment program in your practice-the very first step-is to make a decision and a commitment to begin. In today’s busy world, where the demands of a medical practice have become so complex, adding another project may seem out of the question. Therefore, your decision must derive from personal or practice-based motivations that you anticipate will be productive and satisfying. These may range from demonstrating to your patients that you are concerned about their overall health-related quality of life; gathering evidence of the effectiveness of a procedure for patient education, marketing, or reimbursement purposes; collecting data for your Triological Society thesis; and/or simply introducing an interesting new element into a practice that has grown predictable and unexciting. Enlisting the collaboration of one or more of your colleagues can often sustain the impetus and motivation through all phases of the project and ensure that critical details are not overlooked.
Next, decide what you want to investigate. Identify the condition or procedure in which you are interested and define the research question. These two critical steps will help you focus your efforts and allow you to design a project that has a beginning and an end, rather than simply having an open-ended data-collection machine that is mined for information at some point in the future. Selecting a topic may be as simple as pursuing an interesting association or effect that you’ve noticed clinically, or delving into a problem that has concerned or frustrated you. Maybe you’ve developed a new approach or modification to an existing procedure and want to evaluate its effectiveness in a more structured way. Ask yourself, What do I want to know? Why is it important to know this?
Once you have decided on a general topic, it is time to narrow it down to a specific question. Keep some fairly obvious pointers in mind at this stage:
- Ask a question that truly interests you and will sustain your interest through the long haul of collecting and analyzing the data.
- Ask a question that has clinical relevance, not just clinical application-does it have the big so-what?
- Ask a question in such a way that either a positive or negative answer will be of interest.
- Be specific in the way you construct your question.
A useful device for constructing the research question is to use the PICO approach. This acronym, common in evidence-based medicine, will remind you to construct a research question that specifies the Patient or Population of interest, the Intervention that will be applied, the Comparison population or intervention (if one is used; this is not always necessary); and the Outcome of interest. Do not neglect this step. It is a necessary one to focus your study, narrowing it to its essentials-the structure of the question will define the research design (cross-sectional, cohort, case-control) and hence the type of statistical analysis that will be necessary, and it will help you avoid the collection of extraneous data.
In designing your question, make sure that you have selected one for which you have the adequate resources-access to an adequate number of the clinical patients of interest being paramount. Otherwise, you may find yourself a victim of Lasagna’s Law: The incidence of patient availability sharply decreases when a [clinical study] begins and returns to its original level as soon as the trial is completed.1 Define your patient population carefully, specifying inclusion and exclusion criteria for participating in the study-inclusion criteria that are too strict will make it very difficult to recruit sufficient numbers of subjects; criteria that are too broad will result in including subjects who have comorbidities or other factors that may confound the results of the study.
Having defined your area of interest and narrowed that down to a specific, focused research question, you are ready to define the scope of the project. This will require making several decisions related to numbers, time, and complexity.
How many patients do you want to recruit for your study? It is important to specify a number at the beginning, and this number should not be selected randomly. The sample needs to be large enough to provide some assurance that if an effect is found, either between two groups or before and after treatment, they represent a real effect and not samples drawn at random from the same statistical population. If the sample is too small, you may fail to detect a clinically important difference; too large, and the study will be longer and more expensive than necessary. An excellent method for determining sample size can be found in Norman and Streiner’s PDQ Statistics.2 The gold standard, however, is to invest in several hours of consulting time with a biostatistician from a local university or medical school. Based on the question you want to investigate, the statistician will reinforce your choice of design (or suggest options); calculate the appropriate sample size needed based on the population variance, effect size, and desired power; and, if you have established a satisfactory working relationship, can assist with statistical analysis of the data at the conclusion of the study.
For the time parameters, decide for how long you will actively recruit patients into your study, and how long the follow-up period for each will be, if you have decided on a design other than a cross-sectional study. For the newcomer to clinical research, designing and undertaking a simple, straightforward study with a limited number of variables that can be quantified and measured objectively is always a better way to start.
Foundation and Framework for the Study
To be successful, any clinical research project needs a strong infrastructure; the heart of that infrastructure is a project coordinator. Although the physician conceptualizes the project and has ultimate responsibility for it, the project coordinator-usually a nurse or other health care professional in the office-brings together all the skills and resources to carry out the project. He or she serves as a task manager and liaison among clinicians, clinical staff, and patients, and must have superb interpersonal communication skills. Invite someone to serve in this role who has not only the skills, but also the willingness, enthusiasm, and dependability to see the project through.
Once you have selected the topic of your question and focused it with a research question, decided on the numbers of patients that you need, selected an outcomes instrument, set the time parameters of your study, and appointed a project coordinator, you should consider and formalize the logistics of the study so that it will interfere as little as possible with normal office routine. This would include:
- When will the data be collected (before and after a treatment, prior to an encounter)?
- How often will be data be collected (every office visit, every three months, once a year)?
- Where will the data be collected (in the office, sent out by mail or e-mailed)?
- Who will analyze the data collected from the patient questionnaires (the physician, the project coordinator, etc.)?3
Any time data are collected from or about patients for research purposes, they must give informed consent to participate in that research and the investigator must obtain institutional review board (IRB) consent for the research. All procedures and consent forms must adhere to HIPAA privacy regulations. If you are not affiliated with a university, IRB review can be obtained through a hospital or an independent board such as the Western IRB (WIRB). For a superb reference, including sample patient consent forms and procedures for IRB applications, see Stone’s Conducting Clinical Research.4 It is a refreshingly pragmatic handbook/guide to every step of the process by an experienced physician researcher.
Conducting the Study
Once collected, the data should be entered to a database for easy analysis. The most familiar and commonly available is Excel™, although others such as Access™ and Paradox™ or others could be used, depending on the investigator’s preference. Data must be checked periodically for accuracy, completeness, and logic-if possible, the database should be programmed to check for values that are missing, out of the measurement range (e.g., entering 100, for an actual range of 1-10), or illogical.
Be sure to check the clinic schedule periodically to assure that all eligible patients have been included in the project. Providing office personnel with a laminated, pocket-size card printed with eligibility criteria may help. A good eye for quality control is essential to ensure the integrity of the data set.
Most database programs have the ability to conduct simple analyses of the data: means, medians, standard deviations, simple t-tests, correlations, and analysis of variance (ANOVA). Data analysis may be aggregate or patient-specific, and you would need to consider whether the data are normally distributed before determining which statistical treatment would be appropriate.
Consultation at the beginning, planning stages of the project with a professional biostatistician is undoubtedly a wise investment for the clinician who does not have a working knowledge of statistics. Data analysis requires statistical expertise, and interpretation of findings requires clinical expertise.5 It is always important to bear in mind that statistical significance may or may not coincide with clinical significance.
The presentation of results will depend on the original motivation for the study; you may want to generate a report in layperson’s language for your patients, especially if the results demonstrate positive impact of an intervention on quality of life and/or activities of daily living. The ultimate goal of patient-reported outcomes assessment, however, is to evaluate and improve the effectiveness of the care process; linking outcome information to clinical process will identify opportunities for process improvement, meet and perhaps exceed patient expectations for satisfaction and improved health status, and contribute higher level evidence supporting the effectiveness of otolaryngology-head and neck surgery interventions to the literature.
To Reach ENT Today
For Editorial, Permissions, or Publishing Matters:
Lippincott Williams & Wilkins 333 7th Avenue, 19th Floor New York, NY 10001 Phone: (646) 674-6544 Fax: (646) 674-6500 ENToday@lwwny.com
For Advertising Sales:
Michael GuireLippincott Williams & Wilkins1300 Virginia Drive, Suite 400Ft. Washington, PA 19034Phone: (215) 643-8140 Fax: (215) 643-3902 Michael.Guire@wolterskluwer.com
- Spilker B. Guide to Clinical Trials. New York: Raven Press, 1991, p. 87.
- Norman GR, Streiner DL. PDQ Statistics (3rd ed.) Hamilton, Ontario: BC Decker, 2003.
- Davies AR, Doyle MAT, Lansky D, Rutt W, et al. Outcomes assessment in clinical settings: A consensus statement on principles and best practices in project management. J Quality Improvement 1994;20(1):5-16.
- Stone J. Conducting Clinical Research: A Practical Guide for Physicians, Nurses, Study Coordinators, and Investigators. Cumberland, MD: Mountainside MD Press, 2006.
- Davies et al., op cit.
©2008 The Triological Society