Skip to Main Content

Research Literature Reviews: How?

Fink's Seven Steps to Success

  1. Select your research question(s):  a precisely stated question that guides the review. (p 21)

     

    1. Example A:  How does alcohol use affect breast cancer? (less specific)
    2. Example B:  What is the relationship between drinking two or more alcoholic beverages daily in women 65 years of age and older and breast cancer? (more specific)    

      Example B is a better research question due its clarity of variable (two or more alcoholic drinks per day) and target population of interest (women 65 years of age and over) (p 21).


Be specific in your vision!

Select bibliographic or articles databases, websites, and other sources (p 18-19).

Since Massasoit is a community college,  students should take advantage of the free-to-search databases on the internet

  1. Google Scholar - 150 million sources - free site that mirrors Web of Science - interdisciplinary
  2. Mendeley - 100 million sources - crowdsourced by authors/researchers - interdisciplinary
  3. BioOne - 180 scientific journals with only free abstracts/references - life sciences.
  4. Science.gov - US government clearinghouse for federal agency research - interdisciplinary
  5. PubMed - US Library of Medicine - biomedical & health sciences

Choose search terms: (p.21-26)

  1. Keywords - generated by the researcher from initial research question
    1. What is the relationship between drinking two or more alcoholic beverages daily in women 65 years of age and older and breast cancer? 
      1. keywords:  women 65 years of age or older, breast cancer, alcoholic beverages
      2. 50,022 results in PubMed
         
  2. Thesaurus - controlled vocabulary issued by the publisher to index the database articles across disciplinary fields that may use different terms for the same concept;  generally called descriptors, subject headings, MESH terms, identifiers, etc. 
    1. What is the relationship between drinking two or more alcoholic beverages daily in women 65 years of age and older and breast cancer?
      1. PubMed's MESH terms: ("Aged"[Mesh] OR "Aged, 80 and over"[Mesh]) AND "Alcoholic Beverages"[Mesh] AND "Breast Neoplasms"[Mesh]
      2. 21 results in PubMed

Apply practical screening criteria - limits scope of literature reviewed by adding parameters for article inclusion or exclusion: (p. 53-54)

  1. Publication language                                  7. Research design
  2. Journal                                                        8. Sampling
  3. Author                                                         9. Date of publication
  4. Setting                                                        10. Date of data collection
  5. Participants/subjects                                  11. Content including topics & variables
  6. Program/intervention                                  12. Sources of funding

Apply methodological screening criteria to select highest quality studies possible: (p. 56-63) 

To find the highest quality articles,  reviewers should ask the following questions first (p. 56):

  1. Is this study's research design internally & externally valid?
  2. Are the data sources used in the study reliable & valid?
  3. Are the analytical methods appropriate given the characteristics & quality of the study's data?
  4. Are the results meaningful in practical & statistical terms?

Factors to keep in mind (p. 57-98):

  1. Determine research study design - the way the subjects of a study are organized and measured
    1. Experimental - uses a control group vs subjects undergoing the new program or intervention - adds new knowledge - more vigorous.  Examples include:
      1. Randomized controlled trial - subjects assigned to groups randomly
      2. Parallel controls or nonrandomized controlled trials - subjects not assigned to groups randomly 
      3. Self-control or longitudinal - pretests before and posttests after intervention
      4. Historical Control - use of normative (preexisting) data compared to a group
         
    2. Observational - uses existing information to draw conclusions on past or existing conditions & activities - describes current conditions - less vigorous
      1. Cohort - provide data about changes in a specific population
      2. Case Controls - determine similarities & differences between two groups (i.e. one with a medical condition,  the other without)
      3. Cross-section - descriptive or survey data at one fixed point in time
         
  2. Address bias within the study - arise from unanticipated and/or unrecognized characteristics
    1. Selection bias -  occurs when selection of experiment subjects leads to a result that is systematically different to the target population.
    2. Randomization - gold standard in reducing biases at the start of the experiment
    3. Blinding - gold standard for reducing bias during the experiment
       
  3. External validity - experimental results apply to the target population - avoid the following: (p. 82-84)
    1. Reactive effects of testing - a premeasure can sensitize participants to the aims of an intervention
    2. Interactive effects of selection - an intervention and the study's participants are a unique mixture 
    3. Reactive effects of innovation - study subjects act uncharacteristically due to the artificiality of the experiment
    4. Multiple-program interference - difficulty in isolating the effects of the intervention because of the possibility that subjects are in other complementary activities or programs. 
       
  4. Internal validity means the study is free from nonrandom error or bias  - below are the potential threats to study accuracy (p. 81-82)
    1. Maturation - age of subjects over course of experiment & aging's effect on the results
    2. Selection - how people are chosen & assigned to groups for the study 
    3. History - events in the world can bias study results
    4. Instrumentation - measures used to collect data are dependable
    5. Statistical regression - tendency of very high or low values to move toward the mean or average.
    6. Attrition - loss of data from subject's inability to finish study (they move, die, etc.)
       
  5. Sampling - a portion or subset of a larger group called a population (p 84-98)
    1. Random sampling - every member of a target population has an equal chance of being selected - least biased
    2. Systematic sampling - selecting every 5th, 10th or 100th subject from a randomized list
    3. Stratified sampling - population is divided into subgroups (strata) and a random sample then selected from each strata
    4. Cluster sampling - naturally occurring group such as schools, clinics, cities, states, etc.
    5. Convenience samples - probability of selection is unknown

Two types of reviewed Information to ensure quality of evidence within article:

  1. Methods
    1. research design
    2. sampling
    3. data collection
    4. analysis
  2. Content
    1. objectives
    2. participants
    3. settings
    4. interventions
    5. results
    6. findings
    7. conclusion
  3.  
  4. Eligibility
    1. contains relevant information us accessible
    2. meets preset standards for methodological quality
    3. does not have any features that justify its exclusion
  5. Actuality

Quick Links

More Methodological Concerns

I. Data Collection & Sources:  Methods & Measurement 

 Checklist - Evaluating Reliability & Validity in Data Collection

  • Are the data collection methods adequately described?
  • Define all key variables
  • Provide information on measure type, content, length
  • Explain & justify intervals between administrations 
  1. Reliability - free of measurement error 
    1. Test-retest reliability:  high correlation between scores on occasion
    2. Equivalence or alternate-form reliability: two assessments measure the same concepts at the same level of difficulty
    3. Homogeneity or internal consistency: all items or questions assess the same skill, characteristic or quality
    4. Interrater reliability: extent to which two or more individuals agree on their measurement of an item.
       
  2. Validity - degree to which a measure accurately assesses its intended measurement
    1. Content validity - degree to which a measure thoroughly & appropriately assess the skills or characteristics it intended to measure
    2. Face validity - extent to which a measure appears on the surface
    3. Criterion validity -  two subcategories:
      1. Predictive validity - extent to which a measure forecasts future performance
      2. Concurrent validity - demonstrated when two assessments agree, or a new measure compares favorably with one already found valid
    4. Construct validity - established experimentally to demonstrate a measure distinguishes between people who do and do not have certain characteristics - established in at least two ways:
      1. Convergent validity - new measure correlates with one or more measures of a similar characteristics
      2. Discriminant validity - new measure does not correlates with dissimilar characteristics

 

Data Analysis: Statistical Methods

II. Data Analysis  

 Checklist - Evaluating a Study's Data Analysis

  • Are research questions clearly stated?
  • Are the independent (predictor) variables defined? Are the dependent (outcome) variables defined?
  • Do the researchers explain the type of data obtained from measures of the independent and dependent variables?
  • Are statistical methods adequately described?
  • Is a reference provided for the statistical program used to analyze the data?
  • Are statistical methods justified?
  • Is the purpose of the analysis clear? 
  • Are scoring systems described?
  • Are potential confounders adequately controlled for in the analysis?
  • Are analytic specifications of the independent and dependent variables consistent with the evaluation questions or hypotheses under study?
  • Is the unit of analysis specified clearly?
  • If statistical tests are used t6o determine difference, is practical significance discussed?
  • Is statistical tests are used to determine differences, is the actual p value given?
  • If the study is concerned with differences among groups, are confidence limits given describing the magnitude of any observed differences?