Systematic Review

The importance of a protocol

The Cochrane handbook expresses the importance of the development of a protocol.

“Publication of a protocol for a review prior to knowledge of the available studies reduces the impact

of review authors’ biases, promotes transparency of methods and processes, reduces the potential for

duplication, allows peer review of the planned methods before they have been completed, and

offers an opportunity for the review team to plan resources and logistics for undertaking the review itself”.

Cochrane Handbook, Section 1.5

 

PLoS Medicine editors published the following article on the importance of protocols in systematic reviews:

Best Practice in Systematic Reviews: The Importance of Protocols and Registration (2011) PLoS Med 8(2): e1001009. doi:10.1371/journal.pmed.1001009

Developing a Systematic Review Protocol

The development of a review protocol not only helps in establishing the researchers approach to the systematic review, but can also act as a 'live' document that the researcher can draw on and use during the lifetime of the review.

A systematic review is an important piece of research therefore a protocol should clearly define the journey the systematic review will take.

The protocol should include:

  • Background
  • Summary of current literature
  • The research question and the aims of the research
  • Method
    • The search strategy
      • Inclusion and exclusion
      • Resources to be searched
    • Screening and selecting
      • Who is going to be involved in this process
      • Criteria to be implemented during this process
    • Quality assessment
      • How do you plan to assess the quality of the included studies
    • Data Extraction
      • What tool you plan to use for extracting data
    • Data Analysis
      • Need to clarify if you intend to undertake a meta-analysis
    • Time frame
      • How long to you expect to take in completing the review
      • Development of a Gantt chart can assist in keeping to timeframes.

 

There are several standards and guidelines available to assist in developing the protocol for the systematic review.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)

Cochrane

Campbell Collaboration

Joanna Briggs Institute

Inclusion and Exclusion Criteria

The inclusion and exclusion criteria are critical components in a systematic review, as the search needs to be replicable. Researchers need to ensure that they are clear on the exact results they want to locate from their search.

The criteria help in eliminating any bias during the selection and extraction phases.

Some examples of what you may include in criteria might be:

 

age groups, language, publication year, study design

Tools for consideration

Below are links to tools that can be used in the processes of screening and selection, quality assessment and data extraction.

Critical Appraisal Tools – University of South Australia

Understanding critical appraisal – critical appraisal tools

Covidence – collaborative screening and data extraction tool

EndNote – reference management software

Protocol draft and amendments

While the protocol is in the draft phase, advice needs to be sought from other members of the review team, (experts in the topic area, statistician and key stakeholders), enabling researchers to make any changes to the protocol before the commencement of the research.

However, it can be difficult to stick rigidly to the original protocol. Primary research may raise questions that were not evident during the writing of the original protocol. When researchers identify a change is required, e.g. the result may not be deemed useful to the end user, the researcher must write a clear justification for making amendments to the protocol.

When making amendments researchers will need to consider the impact these amendments will have on all areas of the review process such as search strategy, data extraction and the initial timeframe of the review process.

It is not appropriate to make modifications to the original protocol. Amendments should be recorded clearly so they are identifiable. Making changes to the protocol throughout the systematic review process can be viewed as presenting a bias, affecting the validity of the conclusions presented by the authors.

When writing up the results of their systematic review, researchers need to make sure any amendments made to protocol from the original research question are discussed.

Registering your protocol

Once the protocol has been developed it is important to register the systematic review within databases such as the International prospective register of systematic reviews (PROSPERO)