The Cochrane handbook expresses the importance of the development of a protocol.
“Publication of a protocol for a review prior to knowledge of the available studies reduces the impact
of review authors’ biases, promotes transparency of methods and processes, reduces the potential for
duplication, allows peer review of the planned methods before they have been completed, and
offers an opportunity for the review team to plan resources and logistics for undertaking the review itself”.
PLoS Medicine editors published the following article on the importance of protocols in systematic reviews:
Best Practice in Systematic Reviews: The Importance of Protocols and Registration (2011) PLoS Med 8(2): e1001009. doi:10.1371/journal.pmed.1001009
The development of a review protocol not only helps in establishing the researchers approach to the systematic review, but can also act as a 'live' document that the researcher can draw on and use during the lifetime of the review.
A systematic review is an important piece of research therefore a protocol should clearly define the journey the systematic review will take.
The protocol should include:
There are several standards and guidelines available to assist in developing the protocol for the systematic review.
The inclusion and exclusion criteria are critical components in a systematic review, as the search needs to be replicable. Researchers need to ensure that they are clear on the exact results they want to locate from their search.
The criteria help in eliminating any bias during the selection and extraction phases.
Some examples of what you may include in criteria might be:
Below are links to tools that can be used in the processes of screening and selection, quality assessment and data extraction.
Covidence – collaborative screening and data extraction tool
EndNote – reference management software
While the protocol is in the draft phase, advice needs to be sought from other members of the review team, (experts in the topic area, statistician and key stakeholders), enabling researchers to make any changes to the protocol before the commencement of the research.
However, it can be difficult to stick rigidly to the original protocol. Primary research may raise questions that were not evident during the writing of the original protocol. When researchers identify a change is required, e.g. the result may not be deemed useful to the end user, the researcher must write a clear justification for making amendments to the protocol.
When making amendments researchers will need to consider the impact these amendments will have on all areas of the review process such as search strategy, data extraction and the initial timeframe of the review process.
It is not appropriate to make modifications to the original protocol. Amendments should be recorded clearly so they are identifiable. Making changes to the protocol throughout the systematic review process can be viewed as presenting a bias, affecting the validity of the conclusions presented by the authors.
When writing up the results of their systematic review, researchers need to make sure any amendments made to protocol from the original research question are discussed.