How to do a systematic review: steps and example

A systematic review answers one focused question using all the studies that meet rules you set before you start. The search, the choice of studies and the analysis are planned in advance and written down, so another researcher could follow the same steps and reach the same set of studies.

Neither is better in every case. A narrative review suits a wide topic; a systematic review suits a precise question where you want the whole body of evidence. Our blog on types of literature review compares them with scoping and umbrella reviews.

Zoya's first question was "Is oral health education useful?" Dr. Patil asked: useful for whom, compared with what, and measured how? The Cochrane Handbook, the standard guide for health reviews, says the scope of a review is defined by the population, the interventions and comparisons, and the outcomes, and uses the acronym PICO as a reminder.

Her new question read: "In school children aged 6 to 12, does oral health education in school reduce dental plaque compared with no education?" Our guide to writing a research question explains how to test a question like this.

The Handbook adds that eligibility criteria are based on the PICO elements plus the types of study that have addressed the question. Zoya added one more rule: randomised and non-randomised controlled trials only. It also warns that outcomes are rarely used to include or exclude studies, so a trial that measured plaque but did not report it clearly should still be found and noted.

A protocol is the written plan of your review. It fixes your decisions before you see any results, so you cannot quietly change the rules to suit what you find. Zoya's protocol was six pages long and covered:

• the question and the PICO, in full
• the inclusion and exclusion criteria
• the databases and the draft search strategy
• how studies will be screened, and by how many people
• what data will be extracted, on a sample form
• the risk of bias tool
• how results will be combined, and when a meta-analysis will or will not be done

Health reviews are often registered on PROSPERO, an international register of systematic reviews run from the University of York. When its founders described it in 2012, registration was free, it covered reviews with a health-related outcome, and it advised registering before formal screening began. Check the current rules on its website before you submit.

Outside health, ask your guide where reviews in your field record their protocols. If there is no register, attach the dated protocol to your thesis or share it with your guide by email, so the plan exists before the results do.

A systematic review lives or dies by its search. The Cochrane Handbook says that a search of MEDLINE alone is not considered adequate, and that searches should aim for high sensitivity, even if that means many irrelevant results. In other words, it is better to screen out 900 unrelated papers than to miss one that matters.

1. Build the search from your PICO.List synonyms for each element, join synonyms with OR and join the elements with AND. Our blog on searching academic databases shows how Boolean searching works.
2. Use subject headings as well as words.The Handbook asks for both free text and subject headings, such as MeSH in PubMed, because authors describe the same thing in different words.
3. Search several databases.Zoya searched PubMed, Scopus, the Cochrane Library and Google Scholar. Which databases suit you depends on your field.
4. Look beyond journals.The Handbook suggests grey literature such as reports, dissertations, theses and conference abstracts, and checking the reference lists of included studies and earlier reviews. Zoya searched Shodhganga for Indian theses too.
5. Record every search.The database, the date, the exact search string and the number of results. You will need these numbers for the PRISMA flow diagram.

The Handbook also strongly recommends that the search strategy be peer reviewed before it is run, ideally with a librarian or information specialist. Zoya asked her college librarian, who added three synonyms she had missed.

Zoya's searches found 1,146 records. After removing duplicates she had 812. Now she had to decide, record by record, which ones met her criteria. The Handbook describes the process in two main stages.

1. Titles and abstracts.Remove reports that are obviously irrelevant. The Handbook says to be over-inclusive here: when in doubt, keep the record for the next stage.
2. Full texts.Read each remaining report in full and check it against every criterion. A single failed criterion is enough to exclude a study. Write down the reason each time.

Cochrane requires at least two people working independently to decide whether each study meets the criteria, with the method for settling disagreements fixed in advance. Title and abstract screening by two people is ideal, but one person is acceptable at that first stage. The full-text decision must be made by two.

Zoya screened with Amrita, a classmate, and they each worked alone before comparing. They disagreed on nine studies, and settled all of them by discussion, with Dr. Patil as the tiebreaker the protocol named.

For each included study, Zoya filled the same extraction form: authors and year, country, number and age of children, what the education involved, how long it lasted, the plaque index used, and the results. A literature review matrix is a good shape for this, one study per row. Test the form on two or three studies first and fix it before doing the rest.

Next, judge how much you can trust each result. For randomised trials, Cochrane recommends the RoB 2 tool. It looks at five domains of bias:

1. bias arising from the randomisation process
2. bias due to deviations from the intended interventions
3. bias due to missing outcome data
4. bias in measurement of the outcome
5. bias in selection of the reported result

Each domain is judged as low risk, some concerns or high risk. The overall judgement for a result is the least favourable one across the domains, so one high risk domain makes the whole result high risk. Non-randomised studies need a different tool, and your protocol should name it.

Synthesis means combining what the studies found. If the studies are similar enough in children, programme and measurement, a meta-analysis can pool their numbers into one estimate. If they are not, you compare them in words and tables, grouped in a sensible way. Zoya's trials used different plaque indices and very different programmes, so she combined them in words and tables, without pooling the numbers.

Then report the review with PRISMA 2020, the reporting guideline published by Matthew Page and colleagues in the BMJ in 2021. Its checklist has 27 items in seven sections, from the title and abstract to the discussion and other information. Item 8, for example, asks you to say how many reviewers screened each record and whether they worked independently.

The PRISMA flow diagram maps the number of records identified, included and excluded, with the reasons for exclusion. The PRISMA website offers four templates: for new or updated reviews, and for searches of databases and registers only or of other sources as well. Here is the shape of Zoya's.

The numbers are made up for this blog, but the arithmetic is the point: every box must add up. If 812 records were screened and 761 excluded, exactly 51 full texts go forward.

Zoya's review found that most trials reported less plaque after school education, but many had concerns about measurement, so her conclusion was cautious. She went back to the headmaster with a careful answer: probably yes, in the short term, and better trials are needed. For writing up the background and discussion around such a review, see how to write a literature review.