Realist review of low- to upper-middle-income country experiences on integration of HPV vaccination with other adolescent health services
Introduction In 2020, the World Health Assembly adopted a global strategy to accelerate the elimination of cervical cancer as a public health problem, aiming to shift the needle on one of the most common cancers and causes of cancer-related death in women across the globe, through three pillars: vaccination, screening, and treatment, setting specific scale-up targets for each by 2030 . However, progress toward these targets has been uneven, particularly in low- and lower-middle-income countries (LMICs), where women experience 84 % of the ∼660,000 new annual cases and between 87 and 90 % of the annual 350,000 deaths - incidence that is double and death rates that are roughly triple those of high-income countries (2022 data) . Along with low rates of screening and treatment, HPV vaccination coverage remains low, at 10 % across the 57 Gavi-supported countries in 2022 . Varied stakeholders have proposed that integration of HPV vaccination into Primary Health Care (PHC) and routine vaccination programs may enhance HPV vaccine coverage, programme sustainability , and ability to support uptake among local priority populations such as out-of-school girls, migratory or conflict-affected populations, or youth living with HIV , while harnessing new resources to better equip health systems to meet the underserved needs of adolescent populations . Although primary health care services available for adolescents span a wide range, from nutrition and vision screening to age-appropriate sexual and reproductive health screening and education , a common factor across countries is low and inconsistent uptake . A number of low- to upper-middle-income countries have piloted  or undertaken the integration of HPV vaccine into existing adolescent health programs, or have undertaken the integration of other adolescent health services into HPV vaccine delivery efforts [ , , , ]. However, the theoretical constructs describing why an integrated service delivery approach may contribute to intended outcomes, in which contexts, and for which populations, have not been fully explored; there have been limited investments; and current policy guidance lacks implementation evidence . As countries develop strategies to revitalize their national HPV vaccine service delivery, an action-oriented synthesis of operational evidence is urgently needed to articulate how and why integrated approaches might strengthen both adolescent health programs as well as HPV vaccine programs, now and into the future, and how integration may facilitate pro-equity and sustainable programs and health systems. To address this gap in data to guide policy and programs on integrated service delivery for HPV vaccination, we conducted a realist synthesis review to address two specific implementation research questions: 1. Through which mechanisms is HPV vaccination being effectively integrated with other adolescent health services? 2. What contextual factors show evidence of contributing to the outcomes documented for integrated approaches? 

Methods A realist synthesis methodology is uniquely appropriate to generate an action-oriented synthesis of a complex body of data , and the method places particular emphasis on understanding causal pathways . We selected this methodology and approach to develop a better understanding of how and why integrated service delivery may lead to desired programmatic outcomes for HPV vaccine programs and adolescent health services. To ground our realist synthesis approach in practical theory and to align with common realist practice , our review team identified and invited a selected group of stakeholders who are experts in the field of HPV vaccine implementation and cervical cancer prevention, drawing representatives from the Coalition to strengthen the HPV Immunization Community (CHIC) , US Agency for International Development (USAID), Gavi and WHO, to serve as review commissioners. Following the realist synthesis approach, we worked with these commissioners to develop and iterate a programme theory of change (TOC) that articulates the specific context , such as service delivery setting, in which a particular mechanism of change, such as co-delivery of deworming medications alongside HPV vaccination, is associated with a relevant outcome – such as a change in awareness or uptake of services . Commissioners supported formulation of research questions; advised on research methods; and responded to findings of the review in an ongoing manner. This shaped interpretation and synthesis of results, and guided testing of the TOC, aiming to discern where implementation evidence may be sufficient to translate into action, where there are gaps, and how to evolve the programme TOC – all intended to deepen understanding of how context and causal processes (“mechanisms”) interact to contribute to an outcome . With commissioner input, we developed an initial search strategy to query peer reviewed databases, to select a small number of representative “reference” articles, and to assess relevant grey literature sources. Commissioners submitted lists and websites with potential grey literature to the review team, including programme evaluations, policy documents, and reports from HPV vaccine integration experiences. We worked with an academic library informationist to develop a search strategy of MeSH terms and Boolean operators for PubMed, Embase, and Scopus databases (Appendix I). We validated our search findings and used a “snowball” approach to search reference lists of retrieved articles and to screen bibliographies of a short list of relevant publications [ , , , ] identified by commissioners. We managed screening workflow for peer reviewed articles with Covidence, a web-based collaboration software platform that streamlines literature review processes . We ran the peer-reviewed database query March 03, 2022 and implemented the full search strategy between March 2022 and September 2022. We included studies that: • documented an intervention involving the integration of HPV vaccines with other adolescent health services; • delivered services to an adolescent or youth population; • measured a programme outcome (examples included: feasibility, acceptability, cost, coverage, uptake); • were original research; and • were considered of admissible quality. Review articles, studies involving only adults, studies reporting work conducted only in high-income countries as per World Bank's 2021 categories, and non-human research studies were excluded. We assessed these criteria in the initial screening phase through a list of 9 questions, refining the question list after piloting, and using a similar, refined set of questions for article selection and final appraisal for data extraction. We developed a data abstraction and analysis matrix to identify key context, mechanism and outcome factors from each article. Commissioners provided input and discussion to guide the definition of these terms and the factors that we extracted, throughout the process of extraction and during initial analysis. Two reviewers independently extracted data from each paper. A third reviewer arbitrated the reconciliation of differences in data extracted. In order to assess study quality, we classified each article by its design, according to the validated Cochrane Effective Practice and Organization of Care group (EPOC) criteria . We then assessed quality using a modified GRADE  and CASP-Qualitative study quality assessment tool , according to study design classification. Included studies were those that passed general quality and general acceptability screening for these tools. We mapped extracted evidence into our initial programme TOC (supplemental files), iterating with inputs from each article's specific country experience to provide insight into potential relationships between the context, integration mechanism, and associated outcomes. In order to provide useful guidance to efforts to further develop the research base on this topic, we also mapped availability and absence of evidence onto our theory of change. 

Results Articles via Expert Referral and Citation Searching: Fifteen grey literature articles were identified at the outset of the grey literature search through expert referral, through an active process conducted by our commissioners and authors; one additional article was identified in iterative searching ( ). All sixteen underwent full text review. All but one  were excluded, with reasons for exclusion presented in . Two hundred and seventy titles and abstracts were identified from the reference lists of the four selected relevant articles (listed above). Of these only three articles were not captured in the database search (below) and met criteria to undergo full text screening. None of these three progressed to data extraction because they were either not focused on HPV vaccine delivery ( n = 1), not original research (n = 1), or there was insufficient data for extraction (n = 1). Articles from Databases: Our search of peer-reviewed academic databases yielded 1,961 articles. We imported these into Covidence for screening, removing 428 duplicates. Two reviewers independently screened title and abstracts against inclusion criteria for the resulting 1,533 records, excluding 1,497. A third reviewer arbitrated discrepancies in screening determination. Two reviewers independently reviewed the full text of the resulting 36 articles, excluding 28 with reasons listed in . Three of the initially resulting 11 articles were excluded during the final iteration of data extraction and analysis. Of these, one was a prospective non-controlled case study in Brazil which did not explicitly bundle or link HPV vaccine with other services ; another was a qualitative programme assessment of lessons learned after HPV vaccine introduction in four unnamed countries, one of which briefly described an integration approach but in insufficient detail for data extraction ; and one was a cross-sectional qualitative article, a lessons learned programme description of the national roll-out of South Africa's HPV vaccine programme that integrated other health education topics with vaccine delivery, which provided too few data on integrated services for extraction . These were assessed to be low quality/generalizability, medium quality/generalizability, and low quality/generalizability, respectively. Eight articles were assessed as suitable for data abstraction and inclusion in the final analysis ( ). Two of the final included studies were ascertained to be of good  or high  quality; the remaining six studies were of low or very low  quality and generalizability. Despite the varying quality and rigor of the articles, the research team and commissioners came to consensus that all eight of the final studies selected for inclusion provided meaningful inputs to test against our TOC. 3.1 Overview of included articles The eight included articles were conducted across six countries, five in sub-Saharan Africa (Uganda – 2, Togo – 1, South Africa – 1, Rwanda – 1, Tanzania – 1) and one in Southeast Asia (Malaysia – 2) ( ). Six were observational programme evaluations ( n = 6), one was a cross-sectional study, and one was a qualitative study. Implementation described in the articles spanned 13 years, from 2008 through 2021. Articles reported on study or programme evaluation periods ranging from 1 to 7 years, with a median of 1.5 years. 3.2 Cross-cutting findings and iteration of theory of change We synthesize findings as presented in our iterated TOC ( ), followed by discussion of context-mechanism-outcomes (C-M—O) configurations and learnings from each of the six countries. 3.3 Evidence on ‘Context’ Included articles provided evidence for each of the seven ‘context’ factors postulated in the initial TOC, and iteration supported the addition of one new factor ( ). The most frequently reported context for an integrated approach was school-based outreach delivery, presented in all eight articles. Three articles assessed integrated service delivery within a vaccination campaign [ , , ] (Uganda, Rwanda, Togo). Two articles briefly described the use of a facility-based approach, without specifying if integrated services were available  (Uganda and Rwanda). One article reported on an integrated community-based outreach approach  (Uganda). One article assessed gender inequity  (Tanzania), and seven articles assessed other socio-cultural contexts affecting services for adolescents, such as religion, high HIV prevalence, or rates of school enrollment [ , , , , , , ] (South Africa, Malaysia, Rwanda, Tanzania, Uganda). TOC iterations differentiated these factors. Three articles documented context challenges to vaccine confidence, reporting on rumors, myths, and steps taken to reduce vaccine hesitancy [ , , ] (Malaysia, Uganda). One new context factor emerged through TOC iteration: seven articles reported on the contextual role of government and/or political endorsement of an integrated programme approach, either through official communications of government support [ , , ] as in South Africa, Uganda and Togo, or in relation to an emergent mechanism factor (below), of intentional multisectoral cooperation [ , , , ] in Malaysia, Tanzania, and Rwanda. 3.4 Evidence on ‘Mechanism’ Included articles provided evidence for six of the eight mechanisms postulated in the initial TOC, and four mechanisms were added through TOC iteration ( ). Four articles described multisectoral cooperation for policy and planning [ , , , ] in Malaysia, Rwanda, and Tanzania; TOC iterations adopted this mechanism. One article reported the use of an integrative mechanism involving explicit inclusion of parental and adolescent preference to design an integrated service delivery package and program  in Tanzania; we retained this factor in TOC iterations. We found no evidence of “adolescent friendly” facilities; however, we reached consensus to retain this factor in TOC iteration. The initial TOC postulated both a cost-sharing and a decreased delivery cost mechanism. Four articles reported cost sharing between adolescent health and vaccination programs [ , , , , ] in Malaysia, South Africa, Tanzania and Uganda, but TOC iterations captured changes in delivery cost as an outcome (see below). Three articles reported integration of supply chain and distribution [ , , ], in Malaysia, Tanzania, and Uganda; TOC iterations retained this factor. Three articles described a mechanism of integrating communications between adolescent health and HPV vaccination programs [ , , ], in Rwanda, Tanzania and Togo; TOC iterations retained this factor. The most frequently reported mechanism of integration was simultaneous delivery of HPV vaccine and other services, presented in seven articles [ , , , , , , ] (all six countries) and referenced indirectly in the eighth . TOC iterations parsed novelty of the integrated package from staffing concepts. All countries used existing school health or routine immunization staff, with one article reporting the use of non-vaccinating healthcare providers to deliver the integrated services package  (South Africa). The TOC was refined to include a new factor, on whether an integrated approach used a novel [ , , , ] (Rwanda, South Africa, Tanzania, Togo), or an existing package [ , , , , ] (Malaysia, also Rwanda, and Uganda) of other adolescent health services. Further TOC iterations yielded an additional context factor – which system ‘hosted’ the integrated services. Three countries integrated services into the routine immunization or primary health care programme (Rwanda, Tanzania and Uganda) [ , , , ], and four integrated into the school health programme (Malaysia, South Africa and Togo) [ , , , ]. A grey TOC box indicates this close relationship ( ). Finally, one article (Malaysia) reported the integration of programme monitoring ; TOC iterations adopted this factor. 3.5 Evidence on ‘Outcomes’ Nine ‘outcome’ factors were postulated in the initial TOC, including four metrics intended to capture nuances of coverage. Through TOC iteration, we discarded two and retained four outcome factors, while adopting four new factors ( ), effectively condensing coverage metrics. A “change from baseline HPV vaccine coverage” was measured in all eight articles [ , , , , , , , ], largely describing initial coverage in new programs, while a “sustained change in HPV vaccine coverage” metric measured in three articles captured coverage over a longer time course, over 7 years in Malaysia and 14 months in Tanzania [ , , ]. There was no evidence of measurement of adolescent health-seeking behaviors (from initial TOC), but two articles did report on change in participation in integrated programs (also in initial TOC), in Tanzania and Togo ; this was captured in TOC revisions. We also saw no evidence of the HPV vaccine ‘demand’ outcome from the initial TOC, but we reached consensus to retain the concept in TOC iteration – acknowledging that ‘demand’ and ‘health seeking’ concepts overlap. Similarly, we found no evidence of HPV and cervical cancer knowledge outcomes from the initial TOC, although one article reported on change in knowledge of the integrated services, in Togo . The refined TOC presents these as two separate knowledge measures. Three articles reported on a change in attitudes [ , , ], capturing attitudes about the combined, integrated services (acceptance, parent willingness to consent), rather than an attitude unique to the vaccine or other services, in Tanzania, Togo and Malaysia. TOC iterations present a combined “attitude” factor. TOC iterations adopted the outcome of costs of integrated service delivery, reported in Malaysia, Tanzania, and Uganda [ , , ]. TOC iterations also adopted a programme efficiency outcome (stock-outs, school disruption, from Tanzania and Uganda) . No articles presented data on quality of care or services, however, we reached consensus to retain this concept in TOC iterations. Similarly, we found no use of a zero dose outcome; TOC iterations discarded this concept, with Commissioners noting its decreasing importance as countries adopt a single-dose schedule. 3.6 Malaysia In 2010, Malaysia incorporated its new national HPV vaccine program into the existing school health programme ( ), which had a strong track record delivering other vaccines (measles/rubella and diptheria/tetanus toxoid boosters) to younger students (age 7) . Both included articles reported on the socio-cultural context of religiously influenced beliefs held across the country's sizeable Islamic population, describing the intentional use of culturally tailored demand generation messaging ahead of introduction . Malaysia used multiple mechanisms for an integrated approach: HPV vaccine was delivered simultaneously alongside other existing adolescent health services, including other vaccines, embedded into the school health program; the package was delivered by vaccinators; staffing costs were shared between the school health and vaccination programs; and the country integrated budgetary processes, supply chain logistics and distribution programmes, and programme monitoring workflows and infrastructure . Malaysia also explicitly used a mechanism of multi-sectoral cooperation for policy and planning, discussed by one included article as a key factor underpinning success of the integrated approach , and apparent in programme planning, communications, and monitoring. Neither article assessed explicit relationships between these mechanisms and reported outcomes, but both reported an increase in population coverage and number of girls vaccinated, along with a sustained level of high coverage, and a sustained change in attitude (parental consent and acceptance) ( ). One article reported generally on comparative programme costs, linking multisectoral coordination to the concept of reducing costs in the author's discussion . The Malaysian example, which continues today, particularly supports the addition of the multi-sectoral collaboration mechanism to our iterated TOC. 3.7 Rwanda Rwanda was the first low-income country to introduce HPV vaccine at the national level, using a phased approach. The one included article reports outcomes from the initial, 2011 phase of introduction, also describing context and mechanisms used in national scale-up through 2014 ( ) , which continue to influence the current program. Context included a small proportion of out-of-school girls, who were directed to present to local health facilities for vaccination. The article does not suggest these girls received other integrated services. The involvement and support of Rwanda's then-First Lady provides an example of government leader endorsement. Delivery context involved school-based, campaign-style delivery that worked through a mechanism of creating novel “health days”, embedded into the primary health care system, where teachers and local health care professionals collaborated to educate girls being vaccinated on topics such as hygiene, nutrition, infectious diseases, and reproductive health . The article also clearly describes the mechanism of multi-sectoral cooperation for policy and planning, at Ministry and National Immunization Technical Advisory Group (NITAG) levels, which created new technical working groups of experts in education, gender and family promotion, and cancer care. Multi-sectoral collaboration extended to programme planning and readiness, cold chain logistics, vaccine procurement, target population enumeration, and data collection. Reported outcomes included high rates of coverage. However, study design did not support attribution of effect. 3.8 South Africa An integrated approach from one sub-national district in 2011 in South Africa ( ) used school-based outreach delivery in the socio-cultural context of a high HIV/AIDS prevalence and government endorsement, benefiting from the context of a national policy and practice guideline that had recently been put into place supporting an integrated approach . The integrated approach was embedded in a school-based primary health care package of services incorporating novel and existing components , through simultaneous delivery with other services, including a general clinical and nutritional appraisal, by existing school health staff. We categorized the emphasis on shared staffing as a cost-sharing mechanism. This somewhat limited case reported the outcome of an increase in number of vaccinated girls. This article provides a number of negative or “counter-factual” examples, with authors reporting lack of cold chain integration in the setting of challenges with storage capacity; lack of integrated enumeration in the setting of challenges with correct identification of girls for follow-up doses; and lack of integration of data capture into existing school health programs in the setting of programme monitoring challenges due to the use of separate, paper-based vaccine records. This set of “counter-factual” examples supported a revision of our TOC that includes these mechanisms. 3.9 Tanzania Tanzania integrated HPV vaccine in schools within a novel “HPV-Plus” package of adolescent primary health care screening, deworming, and educational content ( ) delivered across six sub-national districts starting in 2020  ( ). Facility-based delivery of the package did not meet feasibility criteria; services were delivered in a school-based outreach delivery context. The programme addressed the context of communities affected by HIV/AIDS in planning efforts; addressed the context of potential gender inequity by offering an educational package for boys as well as girls; and benefited from the context of a national policy that had been put into place supporting an integrated approach . The country used the mechanism of multi-sectoral cooperation, wherein planning and monitoring activities were integrated between the vaccine programme and ongoing broader primary health care efforts. Preferences of adolescents and parents were explicitly incorporated into programme design ( ), through a participatory human-centered design methodology. The programme was embedded into the routine immunization program; of note, this integrated approach has been scaled up to additional regions in the current program. Existing routine immunization workers simultaneously delivered HPV vaccines with the integrated service package, which we categorized as a cost-sharing mechanism. The program also leveraged supply chain and distribution systems as well as communications from both school health and primary health care / routine immunization programmes. The article reported increases in HPV vaccine coverage, and, uniquely among other articles, reported increased participation with the integrated services, particularly nutrition and vision screening. Additional outcomes included positive attitudes toward the integrated services, and costs, although a detailed cost comparison was not performed. The article did report challenges with stock-out of the deworming medication included in the package, noting inadequate integration of procurement mechanisms. Various staffing constraints were also noted. TOC iterations captured these challenges as “comparative programme efficiencies”. 3.10 Togo Togo conducted a sub-national two-district pilot in 2016 and 2017 that integrated a novel age-appropriate adolescent health educational curriculum on puberty education, menstrual hygiene and hand-washing with the simultaneous delivery of HPV vaccination in a school-based campaign approach . The programme addressed the context of potential gender inequity by providing the educational package to boys, and responded to the context of needing to serve out-of-school girls with a tailored outreach strategy. Responsiveness to these factors seems to have been enabled by the use of a multi-sectoral cooperation mechanism for planning. The intervention was embedded in the school health program, and delivered by a differing configuration of workers (teachers and vaccinators in one, vaccinators only in the other). The article reported a favorable effect on HPV vaccine coverage, but study design did not support attribution of effect to an integrated approach. The paper did report positive attitudes from multiple stakeholders in response to the integration experience, and presented evidence of a positive impact of the integrated approach on youth participation in the educational sessions, and, uniquely, on youth knowledge and recall of session content. 3.11 Uganda Uganda conducted a sub-national pilot in 2009, using schools as delivery sites for HPV vaccine alongside deworming medicines and vitamin A in one district through an existing community-based “Child Days Plus” programme . The programme responded to the context of needing to serve out-of-school girls by referring to facilities or other routine mop-up services; neither paper suggests these girls received the other integrated services. Local government did endorse and promote the integrated approach, which one article linked to increased buy-in among health care and education workers . The programme was embedded into routine immunization services, sharing costs by using immunizers to deliver the integrated package, and leveraging existing immunization progamme cold chain, supply and distribution routes for HPV vaccine shipments . Both articles documented relatively high initial coverage over approximately one year, but neither article was designed to accurately assess a coverage effect due to an integrated approach – differences could not be parsed from the use of an age-based versus a grade-based approach to enumerate the target population. One article did report experiences with programme dropout and school disruptions ; TOC iterations captured these as an “efficiency” concept. 

Overview of included articles The eight included articles were conducted across six countries, five in sub-Saharan Africa (Uganda – 2, Togo – 1, South Africa – 1, Rwanda – 1, Tanzania – 1) and one in Southeast Asia (Malaysia – 2) ( ). Six were observational programme evaluations ( n = 6), one was a cross-sectional study, and one was a qualitative study. Implementation described in the articles spanned 13 years, from 2008 through 2021. Articles reported on study or programme evaluation periods ranging from 1 to 7 years, with a median of 1.5 years. 

Cross-cutting findings and iteration of theory of change We synthesize findings as presented in our iterated TOC ( ), followed by discussion of context-mechanism-outcomes (C-M—O) configurations and learnings from each of the six countries. 

Evidence on ‘Context’ Included articles provided evidence for each of the seven ‘context’ factors postulated in the initial TOC, and iteration supported the addition of one new factor ( ). The most frequently reported context for an integrated approach was school-based outreach delivery, presented in all eight articles. Three articles assessed integrated service delivery within a vaccination campaign [ , , ] (Uganda, Rwanda, Togo). Two articles briefly described the use of a facility-based approach, without specifying if integrated services were available  (Uganda and Rwanda). One article reported on an integrated community-based outreach approach  (Uganda). One article assessed gender inequity  (Tanzania), and seven articles assessed other socio-cultural contexts affecting services for adolescents, such as religion, high HIV prevalence, or rates of school enrollment [ , , , , , , ] (South Africa, Malaysia, Rwanda, Tanzania, Uganda). TOC iterations differentiated these factors. Three articles documented context challenges to vaccine confidence, reporting on rumors, myths, and steps taken to reduce vaccine hesitancy [ , , ] (Malaysia, Uganda). One new context factor emerged through TOC iteration: seven articles reported on the contextual role of government and/or political endorsement of an integrated programme approach, either through official communications of government support [ , , ] as in South Africa, Uganda and Togo, or in relation to an emergent mechanism factor (below), of intentional multisectoral cooperation [ , , , ] in Malaysia, Tanzania, and Rwanda. 

Evidence on ‘Mechanism’ Included articles provided evidence for six of the eight mechanisms postulated in the initial TOC, and four mechanisms were added through TOC iteration ( ). Four articles described multisectoral cooperation for policy and planning [ , , , ] in Malaysia, Rwanda, and Tanzania; TOC iterations adopted this mechanism. One article reported the use of an integrative mechanism involving explicit inclusion of parental and adolescent preference to design an integrated service delivery package and program  in Tanzania; we retained this factor in TOC iterations. We found no evidence of “adolescent friendly” facilities; however, we reached consensus to retain this factor in TOC iteration. The initial TOC postulated both a cost-sharing and a decreased delivery cost mechanism. Four articles reported cost sharing between adolescent health and vaccination programs [ , , , , ] in Malaysia, South Africa, Tanzania and Uganda, but TOC iterations captured changes in delivery cost as an outcome (see below). Three articles reported integration of supply chain and distribution [ , , ], in Malaysia, Tanzania, and Uganda; TOC iterations retained this factor. Three articles described a mechanism of integrating communications between adolescent health and HPV vaccination programs [ , , ], in Rwanda, Tanzania and Togo; TOC iterations retained this factor. The most frequently reported mechanism of integration was simultaneous delivery of HPV vaccine and other services, presented in seven articles [ , , , , , , ] (all six countries) and referenced indirectly in the eighth . TOC iterations parsed novelty of the integrated package from staffing concepts. All countries used existing school health or routine immunization staff, with one article reporting the use of non-vaccinating healthcare providers to deliver the integrated services package  (South Africa). The TOC was refined to include a new factor, on whether an integrated approach used a novel [ , , , ] (Rwanda, South Africa, Tanzania, Togo), or an existing package [ , , , , ] (Malaysia, also Rwanda, and Uganda) of other adolescent health services. Further TOC iterations yielded an additional context factor – which system ‘hosted’ the integrated services. Three countries integrated services into the routine immunization or primary health care programme (Rwanda, Tanzania and Uganda) [ , , , ], and four integrated into the school health programme (Malaysia, South Africa and Togo) [ , , , ]. A grey TOC box indicates this close relationship ( ). Finally, one article (Malaysia) reported the integration of programme monitoring ; TOC iterations adopted this factor. 

Evidence on ‘Outcomes’ Nine ‘outcome’ factors were postulated in the initial TOC, including four metrics intended to capture nuances of coverage. Through TOC iteration, we discarded two and retained four outcome factors, while adopting four new factors ( ), effectively condensing coverage metrics. A “change from baseline HPV vaccine coverage” was measured in all eight articles [ , , , , , , , ], largely describing initial coverage in new programs, while a “sustained change in HPV vaccine coverage” metric measured in three articles captured coverage over a longer time course, over 7 years in Malaysia and 14 months in Tanzania [ , , ]. There was no evidence of measurement of adolescent health-seeking behaviors (from initial TOC), but two articles did report on change in participation in integrated programs (also in initial TOC), in Tanzania and Togo ; this was captured in TOC revisions. We also saw no evidence of the HPV vaccine ‘demand’ outcome from the initial TOC, but we reached consensus to retain the concept in TOC iteration – acknowledging that ‘demand’ and ‘health seeking’ concepts overlap. Similarly, we found no evidence of HPV and cervical cancer knowledge outcomes from the initial TOC, although one article reported on change in knowledge of the integrated services, in Togo . The refined TOC presents these as two separate knowledge measures. Three articles reported on a change in attitudes [ , , ], capturing attitudes about the combined, integrated services (acceptance, parent willingness to consent), rather than an attitude unique to the vaccine or other services, in Tanzania, Togo and Malaysia. TOC iterations present a combined “attitude” factor. TOC iterations adopted the outcome of costs of integrated service delivery, reported in Malaysia, Tanzania, and Uganda [ , , ]. TOC iterations also adopted a programme efficiency outcome (stock-outs, school disruption, from Tanzania and Uganda) . No articles presented data on quality of care or services, however, we reached consensus to retain this concept in TOC iterations. Similarly, we found no use of a zero dose outcome; TOC iterations discarded this concept, with Commissioners noting its decreasing importance as countries adopt a single-dose schedule. 

Malaysia In 2010, Malaysia incorporated its new national HPV vaccine program into the existing school health programme ( ), which had a strong track record delivering other vaccines (measles/rubella and diptheria/tetanus toxoid boosters) to younger students (age 7) . Both included articles reported on the socio-cultural context of religiously influenced beliefs held across the country's sizeable Islamic population, describing the intentional use of culturally tailored demand generation messaging ahead of introduction . Malaysia used multiple mechanisms for an integrated approach: HPV vaccine was delivered simultaneously alongside other existing adolescent health services, including other vaccines, embedded into the school health program; the package was delivered by vaccinators; staffing costs were shared between the school health and vaccination programs; and the country integrated budgetary processes, supply chain logistics and distribution programmes, and programme monitoring workflows and infrastructure . Malaysia also explicitly used a mechanism of multi-sectoral cooperation for policy and planning, discussed by one included article as a key factor underpinning success of the integrated approach , and apparent in programme planning, communications, and monitoring. Neither article assessed explicit relationships between these mechanisms and reported outcomes, but both reported an increase in population coverage and number of girls vaccinated, along with a sustained level of high coverage, and a sustained change in attitude (parental consent and acceptance) ( ). One article reported generally on comparative programme costs, linking multisectoral coordination to the concept of reducing costs in the author's discussion . The Malaysian example, which continues today, particularly supports the addition of the multi-sectoral collaboration mechanism to our iterated TOC. 

Rwanda Rwanda was the first low-income country to introduce HPV vaccine at the national level, using a phased approach. The one included article reports outcomes from the initial, 2011 phase of introduction, also describing context and mechanisms used in national scale-up through 2014 ( ) , which continue to influence the current program. Context included a small proportion of out-of-school girls, who were directed to present to local health facilities for vaccination. The article does not suggest these girls received other integrated services. The involvement and support of Rwanda's then-First Lady provides an example of government leader endorsement. Delivery context involved school-based, campaign-style delivery that worked through a mechanism of creating novel “health days”, embedded into the primary health care system, where teachers and local health care professionals collaborated to educate girls being vaccinated on topics such as hygiene, nutrition, infectious diseases, and reproductive health . The article also clearly describes the mechanism of multi-sectoral cooperation for policy and planning, at Ministry and National Immunization Technical Advisory Group (NITAG) levels, which created new technical working groups of experts in education, gender and family promotion, and cancer care. Multi-sectoral collaboration extended to programme planning and readiness, cold chain logistics, vaccine procurement, target population enumeration, and data collection. Reported outcomes included high rates of coverage. However, study design did not support attribution of effect. 

South Africa An integrated approach from one sub-national district in 2011 in South Africa ( ) used school-based outreach delivery in the socio-cultural context of a high HIV/AIDS prevalence and government endorsement, benefiting from the context of a national policy and practice guideline that had recently been put into place supporting an integrated approach . The integrated approach was embedded in a school-based primary health care package of services incorporating novel and existing components , through simultaneous delivery with other services, including a general clinical and nutritional appraisal, by existing school health staff. We categorized the emphasis on shared staffing as a cost-sharing mechanism. This somewhat limited case reported the outcome of an increase in number of vaccinated girls. This article provides a number of negative or “counter-factual” examples, with authors reporting lack of cold chain integration in the setting of challenges with storage capacity; lack of integrated enumeration in the setting of challenges with correct identification of girls for follow-up doses; and lack of integration of data capture into existing school health programs in the setting of programme monitoring challenges due to the use of separate, paper-based vaccine records. This set of “counter-factual” examples supported a revision of our TOC that includes these mechanisms. 

Tanzania Tanzania integrated HPV vaccine in schools within a novel “HPV-Plus” package of adolescent primary health care screening, deworming, and educational content ( ) delivered across six sub-national districts starting in 2020  ( ). Facility-based delivery of the package did not meet feasibility criteria; services were delivered in a school-based outreach delivery context. The programme addressed the context of communities affected by HIV/AIDS in planning efforts; addressed the context of potential gender inequity by offering an educational package for boys as well as girls; and benefited from the context of a national policy that had been put into place supporting an integrated approach . The country used the mechanism of multi-sectoral cooperation, wherein planning and monitoring activities were integrated between the vaccine programme and ongoing broader primary health care efforts. Preferences of adolescents and parents were explicitly incorporated into programme design ( ), through a participatory human-centered design methodology. The programme was embedded into the routine immunization program; of note, this integrated approach has been scaled up to additional regions in the current program. Existing routine immunization workers simultaneously delivered HPV vaccines with the integrated service package, which we categorized as a cost-sharing mechanism. The program also leveraged supply chain and distribution systems as well as communications from both school health and primary health care / routine immunization programmes. The article reported increases in HPV vaccine coverage, and, uniquely among other articles, reported increased participation with the integrated services, particularly nutrition and vision screening. Additional outcomes included positive attitudes toward the integrated services, and costs, although a detailed cost comparison was not performed. The article did report challenges with stock-out of the deworming medication included in the package, noting inadequate integration of procurement mechanisms. Various staffing constraints were also noted. TOC iterations captured these challenges as “comparative programme efficiencies”. 

Togo Togo conducted a sub-national two-district pilot in 2016 and 2017 that integrated a novel age-appropriate adolescent health educational curriculum on puberty education, menstrual hygiene and hand-washing with the simultaneous delivery of HPV vaccination in a school-based campaign approach . The programme addressed the context of potential gender inequity by providing the educational package to boys, and responded to the context of needing to serve out-of-school girls with a tailored outreach strategy. Responsiveness to these factors seems to have been enabled by the use of a multi-sectoral cooperation mechanism for planning. The intervention was embedded in the school health program, and delivered by a differing configuration of workers (teachers and vaccinators in one, vaccinators only in the other). The article reported a favorable effect on HPV vaccine coverage, but study design did not support attribution of effect to an integrated approach. The paper did report positive attitudes from multiple stakeholders in response to the integration experience, and presented evidence of a positive impact of the integrated approach on youth participation in the educational sessions, and, uniquely, on youth knowledge and recall of session content. 

Uganda Uganda conducted a sub-national pilot in 2009, using schools as delivery sites for HPV vaccine alongside deworming medicines and vitamin A in one district through an existing community-based “Child Days Plus” programme . The programme responded to the context of needing to serve out-of-school girls by referring to facilities or other routine mop-up services; neither paper suggests these girls received the other integrated services. Local government did endorse and promote the integrated approach, which one article linked to increased buy-in among health care and education workers . The programme was embedded into routine immunization services, sharing costs by using immunizers to deliver the integrated package, and leveraging existing immunization progamme cold chain, supply and distribution routes for HPV vaccine shipments . Both articles documented relatively high initial coverage over approximately one year, but neither article was designed to accurately assess a coverage effect due to an integrated approach – differences could not be parsed from the use of an age-based versus a grade-based approach to enumerate the target population. One article did report experiences with programme dropout and school disruptions ; TOC iterations captured these as an “efficiency” concept. 

Discussion 4.1 Summary of main findings To our knowledge this is the first realist synthesis describing integrated approaches to delivery of HPV vaccines and other adolescent health services. Our iterative realist process yielded an iterated Theory of Change (TOC) developed with guidance from policy, programme and research experts in the field. A truly linear and connected TOC model is not yet possible, given the diversity of potential interventions and the limited set of evidence available. Instead, our TOC is necessarily more open and allows mapping of potentially beneficial combinations of context and mechanisms, as well as where there are major gaps in the current evidence base. This theory can guide researcher and practitioner efforts to fill current evidence gaps in understanding how and why integrated approaches may yield reported results in this space. By examining available country experience, we have also described how varying combinations of context, mechanisms and outcomes have been implemented and evaluated in real-world settings. Our synthesis, captured in TOC iterations, has notable implications for future research and practice. While integrated approaches were largely delivered to a population of in-school 9–10 year olds, the more recently published experiences addressed wider ranges, up to 13 or 14 years (Togo and Tanzania). As many programs shift to a 9 year old target age, future integration efforts will need to distinguish health needs of younger compared to older adolescents. While most countries described some level of strategy to reach out of school girls, data on reach and effectiveness of integrated mechanisms to meet their needs was sparse. Future research and practice should address this significant gap, as out-of-school girls are likely to have unique health needs . Despite growing consensus that HPV vaccine programs should emphasize strategies for adolescents with HIV , we found little evidence of programme response to this context. We did see evidence of countries acknowledging the context of gender inequity, generally through explicit inclusion of boys in programming; those papers also disaggregated relevant outcomes by gender. When the local context involved government endorsement, it took two forms: that of a new, national policy, where HPV vaccine was integrated with a novel package of services; or of public messages of governmental support for an integrated approach, in which case HPV vaccine was added to an existing package of services. Integrated packages included a limited range of other services, ranging from booster doses of other childhood vaccines (standard national schedule boosters of Measles, Rubella and Diptheria Tetanus administered to seven year olds as part of the school health services program) in Malaysia; deworming medications in Tanzania and Uganda; nutrition and development screening in Malaysia, Tanzania, and South Africa; vitamin A in Uganda; and primary care or sexual/reproductive health programming in Malaysia, Rwanda, Togo, and to some extent in Tanzania. All of these options had been described in a 2013 comprehensive review of evidence-based options ; thus, this guidance remains relevant, and countries should develop integrated service packages according to the context of local needs, available existing programs to leverage, and emerging national priorities. While each country used multiple integrated mechanisms, the use of a multisectoral cooperation mechanism appeared to be linked to the use of a large number of other mechanisms, including the integration of cost-sharing, supply chain and distribution, and programme monitoring, as seen particularly in Malaysia, Tanzania and Togo. We note recent attention to this concept in other HPV vaccine literature . Our findings highlight the importance of studying not just coverage, but outcomes related to understanding of and confidence in public health interventions, such as knowledge and acceptability of an integrated approach. Understanding the effect of an integrated service delivery package on such outcomes is critical to understanding how integrated approaches may work, to vaccines and across a broader context [ , , ]. 4.2 Policy and Practice implications for the implementation research agenda To our knowledge, three of the included countries have continued, or scaled up, their integrated approaches – Malaysia, Tanzania and Rwanda, each integrating into a primary health care package of services for adolescents, using school-based delivery. We show a clear need to design integrated interventions to measure outcomes in a way that allows attribution of effect (e.g. if change in uptake is truly due to an integrated mechanism, instead of enhanced demand generation). We note particular gaps in the evidence around outcomes of integration for priority populations with unique needs, such as adolescents living with HIV, out of school, or otherwise not consistently reached by routine health services (e.g. in migratory or conflict-affected communities). Gains in school attendance over recent decades are in decline, particularly among adolescents , and many out of school youth are increasingly concentrated in small subnational communities . Thus, there is also a clear agenda for studying integrated approaches through subnational studies. We also note a gap in knowledge on how integrated packages may be delivered through community and health facility access points. Closing these gaps will require mixed methods, perhaps drawing on realist evaluation concepts, to understand underlying drivers of success or failure of an integrated approach. Finally, an integrated approach is often cited as a means to improve efficiency and reach, and contribute to programme sustainability [ , , ]. However, we found no usable data on costs – a core component of sustainability. We recommend that the research agenda include health economics perspectives and objectives, so that costs, and potential variance with an integrated approach, are measured appropriately and comprehensively. 4.3 Strengths and limitations An important strength of this review is its flexible, pragmatic realist synthesis methodology, aimed not at an exhaustive review of the literature, but on providing a practical framework ready for use by practitioners. We note increasing use of this methodology to synthesize evidence on complex interventions [ , , ] – to help describe in practical terms not only whether an intervention works, but how, and under what conditions. This methodology permitted us to admit evidence of lesser quality, including project evaluations and grey literature, harnessing learnings that may otherwise be overlooked. Consistent with other realist synthesis experiences, we noted that context and mechanism factors were often intertwined and resistant to separation , making it difficult to parse context and mechanism factor effects on particular outcomes. Therefore, we cannot recommend specific combinations of context and mechanism to yield an outcome of interest. Rather, our TOC provides a framework to guide the incorporation of these multiple factors into programme, policy, and research protocol design. Our findings are drawn from a notably small number of countries (six) representing a limited geography (five in Africa, one in Southeast Asia), limiting the transferability of challenges and opportunities we report. An additional limitation involves our dependence upon article availability. Not all country experience with integrated approaches has been published, even in the grey literature that we with our commissioner experts considered. Many articles screened lacked sufficient data to extract to inform our TOC [ , , , ]. And, perhaps because of the prior absence of a TOC framework, many included articles did not focus on evaluating how and why an integrated approach might work. 

Summary of main findings To our knowledge this is the first realist synthesis describing integrated approaches to delivery of HPV vaccines and other adolescent health services. Our iterative realist process yielded an iterated Theory of Change (TOC) developed with guidance from policy, programme and research experts in the field. A truly linear and connected TOC model is not yet possible, given the diversity of potential interventions and the limited set of evidence available. Instead, our TOC is necessarily more open and allows mapping of potentially beneficial combinations of context and mechanisms, as well as where there are major gaps in the current evidence base. This theory can guide researcher and practitioner efforts to fill current evidence gaps in understanding how and why integrated approaches may yield reported results in this space. By examining available country experience, we have also described how varying combinations of context, mechanisms and outcomes have been implemented and evaluated in real-world settings. Our synthesis, captured in TOC iterations, has notable implications for future research and practice. While integrated approaches were largely delivered to a population of in-school 9–10 year olds, the more recently published experiences addressed wider ranges, up to 13 or 14 years (Togo and Tanzania). As many programs shift to a 9 year old target age, future integration efforts will need to distinguish health needs of younger compared to older adolescents. While most countries described some level of strategy to reach out of school girls, data on reach and effectiveness of integrated mechanisms to meet their needs was sparse. Future research and practice should address this significant gap, as out-of-school girls are likely to have unique health needs . Despite growing consensus that HPV vaccine programs should emphasize strategies for adolescents with HIV , we found little evidence of programme response to this context. We did see evidence of countries acknowledging the context of gender inequity, generally through explicit inclusion of boys in programming; those papers also disaggregated relevant outcomes by gender. When the local context involved government endorsement, it took two forms: that of a new, national policy, where HPV vaccine was integrated with a novel package of services; or of public messages of governmental support for an integrated approach, in which case HPV vaccine was added to an existing package of services. Integrated packages included a limited range of other services, ranging from booster doses of other childhood vaccines (standard national schedule boosters of Measles, Rubella and Diptheria Tetanus administered to seven year olds as part of the school health services program) in Malaysia; deworming medications in Tanzania and Uganda; nutrition and development screening in Malaysia, Tanzania, and South Africa; vitamin A in Uganda; and primary care or sexual/reproductive health programming in Malaysia, Rwanda, Togo, and to some extent in Tanzania. All of these options had been described in a 2013 comprehensive review of evidence-based options ; thus, this guidance remains relevant, and countries should develop integrated service packages according to the context of local needs, available existing programs to leverage, and emerging national priorities. While each country used multiple integrated mechanisms, the use of a multisectoral cooperation mechanism appeared to be linked to the use of a large number of other mechanisms, including the integration of cost-sharing, supply chain and distribution, and programme monitoring, as seen particularly in Malaysia, Tanzania and Togo. We note recent attention to this concept in other HPV vaccine literature . Our findings highlight the importance of studying not just coverage, but outcomes related to understanding of and confidence in public health interventions, such as knowledge and acceptability of an integrated approach. Understanding the effect of an integrated service delivery package on such outcomes is critical to understanding how integrated approaches may work, to vaccines and across a broader context [ , , ]. 

Policy and Practice implications for the implementation research agenda To our knowledge, three of the included countries have continued, or scaled up, their integrated approaches – Malaysia, Tanzania and Rwanda, each integrating into a primary health care package of services for adolescents, using school-based delivery. We show a clear need to design integrated interventions to measure outcomes in a way that allows attribution of effect (e.g. if change in uptake is truly due to an integrated mechanism, instead of enhanced demand generation). We note particular gaps in the evidence around outcomes of integration for priority populations with unique needs, such as adolescents living with HIV, out of school, or otherwise not consistently reached by routine health services (e.g. in migratory or conflict-affected communities). Gains in school attendance over recent decades are in decline, particularly among adolescents , and many out of school youth are increasingly concentrated in small subnational communities . Thus, there is also a clear agenda for studying integrated approaches through subnational studies. We also note a gap in knowledge on how integrated packages may be delivered through community and health facility access points. Closing these gaps will require mixed methods, perhaps drawing on realist evaluation concepts, to understand underlying drivers of success or failure of an integrated approach. Finally, an integrated approach is often cited as a means to improve efficiency and reach, and contribute to programme sustainability [ , , ]. However, we found no usable data on costs – a core component of sustainability. We recommend that the research agenda include health economics perspectives and objectives, so that costs, and potential variance with an integrated approach, are measured appropriately and comprehensively. 

Strengths and limitations An important strength of this review is its flexible, pragmatic realist synthesis methodology, aimed not at an exhaustive review of the literature, but on providing a practical framework ready for use by practitioners. We note increasing use of this methodology to synthesize evidence on complex interventions [ , , ] – to help describe in practical terms not only whether an intervention works, but how, and under what conditions. This methodology permitted us to admit evidence of lesser quality, including project evaluations and grey literature, harnessing learnings that may otherwise be overlooked. Consistent with other realist synthesis experiences, we noted that context and mechanism factors were often intertwined and resistant to separation , making it difficult to parse context and mechanism factor effects on particular outcomes. Therefore, we cannot recommend specific combinations of context and mechanism to yield an outcome of interest. Rather, our TOC provides a framework to guide the incorporation of these multiple factors into programme, policy, and research protocol design. Our findings are drawn from a notably small number of countries (six) representing a limited geography (five in Africa, one in Southeast Asia), limiting the transferability of challenges and opportunities we report. An additional limitation involves our dependence upon article availability. Not all country experience with integrated approaches has been published, even in the grey literature that we with our commissioner experts considered. Many articles screened lacked sufficient data to extract to inform our TOC [ , , , ]. And, perhaps because of the prior absence of a TOC framework, many included articles did not focus on evaluating how and why an integrated approach might work. 

Conclusions and recommendations This realist synthesis review helps to describe and fill important gaps in available evidence on how integrated approaches might effectively yield outcomes of interest for HPV vaccines and other adolescent health services. We call on implementation researchers and funders to invest in more rigorous study designs that can provide more definitive insights into how integrated approaches in varying contexts may be linked to vaccination outcomes – particularly coverage. Our synthesis highlights the need to evaluate how an integrated approach may address programmatic gender inequities, and meet unique needs of locally relevant underserved populations including out of school youth, those affected by HIV, migration, or conflict, and others. Our iterated TOC can guide future implementation research, particularly on the effect an integrated approach may have on other adolescent health services, health knowledge and acceptability. While our findings are oriented to HPV vaccination programs, they have relevance to a broader package of primary health care services, including other vaccines for adolescents. The grounding of our proposed TOC in country experiences will help programme implementers, policy-makers, implementation scientists, and funders base new efforts in a practical theoretical framework – and prioritize filling key gaps in the existing evidence base. 

CM and MCJ contributions to this work were supported by the Bill & Melinda Gates Foundation [grant number INV-006006]. CW and SN contributions to this work were made possible by the generous support of the American people through the U.S. Agency for International Development (USAID) under the terms of the Cooperative Agreement [#7200AA20CA00002], led by Jhpiego and partners. The contents of this manuscript represent the views and opinions of the authors and do not necessarily reflect the views and opinions of the U.S. Agency for International Development (USAID) or the United States Government. Although one commissioner is employed by Public Health Institute, via USAID's Global Health Training, Advisory and Support Contract (GHTASC), that role was on an individual basis. The Sponsors were not involved in study design, data collection, analysis or interpretation of data, or in the writing of this manuscript report and decision to submit for publication. 

Mary Carol Jennings: Writing – review & editing, Writing – original draft, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Sarah Nabia: Writing – review & editing, Writing – original draft, Visualization, Investigation, Formal analysis, Data curation. Christopher Morgan: Writing – review & editing, Supervision, Methodology, Funding acquisition, Conceptualization. Chinelo Cynthia Nduka: Writing – original draft, Formal analysis, Data curation. Julia Brotherton: Writing – review & editing, Validation. Megan Holloway: Writing – review & editing, Validation. Katharine Bagshaw: Writing – review & editing, Validation. Paul Bloem: Writing – review & editing, Validation. Chizoba Wonodi: Writing – review & editing, Writing – original draft, Supervision, Methodology, Funding acquisition, Conceptualization. 

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Two authors are also co-authors on two of the articles retained in the final analysis [8, 25] – P.B., C.M. The remaining authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. 

