Introduction
============

Over recent decades, immunotherapy has become one of the most important breakthroughs in cancer treatment, especially, but not only, for patients with advanced, recurrent, and metastatic malignancy.[@ref1] Immune checkpoint inhibitor (ICI) drugs enhance self immune functions against cancer cells through a unique mechanism that blocks negative regulators expressed on immune or tumour cells. These regulators include cytotoxic T lymphocyte associated antigen 4, programmed cell death 1, and programmed cell death ligand 1.[@ref2] Although tumour cells invalidate immune surveillance and develop malignancies by taking advantage of programmed cell death 1, programmed cell death ligand 1, or cytotoxic T lymphocyte associated antigen 4, these immune checkpoints are normally essential in preventing over activation of the immune system and maintaining immunologic homoeostasis.[@ref1] [@ref3] Therefore, although monoclonal antibodies against programmed cell death 1 (nivolumab and pembrolizumab), programmed cell death ligand 1 (atezolizumab), and cytotoxic T lymphocyte associated antigen 4 (ipilimumab and tremelimumab) provide impressive anti-tumour activity in many solid tumours, they also have associated adverse events, known as immune-related adverse events.[@ref4]

Chemotherapy has long been the first line treatment for advanced cancers. Doctors often use multiple chemotherapy drugs and different combinations (that is, investigator's choice chemotherapy) for patients as their condition changes. Chemotherapy can have serious adverse events that lead to low compliance, dose reductions, treatment delays, and treatment refusals. The toxicities of many chemotherapy drugs have been clearly shown, owing to long term clinical use. The National Comprehensive Cancer Network has published several guidelines addressing adverse events associated with chemotherapy.[@ref5] Compared with the toxicities caused by conventional therapy (chemotherapy, targeted therapy, and their combinations), immune-related adverse events from ICI drugs are unique in terms of the organs involved, onset patterns, and severity.[@ref6] Numerous randomised controlled trials have outlined a crude profile of immune-related adverse events, including skin, gastrointestinal, pulmonary, hepatic, and endocrine toxicities.[@ref7] [@ref8] [@ref9] [@ref10] [@ref11] [@ref12] [@ref13] [@ref14] [@ref15] [@ref16] [@ref17] [@ref18] [@ref19] [@ref20] [@ref21] [@ref22] [@ref23] [@ref24] [@ref25] [@ref26] [@ref27] [@ref28] [@ref29] [@ref30] [@ref31] [@ref32] [@ref33] [@ref34] [@ref35] [@ref36] [@ref37] [@ref38] [@ref39] [@ref40] [@ref41] [@ref42] Although most immune-related adverse events can be well controlled by supportive treatment and glucocorticoids, fatal immune-related adverse events are an increasing concern regarding the safety of ICI drugs and patients' tolerability.[@ref43] [@ref44] The mechanism of immune-related adverse events remains unclear. Different ICI drugs and different doses of the same ICI drug may result in different adverse events. The United States Food and Drug Administration has approved several ICI drug products for the treatment of cancers with comparatively poor prognosis (eg, melanoma, lung cancer, urothelial cancer, renal cell carcinoma, and gastric cancer), and there has been an increase in combined treatment (that is, use of two ICI drugs, or use of one ICI drug with conventional therapy).[@ref8] [@ref9] [@ref15] [@ref16] [@ref18] [@ref26] [@ref27] [@ref28] [@ref29] [@ref38] [@ref39] [@ref40] Robust evidence indicates that ICI drugs are superior to traditional standard chemotherapy in first line treatment for some types of cancer (eg, lung cancer and melanoma) and in second line and subsequent anti-cancer treatment for others, and confer significantly higher survival benefits.[@ref17] [@ref19] [@ref22] [@ref25] [@ref30] [@ref32] [@ref33] [@ref34] [@ref35] In particular, owing to its noticeably improved efficacy in lung cancer, pembrolizumab was promoted to first line therapy in October 2016 and now has expanded indications.[@ref45] However, it is difficult to acquire a complete picture of immune-related adverse events from randomised controlled trials due to limitations in the study design and realistic practicalities. For example, nivolumab and pembrolizumab have never been head-to-head compared, because clinical trials about them are initiated or sponsored by different and competitive pharmaceutical companies. Past meta-analyses on this issue focused on a specific immune-related adverse event or an ICI drug (or both), failing to cover all possible comparisons.[@ref46] [@ref47] In addition, these meta-analyses included phase I, single arm, or placebo control studies, which also diminished the reliability and validity of their results.

High quality studies focusing on adverse events are required to aid clinicians in improving early management and recognition of immune-related adverse events. Researchers also need to be able to optimise the design of future trials with enhanced anti-tumour efficacy and safety as two major criteria of success. Here, we report a comprehensive dose and drug based network meta-analysis of the comparative safety of ICI drugs, with the aim of providing a complete toxicity profile, toxcicity spectrum, and a safety ranking of ICI drugs alone or in combination with conventional therapy.

Methods
=======

This network meta-analysis was reported in line with the preferred reporting items for systematic reviews and meta-analysis (PRISMA) and the PRISMA extension statement for network meta-analysis (supplementary materials, table S1).[@ref48] [@ref49] A prospective protocol was created in advance and uploaded to the PROSPERO online platform, with the registration number CRD42017082553.

Data sources and searches
-------------------------

We searched the PubMed, Embase, Cochrane Library, and Web of Science databases to include relevant studies published in English between January 2007 and February 2018. Search terms and their combinations used in the search strategy included cancer, carcinoma, neoplasm, immune checkpoint inhibitors (anti-programmed cell death 1, anti-programmed cell death ligand 1, and anti-cytotoxic T lymphocyte associated antigen 4), and specific ICI drug names (nivolumab, pembrolizumab, ipilimumab, tremelimumab, and atezolizumab). The search strategy is detailed in the supplementary materials. After the preliminary screening of titles or abstracts, two independent reviewers (CX and X-JD) assessed the full text and reference lists of relevant publications for final inclusion; articles cited as references that were considered to be potentially relevant were also retrieved and assessed in full.

Study selection
---------------

Only head-to-head phase II and III randomised controlled trials comparing two or more treatments including an ICI drug (that is, one ICI drug, two ICI drugs, and one ICI drug with conventional therapy), or comparing an ICI drug with conventional therapy in patients with cancer were eligible. Studies that compared different doses or usages (such as pharmaceutical forms and administration sites) of one ICI drug were also included. We excluded conference abstracts, posters, and presentations of ongoing randomised controlled trials because these brief reports did not contain detailed safety data. Studies providing a summary but no site, organ, or system level toxicity data were also excluded. In addition, high quality, single arm trials and placebo controlled trials meeting certain criteria were selected to establish a validation group (supplementary materials and tables S2 and S3).

Data extraction
---------------

Although immune-related adverse events were thought to be the outcome of interest, this term was inappropriate to describe the toxicity from conventional therapy. We selected treatment-related adverse events as the primary outcome, because these were compatible with most studies and deemed to be a suitable alternative to immune-related adverse events. If treatment-related adverse events were not available in a study, all adverse events were used. We evaluated the main text, supplementary materials, and all possible information available at ClinicalTrials.gov to implement an extensive and detailed data extraction. Although both original and updated studies derived from one trial were included in this network meta-analysis, we extracted treatment-related adverse events data from the study with the most detailed report; the other data were used to supplement basic information. Two reviewers (CX and X-JD) used a standardised form independently to extract and summarise the following data: first author, year of publication, study ID, region, cancer type, study design, total number of patients, number of patients in safety analysis, arms and treatment regimens, version of Common Terminology Criteria for Adverse Events, follow-up time, treatment-related adverse event reporting rate, and the frequency of each specific grade 1-5 and grade 3 or 4 adverse event. The Common Terminology Criteria for Adverse Events is the most commonly used tool for evaluating adverse event type and severity in clinical practice with a grading scale and clear definitions. All grades, grade 3, and grade 4 adverse events indicate complete, severe, and life threatening toxicity, respectively.[@ref50]

Quality assessment
------------------

Two reviewers (CX and X-JD) assessed the risk of bias based on the original study, possible updated study, and supplementary materials by using the tool recommended by the Cochrane Collaboration Handbook. The adequacy of five aspects was assessed: random sequence generation, allocation concealment, blinding method, assessment of outcomes, and reporting of results.[@ref51] Each aspect was assigned an assessment index associated with the risk of bias classified as yes, no, or unclear. We scored the method quality of studies quantitatively by using the modified Jadad scale (high quality, four points and over; or low quality, three points and under).[@ref52] All disagreements in the study selection, data extraction, and quality assessment were resolved by discussion to achieve a consensus.

Data synthesis and statistical analysis
---------------------------------------

Treatment-related adverse events were assessed from two different perspectives: overview and detail, based on the total number of all treatment-related adverse events and the number of each specific treatment-related adverse event, respectively. Regardless of the adverse event grading, general safety was used to indicate the overview of treatment-related adverse events without distinguishing between their specific classifications. We used odds ratios and 95% credibility intervals as summary statistics to quantify the effect of dose (of ICI drug) or drug on the risk of grade 1-5 and grade 3 or 4 adverse events in the network meta-analysis. Odds ratios greater than one represented a safety benefit favouring the control group. Two sided P\<0.05 was considered significant. If a study reported zero adverse events in any arm, the classic half integer continuity correction (adding a 0.5 to each cell) was applied for data preparation.

We generated plots depicting the network geometry using Stata version 12.0 (StataCorp).[@ref53] Both dose and drug based network meta-analysis were performed in a Bayesian framework using Markov chain Monte Carlo methods in WinBUGS version 1.4.3 (MRC Biostatistics Unit, Cambridge, UK).[@ref54] After completion of the dose based network meta-analysis, we re-prepared the data for drug based network meta-analysis for each study by combining multiple treatment arms that represent different doses of an ICI drug into one arm that represents one ICI drug; we excluded studies with only one treatment arm after the combination. We used random effects and consistency models to calculate odds ratios and 95% credibility intervals; these models are thought to be the most conservative approach to dealing with between-study heterogeneity.[@ref55] We used non-informative prior distributions and over dispersed initial values (with a scaling of 5) in four chains to fit the model,[@ref54] yielding 100 000 iterations (including 20 000 tuning iterations) and a thinning interval of 10 for each chain; this method was used to generate posterior distributions of model parameters. Convergence of iterations was assessed with the Gelman-Rubin-Brooks statistic.[@ref56] According to odds ratios and the posterior probabilities, we ranked the probability of each treatment being the safest, followed by the second, and the third, and so on.

Outstanding consistency, represented by the agreement between the direct and indirect results, is the key to robust results. Appraisal of inconsistency was achieved by using a three step method. Firstly, we made a rough comparison of the fit of the consistency model with that of the inconsistency model.[@ref57] Secondly, we tested inconsistency in the entire network on particular comparisons (nodes) by node splitting analysis; P\<0.05 indicated significant inconsistency.[@ref57] [@ref58] Thirdly, pairwise meta-analysis was conducted in a Frequentist framework by use of Review Manager version 5.3 (Cochrane Collaboration) to provide direct estimates (that is, odds ratios and 95% confidence intervals). Between-study heterogeneity was assessed using the χ^2^and Ι^2^ tests, with significance set at P\<0.10; Ι^2^ more than 25%, 50%, or 75% indicated low, moderate, or high heterogeneity, respectively.[@ref59] We used the random effects model if significant heterogeneity existed; otherwise, we used the fixed effects model. Network meta-analysis results (indirect results) were compared with pairwise meta-analysis results (direct results) to explore the source of inconsistency.

We generated the pooled incidence of grade 1-5 and grade 3 or 4 adverse events for treatments by using Meta-Analyst.[@ref60] Subgroup analyses were performed based on each specific treatment-related adverse event and cancer type. We conducted sensitivity analyses to assess the stability of results by restricting the original network meta-analysis to phase III randomised controlled trials, high quality studies, studies explicitly reporting treatment-related adverse events, studies on programmed cell death 1 and programmed cell death ligand 1 ICI drugs, studies using the current recommended dose of ICI drugs, and studies that included patients who were previously treated or untreated.

Patient and public involvement
------------------------------

No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for design or implementation of the study. No patients were asked to advise on interpretation or writing up of results. There are no plans to disseminate the results of the research to study participants or the relevant patient community. It was not evaluated whether the studies included in the review had any patient involvement.

Results
=======

Systematic review and characteristics
-------------------------------------

[Figure 1](#f1){ref-type="fig"} shows a flowchart of the study selection procedure. The electronic searches yielded 1794 potentially relevant studies, of which 92 potentially eligible articles were assessed. [Figure 2A](#f2){ref-type="fig"} shows that 36 randomised controlled trials (n=15 370) evaluating 15 treatments with different doses (nivolumab 2-3 mg/kg every two weeks, 0.3 mg/kg every three weeks, and 10 mg/kg every three weeks; ipilimumab 3 mg/kg or 10 mg/kg every three weeks and 3 mg/kg every four weeks; tremelimumab 10 mg/kg every 90 days; pembrolizumab 10 mg/kg every two weeks, 2 mg/kg or 10 mg/kg every three weeks, and 200 mg every three weeks; atezolizumab 1200 mg every three weeks; two ICI drugs; one ICI drug with conventional therapy; and conventional therapy) were included in the dose based network meta-analysis.[@ref7] [@ref8] [@ref9] [@ref10] [@ref11] [@ref12] [@ref13] [@ref14] [@ref15] [@ref16] [@ref17] [@ref18] [@ref19] [@ref20] [@ref21] [@ref22] [@ref23] [@ref24] [@ref25] [@ref26] [@ref27] [@ref28] [@ref29] [@ref30] [@ref31] [@ref32] [@ref33] [@ref34] [@ref35] [@ref36] [@ref37] [@ref38] [@ref39] [@ref40] [@ref41] [@ref42] [Figure 2B](#f2){ref-type="fig"} shows that 32 randomised controlled trials (n=14 252) evaluating eight treatments with different drugs (nivolumab, ipilimumab, tremelimumab, pembrolizumab, atezolizumab, two ICI drugs, one ICI drug with conventional therapy, and conventional therapy) were included in the drug based network meta-analysis.[@ref7] [@ref8] [@ref9] [@ref10] [@ref11] [@ref12] [@ref13] [@ref14] [@ref15] [@ref16] [@ref17] [@ref18] [@ref19] [@ref21] [@ref22] [@ref23] [@ref25] [@ref26] [@ref27] [@ref28] [@ref29] [@ref30] [@ref31] [@ref32] [@ref33] [@ref34] [@ref35] [@ref36] [@ref37] [@ref38] [@ref39] [@ref40] Both network meta-analyses included five updated randomised controlled trials without duplicate counting of the sample sizes.[@ref9] [@ref11] [@ref14] [@ref16] [@ref29] Ipilimumab 0.3 mg/kg every three weeks was reported to cause disease stabilisation only, which is obviously different from the enhanced effect of the objective response at the dose of 3 mg/kg and 10 mg/kg every three weeks; therefore, we discarded the group receiving ipilimumab 0.3 mg/kg every three weeks in the study by Wolchok and colleagues*.* [@ref42] [Figure 2A](#f2){ref-type="fig"} shows that among the 15 370 patients included in the dose based network meta-analysis, 2733 received nivolumab, 2113 received ipilimumab, 325 received tremelimumab, 2014 received pembrolizumab, 1210 received atezolizumab, 407 received two ICI drugs, 1429 received one ICI drug with conventional therapy, and 5139 received conventional therapy.

![Flowchart of study selection and design](xuc045146.f1){#f1}

![Network plots of comparisons for dose (A) and drug (B) based network meta-analyses. Each circular node represents a type of treatment. The circle size is proportional to the total number of patients (in parentheses). The width of lines is proportional to the number of studies performing head-to-head comparisons in the same study. ICI=immune checkpoint inhibitor](xuc045146.f2){#f2}

[Table 1](#tbl1){ref-type="table"} shows the baseline characteristics of the 31 studies (36 randomised controlled trials). Nineteen studies (61.3%) were phase III trials. Twenty eight (90.3%) were multinational trials. Cancer types tested in these studies included lung cancer (n=12), melanoma (13), urinary system cancer (4), head and neck cancer (1), and digestive system cancer (1). Two arm studies predominated (n=23, 74.2%) with only 25.8% (n=8) having three arms; the most common treatment arm was conventional therapy (n=22, 71.0%). Modified Jadad scores indicated that 25 studies were high quality and six studies were relatively low quality (supplementary materials, table S4).[@ref12] [@ref30] [@ref31] [@ref34] [@ref35] [@ref36]

###### 

Baseline characteristics of 31 studies (36 randomised controlled trials) for Bayesian network meta-analysis by cancer type

  First author, year                                                                                           Study ID        Region                                                                                                                         Trial phase   Total No   Safety analysis No   Arm   Treatment (median follow-up time, months)                                                                          CTCAE version   TrAE reporting rate[\*](#t1n1){ref-type="table-fn"}
  ------------------------------------------------------------------------------------------------------------ --------------- ------------------------------------------------------------------------------------------------------------------------------ ------------- ---------- -------------------- ----- ------------------------------------------------------------------------------------------------------------------ --------------- -----------------------------------------------------
  **Melanoma**                                                                                                                                                                                                                                                                                                                                                                                                                                       
  Resected stage III/IV melanoma:                                                                                                                                                                                                                                                                                                                                                                                                                    
   Weber, 2017[@ref7]                                                                                          CheckMate 238   MN                                                                                                                             III           906        452                  1     NIV 3 mg/kg every 2 weeks (19.5)                                                                                   4.0             ≥5
  453                                                                                                          2               IPI 10 mg/kg every 3 weeks (19.5)                                                                                                                                                                                                                                                                                     
  Untreated unresectable stage III/IV melanoma:                                                                                                                                                                                                                                                                                                                                                                                                      
   Wolchok, 2017[@ref8]/ Larkin, 2015[@ref9]                                                                   CheckMate 067   MN                                                                                                                             III           945        313                  1     NIV 3 mg/kg every 2 weeks (35.7)                                                                                   4.0             ≥2
  313                                                                                                          2               NIV 1 mg/kg and IPI 3 mg/kg every 3 weeks, followed by NIV 3 mg/kg every 2 weeks (38.0)                                                                                                                                                                                                                               
  311                                                                                                          3               IPI 3 mg/kg every 3 weeks (18.6)                                                                                                                                                                                                                                                                                      
   Robert (2), 2015[@ref32]                                                                                    CheckMate 066   MN                                                                                                                             III           418        206                  1     NIV 3 mg/kg every 2 weeks (8.9)                                                                                    4.0             ≥2
  205                                                                                                          2               DTIC 1000 mg/m^2^ every 3 weeks (6.8)                                                                                                                                                                                                                                                                                 
   Robert, 2011[@ref15]/ Maio, 2015[@ref16]                                                                    CA184-024       MN                                                                                                                             III           502        247                  1     IPI 10 mg/kg every 3 weeks and DTIC 850 mg/m^2^(Range: 36.6-54.0)                                                  3.0             ≥10
  251                                                                                                          2               DTIC 850 mg/m^2^(Range: 36.6-54.0)                                                                                                                                                                                                                                                                                    
   Postow, 2015[@ref28]/ Hodi, 2016[@ref29]                                                                    CheckMate 069   USA, France                                                                                                                    II            142        94                   1     NIV 1 mg/kg every 3 weeks and ipilimumab 3 mg/kg every 3 weeks (min 11.0)                                          4.0             ≥10
  46                                                                                                           2               IPI 3 mg/kg every 3 weeks (min 11.0)                                                                                                                                                                                                                                                                                  
   Ribas, 2013[@ref37]                                                                                         A3671009        MN                                                                                                                             III           655        325                  1     TRE 15 mg/kg every 90 days (11.8)                                                                                  3.0             ≥10
  316                                                                                                          2               ICC (oral TEM 200 mg/m^2^ every 4 weeks or DTIC 1000 mg/m^2^ every 3 weeks) (10.7)                                                                                                                                                                                                                                    
  Treated unresectable stage III/IV melanoma:                                                                                                                                                                                                                                                                                                                                                                                                        
   Robert (1), 2015[@ref10]/ Schachter, 2017[@ref11]                                                           Keynote 006     MN                                                                                                                             III           834        278                  1     PEM 10 mg/kg every 2 weeks (22.9)                                                                                  4.0             ≥10
  277                                                                                                          2               PEM 10 mg/kg every 3 weeks (22.9)                                                                                                                                                                                                                                                                                     
  256                                                                                                          3               IPI 3 mg/kg every 3 weeks (22.9)                                                                                                                                                                                                                                                                                      
   Ascierto, 2017[@ref24] [†](#t1n2){ref-type="table-fn"}                                                      CA184-169       MN                                                                                                                             III           727        364                  1     IPI 10 mg/kg every 3 weeks (14.5)                                                                                  3.0             NR
  362                                                                                                          2               IPI 3 mg/kg every 3 weeks (11.2)                                                                                                                                                                                                                                                                                      
   Wolchok, 2010[@ref42] [†](#t1n2){ref-type="table-fn"} [‡](#t1n3){ref-type="table-fn"}                       CA184-022       MN                                                                                                                             II            217        72                   1     IPI 10 mg/kg every 3 weeks (10.7)                                                                                  3.0             ≥5
  71                                                                                                           2               IPI 3 mg/kg every 3 weeks (8.7)                                                                                                                                                                                                                                                                                       
  71                                                                                                           3               IPI 0.3 mg/kg every 3 weeks (8.3)                                                                                                                                                                                                                                                                                     
  Ipilimumab refractory stage III/IV melanoma:                                                                                                                                                                                                                                                                                                                                                                                                       
   Larkin, 2017[@ref13]/ Weber, 2015[@ref14]                                                                   CheckMate 037   MN                                                                                                                             III           405        268                  1     NIV 3 mg/kg every 2 weeks (≈24.0)                                                                                  4.0             ≥1
  102                                                                                                          2                        ICC (DTIC 1000 mg/m^2^ every 3 weeks or CBP AUC=6 and PTX 175 mg/m^2^ every 3 weeks) (≈24.0)                                                                                                                                                                                                                 
   Ribas, 2015[@ref33]                                                                                         Keynote 002     MN                                                                                                                             II            540        178                  1     PEM 2 mg/kg every 3 weeks (10.0)                                                                                   4.0             ≥5
  179                                                                                                          2               PEM 10 mg/kg every 3 weeks (10.0)                                                                                                                                                                                                                                                                                     
  171                                                                                                          3               ICC (PTX and CBP, PTX, CBP, DTIC, oral TEM) (10.0)                                                                                                                                                                                                                                                                    
  Untreated advanced melanoma:                                                                                                                                                                                                                                                                                                                                                                                                                       
   Hersh, 2011[@ref40]                                                                                         MDX010-08       USA                                                                                                                            II            76         39                   1     IPI 3 mg/kg every 4 weeks (16.4)                                                                                   2.0             ≥10
  35                                                                                                           2               IPI 3 mg/kg every 4 weeks and DTIC 1250 mg/m^2^ every 3 weeks (20.9)                                                                                                                                                                                                                                                  
  Treated/untreated unresectable stage III/IV melanoma:                                                                                                                                                                                                                                                                                                                                                                                              
   Hamid, 2011[@ref41] [†](#t1n2){ref-type="table-fn"}                                                         CA184-004       MN                                                                                                                             II            82         40                   1     IPI 3 mg/kg every 3 weeks (8.9)                                                                                    3.0             NR
  42                                                                                                           2               IPI 10 mg/kg every 3 weeks (8.6)                                                                                                                                                                                                                                                                                      
  **Lung cancer**                                                                                                                                                                                                                                                                                                                                                                                                                                    
  Treated advanced/metastatic (stage IIIB/IV or recurrent) squamous/non-squamous non-small-cell lung cancer:                                                                                                                                                                                                                                                                                                                                         
   Rittmeyer, 2017[@ref12]                                                                                     OAK             MN                                                                                                                             III           1225       609                  1     ATE 1200 mg every 3 weeks (21.0)                                                                                   4.0             ≥10
  578                                                                                                          2               DOC 75 mg/m^2^ every 3 weeks (21.0)                                                                                                                                                                                                                                                                                   
   Fehrenbacher, 2016[@ref31]                                                                                  POPLAR          MN                                                                                                                             II            287        142                  1     ATE 1200 mg every 3 weeks (14.8)                                                                                   4.0             ≥5
  135                                                                                                          2               DOC 75 mg/m^2^ every 3 weeks (15.7)                                                                                                                                                                                                                                                                                   
  Untreated stage IV/recurrent squamous non-small-cell lung cancer:                                                                                                                                                                                                                                                                                                                                                                                  
   Govindan, 2017[@ref18]                                                                                      CA184-104       MN                                                                                                                             III           749        388                  1     PTX, CBP, and IPI 10 mg/kg every 3 weeks (12.5)                                                                    3.0             ≥5
  361                                                                                                          2               PTX and CBP (11.8)                                                                                                                                                                                                                                                                                                    
  Untreated PD-L1 positive stage IV/recurrent squamous/non-squamous non-small-cell lung cancer:                                                                                                                                                                                                                                                                                                                                                      
   Carbone, 2017[@ref21]                                                                                       CheckMate 026   MN                                                                                                                             III           541        267                  1     NIV 3 mg/kg every 2 weeks (13.5)                                                                                   4.0             ≥5
  263                                                                                                          2               ICC, Platinum based chemotherapy, 6 cycles (13.5)                                                                                                                                                                                                                                                                     
   Reck (1), 2016[@ref25]                                                                                      Keynote 024     MN                                                                                                                             III           305        154                  1     PEM 200 mg every 3 weeks (11.2)                                                                                    4.0             ≥10
  150                                                                                                          2               ICC (CBP and pemetrexed, DDP and pemetrexed, CBP and GEM, DDP and GEM, CBP and PTX) (11.2)                                                                                                                                                                                                                            
  Untreated extensive-disease small-cell lung cancer:                                                                                                                                                                                                                                                                                                                                                                                                
   Reck (2), 2016[@ref26]                                                                                      CA184-156       MN                                                                                                                             III           954        478                  1     IPI 10 mg/kg every 3 weeks, ETO, and DDP or CBP (10.5)                                                             3.0             ≥5
  476                                                                                                          2               ETO and DDP or CBP (10.2)                                                                                                                                                                                                                                                                                             
   Reck, 2013[@ref38]                                                                                          CA184-041       MN                                                                                                                             II            130        42                   1     CR (IPI 10 mg/kg every 2 weeks and PTX 175 mg/m^2^every 3 weeks or CBP AUC=6, followed by PTX or CBP) (min 11.1)   3.0             ≥15
  42                                                                                                           2               PR (PTX 175 mg/m^2^ every 3 weeks or CBP AUC=6, followed by IPI 10 mg/kg every 2 weeks and PTX or CBP) (min 11.1)                                                                                                                                                                                                     
  44                                                                                                           3               CR (PTX 175 mg/m^2^ every 3 weeks or CBP AUC=6) (min 11.1)                                                                                                                                                                                                                                                            
  Untreated stage IIIB/IV non-squamous non-small-cell lung cancer:                                                                                                                                                                                                                                                                                                                                                                                   
   Langer, 2016[@ref27]                                                                                        Keynote 021     USA, Taiwan                                                                                                                    II            123        59                   1     PEM 200 mg every 3 weeks, CBP AUC=5, and pemetrexed 500 mg/m^2^ every 3 weeks (10.6)                               4.0             ≥10
  62                                                                                                           2               CBP AUC=5 and pemetrexed 500 mg/m^2^ every 3 weeks (10.6)                                                                                                                                                                                                                                                             
  Treated PD-L1 positive advanced squamous/non-squamous non-small-cell lung cancer:                                                                                                                                                                                                                                                                                                                                                                  
   Herbst, 2016[@ref30]                                                                                        Keynote 010     MN                                                                                                                             II/III        1034       339                  1     PEM 2 mg/kg every 3 weeks (13.1)                                                                                   4.0             ≥10
  343                                                                                                          2               PEM 10 mg/kg every 3 weeks (13.1)                                                                                                                                                                                                                                                                                     
  309                                                                                                          3               DOC 75 mg/m^2^ every 3 weeks (13.1)                                                                                                                                                                                                                                                                                   
  Treated advanced/metastatic (stage IIIB/IV or recurrent) squamous non-small-cell lung cancer:                                                                                                                                                                                                                                                                                                                                                      
   Brahmer, 2015[@ref34]                                                                                       CheckMate 017   MN                                                                                                                             III           272        131                  1     NIV 3 mg/kg every 2 weeks (min 11.0)                                                                               4.0             ≥5
  129                                                                                                          2               DOC 75 mg/m^2^ every 3 weeks (min 11.0)                                                                                                                                                                                                                                                                               
  Treated advanced/metastatic (stage IIIB/IV or recurrent) non-squamous non-small-cell lung cancer:                                                                                                                                                                                                                                                                                                                                                  
   Borghaei, 2015[@ref35]                                                                                      CheckMate 057   MN                                                                                                                             III           582        287                  1     NIV 3 mg/kg every 2 weeks (min 13.2)                                                                               4.0             ≥5
  268                                                                                                          2               DOC 75 mg/m^2^ every 3 weeks (min 13.2)                                                                                                                                                                                                                                                                               
  Untreated stage IIIB/IV squamous/non-squamous non-small-cell lung cancer:                                                                                                                                                                                                                                                                                                                                                                          
   Lynch, 2012[@ref39]                                                                                         CA184-041       MN                                                                                                                             II            204        71                   1     CR (IPI 10 mg/kg every 2 weeks and PTX 175 mg/m^2^ every 3 weeks or CBP AUC=6, followed by PTX or CBP) (NR)        3.0             ≥15
  67                                                                                                           2               PR (PTX 175 mg/m^2^ every 3 weeks or CBP AUC=6, followed by IPI 10 mg/kg every 2 weeks and PTX or CBP) (NR)                                                                                                                                                                                                           
  65                                                                                                           3               CR (PTX 175 mg/m^2^ every 3 weeks or CBP AUC=6) (NR)                                                                                                                                                                                                                                                                  
  **Head and neck cancer**                                                                                                                                                                                                                                                                                                                                                                                                                           
  Treated recurrent/metastatic head and neck squamous-cell carcinoma:                                                                                                                                                                                                                                                                                                                                                                                
   Ferris, 2016[@ref17]                                                                                        CheckMate 141   MN                                                                                                                             III           361        236                  1     NIV 3 mg/kg every 2 weeks (5.1)                                                                                    4.0             ≥5
  111                                                                                                          2               ICC, standard single agent (MTX 40-60 mg/m^2^, DOC 30-40 mg/m^2^, CET 250 mg/m^2^ after a loading dose of 400 mg/m^2^) (5.1)                                                                                                                                                                                          
  **Urinary system cancer**                                                                                                                                                                                                                                                                                                                                                                                                                          
  Treated advanced renal cell carcinoma:                                                                                                                                                                                                                                                                                                                                                                                                             
   Motzer (1), 2015[@ref19]                                                                                    CheckMate 025   MN                                                                                                                             III           821        406                  1     NIV 3 mg/kg every 2 weeks (min 14.0)                                                                               4.0             ≥10
  397                                                                                                          2               EVE 10 mg orally once daily (min 14.0)                                                                                                                                                                                                                                                                                
   Motzer (2), 2015[@ref20] [†](#t1n2){ref-type="table-fn"}                                                    CA209-010       MN                                                                                                                             II            168        59                   1     NIV 0.3 mg/kg every 3 weeks (min 24.0)                                                                             4.0             ≥10
  54                                                                                                           2               NIV 2 mg/kg every 3 weeks (min 24.0)                                                                                                                                                                                                                                                                                  
  54                                                                                                           3               NIV 10 mg/kg every 3 weeks (min 24.0)                                                                                                                                                                                                                                                                                 
  Treated locally advanced/metastatic urothelial carcinoma:                                                                                                                                                                                                                                                                                                                                                                                          
   Bellmunt, 2017[@ref22]                                                                                      Keynote 045     MN                                                                                                                             III           542        266                  1     PEM 200 mg every 3 weeks (14.1)                                                                                    4.0             ≥5
  255                                                                                                          2                        ICC (PTX 175 mg/m^2^ every 3 weeks, DOC 75 mg/m^2^ every 3 weeks, or VIN 320 mg/m^2^ every 3 weeks) (14.1)                                                                                                                                                                                                   
   Powles, 2017[@ref36]                                                                                        IMvigor211      MN                                                                                                                             III           931        459                  1     ATE 1200 mg every 3 weeks (17.3)                                                                                   4.0             ≥5
  443                                                                                                          2               ICC (VIN 320 mg/m^2^ every 3 weeks, PTX 175 mg/m^2^ every 3 weeks, or DOC 75 mg/m^2^ every 3 weeks) (17.3)                                                                                                                                                                                                            
  **Digestive system cancer**                                                                                                                                                                                                                                                                                                                                                                                                                        
  Treated advanced/metastatic gastric or gastroesophageal junction cancer:                                                                                                                                                                                                                                                                                                                                                                           
   Bang, 2017[@ref23]                                                                                          CA184-162       MN                                                                                                                             II            114        57                   1     IPI 10 mg/kg every 3 weeks, and 10 mg/kg every 12 weeks maintenance (9.0)                                          3.0             ≥10
  45                                                                                                           2               Best supportive care (continuation of fluoropyrimidine chemotherapy or no active maintenance treatment) (12.0)                                                                                                                                                                                                        

CTCAE=Common Terminology Criteria for Adverse Events; TrAE=treatment-related adverse event; MN=multinational; PD-L1=programmed cell death ligand 1; NIV=nivolumab; IPI=ipilimumab; DTIC=dacarbazine; PEM=pembrolizumab; NR=not reported; ICC=investigator's choice chemotherapy; CBP=carboplatin; AUC=area under the curve; PTX=paclitaxel; TEM=temozolomide; TRE=tremelimumab; ATE=atezolizumab; DDP=cisplatin; GEM=gemcitabine; ETO=etoposide; DOC=docetaxel; CR=concurrent regimen; PR=phased regimen; MTX=methotrexate; CET=cetuximab; EVE=everolimus; VIN=vinflunine

Studies only reported the TrAE with an incidence in any arm equal to or greater than the listed rate; TrAEs were not available in the study by Robert-2011[@ref15]/Maio-2015,[@ref16] so all adverse events were used.

The four trials were excluded from the drug based network meta-analysis.

Arm 3 (IPI 0.3 mg/kg every 3 weeks) was not included in the network meta-analysis.

Dose based network meta-analysis
--------------------------------

The upfront dose based network meta-analysis was conducted to investigate the comparability of doses of the same drug. In the consistency model, we found non-significant differences in the risk of grade 1-5 or grade 3 or 4 adverse events between any two different doses of the same ICI drug, although ipilimumab 10 mg/kg every three weeks had a higher risk of grade 3 or 4 adverse events than 3 mg/kg every three weeks (odds ratio 3.08, 95% credibility interval 1.52 to 6.32). The results obtained using the consistency model fit well with the results using the inconsistency model; node splitting analyses showed no significant inconsistency (all P\>0.05; supplementary materials, figs S1 and S2 and table S5). These results indicated that the safety profile for each ICI drug, except for ipilimumab, was the same for each dose; this is a prerequisite for the drug based network meta-analysis.

Drug based network meta-analysis and inconsistency
--------------------------------------------------

The safety profile in the consistency model ([fig 3A](#f3){ref-type="fig"}) showed a noticeably lower risk of grade 1-5 adverse events favouring atezolizumab or nivolumab over ipilimumab, tremelimumab, two ICI drugs, one ICI drug with conventional therapy, and conventional therapy; ipilimumab or conventional therapy over one ICI drug with conventional therapy; and pembrolizumab over tremelimumab, one ICI drug with conventional therapy, and conventional therapy. [Figure 3A](#f3){ref-type="fig"} shows that for grade 3 or 4 adverse events, atezolizumab or nivolumab was significantly safer than ipilimumab, tremelimumab, two ICI drugs, one ICI drug with conventional therapy, and conventional therapy; ipilimumab was safer than two ICI drugs and one ICI drug with conventional therapy; pembrolizumab was safer than two ICI drugs, one ICI drug with conventional therapy, and conventional therapy; conventional therapy was safer than one ICI drug with conventional therapy. [Figure 3A](#f3){ref-type="fig"} shows that after combining the validation group (two high quality placebo controlled trials and six high quality single arm trials[@ref61] [@ref62] [@ref63] [@ref64] [@ref65] [@ref66] [@ref67] [@ref68]; supplementary materials, table S2 and S3), the pooled incidence of grade 1-5 adverse events for atezolizumab, nivolumab, pembrolizumab, and ipilimumab was 66.4%, 71.8%, 75.1%, and 86.8%, respectively. The pooled incidence of grade 3 or 4 adverse events for atezolizumab, nivolumab, pembrolizumab, and ipilimumab was 15.1%, 14.1%, 19.8%, and 28.6%, respectively. It was not applicable to pool the incidence of treatment-related adverse events for tremelimumab, as only one study was available.[@ref37] [Figure 3B](#f3){ref-type="fig"} shows that atezolizumab was associated with the best safety ranking for grade 1-5 adverse events (probability=76%), followed by nivolumab (56%), pembrolizumab (55%), ipilimumab (55%), conventional therapy (51%), two ICI drugs (30%), one ICI drug with conventional therapy (48%), and finally tremelimumab (54%). The safety ranking for grade 3 or 4 adverse events from high to low was: atezolizumab (49%), nivolumab (46%), pembrolizumab (72%), ipilimumab (51%), conventional therapy (54%), tremelimumab (28%), one ICI drug with conventional therapy (45%), and two ICI drugs (47%). The probabilities are detailed in the supplementary materials, table S6.

![Safety profile (A) and ranking curves (B) according to the drug based network meta-analysis (NMA) in the consistency model. Each cell of the safety profile contains the pooled odds ratios and 95% credibility intervals for grade 1-5 adverse events and grade 3 or 4 adverse events; significant results are in bold. The pooled odds ratios and 95% credibility intervals indicate the result of the top treatment compared with the bottom treatment. Ranking curves indicate the probability of the highest risk of grade 1-5 adverse events and grade 3 or 4 adverse events, the second highest, the third highest, and so on. ICI=immune checkpoint inhibitor](xuc045146.f3){#f3}

The results in the inconsistency model had a generally satisfactory fit compared with that of the results in the consistency model, except for some comparisons based on ipilimumab; node splitting analysis indicated no significant inconsistency (supplementary materials, table S7 and fig S3). [Figure 4](#f4){ref-type="fig"} shows the direct and indirect results for 11 head-to-head comparisons. Pairwise meta-analysis was feasible for six comparisons. Four comparisons (nivolumab, pembrolizumab, atezolizumab, and one ICI drug with conventional therapy) were versus conventional therapy. Two comparisons (nivolumab and two ICI drugs) were versus ipilimumab. The pairwise meta-analysis results showed outstanding consistency in both significance and tendency with respect to the corresponding network meta-analysis results. Significant between-study heterogeneity was found in almost all pairwise meta-analysis comparisons (Ι^2^=73% to 96%; all P\<0.05), but not atezolizumab versus conventional therapy (supplementary materials, fig S4 and S5). Among the remaining five comparisons based on one study, only ipilimumab versus conventional therapy showed obvious inconsistency between the direct (safety favoured conventional therapy) and indirect (safety favoured ipilimumab) results, representing the major source of inconsistency.

![Forest plots depicting the direct and indirect results of head-to-head comparisons. ICI=immune checkpoint inhibitor; NA=not applicable. \*Values in brackets are 95% CrIs.](xuc045146.f4){#f4}

Subgroup analysis based on specific treatment related adverse events
--------------------------------------------------------------------

Ten to 27 studies (4585 to 14 139 patients) were included in the subgroup analysis based on each specific grade 1-5 adverse event. [Table 2](#tbl2){ref-type="table"} shows that nivolumab was safer than tremelimumab for diarrhoea, and safer than ipilimumab for diarrhoea and colitis. Ipilimumab was safer than nivolumab for hyperthyroidism, and safer than pembrolizumab for arthralgia. Pembrolizumab and atezolizumab were safer than ipilimumab and tremelimumab for diarrhoea. All ICI drugs except for tremelimumab had a wider safety profile than two ICI drugs or one ICI drug with conventional therapy. Compared with conventional therapy, ICI drugs were associated predominantly with pruritus, rash, diarrhoea, colitis, hypothyroidism, hyperthyroidism, and pneumonitis; two ICI drugs or one ICI drug with conventional therapy was associated specifically with hepatic toxicities. [Figure 5](#f5){ref-type="fig"} shows that the ranking profile was generally consistent with the original network meta-analysis ranking with some new findings. Among the five ICI drugs, atezolizumab had the highest risk for causing hypothyroidism, nausea, and vomiting. Tremelimumab had the highest risk for causing rash, diarrhoea, and fatigue. The main adverse events caused by pembrolizumab were arthralgia, pneumonitis, and hepatic toxicities. Nivolumab and ipilimumab had non-overlapping toxicity results. Nivolumab mainly caused endocrine toxicities. Ipilimumab mainly caused skin, gastrointestinal, and renal toxicities. Integrated evidence implied that nivolumab was the best option for safety, relying on a narrow and mild toxicity spectrum.

###### 

Effect of treatment on each specific grade 1-5 adverse event. Values are odds ratios\* unless stated otherwise

  --------------------------------------------------- ------------- -------------- ---------- --------------- ------------- ------------ -------------- ------------------- ------------------- ----------------- ------------------- -------------------- --------------------- ----------------
  **Group**                                           **Fatigue**   **Pruritus**   **Rash**   **Diarrhoea**   **Colitis**   **Nausea**   **Vomiting**   **ALT increased**   **AST increased**   **Pneumonitis**   **CRE increased**   **Hypothyroidism**   **Hyperthyroidism**   **Arthralgia**
  No of studies                                       26            24             24         27              16            26           19             16                  16                  14                10                  19                   14                    17
  No of patients                                      13 641        12 371         12 371     14 139          8729          13 641       9761           7666                7666                6855              4585                9252                 7562                  9762
  Nivolumab as control                                                                                                                                                                                                                                                           
   Ipilimumab                                         1.14          0.48           0.69       *0.42*          *0.17*        0.75         0.43           0.63                0.68                1.02              0.33                2.68                 *2.58*                1.96
   Tremelimumab                                       0.84          0.55           0.24       *0.15*          0.99          0.61         0.52           NA                  NA                  NA                NA                  1.49                 0.72                  NA
   Pembrolizumab                                      1.22          0.82           0.87       0.93            0.66          0.88         1.09           0.47                0.49                0.37              1.85                0.96                 1.12                  0.60
   Atezolizumab                                       1.26          1.21           1.13       1.28            NA            0.61         0.49           NA                  NA                  0.58              NA                  0.67                 NA                    0.82
   Two ICI drugs                                      0.95          0.43           0.47       *0.29*          *0.13*        0.48         0.29           *0.12*              *0.15*              0.24              0.10                0.73                 *0.32*                0.96
   One ICI drug with conventional therapy             *0.55*        0.79           0.47       *0.31*          0.24          *0.23*       *0.28*         0.49                0.72                0.19              0.71                3.73                 1.53                  0.55
   Conventional therapy                               *0.70*        *4.76*         *2.19*     0.76            *4.80*        *0.31*       *0.38*         1.58                1.86                2.26              1.14                *15.69*              *7.58*                0.77
  Ipilimumab as control                                                                                                                                                                                                                                                          
   Tremelimumab                                       0.75          1.15           0.36       0.37            5.66          0.82         1.20           NA                  NA                  NA                NA                  0.56                 0.28                  NA
   Pembrolizumab                                      1.08          1.71           1.28       *2.24*          3.79          1.17         2.53           0.74                0.72                0.37              5.63                0.36                 0.44                  *0.31*
   Atezolizumab                                       1.11          2.49           1.65       *3.08*          NA            0.82         1.14           NA                  NA                  0.57              NA                  0.25                 NA                    0.42
   Two ICI drugs                                      0.84          0.90           0.68       0.70            0.74          0.65         0.67           *0.19*              *0.22*              0.24              0.32                0.27                 *0.13*                0.49
   One ICI drug with conventional therapy             *0.48*        1.65           0.68       0.74            1.37          *0.31*       0.64           0.77                1.06                0.19              2.05                1.38                 0.59                  0.28
   Conventional therapy                               0.62          *9.95*         *3.20*     *1.83*          *27.91*       *0.42*       0.88           2.51                2.74                2.23              3.47                *5.84*               2.92                  0.40
  Tremelimumab as control                                                                                                                                                                                                                                                        
   Pembrolizumab                                      1.44          1.48           3.56       *6.04*          0.65          1.45         2.07           NA                  NA                  NA                NA                  0.65                 1.57                  NA
   Atezolizumab                                       1.48          2.16           4.64       *8.32*          NA            1.00         0.94           NA                  NA                  NA                NA                  0.45                 NA                    NA
   Two ICI drugs                                      1.12          0.78           1.91       1.89            0.13          0.80         0.56           NA                  NA                  NA                NA                  0.50                 0.44                  NA
   One ICI drug with conventional therapy             0.65          1.43           1.92       1.99            0.24          0.38         0.53           NA                  NA                  NA                NA                  2.51                 2.11                  NA
   Conventional therapy                               0.83          *8.63*         *8.93*     *4.96*          5.04          0.51         0.73           NA                  NA                  NA                NA                  *10.58*              *10.57*               NA
  Pembrolizumab as control                                                                                                                                                                                                                                                       
   Atezolizumab                                       1.03          1.47           1.29       1.37            NA            0.69         0.45           NA                  NA                  1.57              NA                  0.69                 NA                    1.35
   Two ICI drugs                                      0.78          0.53           0.53       *0.31*          0.19          0.55         0.27           0.26                0.31                0.64              0.06                0.76                 0.28                  1.58
   One ICI drug with conventional therapy             *0.45*        0.97           0.54       *0.33*          0.36          *0.26*       *0.25*         1.04                1.48                0.53              0.38                3.85                 1.35                  0.92
   Conventional therapy                               *0.57*        *5.79*         *2.50*     0.82            *7.36*        *0.36*       *0.35*         3.38                3.81                *6.06*            0.62                *16.21*              *6.71*                1.29
  Atezolizumab as control                                                                                                                                                                                                                                                        
   Two ICI drugs                                      0.75          0.36           0.41       *0.23*          NA            0.80         0.59           NA                  NA                  0.42              NA                  1.12                 NA                    1.18
   One ICI drug with conventional therapy             *0.43*        0.66           0.41       *0.24*          NA            *0.38*       0.56           NA                  NA                  0.33              NA                  5.57                 NA                    0.68
   Conventional therapy                               *0.56*        3.96           1.93       0.59            NA            *0.51*       0.77           NA                  NA                  3.91              NA                  *23.47*              NA                    0.95
  Two ICI drugs as control                                                                                                                                                                                                                                                       
   One ICI drug with conventional therapy             0.57          1.83           1.00       1.06            1.89          0.48         0.95           4.03                4.76                0.79              6.67                5.01                 4.75                  0.58
   Conventional therapy                               0.74          *10.97*        *4.69*     *2.62*          *38.41*       0.64         1.30           *13.15*             *12.38*             9.41              11.00               *21.54*              *23.79*               0.81
  One ICI drug with conventional therapy as control                                                                                                                                                                                                                              
   Conventional therapy                               1.28          *5.99*         *4.68*     *2.48*          *20.80*       1.33         1.38           *3.24*              *2.59*              11.51             1.69                4.25                 *4.94*                1.40
  --------------------------------------------------- ------------- -------------- ---------- --------------- ------------- ------------ -------------- ------------------- ------------------- ----------------- ------------------- -------------------- --------------------- ----------------

ALT=alanine transaminase; AST=aspartate transaminase; CRE=blood creatinine; ICI=immune checkpoint inhibitor; NA=not applicable.

Significant values are shown in italics.

![Toxicity spectra and rankings in the subgroup analysis based on each specific grade 1-5 adverse event and cancer type. Immune checkpoint inhibitor (ICI) drugs are shown with a dark background. The number in each parenthesis represents the probability of risk to rank; values with the same rank are underlined.](xuc045146.f5){#f5}

Subgroup analysis based on cancer type
--------------------------------------

The subgroup analysis of lung cancer involved 12 studies and 6259 patients. [Figure 6A](#f6){ref-type="fig"} shows that a lower risk of grade 1-5 or grade 3 or 4 adverse events was favoured by nivolumab, pembrolizumab, and atezolizumab over one ICI drug with conventional therapy and conventional therapy; conventional therapy over one ICI drug with conventional therapy; and nivolumab over pembrolizumab. The subgroup analysis of melanoma involved 10 studies and 5318 patients. [Figure 6B](#f6){ref-type="fig"} shows that a lower risk of grade 1-5 or grade 3 and 4 adverse events was favoured by nivolumab and pembrolizumab over two ICI drugs and one ICI drug with conventional therapy; and conventional therapy over one ICI drug with conventional therapy. [Figure 5](#f5){ref-type="fig"} shows that all rankings were identical to those of the original network meta-analysis, with the exception of two substantial changes. Firstly, for lung cancer, nivolumab was safer than atezolizumab with an obviously higher probability in terms of grade 1-5 (70% *v* 60%) and grade 3 or 4 (93% *v* 74%) adverse events. Secondly, for melanoma, ipilimumab was less safe than conventional therapy, with an increased risk of grade 1-5 and grade 3 or 4 adverse events.

![Safety profiles in the subgroup analysis based on lung cancer (A) and melanoma (B). Each cell contains the pooled odds ratios and 95% credibility intervals for grade 1-5 adverse events or grade 3 or 4 adverse events; significant results are in bold. ICI=immune checkpoint inhibitor](xuc045146.f6){#f6}

Sensitivity analysis
--------------------

Individual sensitivity analysis of 21 high quality studies,[@ref7] [@ref8] [@ref9] [@ref10] [@ref11] [@ref13] [@ref14] [@ref15] [@ref16] [@ref17] [@ref18] [@ref19] [@ref21] [@ref22] [@ref23] [@ref25] [@ref26] [@ref27] [@ref28] [@ref29] [@ref32] [@ref33] [@ref37] [@ref38] [@ref39] [@ref40] 26 studies explicitly reporting treatment-related adverse events,[@ref7] [@ref8] [@ref9] [@ref10] [@ref11] [@ref12] [@ref13] [@ref14] [@ref17] [@ref18] [@ref19] [@ref21] [@ref22] [@ref23] [@ref25] [@ref26] [@ref27] [@ref28] [@ref29] [@ref30] [@ref31] [@ref32] [@ref33] [@ref34] [@ref35] [@ref36] [@ref37] [@ref38] [@ref39] [@ref40] and 13 studies including patients who were previously untreated gave stable outcomes without any changes compared with the original network meta-analysis.[@ref8] [@ref9] [@ref15] [@ref16] [@ref18] [@ref21] [@ref25] [@ref26] [@ref27] [@ref28] [@ref29] [@ref32] [@ref37] [@ref38] [@ref39] [@ref40] The ranking order of pembrolizumab was slightly different in the sensitivity analysis based on 21 studies that used the current recommended dosage of ICI drugs,[@ref8] [@ref9] [@ref12] [@ref13] [@ref14] [@ref15] [@ref16] [@ref17] [@ref18] [@ref19] [@ref21] [@ref22] [@ref25] [@ref26] [@ref27] [@ref28] [@ref29] [@ref31] [@ref32] [@ref34] [@ref35] [@ref36] [@ref37] [@ref38] [@ref39] and this alteration only occurred between pembrolizumab and nivolumab. Individual sensitivity analysis based on 13 studies that included patients who were previously treated,[@ref7] [@ref12] [@ref13] [@ref14] [@ref17] [@ref19] [@ref22] [@ref23] [@ref30] [@ref31] [@ref33] [@ref34] [@ref35] [@ref36] 18 phase III studies,[@ref7] [@ref8] [@ref9] [@ref10] [@ref11] [@ref12] [@ref13] [@ref14] [@ref15] [@ref16] [@ref17] [@ref18] [@ref19] [@ref21] [@ref22] [@ref25] [@ref26] [@ref32] [@ref34] [@ref35] [@ref36] [@ref37] and 15 studies on programmed cell death 1 and programmed cell death ligand 1 ICI drugs suggested that nivolumab was safer than atezolizumab with a lower risk of grade 3 or 4 adverse events (supplementary materials, table S8).[@ref12] [@ref13] [@ref14] [@ref17] [@ref19] [@ref21] [@ref22] [@ref25] [@ref27] [@ref30] [@ref31] [@ref32] [@ref33] [@ref34] [@ref35] [@ref36]

Discussion
==========

ICI drugs act through distinct immunologic mechanisms and should not be regarded as one entity. Given that a comprehensive toxicity profile for ICI drugs remains to be clearly defined, we included 36 head-to-head phase II and III randomised controlled trials (15 370 patients) in the first network meta-analysis of the comparative safety of ICI drugs. The general safety of ICI drugs ranked from high to low is: atezolizumab, nivolumab, pembrolizumab, ipilimumab, and tremelimumab. There were differences in the toxicity spectra among atezolizumab (hypothyroidism, nausea, and vomiting), nivolumab (endocrine toxicities), pembrolizumab (arthralgia, pneumonitis, and hepatic toxicities), ipilimumab (skin, gastrointestinal, and renal toxicities), and tremelimumab (rash, diarrhoea, and fatigue). Integrated evidence based on the pooled incidences, subgroup, and sensitivity analyses implied that nivolumab is the safest ICI drug, especially for lung cancer.

Strengths and implications
--------------------------

A good exchangeability assumes that the estimates of the study effects, characteristics of participants, and the common treatment in different trials are similar and exchangeable.[@ref69] Lack of exchangeability is associated with the development of inconsistency and the disregard of similarity and transitivity. Some previous meta-analyses on this topic included placebo controlled trials and phase I trials.[@ref46] [@ref47] [@ref70] [@ref71] [@ref72] Given that patients with mild disease tend to be recruited in placebo controlled trials for ethical and safety reasons,[@ref73] and phase I trials have small sample sizes focusing mainly on pharmacokinetics and pharmacometrics, we included head-to-head phase II and III trials exclusively to avoid potential bias and confounding effects. Furthermore, all these previous meta-analyses ignored the influence of different drug doses.[@ref46] [@ref47] [@ref70] [@ref71] [@ref72] Combining the estimates of different doses of the same drug in a meta-analysis without knowing the individual doses can seriously damage the consistency and quality of the results. With stringent inclusion criteria and a tailored study design (dose and drug based network meta-analysis), our network meta-analysis has an excellent transitivity and the results have high validity and reliability. This information will aid clinicians to manage life threatening treatment-related adverse events, optimise future trial design, and modify the prescribing of ICI drugs.

Similar to observations from previous studies investigating the safety of ICI drugs, we confirmed that anti-cytotoxic T lymphocyte associated antigen 4 ICI drugs have generally higher toxicities than anti-programmed cell death ligand 1 ICI drugs, and one ICI drug with conventional therapy is less safe than ICI drugs on their own.[@ref74] [@ref75] Of note, we have obtained some unique findings. Firstly, some treatment-related adverse events (eg, pneumonitis and hepatic toxicity) are fatal, warranting a thorough investigation of the toxicity profiles for all ICI drugs.[@ref43] [@ref70] However, previous studies were limited to a superficial level, failing to resolve this issue satisfactorily. Khoja and colleagues suggested that pneumonitis, hypothyroidism, and arthralgia were more commonly associated with anti-programmed cell death 1 ICI drugs than with anti-cytotoxic T lymphocyte associated antigen 4 ICI drugs.[@ref71] Focusing specifically on each ICI drug, we found that pembrolizumab had a higher risk of pneumonitis and arthralgia; nivolumab had a higher risk of hypothyroidism. Hypotheses that can account for this finding include tumour microenvironment, expression of immune related genes, and individual response to ICI drugs, although the underlying mechanisms are not yet clear.[@ref76] A recent network meta-analysis showed non-significant differences in anti-tumour efficacy for lung cancer between nivolumab, pembrolizumab, and atezolizumab,[@ref77] particularly highlighting the importance of investigating the safety discrepancy between ICI drugs in that safety is the critical factor for drug evaluation. Secondly, this study indicated atezolizumab had the highest risk of hypothyroidism, nausea, and vomiting among the five ICI drugs, and nivolumab was the best choice based on both general (ranking) and specific (toxicity spectrum) evidence. This is somewhat surprising given that the anti-programmed cell death ligand 1 drug atezolizumab is most commonly considered to be the safest ICI drug owing to its programmed cell death ligand 2 sparing ability that preserves normal immunological homoeostasis. These findings will be helpful in selecting the most appropriate ICI drug for patients with cancer with different clinical backgrounds.

As mentioned previously, the unequal doses of ipilimumab was a major source of inconsistency, especially reflected in the direct results of ipilimumab versus conventional therapy based on one study. Since this study compared ipilimumab at 10 mg/kg every three weeks with the best supportive care (chemotherapy or no active maintenance treatment),[@ref23] the direct results cannot be representative of all doses of ipilimumab. Furthermore, the control group containing no active maintenance treatment is weaker and therefore, has fewer toxicities than full chemotherapy used as a control, which also led to inconsistency. Nonetheless, results of other ICI drugs showed satisfactory consistency and the sensitivity analysis further confirmed the stability of our results, providing assurance that this network meta-analysis has led to a better understanding of treatment-related adverse events of ICI drugs.

Limitations
-----------

Several limitations of this study should be stated. Firstly, some treatment-related adverse events are late onset diseases, at which point, patients may have already been discharged from hospital. Pneumonitis is reported to occur between 7.4 and 24.3 months after taking ICI drugs.[@ref4] Additionally, the median follow-up time was different for all included studies, ranging from 5.1 to 38.0 months, and the updated studies provide long term results. As such, the frequency of treatment-related adverse events may not only be caused by ICI drugs, but also by the confounding effect of time. Secondly, studies only reported the treatment-related adverse event with an incidence above a certain percentage ranging between ≥1% and ≥15%. One possible reason is that the method for recognising treatment-related adverse events has not been standardised, usually depending on investigators' evaluation or self reporting by patients, or both. Thirdly, heterogeneity between the included studies---the weakness of meta-analysis---was generally present in this network meta-analysis, manifesting in the difference of follow-up time, cancer type, programmed cell death ligand 1 expression level, and so on. The studies included in this network meta-analysis used different cut-off values to determine programmed cell death ligand 1 positive patients, while a positive association has been identified between tumour programmed cell death ligand 1 expression levels and survival benefits.[@ref77] In addition, the disparity in the common comparator (eg, chemotherapy) among the included studies could have undermined the exchangeability of the network meta-analysis to some extent. Supplementary materials, table S9 details the severity of toxicity and the safety profile of different chemotherapy strategies in 24 studies.[@ref12] [@ref13] [@ref14] [@ref15] [@ref16] [@ref17] [@ref18] [@ref19] [@ref21] [@ref22] [@ref23] [@ref25] [@ref26] [@ref27] [@ref30] [@ref31] [@ref32] [@ref33] [@ref34] [@ref35] [@ref36] [@ref37] [@ref38] [@ref39] Single agent dacarbazine and the best supportive care was associated with obviously lower incidence of grade 1-5 and grade 3 or 4 adverse events compared with other strategies, in line with a previous meta-analysis that compared the toxicity of five commonly used drugs to treat patients with melanoma.[@ref78] The study indicated that dacarbazine has the lowest incidence rates of adverse events, ranging from 0.00008 to 0.0001 cases per 100 person years.[@ref78] As most studies adopted investigator's choice chemotherapy comprising several drugs and combinations, the chemotherapy strategies had varied and irregular safety profiles. All between-study discrepancies listed here could subject the pairwise meta-analysis results to high heterogeneity. Fourthly, the anti-programmed cell death 1 ICI drug trials were generally started later than the anti-cytotoxic T lymphocyte associated antigen 4 ICI drug trials. As the understanding and management of treatment-related adverse events have improved over time, it could lead to superior safety of anti-programmed cell death 1 ICI drugs over anti-cytotoxic T lymphocyte associated antigen 4 ICI drugs. Another caveat that is a cause for concern is the relation of cancer types and treatment-related adverse events. Patients with lung cancer are reported to have a higher risk of treatment-related pneumonitis from an ICI drug than patients with melanoma.[@ref72] Given that our findings are based entirely on clinical trials without available comorbidity data, future studies are needed for validation in the real world, in which patients with cancer are usually in a less favourable condition owing to high rates of comorbidity and competing risks.

Conclusions
-----------

In this dose and drug based network meta-analysis, clinically important differences in safety exist among ICI drugs for patients with cancer. We found that atezolizumab had the best safety profile in general, and nivolumab had the best safety profile in lung cancer when taking and integrated approach. The safety ranking of treatments based on ICI drugs is modulated by specific treatment-related adverse events.

### What is already known on this topic

1.  Adverse events are a cause for increasing concern regarding the comparative safety of immune checkpoint inhibitor (ICI) drugs in the treatment of patients with cancer

2.  Different ICI drugs and doses can have different toxicity features

3.  Pairwise meta-analyses are limited to comparisons of two drugs, failing to generate a complete picture of the toxicity profile of ICI drugs

### What this study adds

1.  Based on 36 eligible studies and 15 370 patients, the rank of ICI drugs from high to low with respect to general safety assessed by grade 1-5 or grade 3 or 4 adverse events is: atezolizumab (pooled incidence of 66.4% and 15.1%, respectively), nivolumab (71.8% and 14.1%), pembrolizumab (75.1% and 19.8%), ipilimumab (86.8% and 28.6%), and tremelimumab (not applicable)

2.  Nivolumab had the best safety profile in lung cancer when taking an integrated approach

We thank Brigette BY Ma, from the Chinese University of Hong Kong, for her helpful comments on this study.

Extra material supplied by the author

###### 

**Supplementary materials**: Supplementary tables S1-S9 and figures S1-S5

Contributors: The first four authors (CX, Y-PC, X-JD, and J-QL) contributed equally to this work. JM conceived, designed, and supervised the project. JM performed the quality assessment, review, and approval of the manuscript. CX, X-JD, Y-PC, and J-QL contributed to the design of the study, writing the protocol, data preparation, analysis and interpretation, and writing and final approval of the report. CX and X-JD identified eligible trials, extracted the data, and assessed the quality of included trials. CX, J-QL, C-LH, L-LT, LC, G-QZ, W-FL, and Y-PM processed the data and generated the tables and figures. CX, A-HL, and QL were responsible for statistical analysis and interpretation. CX, Y-PC, L-LT, and LC contributed to the interpretation of the data. CX, X-JD, and J-QL drafted the manuscript. CX, Y-PC, YS, and CH revised the manuscript before submission. CX, Y-PC, and J-QL participated in the formal revision, including statistical analysis, data processing, production of figures and tables, and text modification. All authors had full access to the data in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. JM is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: This work was supported by grants from the Natural Science Foundation of Guang Dong Province (No 2017A030312003), Health and Medical Collaborative Innovation Project of Guangzhou City (No 201803040003), Innovation Team Development Plan of the Ministry of Education (No IRT_17R110), and Overseas Expertise Introduction Project for Discipline Innovation (111 Project, No B14035). The sponsors or funders had no involvement in any parts of this study. All authors confirmed the independence of researchers from funding sources.

Competing interests: All authors have completed the ICMJE uniform disclosure form at [www.icmje.org/coi_disclosure.pdf](http://www.icmje.org/coi_disclosure.pdf) and declare: support from the Natural Science Foundation of Guang Dong Province, Health and Medical Collaborative Innovation Project of Guangzhou City, Innovation Team Development Plan of the Ministry of Education, and Overseas Expertise Introduction Project for Discipline Innovation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Ethical approval: Not required.

Data sharing: No additional data available.

The manuscript's guarantor (JM) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned and registered have been explained.
