The value of incorporating patient-consulted medication reconciliation in influencing drug-related actions in the outpatient rheumatology setting
Transitions in healthcare are associated with a high risk of medication errors [ – ]. These errors are often caused by medication discrepancies between patient’s actual medication use and the medication that is listed in a patient’s medical record. Although not all medication discrepancies are clinically relevant, some medication discrepancies result in drug interactions, therapeutic duplication, and other unintended adverse events [ , , ]. If not recognized early, these medication discrepancies will lead to an increased risk of hospital readmissions, emergency room visits and prolonged hospital stay [ , – ]. Up to 100% of the patients admitted to a hospital have at least one medication discrepancy , of which half of these have the potential to harm patients [ – ]. In the outpatient setting, more than half of the patients visiting the outpatient clinic have at least one medication discrepancy [ – ]. Although the proportion of outpatients with a medication discrepancy is lower, the large number of patients visiting the outpatient clinics results in a high number of patients with at least one medication discrepancy. Because of this, it is important to identify outpatients at risk and prevent harm to those patients. Medication reconciliation (MR) has been proven as an effective method to reduce medication discrepancies. Most studies demonstrate that MR approximately halves the number of medication discrepancies both in the inpatient and outpatient setting [ , , , ]. Besides reducing medication discrepancies, MR has the ability to identify medication non-adherence, low medication literacy, and use of non-prescribed medications such as supplements and vitamins . During MR, the best possible medication history (BPMH) is composed according to the standard operating procedure of the High 5 s project of the World Health Organization . The World Health Organization defined the BPMH as “a medication history obtained by a clinician which includes a thorough history of all regular medication use (prescribed and non-prescribed), using a number of different sources of information” . Drug information sources that are usually available in the Netherlands for MR are the electronical medical record (EMR) and the Nationwide Medication Record System (NMRS) (Table ). To obtain the BPMH in this setting, pharmacy technicians combine the information provided by a structured patient interview, information from the EMR and information from the NMRS. Inpatients are assessed in the first 24 hours of their admission . This is important because studies shows that Dutch inpatients have on average three medication discrepancies [ , , ]. Also, in the outpatient setting about 83% of the patients have at least one medication discrepancy . However, outpatient MR, such as for the outpatient rheumatology clinic, is not implemented at all Dutch hospitals. At patient’s discharge, all drug changes are communicated back to the patient’s community pharmacy and general practitioner . According to the World Health Organization, MR is classified as one of the top five strategies to ensure patient safety and it is recommended or even mandated by several guidelines and accreditation standards [ , , , ]. However, given the time spent and cost to perform MR with each patient (up to 30 minutes) and the large number of patients visiting the outpatient clinic, healthcare organizations struggle to implement MR, especially in settings with a high patient turn-over such as the outpatient clinic [ – ]. Because of this, MR should be targeted to outpatients with a high risk of medication discrepancies that affect the clinical decisions made by the prescriber. To achieve a more targeted and consequently efficient implementation of MR, it can be hypothesized that for some (low risk) patients MR based on only digital sources might be sufficient, whereas for some (high risk) patients information from the patient in addition to the digital information is essential for the prescriber to perform drug related actions at the outpatient setting. However, no information is available in the literature how often patient reported information during MR leads to other clinical decisions compared to a situation in which only digital information was available. Therefore, the primary aim of this study is to determine the proportion of outpatient visits in which information obtained with MR led to additional drug-related actions. 

Setting A cross-sectional observational study was conducted at the outpatient rheumatology clinic at the Sint Maartenskliniek, Nijmegen, the Netherlands. During this study, outpatient visits were observed in the period between October 31 and November 29, 2018. Ethical considerations The study (file number: 2021–13,272) was approved by the Medical Research Ethics Committee of Arnhem-Nijmegen, the Netherlands. Informed consent was obtained (by the observed rheumatologists) from all individual participants included in the study. Selection of observation days and healthcare practitioners A sample of consultations was taken by selection of consultations with various rheumatologists or physician assistants on as many different dayparts (morning or afternoon) of the week as possible. There is no difference in patient characteristics and/or type of consultations per part of the day and/or practitioner. The selection of the observation dayparts and practitioners was performed by a medical secretary who was not part of the research team and all practitioners were eligible. All consecutive visits (if patients agreed to participate) of the selected practitioner were observed and each selected practitioner was observed for a maximum of one daypart. Participants All consecutive patients visiting the selected rheumatologist on an observation day were eligible. Inclusion criteria were age ≥ 18 years, able to speak and understand the Dutch language and that MR had been performed in the 2 weeks prior to the outpatient visit. Patients were included after obtaining verbal consent. Patients were excluded when MR was not performed prior to the visit or when the rheumatologist requested not to observe a specific patient (for example if the patient was familiar to the observer or was nervous). Study design A schematic representation of the study design is depicted in Fig. . Prior to the outpatient visit Fourteen to 3 days before the visit, MR was performed with the outpatient by telephone or online using a personal health record. During MR, the BPMH was created based on patient’s input and drug information derived from digital sources (the EMR and NMRS). Observation during the visits Each visit was observed by a single pharmacy technician. The nine rheumatologists who conducted the outpatient visits were not blinded about the aim of the study. The pharmacy technician was trained to observe and report all drug-related actions made by the rheumatologist, based on a standardized observation scheme and data collection form (see Additional file ). Drug-related actions performed during an outpatient visit covered changing drugs and discuss drug-related problems (DRPs), defined as events or circumstances involving drug therapies that actually or potentially interfere with an optimal outcome of medical care . Measurements directly after the visits: assessment by the rheumatologists After each outpatient visit, the rheumatologist was interviewed about which information of the BPMH (the presence of a drug, drug strength and/or drug frequency) he/she had used to perform each of the drug-related actions. The rheumatologists did not have the opportunity to discuss the required information with the other rheumatologists. Subsequently, a researcher determined if this required information was also available in the digital sources. Assessment by the expert panel In 2020, an expert panel consisting of two (of the nine observed) rheumatologists and two pharmacists scored which drug-related information was necessary for the drug-related actions that had been performed during the visits. This was done to get insight into inter-individual differences between rheumatologists about the required drug information to perform the drug-related actions and to get insight into potential underreporting of required drug information caused by time constraints during the interviews after the visits. First, the members of the expert panel (VH, BvdB, AvdM and JvZ) were individually asked to construct a generic assessment form (see Additional file ) to consistently define which information is essential to perform 14 observed types of drug-related actions. In 9 of the 14 types of drug-related actions consensus was reached (Additional file ). Based on the created assessment form, one researcher (DN) scored for all individual drug-related actions which belong to those 9 types of drug related-actions, which information of the BPMH was required to perform these actions. For the remaining 5 of the 14 types of drug-related actions with could not be assessed with the generic assessment form, the individual drug-related actions belonging to those 5 types of drug-related actions were assessed by two members of the expert panel (VH and AvdM). In case of disagreement about the required drug-information for an individual drug-related action, the case was discussed with a third member of the expert panel (BvdB) until consensus was achieved. Finally, a researcher (DN) determined if the required information was also available in the digital sources. Outcome measures The primary outcome of the study was the proportion of outpatient visits in which the patient provided essential drug information in addition to the existing digital information (including the EMR and NMRS) during MR that was required for the rheumatologist to perform drug-related actions. The secondary endpoint was the proportion of outpatient visits in which the patient provided essential information during MR that was not available in the EMR only. Data collection Patients’ drug information was collected from three sources: the NMRS, the EMR and the patient (Table ). The drug information of the EMR and the NMRS were combined to reflect the situation that both information sources were available for the rheumatologist. Besides that, the BPMH was created based on drug information obtained from the patient and information derived from the EMR and the NMRS. Drug-related actions performed by the rheumatologists were collected during the visits by the observer. A change in a drug prescription was defined as a change in one of a patient’s drugs as compared to the BPMH. Drug changes were categorized into the following types: start a drug, stop a drug, adjust the dose or frequency of a current drug . DRPs (e.g. experiencing adverse drug reactions, worrying about (potential adverse events of) a drug or using drug in a different way than prescribed), were classified using the DOCUMENT classification system, with modifications as described by Kwint et al. . DRPs were classified independently by two researchers (VH and BvdB). Statistical analysis Three steps were performed to determine the proportion of outpatient visits in which the patient provided essential additional information during MR compared to the digital sources. First, for each drug-related action the researcher determined if the required drug information was available in the EMR, NMRS and BPMH. Second, the number of outpatient visits for which all required drug information was only available in the BPMH (and not in EMR and NMRS) were counted. Third, the proportion of outpatient visits in which patient’s input provided additional essential information compared to the EMR (plus NMRS) was determined. Data were analysed using IBM SPSS Statistics software version 25. Additional descriptive statistics were provided using mean (± standard deviation [SD]) or median (interquartile range [IQR]) values depending on the (non-)parametric distribution of measured variables. 

A cross-sectional observational study was conducted at the outpatient rheumatology clinic at the Sint Maartenskliniek, Nijmegen, the Netherlands. During this study, outpatient visits were observed in the period between October 31 and November 29, 2018. 

The study (file number: 2021–13,272) was approved by the Medical Research Ethics Committee of Arnhem-Nijmegen, the Netherlands. Informed consent was obtained (by the observed rheumatologists) from all individual participants included in the study. 

A sample of consultations was taken by selection of consultations with various rheumatologists or physician assistants on as many different dayparts (morning or afternoon) of the week as possible. There is no difference in patient characteristics and/or type of consultations per part of the day and/or practitioner. The selection of the observation dayparts and practitioners was performed by a medical secretary who was not part of the research team and all practitioners were eligible. All consecutive visits (if patients agreed to participate) of the selected practitioner were observed and each selected practitioner was observed for a maximum of one daypart. 

All consecutive patients visiting the selected rheumatologist on an observation day were eligible. Inclusion criteria were age ≥ 18 years, able to speak and understand the Dutch language and that MR had been performed in the 2 weeks prior to the outpatient visit. Patients were included after obtaining verbal consent. Patients were excluded when MR was not performed prior to the visit or when the rheumatologist requested not to observe a specific patient (for example if the patient was familiar to the observer or was nervous). 

A schematic representation of the study design is depicted in Fig. . Prior to the outpatient visit Fourteen to 3 days before the visit, MR was performed with the outpatient by telephone or online using a personal health record. During MR, the BPMH was created based on patient’s input and drug information derived from digital sources (the EMR and NMRS). Observation during the visits Each visit was observed by a single pharmacy technician. The nine rheumatologists who conducted the outpatient visits were not blinded about the aim of the study. The pharmacy technician was trained to observe and report all drug-related actions made by the rheumatologist, based on a standardized observation scheme and data collection form (see Additional file ). Drug-related actions performed during an outpatient visit covered changing drugs and discuss drug-related problems (DRPs), defined as events or circumstances involving drug therapies that actually or potentially interfere with an optimal outcome of medical care . Measurements directly after the visits: assessment by the rheumatologists After each outpatient visit, the rheumatologist was interviewed about which information of the BPMH (the presence of a drug, drug strength and/or drug frequency) he/she had used to perform each of the drug-related actions. The rheumatologists did not have the opportunity to discuss the required information with the other rheumatologists. Subsequently, a researcher determined if this required information was also available in the digital sources. Assessment by the expert panel In 2020, an expert panel consisting of two (of the nine observed) rheumatologists and two pharmacists scored which drug-related information was necessary for the drug-related actions that had been performed during the visits. This was done to get insight into inter-individual differences between rheumatologists about the required drug information to perform the drug-related actions and to get insight into potential underreporting of required drug information caused by time constraints during the interviews after the visits. First, the members of the expert panel (VH, BvdB, AvdM and JvZ) were individually asked to construct a generic assessment form (see Additional file ) to consistently define which information is essential to perform 14 observed types of drug-related actions. In 9 of the 14 types of drug-related actions consensus was reached (Additional file ). Based on the created assessment form, one researcher (DN) scored for all individual drug-related actions which belong to those 9 types of drug related-actions, which information of the BPMH was required to perform these actions. For the remaining 5 of the 14 types of drug-related actions with could not be assessed with the generic assessment form, the individual drug-related actions belonging to those 5 types of drug-related actions were assessed by two members of the expert panel (VH and AvdM). In case of disagreement about the required drug-information for an individual drug-related action, the case was discussed with a third member of the expert panel (BvdB) until consensus was achieved. Finally, a researcher (DN) determined if the required information was also available in the digital sources. 

Fourteen to 3 days before the visit, MR was performed with the outpatient by telephone or online using a personal health record. During MR, the BPMH was created based on patient’s input and drug information derived from digital sources (the EMR and NMRS). 

Each visit was observed by a single pharmacy technician. The nine rheumatologists who conducted the outpatient visits were not blinded about the aim of the study. The pharmacy technician was trained to observe and report all drug-related actions made by the rheumatologist, based on a standardized observation scheme and data collection form (see Additional file ). Drug-related actions performed during an outpatient visit covered changing drugs and discuss drug-related problems (DRPs), defined as events or circumstances involving drug therapies that actually or potentially interfere with an optimal outcome of medical care . 

After each outpatient visit, the rheumatologist was interviewed about which information of the BPMH (the presence of a drug, drug strength and/or drug frequency) he/she had used to perform each of the drug-related actions. The rheumatologists did not have the opportunity to discuss the required information with the other rheumatologists. Subsequently, a researcher determined if this required information was also available in the digital sources. 

In 2020, an expert panel consisting of two (of the nine observed) rheumatologists and two pharmacists scored which drug-related information was necessary for the drug-related actions that had been performed during the visits. This was done to get insight into inter-individual differences between rheumatologists about the required drug information to perform the drug-related actions and to get insight into potential underreporting of required drug information caused by time constraints during the interviews after the visits. First, the members of the expert panel (VH, BvdB, AvdM and JvZ) were individually asked to construct a generic assessment form (see Additional file ) to consistently define which information is essential to perform 14 observed types of drug-related actions. In 9 of the 14 types of drug-related actions consensus was reached (Additional file ). Based on the created assessment form, one researcher (DN) scored for all individual drug-related actions which belong to those 9 types of drug related-actions, which information of the BPMH was required to perform these actions. For the remaining 5 of the 14 types of drug-related actions with could not be assessed with the generic assessment form, the individual drug-related actions belonging to those 5 types of drug-related actions were assessed by two members of the expert panel (VH and AvdM). In case of disagreement about the required drug-information for an individual drug-related action, the case was discussed with a third member of the expert panel (BvdB) until consensus was achieved. Finally, a researcher (DN) determined if the required information was also available in the digital sources. 

The primary outcome of the study was the proportion of outpatient visits in which the patient provided essential drug information in addition to the existing digital information (including the EMR and NMRS) during MR that was required for the rheumatologist to perform drug-related actions. The secondary endpoint was the proportion of outpatient visits in which the patient provided essential information during MR that was not available in the EMR only. 

Patients’ drug information was collected from three sources: the NMRS, the EMR and the patient (Table ). The drug information of the EMR and the NMRS were combined to reflect the situation that both information sources were available for the rheumatologist. Besides that, the BPMH was created based on drug information obtained from the patient and information derived from the EMR and the NMRS. Drug-related actions performed by the rheumatologists were collected during the visits by the observer. A change in a drug prescription was defined as a change in one of a patient’s drugs as compared to the BPMH. Drug changes were categorized into the following types: start a drug, stop a drug, adjust the dose or frequency of a current drug . DRPs (e.g. experiencing adverse drug reactions, worrying about (potential adverse events of) a drug or using drug in a different way than prescribed), were classified using the DOCUMENT classification system, with modifications as described by Kwint et al. . DRPs were classified independently by two researchers (VH and BvdB). 

Three steps were performed to determine the proportion of outpatient visits in which the patient provided essential additional information during MR compared to the digital sources. First, for each drug-related action the researcher determined if the required drug information was available in the EMR, NMRS and BPMH. Second, the number of outpatient visits for which all required drug information was only available in the BPMH (and not in EMR and NMRS) were counted. Third, the proportion of outpatient visits in which patient’s input provided additional essential information compared to the EMR (plus NMRS) was determined. Data were analysed using IBM SPSS Statistics software version 25. Additional descriptive statistics were provided using mean (± standard deviation [SD]) or median (interquartile range [IQR]) values depending on the (non-)parametric distribution of measured variables. 

Study population Of the 114 selected patients, 83 (73%) patients with a single visit to the outpatient rheumatology clinic were included (Table ). The visits were performed by nine rheumatologists. The most common reason of exclusion (71%) was that MR was not performed prior to the visit (due to capacity problems of the pharmacy technician team) (Fig. ). In the study population, 78 (94%) patients had at least one medication discrepancy between the drug list generated with MR and the NMRS plus EMR. Compared to the EMR alone, 68 (82%) patients had at least one medication discrepancy. Performed drug-related actions Rheumatologists performed at least one drug-related action in 65 (78%) of the outpatient visits (Table ). In total, 123 drug-related actions were performed by the rheumatologists, 67 (54%) of the drug-related actions concerned a change of drugs and 56 (46%) drug-related actions concerned the discussion of DRPs (Table ). In 33 (51%) of the outpatient visits in which a drug-related action was performed, the rheumatologist indicated that he/she needed drug information to perform these drug-related actions. According to the expert panel, in 59 (91%) of the outpatient visits in which a drug-related action was performed, drug information was required to perform the drug-related actions (Table ). Primary outcome: proportion of visits with additional essential information provided by medication reconciliation According to the individual rheumatologists, in none of the outpatient visits the patient provided additional information during MR required for the rheumatologists to perform drug-related actions that was not available in the EMR plus NMRS. According to the expert panel this was the case in 14% of the visits. Compared to the EMR alone, patient’s input during MR resulted in additional essential information according to the rheumatologists in 8% and according to the expert panel in 29% of the outpatient visits (Table ). For example, in one case the rheumatologist stopped etoricoxib. To perform this drug-related action, it is relevant to know that the patient used etoricoxib. However, use of etoricoxib was not recorded in the EMR, whereas the patient did provide this information during MR. In another case, the rheumatologist started celecoxib. For this, the use of other pain medication and a proton pump inhibitor is relevant. Use of paracetamol, etoricoxib, codeine and pantoprazole was not reported in the EMR, but was provided by the patient during MR. We observed that the added value of patient’s input during MR depended on the type of performed drug-related action (Table ). According to the expert panel, the additional information – compared to the EMR and NMRS – provided by the patient, concerned information needed to start a new drug, to change the frequency of a current drug and to discuss DRPs about drug selection, education and monitoring. 

Of the 114 selected patients, 83 (73%) patients with a single visit to the outpatient rheumatology clinic were included (Table ). The visits were performed by nine rheumatologists. The most common reason of exclusion (71%) was that MR was not performed prior to the visit (due to capacity problems of the pharmacy technician team) (Fig. ). In the study population, 78 (94%) patients had at least one medication discrepancy between the drug list generated with MR and the NMRS plus EMR. Compared to the EMR alone, 68 (82%) patients had at least one medication discrepancy. 

Rheumatologists performed at least one drug-related action in 65 (78%) of the outpatient visits (Table ). In total, 123 drug-related actions were performed by the rheumatologists, 67 (54%) of the drug-related actions concerned a change of drugs and 56 (46%) drug-related actions concerned the discussion of DRPs (Table ). In 33 (51%) of the outpatient visits in which a drug-related action was performed, the rheumatologist indicated that he/she needed drug information to perform these drug-related actions. According to the expert panel, in 59 (91%) of the outpatient visits in which a drug-related action was performed, drug information was required to perform the drug-related actions (Table ). 

According to the individual rheumatologists, in none of the outpatient visits the patient provided additional information during MR required for the rheumatologists to perform drug-related actions that was not available in the EMR plus NMRS. According to the expert panel this was the case in 14% of the visits. Compared to the EMR alone, patient’s input during MR resulted in additional essential information according to the rheumatologists in 8% and according to the expert panel in 29% of the outpatient visits (Table ). For example, in one case the rheumatologist stopped etoricoxib. To perform this drug-related action, it is relevant to know that the patient used etoricoxib. However, use of etoricoxib was not recorded in the EMR, whereas the patient did provide this information during MR. In another case, the rheumatologist started celecoxib. For this, the use of other pain medication and a proton pump inhibitor is relevant. Use of paracetamol, etoricoxib, codeine and pantoprazole was not reported in the EMR, but was provided by the patient during MR. We observed that the added value of patient’s input during MR depended on the type of performed drug-related action (Table ). According to the expert panel, the additional information – compared to the EMR and NMRS – provided by the patient, concerned information needed to start a new drug, to change the frequency of a current drug and to discuss DRPs about drug selection, education and monitoring. 

To our knowledge, this is the first study examining the need of MR for prescribing drugs in an outpatient rheumatology clinic. We examined if the patient provided additional essential information during MR that was not available in digital sources, required for rheumatologists to perform drug-related actions during outpatient visits. According to the rheumatologists, patients provided additional essential information during MR in 8 and 0% of the outpatient visits if compared to the EMR and the EMR plus NMRS, respectively. According to the expert panel, these proportions were 29 and 14%. Studies on the added value of MR in the outpatient setting are lacking. Until now, studies in the outpatient clinics have only examined the effect of MR on the identification and reduction of MDs, without taking into account the value of the additional information provided by the patient during MR [ , , , ]. According to the guidelines, it is necessary to perform MR prior to or during the prescribing of drugs . Our results indicate the opposite for the outpatient rheumatology setting when rheumatologists had access to two digital drug sources (EMR and NMRS): patient’s input during MR resulted in additional essential information according to the rheumatologist in 0% and according to the expert panel in 14% of the outpatient visits. Although the proportion of patients with at least one medication discrepancy was high in our research (94%), it seems not efficient to obtain patient’s input prior to all outpatient visits. Based on this, we assume that information from identified medication discrepancies is only required to perform drug-related actions in a few cases. Furthermore, we observed that the added value of patient’s input during MR depended on the type of performed drug-related action. According to both the rheumatologists and the expert panel, in about half of the cases the additional essential information provided by the patient was required to start a new drug. This may be explained by the fact that starting new drugs requires information about patient’s current use of both rheumatism-related and non-rheumatism-related drugs, which may not be completely reported in the EMR of rheumatologists. This is supported by other studies demonstrating that performing MR is relevant to prevent drug-drug interactions, which can arise when a new drug is started [ – ]. Also discussing DRPs may require information about non-rheumatism-related drugs. This was confirmed by the observation that the patient provided additional essential information to discuss DRPs concerning drug selection, compliance, monitoring and education. Based on our results, it may be considered to only perform MR with the patient when a drug is started and/or a DRP is discussed. Nevertheless, physicians should keep in mind that, if they use information from the EMR and NRMS, unverified dispensing data of the NMRS often contain too much drug information (e.g. stopped drugs). This potentially results in an inappropriate/unnecessary therapy monitoring or withholding of therapy (based on no longer existing interactions) and an increased chance of unnecessary drug safety alerts . Awareness of these aspects is important and the physician should check the actual drug use by the patient in case of a questionable situation. Strength of our study is that the required drug information to perform drug-related actions was assessed by both individual rheumatologists and an expert panel. We observed discrepancies between the assessments of the rheumatologists and the expert panel: rheumatologists had a lower need for drug information to perform drug-related actions during the visits compared to the expert panel. This could be explained by several causes. First, the need for information was based on a doctor’s individual opinion, and therefore unmeasured subconscious decisions could not be taken into account. Second, there may have been a time constraint during the interviews with the individual rheumatologists after the visits which resulted in underreporting of required drug information. Third, due to time constraints also not all observed drug-related actions were discussed with the rheumatologists by the observer. As a result it was not known for all actions whether the rheumatologist needed information to perform the drug-related action. Fourth, the expert panel also included two pharmacists who had additional drug knowledge compared to rheumatologists which may have affected the joint assessment. As the assessment of the expert panel was subjected to fewer limitations compared to the individual rheumatologists, we assume that the added value of patient’s input during MR based on the assessment of the expert panel gives a more objective representation. This study has several limitations. First, the rheumatologists who conducted the outpatient visits were not blinded about the aim of the study. Before the outpatient visit, the observer asked the rheumatologist to look at the BPMH. So rheumatologists were urged to use the BPMH, whereas in daily practice it is possible that the BPMH is not always used during the outpatient visit by rheumatologists. This might have led to an overestimation of the added value of medication reconciliation for creating the BPMH in clinical practice. Second, it is possible that the rheumatologists had a greater focus on medication because they were observed by a pharmacy technician. This potentially resulted in more drug-related actions and increased the chance of overestimating the added value of performing MR in clinical practice. However, the presence of an observer during the visit may have been a barrier for patients to discuss DRPs with the rheumatologist, which may have resulted in an underestimation of the number of discussed DRPs. Third, there is a chance of recall bias by the rheumatologists of the expert panel, because these rheumatologists also performed 20% of the visits which were observed by the trained pharmacy technician. However, the chance of recall bias is limited because the data were anonymized and there was a large period between performing the observations and consulting the expert panel (2018 versus 2020). Besides that, the assessment made by the (four members of the) expert panel may be different compared to the assessment made by the individual rheumatologist because the expert panel discussed until consensus was achieved. Fourth, the external validity of this research might be limited because the research was performed with a specific population, in a single hospital, and only in one type of department. Possibly, patient characteristics have influenced MR outcome. The median number of drugs in our population was lower compared to other outpatient clinics like the cardiology or geriatric outpatient clinics [ , , ]. Besides that, it is possible that patients visiting the rheumatology clinic have a higher medication adherence compared to patients with less symptomatic diseases due to the direct (pain-relieving) effects of the used medication. This may have resulted in a lower added value of outpatient MR in our specific population. However, interventions and medications used in rheumatology are comparable to other autoimmune diseases like inflammatory bowel disease or psoriasis. Therefore, our results may still be applicable to more outpatient settings. Fifth, the results of this study rely to a large extent on the quality and process of creating the BPMH, EMR and NRMS. So, further research is required to confirm our results in other/larger outpatient clinics. However, we expected that the results of the EMR are also generalizable to other countries (without access to the NRMS) as the EMR includes the electronic prescribing system and the physician’s medical notes which are mostly available in other countries. 

Altogether, we observed that in at most one third of outpatient rheumatology visits, patients provided additional information during MR that was required for the rheumatologist to perform drug-related actions when they only had access to the EMR. However, if both the EMR and NMRS were available, patient’s additional input during MR for rheumatologists to perform drug-related actions was less. The added value of patient’s input during MR is highest when a new drug is started or DRPs are discussed. Consequently, to achieve a more targeted and cost-effective implementation of MR, it could be recommended that MR is only performed (during the visit) on these specific occasions. In addition to this, patients could be stimulated to use a personal health record for MR, decreasing the need for pharmacy technicians in this process. However, before these recommendations can be applied in practice, further research is required to confirm our results in other outpatient clinics. 

Additional file 1. Data collection form.pdf. This document shows the data collection form used during the research. Additional file 2. Assessment form for the expert panel.pdf. This document shows the assessment form used by the expert panel. Additional file 3. Items of the assessment form of the expert panel where consensus was reached.pdf. This document shows the nine items of the assessment form were consensus was achieved by the expert panel. 

