ABSTRACT
Dosing of vancomycin is often guided by therapeutic drug monitoring and population pharmacokinetic models in the intensive care unit (ICU). The validity of these models is crucial, as ICU patients have marked pharmacokinetic variability. Therefore, we set out to evaluate the predictive performance of published population pharmacokinetic models of vancomycin in ICU patients. The PubMed database was used to search for population pharmacokinetic models of vancomycin in adult ICU patients. The identified models were evaluated in two independent data sets which were collected from two large hospitals in the Netherlands (Amsterdam UMC, Location VUmc, and OLVG Oost). We also tested a one-compartment model with fixed values for clearance and volume of distribution, in which a clinical standard dosage regimen (SDR) was mimicked to assess its predictive performance. Prediction error was calculated to assess the predictive performance of the models. Six models plus the SDR model were evaluated. The model of Roberts et al. (J. A. Roberts, F. S. Taccone, A. A. Udy, J.-L. Vincent, F. Jacobs, and J. Lipman, Antimicrob Agents Chemother 55:2704–2709, 2011, https://doi.org/10.1128/AAC.01708-10) performed satisfactorily, with mean and median values of prediction error of 5.1% and −7.5%, respectively, for Amsterdam UMC, Location VUmc, patients, and −12.6% and −17.2% respectively, for OLVG Oost patients. The other models, including the SDR model, yielded high mean values (−49.7% to 87.7%) and median values (−56.1% to 66.1%) for both populations. In conclusion, only the model of Roberts et al. was able to validly predict the concentrations of vancomycin for our data, whereas other models and standard dosing were largely inadequate. Extensive evaluation should precede the adoption of any model in clinical practice for ICU patients.
TEXT
Vancomycin is a glycopeptide antibiotic with activity against Gram-positive bacteria, including methicillin-resistant Staphylococcus aureus. It is widely used in intensive care units (ICUs) for the prevention and treatment of sepsis (1–3). With a mortality rate of over 40% for septic shock, early and appropriate dosing is pivotal (4). Insufficient exposure could increase mortality and, possibly, increase antimicrobial resistance, while excessive exposure may be nephrotoxic (5). However, adequate dosing is a particular challenge in ICU patients because their pharmacokinetics (PK) change markedly and rapidly with disease severity (6).
The antimicrobial effect of vancomycin is time and concentration dependent. Its clinical and microbiological efficacy is therefore best described by the ratio of the area under the plasma concentration-time curve (AUC) during a 24-h time period (AUC0–24) and the MIC, i.e., AUC0–24/MIC (7). The recommended target ratio of an effective dosing regimen of vancomycin is 400 mg · h/liter or greater (1). Trough concentrations may be used to guide dosing, as these are correlated with AUC, although this correlation and the cutoff values of trough concentrations are debated (8–10). Consequently, vancomycin dose adaptation increasingly relies on population pharmacokinetic (PK) models. They describe the full-time course of vancomycin concentrations and can be used to estimate individual PK parameters through Bayesian approaches. Thus, they can be deployed in therapeutic drug monitoring programs to predict vancomycin concentrations and optimize individual vancomycin dosing. In addition, these models are essential for decision support software, such as our AutoKinetics system, that integrates with electronic medical records (EMR) to provide real-time bedside dosing guidance.
Obviously, the validity of the population PK models to be used is crucial to model-based dosing. For vancomycin, many of these models have been published for different populations, ranging from children to adults and from ordinary patients to ICU patients (11–17). Validation of these models for ICU patients is of paramount importance, as these patients are likely to show more PK variability than other populations due to their markedly varying physiological status (7). Ill-considered use of models for ICU patients without external validation may raise safety issues. This creates the urgent need for answering the following question: are published models in ICU patients applicable to external data?
Therefore, we set out to evaluate the predictive performance of published population PK models of vancomycin established in ICU patients using data from two very large cohorts of ICU patients. In addition, we also tested a one-size-fits-all model with a fixed clearance (CL) and volume of distribution (V) to benchmark the effects of standard dosing.
RESULTS
Data extraction.A data set with data for 236 ICU patients with 516 vancomycin concentration-time observations collected from January 2012 to March 2016 was obtained from Amsterdam UMC, Location VUmc (VUmc). The patients received vancomycin with a loading dose of between 1,000 mg and 2,000 mg and maintenance doses of 1,000 mg twice a day by intravenous infusion over 60 min. The maintenance dose was adjusted at the discretion of the treating clinician based on therapeutic drug monitoring of trough levels. The blood samples were drawn 2 or 3 days after the first dose of vancomycin and then twice weekly. Another data set with data for 603 ICU patients with 2,643 vancomycin concentration-time observations collected from January 2011 to May 2017 was obtained from OLVG Oost (OLVG). These patients received vancomycin with a loading dose, after which a continuous infusion was started. Both loading doses and maintenance doses were individualized based on the advice from the in-house-developed AutoKinetics system, which used the model of Del Mar Fernandez De Gatta Garcia et al. (18) but with individual parameters corrected by fitting to the observed vancomycin concentrations. Blood samples were drawn once a day for therapeutic drug monitoring. Creatinine clearance (CLCR) was calculated using the formula from the Modification of Diet in Renal Disease (MDRD) study (19). The characteristics of the patients from both VUmc and OLVG are summarized in Table 1.
Characteristics of the patients in the published studies and this studya
Literature review and model collection.We identified eight published studies and extracted the models from the original articles. Two studies were omitted to comply with the exclusion criteria (18, 20). The number of patients and the number of samples collected in the included studies varied. Three studies created their models in relatively larger data sets with more than 500 observed concentration-time data points (11, 21, 22). Five studies collected vancomycin concentration-time data at either the trough level or the plateau level only, while the study of Llopis-Salvia and Jiménez-Torres (23) collected data randomly. The characteristics of the patients in the included studies are listed in Table 1. Both the model of Mangin et al. (24) and the model of Llopis-Salvia and Jiménez-Torres (23) were two-compartment models, while the other four were one-compartment models. Regarding the covariate effects, all models incorporated body weight, and all models except the model of Udy et al. (25) incorporated renal function (serum creatinine level or CLCR). The model of Udy et al. (25) incorporated only body weight as a covariate. The details of the models, including all covariate effects, are summarized in Table 2.
Summary of the evaluated modelsa
Model evaluation.The models were evaluated in only the renal replacement therapy (RRT) patients or only the non-RRT patients, or both groups of patients, in our data sets, depending on which patient population (RRT or non-RRT, or both) were used for the development of the original models. Table 3 and Fig. 1 show that the model of Roberts et al. (21) was the only model complying with our criterion for validity: concentration-time data could be accurately predicted with both mean and median values of prediction error (PE) lower than 20% for the patients from the two hospitals. In general, the other models yielded considerably high PE values, suggesting that they performed poorly in predicting concentrations for the patients in our study. Noticeably, the models of Revilla et al. (11) and Llopis-Salvia and Jiménez-Torres (23) showed a relatively low root mean square error (RMSE) while showing high inaccuracy. Bland-Altman plots (Fig. 2 and 3) show that there was no explicit trend in the PE for the evaluated models over the whole concentration range observed in our data.
Prediction error of the evaluated modelsa
Box plot of the prediction error (in percent) of the evaluated models with VUmc patients (left) and OLVG patients (right). The bold solid lines within each box are the median values. The lower and upper limits of the box are the first and third quartiles. The red dashed lines are ±20% of the prediction error. SDR is the standard dosage regimen model.
Bland-Altman plot of published models based on the VUmc data set. Red dashed lines are the mean and mean ± 1.96 times the SD of the prediction error. Pred, predicted; Obs, observed.
Bland-Altman plot of published models based on the OLVG data set. Red dashed lines are the mean and mean ± 1.96 times the SD of the prediction error. Pred, predicted; Obs, observed.
SDR model.The standard dosage regimen (SDR) model predicted concentration-time data poorly for the patients from both hospitals, with mean and median values of PE of −22.6% and −45.5%, respectively, for VUmc patients, and −49.7% and −56.1%, respectively, for OLVG patients (Table 3 and Fig. 1). No trend of bias of prediction was present in the SDR model (Fig. 2 and 3).
DISCUSSION
Population PK modeling has become increasingly prevalent since the innovative breakthrough by Sheiner et al. (26) and Beal and Sheiner (27). Such a technique can be used to estimate individual PK parameters through the implementation of empirical Bayes estimation with sparsely sampled data, e.g., the trough concentration only. With the individual parameters obtained, population PK models can in turn be used to predict the plasma concentrations of drugs for individuals following a dosage regimen under consideration. Further, predicted concentrations can be made use of to calculate PK exposure, e.g., the AUC for optimizing the vancomycin dose in clinical practice (10). Over the past few decades, population PK models of vancomycin have been extensively reported (11–17), and among these, some have described the kinetics of vancomycin in ICU patients (11, 21–25). So far, however, these published models have not been sufficiently evaluated with data from external data sets. Deng et al. did evaluate published vancomycin models but did so with data for a population of mainly general patients and with no discrimination of the patient population in which the published models were created (28). One may assume that a population pharmacokinetic model should be able to predict concentration-time data properly for external subjects that belong to the same population in which the model was created. However, our study suggests that this is not necessarily true. The results warn us that it is crucial to evaluate a model in the target populations before applying it in clinical practice. To the best of our knowledge, our study is the first to externally evaluate the published vancomycin models specifically for ICU patients.
The predictive performance of the models evaluated in our study varied considerably and seemed to be very random, as the models of Mangin et al. (24) and Medellín-Garibay et al. (22) tended to overpredict our data, while other models tended to underpredict our data. Among all models evaluated in this study, only the model of Roberts et al. (21) met the predefined criterion for model validity for both VUmc patients and OLVG patients. The reason may be that the characteristics of the patients in our study, which were partly represented in the models by covariate effects, did not fully overlap those of the patients in the published studies, as can be seen in Table 1. Examples are the acute physiology and chronic health evaluation II (APACHE II) and simplified acute physiology II (SAPS II) scores, which were higher in our patient populations than in the populations of patients from the previously published studies, indicating that the patients in our study were more severely ill. When performing an external validation, the effects of covariates in the published models were extrapolated out of the range of values for the original patient population. This may have resulted in a reduction in the accuracy of the model prediction. In addition, a high PE may also be caused by unidentified covariates, which were then, of course, not taken into account in the published studies but which play a role in vancomycin pharmacokinetics in ICU patients. Apart from differences in patient characteristics and covariate effects, poor predictive performance may also be caused by differences in the manner of vancomycin administration in our study and the previously published studies, i.e., applying a model developed with data collected under continuous infusion to a set of data for samples collected during intermittent infusion and vice versa, yet the model of Roberts et al. (21), developed with data collected during continuous infusion, did perform sufficiently with data from the VUmc data set, in which only data for samples collected during intermittent dosing were present.
The result obtained with the SDR model was intended to show how far off vancomycin concentrations could be when treating every patient as if he or she has the same clearance and volume of distribution. The SDR model showed poor PE values, obviously since the SDR did not tailor the dose to the characteristics of ICU patients, like a large V and an often poor or augmented CL. Nevertheless, the SDR model did not perform the worst. Still, our findings provide support for population PK model-based dosing in order to achieve the target concentration and, as such, provide a means that may bring us a step closer to precision medicine; i.e., the use of a model could be better than the use of no model, as long as an external validation is performed.
Some remarks need to be made regarding the design of this study. First, the data that we used in this study were retrospectively collected from clinical settings, in which it was usually hard to precisely record the data. Although data inspection was carried out preceding the model validation, there might remain random errors in the data. However, as the model to be selected will be used in everyday clinical practice, the same imprecision may also be present under such circumstances and, as such, has been automatically accounted for. Second, the models were evaluated using the two data sets separately. The main reason to do so is that the model to be selected was intended to be implemented in the EMRs of both ICUs, where vancomycin is dosed in different regimes (continuous infusion versus intermittent infusion). Additionally, given the imbalance in the quantity of data in the two data sets (the OLVG data set was noticeably larger than the VUmc data set), evaluation of the data sets separately avoided the possibility that the data set with a larger quantity dominated the results. Third, we identified eight models in total from the literature, but two of them were omitted from our study. The model of Del Mar Fernandez De Gatta Garcia et al. was not developed by the use of a nonlinear mixed-effects modeling approach and, thus, did not meet the inclusion criteria (18). The other model that was omitted was from the study of Escobar et al., which incorporated a high-volume hemodialysis filtration rate as the key covariate on clearance and intercompartmental clearance (20). This covariate was not present in our data set. Fourth, in order to evaluate the predictive performance of the published models for the population rather than for particular individuals, population predictions (PRED) instead of individual predictions (IPRED) were used to calculate the PE for each patient. As such, the PE obtained from the models is built up out of the predictive ability of the models and the unexplained variabilities (random effects). Lastly, the reason that we chose 20% as the cutoff value for the prediction error was based on our clinical practice, as with such a prediction error, the risk of unrightfully adjusting the dose based on a predicted concentration above or below the target window while the actual concentration is on target is minimal. For example, for continuous infusion, the vancomycin target range is between 15 mg/liter and 25 mg/liter. Given that the actual concentration is 20 mg/liter, the predicted concentration will be between 16 mg/liter and 24 mg/liter with a PE of 20%. In neither case would a dose adjustment be indicated, which could have been the case with a PE of, e.g., 30%. Additionally, the precision of the predictive performance was not considered in the model selection procedure. Our main concern was that precision does not guarantee the accuracy of a model, which is crucial for validity. As seen in the results (Table 3; Fig. 1 and 3), the models of both Revilla et al. (11) and Llopis-Salvia and Jiménez-Torres (23) showed acceptable precision but had poor accuracy with a clear downward bias. As such, these models were precisely inaccurate in predicting our data and, therefore, less suitable for being selected for use in our clinical settings. Nevertheless, the choice of the criteria used to establish model validity is subjective and can surely be debated.
In conclusion, our study has shown that the model of Roberts et al. (21) had the best predictive performance for ICU patients in both hospitals evaluated in the present study. The fact that not all evaluated models passed the external validation procedure serves as an alert that extensive validation should precede the adoption of any model used for clinical care for ICU patients.
MATERIALS AND METHODS
Data extraction.Retrospective therapeutic drug monitoring data were extracted from the EMR systems of the ICUs of the VUmc and OLVG hospitals. These data included dosing records, observations (plasma concentrations of vancomycin), and the covariate items used in the published models. The use of retrospective data was approved by the Medical Ethical Committees of both VUmc and OLVG.
Literature review and model collection.The PubMed database was systematically searched for population pharmacokinetic analyses of vancomycin published up to 1 October 2018. The keywords [vancomycin] AND (pharmacokinetic OR pharmacokinetics OR model) AND (intensive care OR critical care) were used in the search strategy. Additional publications cited in the references of the identified publications were also screened. The publications were included if (i) the study was a population pharmacokinetic analysis of vancomycin in ICU adult patients and (ii) the article was written in English. The publications were excluded if (i) the model contained any covariate that was not available in our data set or (ii) the model was not created by the use of a nonlinear mixed-effects modeling approach.
Model evaluation.The candidate models were evaluated separately with data from both data sets extracted from the two hospitals mentioned above. Population parameters were adopted for each model to calculate the predicted concentrations at sampling times identical to those of our own data. Prediction error (PE) was used to assess the predictive performance of the models and was the primary endpoint of this study. PE for each predicted concentration (PEi) was defined through the following equation:
Standard dosage regimen model.The clinical standard one-size-fits-all dosage regimen of vancomycin of 1,000 mg every 12 h was chosen, as this is the most frequently applied regimen which physicians are likely start with. Accordingly, we built a one-compartment model to represent the assumptions silently underlying this clinical practice, being a population clearance (CL) of 5 liters/h and a population volume of distribution (V) of 50 liters. The population CL of 5 liters/h was calculated by the following equation:
ACKNOWLEDGMENTS
This work is part of the Right Dose, Right Now project by Amsterdam UMC, Location VUmc, and OLVG Oost and was partially funded by the ZonMw Rational Pharmacotherapy program.
We thank Ronald Driessen for kindly helping us with data extraction.
We declare no conflicts of interest.
FOOTNOTES
- Received 17 December 2018.
- Returned for modification 17 January 2019.
- Accepted 22 February 2019.
- Accepted manuscript posted online 4 March 2019.
- Copyright © 2019 American Society for Microbiology.