Comparison of Censored Regression and Standard Regression Analyses for Modeling Relationships between Antimicrobial Susceptibility and Patient- and Institution-Specific Variables

In order to identify patients likely to be infected with resistantbacterial pathogens, analytic methods such as standard regression(SR) may be applied to surveillance data to determine patient-and institution-specific factors predictive of an increasedMIC. However, the censored nature of MIC data (e.g., MIC $≤$ 0.5mg/liter or MIC > 8 mg/liter) imposes certain limitationson the use of SR. In order to investigate the nature of theselimitations, simulations were performed to compare a regressiontailored for censored data (censored regression [CR]) and onetailored for an SR. By using a model relating piperacillin-tazobactamMICs against Enterobacter spp. to patient age and hospital bedcapacity, 200 simulations of 500 isolates were performed. VariousMIC censoring patterns were imposed by using 26 left- or right-censored(L,R) pairs (i.e., MICs $≤$ 2 mg/liter^L [2^L] or MICs > 2 mg/liter^R[2^R], respectively). Data were fit by CR and SR for which censoredMICs were either (i) excluded, (ii) replaced by 2^L or 2^R, or(iii) replaced by 2^{L – 1} or 2^{R + 1}. Total censoring forthe 26 pairs ranged from 7 to 86%. By CR, deviations of averageparameter estimates from the true parameter values were <0.10log₂ (mg/liter) for all parameters for each of the 26 pairs.By SR, these deviations were >0.10 log₂ (mg/liter) for atleast 18 of the 26 pairs for all but one parameter. Two-standard-errorconfidence intervals for individual parameters contained aslittle as 0% of cases for all SR approaches but $≥$ 91.5% of casesfor the CR approach. When censored MIC data are modeled, CRmay reduce or eliminate biased parameter estimates obtainedby SR.

INTRODUCTION

Top

Introduction

One challenge in the clinical study of resistant bacteria hasbeen the difficulty in identifying patients likely to be infectedwith such pathogens. In an effort to identify factors predictiveof resistance, linear regression has frequently been appliedto epidemiologic data collected across multiple institutions(1, 2, 4, 6; C. K. Johnson, and R. Polk, Abstr. 43rd Intersci.Conf. Antimicrob. Agents Chemother., abstr. K-1399, 2003). Inthese analyses, the outcome variable is typically the percentageof isolates that are nonsusceptible or resistant to a givenantimicrobial agent. However, the use of qualitative susceptibilitydata places undue restrictions upon the sensitivity of an analysis:significant increases in MICs within any of the qualitativesusceptibility categories (susceptible, intermediate, or resistant)cannot be detected (S. M. Bhavnani, Abstr. 41st Infect. Dis.Soc. Am., Session 125, abstr. 1005, 2003). Epidemiologic analysesinvolving quantitative MIC data overcome the latter limitation.

Most regression analyses for quantitative data are performedby using a dependent variable for which all observations areknown. However, the analysis of MICs which include censoredvalues within and at the lower margins of the MIC distributionentails certain challenges. Typically, a high proportion ofleft-censoring (i.e., MICs $≤$ x) is common with highly activedrugs. As reported previously, the total proportion of censoredMICs was as high as 90% for MICs for cefepime against Klebsiellapneumoniae (3). In such cases, one option is to simply replacethe censored value with an actual value (e.g., replace a MICof >2 mg/liter with a MIC of 2 mg/liter, or replace a MICof >2 mg/liter with a MIC of 4 mg/liter). Alternatively,all observations for which the dependent variable is censoredmay be removed from the analysis. A third approach is to retaincensored MIC data and use maximum-likelihood estimation. Inthis manner, a multiple-regression model is fit to the data,and the model incorporates censored MIC observations into thelikelihood function by using the tail probabilities of the errordistribution (3, 7). By the use of such an approach, the uncertaintyof the censored MICs is preserved.

In order to compare the impact of general linear regressionthat accounts for the censored pattern of MICs (i.e., "censoredregression") with standard regression, which excludes or replacescensored MICs, to correctly estimate model parameters, we conducteda series of simulations using data with various proportionsof left- and right-censored MICs.

MATERIALS AND METHODS

Top

Materials and Methods

Data simulation. Simulations were based on the MICs generated by using a simplifiedapproximation of a multiple-regression model for piperacillin-tazobactamMICs against Enterobacter species reported previously (3). Whileother significant relationships were found, important independentvariables associated with the MIC included the categories ofpatient age and hospital bed size. The form of the regressionmodel used for the simulations is described by equation 1:

Formula 1 (1)
where the intercept (parameter1) is equal to 0.8; the age effect is equal to 0.8 if the ageis $≤$ 18 years (parameter 2), 1.2 if 41 years $≤$ age $≤$ 60 years (parameter3), 0.7 if 61 years $≤$ age $≤$ 75 years (parameter 4), and 0.6 ifage is >75 years (parameter 5); and the hospital bed sizeeffect is equal to –1.1 if 401 $≤$ bed size $≤$ 900 (parameter6), –0.1 if 901 $≤$ bed size $≤$ 1,350 (parameter 7), and 1.1if bed size is >1,350 (parameter 8). The reference categorieswere 19 to 40 years for age and 0 to 400 beds for hospital bedsize.

By using SAS version 8.2, a total of 200 simulated data setswere generated. Each simulated data set contained 500 isolateswith MICs determined by equation 1. A random error was assignedaccording to a normal probability distribution with a mean of0 and a standard deviation of 1.9 log₂ (mg/liter). Random errorsfrom different observations were assumed to be statisticallyindependent. MICs were log transformed (log₂) and rounded tothe nearest integer to create MIC data of the same quantitativeand integer-valued nature as the original observed and modeledMIC data. For each simulation, patient age and hospital bedsize were randomly generated according to a joint probabilitydistribution of which the probability for any combination ofthe age and the bed size variables was equal to the proportionof the combination observed within the original data set (3).

Imposed censoring. The proportion of left and right censoring of MICs in a dataset is dependent upon the integer-valued censoring pair whichdefines the lower and the upper margins of susceptibility tested.In order to compare censored regression and standard regressionmethods for a wide range of censoring patterns, 26 distinctcensoring conditions [i.e., 26 censoring pairs, left and right(L,R)] were applied to the MICs in each simulated data set.For each censoring pattern, the proportions of total censoringand the balance between left and right censoring were averagedacross the 200 simulations to describe the extent of censoring.

General linear modeling. For each of the 200 simulated data sets and each censoring pattern,censored regression and each of three standard regression methodswere used to fit the regression model in equation 1 with eightcoefficient parameters (the intercept, the four age categoryparameters, and the three bed size category parameters). Inorder to provide MICs suitable for use in multiple-regressionmodeling of only uncensored dependent variable values, the followingthree standard regression methods were used to handle censoring:(i) multiple regression with censored MICs excluded from theanalyses (exclude observations); (ii) multiple regression withcensored MICs of the form MIC $≤$ 2 mg/liter^L (2^L) or MIC >2 mg/liter^R (2^R) replaced by the censoring boundaries 2^L or2^R (ignore inequality); and (iii) multiple regression with censoredMICs replaced by 2^L ^– ¹ or 2^{R +} ¹ (adjust by 1). The modelfits for censored regression were obtained by using the LIFEREGprocedure in SAS version 8.2 (SAS Institute), while the fitsfor the standard regression methods were obtained by using theREG procedure.

Evaluation of parameter estimation performance. For any given modeling method, the difference between the meanof a model parameter estimator's probability distribution andthe true parameter value represented estimator bias. For eachof the four regression methods evaluated, each of the 26 censoringconditions, and each of the eight model parameters, the parameterestimate average (PEAV) over the 200 simulated data sets wascomputed. PEAV values were plotted in three dimensions againstthe censoring conditions imposed (showing both quantity andbalance) as a result of the 26 censoring conditions. Deviations(in absolute values) between PEAV and the true parameter valueswere computed to estimate the bias. Although the results forindividual censoring conditions are the most applicable to aspecific modeling situation, the average over the 26 censoringconditions of the deviations between the PEAV and true parametervalues was used to measure the overall performance of an individualregression method. In addition to the average difference betweenPEAV and the true values, the percentage of PEAV values amongthe 26 pairs within 0.10 log₂ (mg/liter) of the true parametervalue was also summarized for each of the four regression methods.The graphical and numerical summaries of PEAV were generatedby using S-Plus UNIX version 6.0.1 (Insightful).

When assumptions of negligible bias and approximate normalityof a parameter estimate are satisfied, a two-standard-error(2 · SE) deviation on either side of the parameter estimateprovides an approximate 95% confidence interval for the trueparameter value. The percentage of the 200 simulated data setsfor which the 2 · SE interval contained a given parameter(or coverage percentage) was also used to assess the parameterestimation performance. The coverage percentage for such intervalsshould be close to 95%. The 2 · SE confidence intervalswithin each simulation were computed for each parameter andeach of the 26 censoring conditions. Confidence intervals werecomputed in conjunction with the general linear regression modelingby using the LIFEREG procedure (SAS version 8.2), and numericaland graphical summaries of the coverage percentages were generatedby using S-Plus.

RESULTS

Top

Results

Censoring proportions. The histogram of all simulated MICs and the summary of censoringconditions for the 26 (L,R) censoring pairs, averaged acrossthe 200 simulated data sets, is shown in Fig. 1. For any givenpair, the MICs to the left of the closed datum point and theMICs to the right of the open datum point were censored. Theaverage proportion of the total sample censored ranged from7 to 86%, with the average left censoring of the MIC (as a percentageof the censored observations) ranging from 0.9 to 92%.

View larger version (31K):
[in this window]
[in a new window]
FIG. 1. Simulated MIC histogram with censoring conditions for the 26 (L,R) censoring pairs. The (L,R) censoring pairs are represented by closed and open dots, respectively.

PEAV summaries. Three-dimensional graphical summaries of PEAV for model parameters1 and 2 for each of the four regression methods are shown inFig. 2. Within each graph, the 26 datum points represented thecensoring pairs, which demonstrated the total proportion andbalance (between left and right) of censoring. A second-orderregression surface was fit to these datum points and is displayedas a grid to assist in the visual description. Although theyare not shown, graphs for parameters 3 through 8 demonstratedsurface shapes which were the same as those seen in the seriesof graphs for parameter 2. The patterns for parameters 6 and7, however, were flipped upside down because their true valuesare negative, unlike the positive value of parameter 2 and theother parameters.

View larger version (23K):
[in this window]
[in a new window]
FIG. 2. Comparison of PEAV values across a range of censoring conditions for parameters 1 and 2. A surface was fit to PEAV by a two-order regression equation. Portions of the surface (PEAV) within 0.10 log₂ (mg/liter) of the true value are displayed in black. Gray surfaces represent PEAV greater than 0.1 log₂ (ml).

The graphs in Fig. 2 for both the intercept parameter (parameter1) and an example nonintercept parameter (parameter 2) showthat as the total proportion of censored observations approached0, the PEAV values converge toward the true parameter value.The surfaces in Fig. 2 describing the PEAV against the censoringconditions take on different characteristics between the interceptparameter and the nonintercept parameter. For the intercept,the bias appeared to be low for the standard regression methodswhen the proportion of left censoring versus that of right censoringwas balanced. A greater proportion of left censoring than rightcensoring showed a trend toward overestimates of the interceptfor the exclude observations and ignore inequality methods.The adjust-by-1 method had an opposite and less dramatic effectfor these cases. In cases where there were equal proportionsof left and right censoring, these trends were reversed (i.e.,estimates for the intercept were low for the exclude observationsand ignore inequality methods and, to a lesser degree, highfor the adjust-by-1 method). In contrast to the three standardregression methods, censored regression consistently producedestimates of the intercept with no perceivable bias across allcensoring conditions.

The graphs in Fig. 2 for the nonintercept parameter demonstratednegative bias estimates for the exclude observations and ignore-inequalitystandard regression methods, the degree of which increased asthe total proportion of censored observations increased. Althoughnot as severe, the bias for the adjust-by-1 method was in thepositive direction. There appeared to be some improvement inthis bias as the total proportion of censored observations increased.Visual inspection of surfaces for the exclude observations andadjust-by-1 methods revealed a more severe bias when there wasan even balance of left and right censoring. In comparison,there was no visual indication that bias was affected for theignore-inequality approach when censoring was balanced. As inthe case of the intercept parameter, censored regression consistentlyproduced estimates with no perceivable bias across the censoringconditions.

Table 1 summarizes the numerical measures of bias in the parameterestimates for each regression method. When the absolute deviationbetween the PEAV and the true parameter values is examined,censored regression provided a deviation less than 0.1 log₂(mg/liter) for each of the eight parameters under all of the26 censoring conditions. Simulations for the standard regressionmethods showed that deviations within 0.1 log₂ (mg/liter) ofthe true value were attained or approached 100% of the censoringconditions for parameter 7 only, the true value of which was–0.1 log₂ (mg/liter). The magnitudes of the deviationsfrom the true value, irrespective of the regression method used,were generally smaller for parameters with true values closerto 0. Parameters with true values of 0.6 log₂ (mg/liter) orlarger, which included all but parameter 7, were estimated witha bias of less than 0.1 log₂ (mg/liter) for only 30.8% of thecensoring conditions or less.

View this table:
[in this window]
[in a new window]
TABLE 1. Summary statistics for PEAV

Confidence interval coverage. Table 2 shows summaries of the 2 · SE confidence intervalcoverage percentage for individual parameters for each of theregression methods. For most parameters and for each of thestandard regression methods, there were censoring conditionsunder which none of the 200 simulated confidence intervals containedthe true parameter value. In contrast, confidence intervalsfor censored regression consistently contained the true parametervalues for close to 95% of the simulations. Figure 3, whichdisplays the estimated coverage percentage against the averageproportion of the total sample censored, demonstrates a lowcoverage percentage for the 2 · SE confidence intervalsbased on each of the standard regression methods when the proportionof censoring was high. The confidence intervals for censoredregression, however, consistently covered the true parametervalues in nearly 95% of the simulations, regardless of the amountof the sample censored. The relationships between the proportionof confidence intervals that covered the true parameter valuesand the balance between left and right censoring were also explored,but there appeared to be no relationship between the two forany of the methods. Figure 4 shows graphs of the standard errorsfor parameter estimates. When the three standard regressionmethods were used, the standard error tended to decrease asthe proportion of censored data increased. In contrast, thestandard error increased with increasing proportions of censoreddata when censored regression was used.

View this table:
[in this window]
[in a new window]
TABLE 2. Range of coverage percentage for 2 · SE confidence intervals

View larger version (18K):
[in this window]
[in a new window]
FIG. 3. Comparison of coverage percentage for a 2 · SE confidence interval (CI) across a range of censoring conditions for parameters 1 and 2.

View larger version (16K):
[in this window]
[in a new window]
FIG. 4. Comparison of the parameter estimate standard error across a range of censoring conditions for parameters 1 and 2.

DISCUSSION

Top

Discussion

The results of the simulations presented herein, which werebased on a simplified approximation of a multiple-regressionmodel for piperacillin-tazobactam MICs against Enterobacterspecies (3), allowed the evaluation of four regression approachesfor different patterns of MIC censoring and a range of censoringconditions. As expected, the performance of each of the approacheswas comparable under conditions of very limited or no censoring(i.e., the PEAV converged toward the true parameter value).However, irrespective of the proportion of the sample that wascensored or the pattern of censoring, censored regression provedto be superior to the three standard regression methods. Deviationsbetween PEAV and true parameter values were within 0.1 log₂(mg/liter) for 100% of the censoring patterns for all parametersusing this approach. The rank order of performance for the threestandard regression methods evaluated was as follows: adjustby 1, ignore inequality, and exclude observations. Of thesethree methods, adjustment by ±1 on the log₂ scale wasperhaps the most arbitrary. Nonetheless, the adjust-by-1 methodwould be considered a reasonable approach for consideration,since an adjustment of ±1 or any other integer preservesthe integer nature of the MIC on the log₂ scale. The superiorperformance of censored regression was likely the result ofthe inclusion of all information for the dependent variablewithout the imposition of specific nonobserved values when theexact value is unknown.

The low coverage percentages of the 2 · SE confidenceintervals under some censoring patterns for the standard regressionmethods was striking. One (and perhaps the most important) contributingfactor to this finding was that the presence of censoring ledto biased parameter estimates, as shown by the PEAV results.Interestingly, as the proportion of censored data increased,the standard error for the parameter estimates based on standardregression methods decreased. This finding was counterintuitive,since increased censoring of a dependent variable results inthe loss of specific information and thus should result in anincrease in the standard error. Stated more generally, lessinformation intuitively implies more uncertainty. In practice,one might not be aware of this counterintuitive behavior ofthe standard error and the resulting low coverage percentageof the confidence intervals obtained by using standard regressionmethods with a modest or large amount of censored dependentvariable values. The danger is that confidence intervals thatvery likely do not contain the true parameter value may be reportedwith an assumed confidence level (typically near 95%). The findingsof such analyses could be seriously misleading.

Through the analyses undertaken, we demonstrated both the usefulnessand the potential benefit of the use of linear regression modelswhich appropriately accommodate a dependent variable with censoredobservations. In addition to MICs, concentration and CFU dataare frequently censored due to lower limits of detection. Statisticalmethods which can appropriately account for these data wouldbe of great benefit in providing a better understanding of exposure-responserelationships at higher and lower concentrations or colony counts.As demonstrated by these analyses, naïve approaches whichexclude or replace such observations may provide misleadingresults. Unfortunately, many commonly used desktop statisticalpackages do not provide the capability of handling both leftand right censoring of the dependent variable. While commonlyused packages such as SAS and S-Plus do provide the capabilityto perform regression procedures for censored data, their useis currently limited to linear models with independent errorstructures. Moreover, only a small selection of independenterror distributions is available. The MIC data in these analyseswere obtained from only single measurements from individualpatients, so the independent error structure was appropriate.Analyses that contain repeated measurements may require theimplementation of censored regression methods that also allowthe use of dependent error structures.

In our analyses of the simulated data, we correctly assumeda normal error distribution, since this was the error distributionimposed upon the data. We did not use alternative error distributionsto thoroughly evaluate the impact of misspecification of theerror structure. We recommend that caution be taken when anappropriate distribution is selected, since the robustness oferror distribution selection has yet to be explored. In addition,selection of the error distribution should be based on the residualsof model fits rather than the original MICs, since independentvariables in the model may explain much of the variation ormultimodal nature of the MIC distribution, particularly in theright tail. If, after careful evaluation, a suitable error distributionis not available using standard statistical packages (i.e.,no near approximation to the supposed error distribution isfound), then the use of alternative approaches (e.g., transformationsor different software) should be considered.

Although the modeling of MIC data, due to their censored nature,is more challenging than the modeling of other end points suchas the percentage of resistant isolates, greater insight maybe gained by evaluating these data, especially when a low burdenof resistant isolates exists. For example, multivariable censoredregression analyses could be applied to the identification ofsubgroups of patients at increased risk for infection arisingfrom organisms with increased MICs. Such information may thenbe used to identify patients of interest for clinical studyas well as patients for whom interventions (e.g., increasedinfection control and/or treatment with a potent antimicrobialregimen) may increase the likelihood of clinical success andreduce the emergence of resistance. A number of large and long-standingsurveillance systems, such as the SENTRY Antimicrobial SurveillanceProgram (5, 9) and the Trust (10) and Mystic (8) surveillanceprograms, have performed extensive MIC testing of isolates collectedglobally. Furthermore, these surveillance systems frequentlyhave more information about the patient and institution fromwhich isolates were collected than is ever reported. Analysesthat use quantitative MIC data, and especially those that canappropriately integrate censored MICs, represent a step forwardin the effort to better understand predictors of resistance.

FOOTNOTES

* Corresponding author. Present address: Institute for Clinical Pharmacodynamics, Ordway Research Institute, 150 New Scotland Avenue, Albany, NY 12208 Phone: (518) 641-6473. Fax: (518) 641-6304. E-mail: SBhavnani-ICPD{at}OrdwayResearch.org.

Present address: Collaborative Biostatistics Center, ClevelandClinic Foundation, Cleveland, OH 44195.

Present address: Institute for Clinical Pharmacodynamics, OrdwayResearch Institute, Albany, NY 12208.

REFERENCES

Top