Previous Article | Next Article ![]()
Antimicrobial Agents and Chemotherapy, November 2006, p. 3734-3739, Vol. 50, No. 11
0066-4804/06/$08.00+0 doi:10.1128/AAC.01618-05
Copyright © 2006, American Society for Microbiology. All Rights Reserved.
Epicentre, Paris, France,1 Department of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, United Kingdom,2 Médecins Sans Frontières, Paris, France,3 Médecins Sans Frontières, Brussels, Belgium,4 Risk Assessment and Field Operations, CSR Office of Alert and Response Operations, WHO, Geneva, Switzerland,5 UNICEF/UNDP/World Bank/WHO Special Programme on Research and Training in Tropical Diseases (TDR), Geneva, Switzerland,6 University of Oxford, Centre for Tropical Medicine and Vaccinology, Churchill Hospital, United Kingdom7
Received 21 December 2005/ Returned for modification 9 February 2006/ Accepted 30 August 2006
|
|
|---|
|
|
|---|
In areas of intense transmission, a second episode of malaria or a recurrent parasitemia during the drug-free follow-up period may be due to the same infection or a different infection (a recrudescence, thus a treatment failure, or a new infection, respectively). To distinguish these two events, polymorphic Plasmodium falciparum genes, such as the merozoite surface protein 1 and 2 genes (msp1 and msp2) and the glutamate-rich protein gene (glurp), can be genotyped by PCR (13). The duration of follow-up required to capture all true treatment failures will depend on a drug's residence time in the body (15) and the level of parasite sensitivity (21).
We analyzed data from 13 antimalarial drug studies with the standard single-agent first- and second-line drugs: chloroquine (CQ), sulfadoxine/pyrimethamine (SP), and amodiaquine (AQ). These studies enrolled 2,576 children in eight African countries from 2001 to 2004. The aims of this analysis were to (i) determine the feasibility of in vivo assessment of antimalarial drugs by using PCR genotyping for resolving recurrent infections (use effectiveness and test performance) and (ii) compare current and alternative methods of data analysis. The current recommendation is that failure rates should be calculated on target days (days 14 and 28) by using PCR-corrected outcomes after excluding both missing values and new infections (19). Specifically, we wanted to compare (i) treatment outcomes expressed as product limit estimates of failure (Kaplan-Meier) versus per-protocol evaluable-patient analyses on target days (days 14 and 28), (ii) analysis of PCR-corrected results allowing or excluding new infections, and (iii) day 14 versus day 28 results.
|
|
|---|
We applied the WHO 2004 endpoint classification (Table 1). When late-clinical or parasitological failure occurred after day 14, we used PCR (2, 11, 12, 13) to compare the genotypic profiles of msp1 and msp2 for post- and pretreatment parasites (in Bundi Bugyo, Uganda, glurp was also analyzed due to the low allelic heterogeneity of local isolates). PCR analyses were conducted at Prince Leopold Institute of Tropical Medicine, Antwerp, Belgium (four sites, 37% of total samples); University of Glasgow, Glasgow, United Kingdom (three sites, 19% of total samples); Hôpital Bichat, Paris, France (two sites, 15% of total samples); Shoklo Malaria Research Unit, Mae Sod, Thailand (one site, 9% of total samples); Kenya Medical Research Institute, Nairobi, Kenya, and Malaria Research and Training Centre, Bamako, Mali (one site each, 8% of total samples); and Faculté de Médicine-Pharmacie, Rouen, France (one site, 4% of total samples). Methods were not standardized across these laboratories, which used different extraction methods, primers, and reaction conditions.
|
View this table: [in a new window] |
TABLE 1. Efficacy endpoints for WHO classification
|
PCR genotyping. For each study, we identified cases requiring PCR genotyping, i.e., children with a recurrence after day 14 (it is generally assumed that, for P. falciparum, recurrences on or before day 14 are recrudescences). These cases were classified as either (i) resolved by PCR and further categorized as recrudescences or new infections or (ii) unresolved by PCR, with the reason recorded (missing sample, failure to extract DNA, or result not interpretable). PCR genotyping was assessed in terms of (i) use effectiveness, defined as the number of cases resolved by PCR divided by the total number of children with recurrences after day 14 (the denominator includes missing samples), and (ii) test performance, defined as the number of cases resolved by PCR out of the total number of paired samples available for genotyping (the denominator does not include missing samples).
Failure rate (day 14 and day 28 evaluable-patient analysis) versus Kaplan-Meier analysis. Failure rates at day 14 and day 28 were calculated using the method of analysis recommended by the WHO (18) and compared to the product limit estimates of failure obtained by Kaplan-Meier analysis. The WHO method takes into account only patients with an outcome on the target day (day 14 or day 28) (evaluable patients). The Kaplan-Meier method allows for dropouts (losses during follow-up and withdrawals). In our Kaplan-Meier analysis, the event measured was "failure" and the time to the event was the duration of the follow-up (either 28 days for patients generating adequate clinical and parasitological response [ACPR] at day 28, x days for patients who failed at day x of follow-up, or the number of days of follow-up until the last visit for patients lost during follow-up or excluded). For both methods, we calculated the number of patients included in the analysis and the risks of failure and compared these figures between the two methods.
Day 28 PCR-corrected results for two methods of analysis. The WHO currently recommend calculation of failure rates on target days by using PCR-corrected outcomes after excluding both missing values and new infections (19).
We genotyped only recurrences occurring after day 14 and assumed all earlier failures to be recrudescences. Two methods of analysis were used. For method 1, all PCR-confirmed new infections were reclassified as successes (ACPR), with individuals unresolved by PCR excluded. For method 2, individuals with new infections or infections unresolved by PCR were excluded from the analysis, per the current WHO approach (19). For both methods, results are presented by drug and study site as probabilities of failure [1 St, where St is probability of survival] on day 28 with 95% confidence intervals (CIs). Since recurrent episodes occurring on or before day 14 were not genotyped, for day 14, we report only crude (uncorrected) results (with no comparisons between data for methods 1 and 2).
Day 14 versus day 28 failures. Estimates of risk of failure by days 14 and 28 were analyzed separately (Table 1), based on results obtained with method 1 and Kaplan-Meier analysis. Comparisons were expressed as absolute differences (summarized across sites as medians and ranges). Correlations between day 14 and 28 results were expressed as coefficients of correlation.
Data analysis was performed using Stata 8.2 (Stata Corp., College Station, Texas).
|
|
|---|
|
View this table: [in a new window] |
TABLE 2. Profile of PCR genotyping of recurrent episodes of malaria during follow-up
|
Of the 9,610 patients screened (based on data available from 12 studies), 4,087 (42.3%) were slide positive; 2,576 (26.8%) were treated with CQ (n = 593), SP (n = 989), or AQ (n = 994) (see Table 3 for details); 112 (4.3%) patients were lost during follow-up (64 before day 14, 48 on or after day 14); and 177 (6.9%) were withdrawn (127 before day 14, 50 on or after day 14). The characteristics of these patients on inclusion are reported elsewhere (1, 3, 4-8, 10).
|
View this table: [in a new window] |
TABLE 3. Estimated risk of failure for CQ, SP, and AQ on day 28, calculated by Kaplan-Meier analysis using method 1 (unresolved PCR excluded from the analysis) and method 2 (both unresolved PCR and new infections excluded)
|
Risk of failure: per-protocol versus Kaplan-Meier analysis. By day 14, 8% of the 2,576 children had been lost during follow-up or withdrawn and are not included in the estimation of risk of failure based on evaluable children, while with the Kaplan-Meier method, only children with no follow-up (i.e., lost at day 0) are not considered (1%) (Table 4) . By day 28, 11% of enrolled children had been lost during follow-up or withdrawn. After PCR correction, the loss from the initial sample size was 6% for Kaplan-Meier and 15% for day 28 evaluable-patient analysis (method 1); losses were 15 and 25%, respectively, after removing new infections (method 2).
|
View this table: [in a new window] |
TABLE 4. Patient attrition with the various analysesa
|
![]() View larger version (27K): [in a new window] |
FIG. 1. Comparison of Kaplan-Meier and evaluability (per-protocol) analyses (failure rates on day 28) using method 1 (PCR corrected, with cases unresolved by PCR excluded) (open symbols) and method 2 (PCR corrected, with cases unresolved by PCR and new infections both excluded) (closed symbols) by drug treatment.
|
Day 28 results for two methods of PCR analysis. Table 3 shows failure rates at day 28 estimated by Kaplan-Meier analysis with the two methods for analysis of PCR-corrected outcomes. Method 2 (unresolved genotypes and new infections removed) yields higher failure rates than method 1: +2.9 (range, +1.6 to +10.7) for CQ, +1.15 (range, 0 to +10.5) for SP, and +2.4 (range, +0.1 to +6.6) for AQ.
Day 14 and day 28 Kaplan-Meier failure estimates. Because method 1 minimizes the number of patients lost in the analysis, we used it to compare risks of failure assessed at days 28 and 14 (Fig. 2); the median differences were 5.9% (range, 15.0 to +15.0) for CQ, +9.8% (range, 15.6 to +16.8) for SP, and +9.8% (range, 1.6 to +18.1) for AQ. The reason for the risk of failure being lower in some instances on day 28 than on day 14 is that we compared outcomes for the analyses of two different data sets (Table 1). Considering all studies, the coefficient of correlation was 0.94 between day 14 and day 28 results but was 0.89 for CQ, 0.67 for SP, and 0.50 for AQ.
![]() View larger version (28K): [in a new window] |
FIG. 2. Failure rates (95% CIs) on day 14 (crude) (white bars) and day 28 (PCR corrected, method 1) (gray bars) for chloroquine, sulfadoxine-pyrimethamine, and amodiaquine.
|
|
|
|---|
We attempted to address a series of methodological issues related to the evaluation of treatment outcomes in pediatric uncomplicated malaria in Africa by analyzing 28 treatment arms enrolling a total of 2,576 children on antimalarial mono-therapy.
We found that, with suitable resources, the currently recommended 28-day follow-up is feasible even in relatively problematical field conditions (dropout rate, <5%; total exclusions by day 28, 11%). Extending the period of observation from 14 to 28 days resulted in higher estimated risks of failure for SP and AQ but not for CQ, for which the failure rates by day 14 were already extremely high. For AQ, the drug with the lowest failure rates at day 14, the estimated failure rate was commonly increased twofold or more when the follow-up was increased to 28 days. The gain in information varied with the drug and the site (as a function of underlying levels of drug resistance and transmission intensity, etc.).
This longer follow-up requires, however, that recurrent isolates be genotyped. This too proved feasible and adequate in these studies. Without PCR genotyping, 36% of the recurrent parasitemias after day 14 (250/696 recurrences) would have been wrongly classified as failures. This would have led to 1,048 cases being considered failures by day 28 (352 by or before day 14 plus 696 between days 14 and 28), thus overestimating the risk of failure overall by about one-third. While not all recurrences could be ascribed to either a new infection or a recrudescence (for 6%, a paired sample was not available, and for an additional 10% of cases, genotypes could not be resolved due to DNA extraction problems or because the result was not interpretable), in these studies, PCR genotyping performed very well (overall use effectiveness, 84%; test performance, 90%). Samples were genotyped in different laboratories by using nonstandardized methods. While this may be seen as a methodological deficiency, it makes these results representative of the variety of conditions applied in real life. In order to optimize the use of PCR genotyping, it is important to calculate the pretest probability of the same genotype occurring in the same individual pre- and posttreatment.
We followed today's prevailing practice of genotyping only post-day 14 recurrences. However, we have (yet unpublished) evidence from Uganda and Burkina-Faso that new infections can be documented also pre-day 14. More research is needed to further clarify the role and conditions under which PCR genotyping should be done and analyzed, including the discrimination power of PCR to correctly classify recurrent parasites.
A number of approaches are available for analysis of PCR-corrected data. We compared current and alternative methods of analyzing outcomes; our data set was large and heterogeneous. Given the high rates of follow-up and the high use effectiveness of PCR achieved, the results obtained from the different approaches were very similar in most instances, but when the WHO protocol is applied in routine conditions, variable dropout rates can be expected. Based on the principle that one should discard as little information as possible, we favor the use of Kaplan-Meier analysis in conjunction with treating new infections as treatment successes (method 1). In our data, this resulted in only 6% of patient data being lost for the entire 28-day study; by contrast, the current method (with patients evaluable at target day and excluding new infections) results in a substantial reduction in sample size (25%). This has consequences for sample size calculations in studies, particularly when dropout rates during follow-up are high (particularly if follow-up is further extended for drugs with long half-lives). A further extension to this approach, to avoid losing all the information provided by patients for whom PCR fails, would be to use multiple imputation, and we believe that this is worthy of further investigation. Indeed, there are pending issues with the criteria used to ascribe a recurrent isolate to a new or recrudescent infection.
Our results are in line with the WHO recommendation for use of life tables in analyzing in vivo study results (19).
When comparing outcomes on days 14 and 28, the correlation coefficient calculated for all studies appears reasonable but is misleading (influenced by the spread of data and the high failure rates for CQ at both day 14 and day 28). There was poor correlation with AQ. It should be noted that these analyses pertain to single-agent treatments which are no longer recommended. The WHO is now encouraging a review of drug policy when PCR-corrected failure rates at day 28 exceed 10% and the use of effective treatments, notably artemisinin-based combinations, including some with drugs with long residence times (20). The new policies will exacerbate the need for extended follow-up and for accurate assessment of true failure rate.
In conclusion, our results support the implementation of the latest WHO protocol (28-day follow-up with PCR genotyping), which proved feasible in field conditions. While this protocol is more demanding for staff and carries extra costs, it is clear that it provides a clearer picture of the level of resistance than a follow-up of only 14 days, notably in the initial phase of the establishment of resistance. To make maximum use of the data, we favor analyzing PCR-corrected outcomes with the Kaplan-Meier method and retaining new infections as treatment successes.
The individual studies were financed by Médecins sans Frontières and by Merlin in one of the sites in Sierra Leone. The present work is a desk-based analysis, supported by Médecins sans Frontières.
The opinions expressed in this paper are the authors' and do not necessarily reflect those of the employing organizations.
Published ahead of print on 5 September 2006. ![]()
|
|
|---|
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Copyright © 2010 by the American Society for Microbiology. For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»