This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrowReprints and Permissions
Right arrow Copyright Information
Right arrow Books from ASM Press
Right arrow MicrobeWorld
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Govorkova, E. A.
Right arrow Articles by Webster, R. G.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Govorkova, E. A.
Right arrow Articles by Webster, R. G.

 Previous Article  |  Next Article 

Antimicrobial Agents and Chemotherapy, December 2004, p. 4855-4863, Vol. 48, No. 12
0066-4804/04/$08.00+0     DOI: 10.1128/AAC.48.12.4855-4863.2004
Copyright © 2004, American Society for Microbiology. All Rights Reserved.

Neuraminidase Inhibitor-Rimantadine Combinations Exert Additive and Synergistic Anti-Influenza Virus Effects in MDCK Cells

Elena A. Govorkova,1,2 Hong-Bin Fang,3 Ming Tan,3 and Robert G. Webster1,4*

Department of Infectious Diseases, St. Jude Children's Research Hospital,1 Department of Pathology, University of Tennessee, Memphis, Tennessee,4 Division of Biostatistics, University of Maryland Greenebaum Cancer Center, Baltimore, Maryland,3 The D. I. Ivanovsky Institute of Virology, Moscow, Russia2

Received 20 January 2004/ Returned for modification 13 June 2004/ Accepted 9 August 2004


arrow
ABSTRACT
 
There is insufficient information about combination therapy with approved anti-influenza agents. We tested combinations that paired a neuraminidase (NA) inhibitor (zanamivir, oseltamivir carboxylate, or peramivir) with rimantadine against infection of MDCK cells with H1N1 and H3N2 subtypes of influenza A virus and characterized their mode of interaction. When reduction of extracellular virus was analyzed by individual regression models and three-dimensional representations of the data, all three combinations showed additive and synergistic effects with no cytotoxicity. Maximum synergy against A/New Caledonia/20/99 (H1N1) virus infection was observed with <2.5 µM rimantadine paired with low concentrations of NA inhibitors. All combinations reduced the extracellular yield of A/Panama/2007/99 (H3N2) influenza virus synergistically. However, our findings were different for the cell-associated virus yield. At some drug concentrations, the yield of cell-associated virus was inhibited antagonistically. Therefore, the method of analysis can be a crucial factor in evaluating the interactions of drugs with different mechanisms. We hypothesize that assays based on cell-associated virus yield may underestimate the efficacies of drug combinations that include an NA inhibitor. Taken together, our results suggest that regimens that combine NA inhibitors and rimantadine exert synergistic anti-influenza effects in vitro. These findings provide baseline information for therapeutic testing of the drug combinations in vivo.


arrow
INTRODUCTION
 
Influenza remains a serious health problem worldwide, causing the deaths of elderly people and young children and imposing substantial economic costs (17). Strategies for dealing with influenza are based on annual immunization and antiviral drugs. Control of emerging and reemerging H5N1 influenza viruses in Asia includes slaughter of poultry in markets and improvements in biosecurity (31, 33). However, the efficacies of influenza vaccines are seriously limited by the continual evolution of the hemagglutinin (HA) and neuraminidase (NA) surface glycoproteins of the viruses. For that reason, anti-influenza drugs are crucial for the control of influenza, and in the face of a pandemic virus they would be the most important short-term resource. Information about the optimal use of the currently available anti-influenza drugs is needed.

Two classes of drugs are approved for influenza prophylaxis and treatment: M2 ion channel blockers (amantadine and its derivative rimantadine) and NA inhibitors. Amantadine and rimantadine block the hydrogen ion channel activity of the M2 protein of influenza A virus (40), inhibiting viral replication by blocking virus entry into cells (4). The genetic stability of the NA enzymatic active center among influenza viruses (6) makes it a promising target for the development of antiviral drugs aimed at protecting humans against all influenza viruses. Knowledge of the NA crystal structure (38) has made possible the synthesis of NA inhibitors, the other class of anti-influenza drugs (18, 20, 39), which interrupt an established infection at a late stage by inhibiting the release of virions from infected cells. They also cause aggregation of the released virions, which are then less able to penetrate mucous secretions and infect other cells (25, 32). Thus, the two classes of available anti-influenza drugs act by different mechanisms and at different stages of the virus replication cycle. The main drawbacks of M2 blockers are the rapid development of drug-resistant variants and inefficacy against influenza B virus (14, 15). NA inhibitors are more costly, but they are active against both influenza A and B viruses (3, 26), and emergence of drug-resistant variants is limited (24).

The combined use of two or more drugs for which there are different mechanisms of resistance can also reduce the effect of resistance to a single drug. The NA inhibitor 4-guanidino-Neu5Ac2en was found to effectively inhibit plaque formation of influenza A clinical isolates that were resistant to amantadine and rimantadine (43), and treatment with zanamivir reportedly ended an outbreak of influenza that amantadine had failed to control (and from which amantadine-resistant variants were isolated) in a nursing home (19).

Therapy with synergistically active antiviral drugs that target different viral proteins and have different mechanisms of action may provide several advantages over single-agent treatment, such as greater potency, superior clinical efficacy, reduction of the drug dosages needed, reduction of respiratory complications requiring antibiotic therapy, reduction of cellular toxicity and side effects, and greater cost-effectiveness. A number of reports address the anti-influenza activity of drug combinations. Combinations of ribavirin and rimantadine were reported to cause additive and, in specific concentrations, synergistic reduction of influenza A/FPV (7), influenza A/Texas/77 (H3N2), and influenza A/USSR/77 (H1N1) virus yield in MDCK cells (11). Human alpha interferon and rimantadine or ribavirin additively or synergistically reduce the yield of clinical H3N2 or H1N1 influenza A isolates in primary rhesus monkey kidney cells (12). In a mouse model, combined rimantadine and ribavirin were associated with enhanced survival and were significantly more effective than either drug alone (13, 42). Combined treatment with rimantadine and the protease inhibitor aprotinin highly protected mice against lethal influenza virus challenge (44).

Only a few studies have tested the new class of antiviral drugs, NA inhibitors, in combination with other agents. Zanamivir combined with rimantadine, ribavirin, or 2'-deoxy-2'-fluoroguanosine showed additive effects against influenza A viruses in MDCK cells (22). The NA inhibitor peramivir was recently shown to interact favorably with ribavirin to reduce influenza A virus infection in cell culture and in mice (35).

An important initial step in evaluating combination therapy is to determine whether the combined agents reduce influenza virus replication additively or synergistically in an in vitro system. We determined the efficacies of the NA inhibitors combined with rimantadine against influenza virus infection in MDCK cells and characterized their modes of interaction. We used H1N1 and H3N2 human influenza virus subtypes that represent antigenically dominant populations included in the 2000-2001 through 2003-2004 influenza vaccines. We found that NA inhibitor-rimantadine combinations additively or synergistically reduce the extracellular virus yield in MDCK cells. Because our studies of cell-associated virus yield showed a different pattern of drug interaction, we discuss the suitability of different experimental assays for the evaluation of drug combinations.


arrow
MATERIALS AND METHODS
 
Compounds. The NA inhibitors zanamivir (4-guanidino-2,4-dideoxy-2,3-dehydro-N-acetylneuraminic acid [GG167]), GS4071 (oseltamivir carboxylate, the active metabolite of oseltamivir [3R,4R,5S]-4-acetamido-5-amino-3-[1-ethylpropoxy]-1-cyclohexane-1-carboxylic acid), and peramivir {[1S,2S,3R,4R,1'S]-3-[1'-acetylamino-2'-ethyl]butyl-4-[(aminoimino)-methyl]amino-2-hydroxycyclopentane-1-carboxylic acid [BCX-1812 or RWJ-270201] } were provided by the R. W. Johnson Pharmaceutical Research Institute (Raritan, N.J.) as lyophilized powder and were maintained at 4°C. They were dissolved in sterile distilled water at a concentration of 1 mg/ml, and aliquots were kept frozen at –70°C until they were diluted in cell culture media just before use. Rimantadine hydrochloride (1-[1-adamantyl]ethylamine hydrochloride) was obtained from Sigma-Aldrich (Milwaukee, Wis.).

Viruses. Influenza A/New Caledonia/20/99 (H1N1) and A/Panama/2007/99 (H3N2) were obtained from the repository of St. Jude Children's Research Hospital. These viruses were isolated and cultivated in the allantoic cavities of 10-day-old embryonated chicken eggs for 48 h.

Cells. Madin-Darby canine kidney (MDCK) cells were obtained from the American Type Culture Collection (Manassas, Va.) and were grown in minimal essential medium supplemented with 5% fetal calf serum, 5 mM L-glutamine, sodium bicarbonate, 100 U of penicillin per ml, 100 µg of streptomycin sulfate per ml, and 100 µg of kanamycin sulfate per ml in a humidified atmosphere of 5% CO2.

Plaque assay. Plaque assay was performed as described previously (11) and was used to determine the input virus dose for the extracellular and cell-associated virus yield reduction assays. Briefly, confluent monolayers of MDCK cells were inoculated with 10-fold dilutions of influenza virus. After 1 h at 37°C, the inoculum was removed and the cells were washed and overlaid with maintenance medium containing 1% agarose, 0.2% serum albumin, and 2.5 µg of L-1-(tosylamido-2-phenyl)ethyl chloromethyl ketone (TPCK)-treated trypsin (Worthington Diagnostics, Freehold, N.J.)/ml. After 3 days of incubation at 37°C in a humidified atmosphere of 5% CO2, the cells were stained with 0.1% crystal violet in 37% formaldehyde solution.

Extracellular virus yield reduction assay. The extracellular virus yield reduction assay was performed as described previously in 24-well plates containing confluent MDCK monolayers (34). The concentrations of rimantadine tested were 2.5, 5, 10, 20, 40, and 80 µM. The concentrations of zanamivir and oseltamivir carboxylate used were 0.001, 0.003, 0.01, 0.03, 0.1, and 0.3 µM. Peramivir was used at concentrations of 0.0001, 0.0003, 0.001, 0.003, 0.01, and 0.03 µM. Three experiments were conducted at each concentration. Drugs, alone or in combination, were added to the 24-well plates. After 30 min of incubation at 37°C, the cells were inoculated with virus at a multiplicity of infection of approximately 0.01 PFU per cell. After 48 h of incubation at 37°C, the medium was removed and centrifuged at 3,200 x g for 5 min. The supernatant was titrated by adding serially diluted samples to four wells (each) in 96-well plates of MDCK cells. To remove residual compound, the medium was replaced 2 h after virus inoculation. Virus replication was detected by HA assay 48 h after inoculation, and titers were expressed as log10 of the 50% tissue culture infectious dose (TCID) by the end point method of Reed and Muench (28). Quantification of the extracellular (supernatant) influenza virus yields in MDCK cells was conducted as described earlier (34).

Cell-associated virus yield assay. Cell-associated virus yield was assayed by a modified microneutralization method and subsequent enzyme-linked immunosorbent assay (ELISA) as described elsewhere (8). Confluent monolayers of MDCK cells in 96-well plates were overlaid with 2x drug-containing medium (0.1 ml/well). After 30 min, the cells were inoculated with influenza virus at a multiplicity of infection of 0.1 PFU/cell and incubated for 18 h at 37°C. Virus replication was determined by measuring viral nucleoprotein on the surface of infected cells. The percent inhibition of virus replication was calculated from the absorbance values determined at 490 nm on a microplate reader (Bio-Rad Laboratories, Hercules, Calif.) after correction for absorbance values of uninfected cultures. The absorbance values for the control wells (without drugs) were considered to indicate 0% inhibition of virus replication.

Drug cytotoxicity. The effects of the drugs on the growth of uninfected MDCK cells in 96-well plates was determined by using a LIVE/DEAD Viability/Cytotoxicity Kit (L-3224) (Molecular Probes, Inc., Eugene, Oreg.), according to the manufacturer's recommendations. Briefly, a confluent monolayer of cells was washed three times with washing medium and then overlaid with growth medium containing the NA inhibitors and rimantadine, individually and combined, at each concentration to be tested. After 48 h of incubation at 37°C, the plates were stained with calcein AM and ethidium homodimer-1 for 30 min at room temperature. The fluorescence of drug-exposed and control cell samples was measured by using two sets of filters. Excitation and emission wavelengths for one set were 485 and 538 nm, respectively; excitation and emission wavelengths for the other set were 544 and 590 nm, respectively. The relative numbers of living and dead cells were expressed in terms of percentages; e.g., the proportion of living cells was expressed as a percentage of the total number of cells.

Statistical analysis. The inhibitory activity of individual agents is analyzed by a regression model. The experimental data for the individual agents show that the models for the individual agents are fitted with an appropriate regression model (e.g., a linear-log concentration-response curve). To test the synergy between two agents in the experimental dose range, an F test is used (37). To explore the dose regions where synergy occurs, we considered the differences among the values for the expected model, and the regression model was fitted to the observed data.

More specifically, the individual concentration-response curves for agents A and B are y = {alpha}A + ß log(zA + {gamma}A) and y = {alpha}B + ß log(zB + {gamma}B), respectively, where y is the virus titer expressed as log10 TCID50/0.2 ml, zA and zB are the concentrations of agents A and B, respectively, and {alpha}, ß, and {gamma} are the regression coefficients. The expected response model for the mixtures of two agents A and B, assuming Loewe additivity (1, 9), should be y = {alpha}A + ß log[zA + {gamma}A + {rho}(zB + {gamma}B)], where {rho} is the potency of the agent B relative to the agent A. To characterize the joint action of two agents, the regression response model that fits the data the best is selected by examining the goodness of fit of the model for the mixtures (27). We chose 0.25 log10 TCID50/0.2 ml empirically as a plausible reference difference of interest to detect and compare the expected model for an additive action with the regression model fitted to the mixture responses. For a given mixture of the two agents, the difference of the values for the expected model and the regression model is compared to the reference difference to show the synergistic (or antagonistic) effect in that region. The contour plots of the differences for these two models show the synergistic areas of the mixture dosages. The t test is used to compare the effects of different treatments on the virus yield for specific dose. All computation is performed by using StatXact and S-PLUS.


arrow
RESULTS
 
Effect of NA inhibitors combined with rimantadine on extracellular influenza virus yield. Three different drug combinations were tested: (i) zanamivir plus rimantadine, (ii) oseltamivir carboxylate plus rimantadine, and (iii) peramivir plus rimantadine. The antiviral efficacies of the NA inhibitors and rimantadine alone and in combinations were evaluated in MDCK cells by two methods: virus yield reduction assay, which allows detection of the extracellular virus yield in cell culture supernatants, and cell ELISA, which determines the level of drug inhibition of cell-associated virus. We tested six different concentrations of NA inhibitor (0.0001 to 0.3 µM) or rimantadine (2.5 to 80 µM) shown by individual dose response curves to give from 0 to 10% reduction to 90 to 100% reduction of the initial virus yield. Tables 1 and 2 show the reduction of extracellular A/New Caledonia/20/99 (H1N1) and A/Panama/2007/99 (H3N2) virus yields, respectively.


View this table:
[in this window]
[in a new window]
  		TABLE 1. . Effect of combinations of rimantadine with NA inhibitors on extracellular yield of A/New Caledonia/20/99 (H1N1) influenza virus in MDCK cells


View this table:
[in this window]
[in a new window]
  		TABLE 2. Effect of combinations of rimantadine with NA inhibitors on extracellular yield of A/Panama/2007/99 (H3N2) virus in MDCK cells

Table 1 shows the extracellular yield of the H1N1 virus at selected drug concentrations. The virus yield in the untreated control wells was 7.5 log10 TCID50/0.2 ml. Single-agent rimantadine at doses of 5 and 10 µM resulted in ~10% and 20% reduction of the extracellular virus yield, respectively. Even the highest rimantadine concentration tested did not completely inhibit H1N1 extracellular virus, despite a reduction of 5.0 log10 TCID50/0.2 ml (70% inhibition). Conversely, single-agent NA inhibitors at certain concentrations—zanamivir and oseltamivir carboxylate at 0.3 µM and peramivir at 0.03 µM—reduced virus replication by 100%. When used in combination, the anti-influenza drugs caused a greater antiviral effect than did each drug alone. Significant reduction of virus yield (P < 0.05) was achieved at a mixture of concentrations: 0.01 to 0.1 µM for zanamivir, <0.03 µM for oseltamivir carboxylate, and <0.003 µM for peramivir in combination with rimantadine (<20 µM). The combination of 5 µM rimantadine with a 0.1 µM concentration of either zanamivir or oseltamivir carboxylate resulted in ~90% and 100% reduction of virus yield, respectively (Table 1). Peramivir was more potent than the other two NA inhibitors; the virus was completely inhibited at 0.01 µM, a dose 10-fold less than the equivalent doses of zanamivir and oseltamivir carboxylate.

Table 2 shows the yield of H3N2 virus at selected drug concentrations. For influenza A/Panama/2007/99 (H3N2) virus, it was shown that in the control wells the virus was replicating to 7.0 log10 TCID50/0.2 ml. Rimantadine at 5 and 10 µM inhibited about 10 and 25% of the virus, respectively. As with the H1N1 virus, the H3N2 virus was not completely inhibited at the highest rimantadine concentration tested (80 µM). Combinations were significantly (P < 0.05) more effective than either agent alone. Complete inhibition was achieved by 80 µM rimantadine in combination with 0.3 µM zanamivir or oseltamivir carboxylate, or 0.01 µM peramivir.

Mode of interaction of anti-influenza drugs in reducing extracellular yield. Data on the inhibition of influenza H1N1 and H3N2 extracellular virus yield were plotted in three dimensions to form a response surface (Fig. 1), and the mode of drug interaction was characterized by the regression models. Individual regression models were determined by experimental data with rimantadine, zanamivir, oseltamivir carboxylate, and peramivir; models that predicted the additive action of two agents were derived from these individual models. According to the results of the combination experiments, we obtained the response regression models for the joint action of zanamivir, oseltamivir carboxylate, and peramivir with rimantadine against H1N1 and H3N2 viruses by using model selection techniques. A comparison of the expected additive model with the corresponding response regression model for each combination of agents and the corresponding contour plots of the differences for these two models yielded the following synergy analysis.



View larger version (43K):
[in this window]
[in a new window]
  		FIG. 1. Three-dimensional response surface showing the reduction of extracellular virus yield in MDCK cells by NA inhibitor-rimantadine combinations. The X and Y axes show the concentrations (in µM) of drugs. Rimantadine was combined with zanamivir (A), oseltamivir carboxylate (B), or peramivir (C) against the A/New Caledonia/20/99 (H1N1) influenza virus. Rimantadine was combined with zanamivir (D), oseltamivir carboxylate (E), or peramivir (F) against the A/Panama/2007/99 (H3N2) influenza virus. Virus yield was measured as log10 TCID50/0.2 ml in culture supernatants.

The overall interaction of zanamivir with rimantadine is synergistic in inhibiting extracellular A/New Caledonia/20/99 (H1N1) virus [F(34,108) = 18.96; P < 0.0001] (Fig. 2A). The joint action of the two drugs reaches maximum synergy when rimantadine at 10 to ~80 µM is combined with zanamivir at 0.02 to ~0.06 µM. Although the interaction of the two drugs was synergistic at most concentrations tested, there are regions of additivity (shown in yellow in Fig. 2A) and antagonism (rimantadine at <5.5 µM and zanamivir <0.0025 µM).



View larger version (37K):
[in this window]
[in a new window]
  		FIG. 2. Contour plots showing interaction of the NA inhibitors with rimantadine in reducing the extracellular yield of the A/New Caledonia/20/99 (H1N1) virus in MDCK cells. The regions shown in yellow represent additive drug interactions. Rimantadine was combined with zanamivir (A), oseltamivir carboxylate (B), or peramivir (C and D). The numbers in the contour curves are the differences of the values from those of the expected model for the mixtures of two agents, assuming Loewe additivity and the regression response model that fits the data.

The interaction of oseltamivir carboxylate with rimantadine in the reduction of extracellular H1N1 virus yield was mainly synergistic [F(34,108) = 16.78; P < 0.0001] (Fig. 2B). Synergistic activity was observed at a wide range of concentrations, reaching a maximum at concentrations of 32 to ~80 µM for rimantadine and <0.06 µM for oseltamivir carboxylate. However, as with the zanamivir and rimantadine combinations, there was a region of antagonistic interaction (rimantadine at <4 µM and oseltamivir carboxylate at <0.005 µM). The two-drug interaction was additive at concentrations of <0.01 µM for oseltamivir carboxylate and <4.5 µM for rimantadine.

The third combination tested (peramivir plus rimantadine) also synergistically reduced the extracellular yield of A/New Caledonia/20/99 (H1N1) influenza virus [F(34,108) = 25.34; P < 0.0001] (Fig. 2C and D). Within the range of concentrations tested, maximum synergy was achieved at rimantadine concentrations of 12 to ~80 µM combined with a peramivir concentration of <0.0025 µM. However, the mode of action of this combination differed from the two described earlier. There appeared to be additive effects at two ranges of concentration (Fig. 2C and D): <0.00015 µM peramivir combined with <6 µM rimantadine and <0.015 µM peramivir combined with <5 µM rimantadine. There were also two ranges of antagonistic interaction, one in which the concentrations of both drugs were low (<9 µM for rimantadine and <0.00015 µM for peramivir) and one in which the concentrations of both drugs were high (20 to ~80 µM for rimantadine and 0.016 to ~0.03 µM for peramivir).

When tested against influenza A/Panama/2007/99 (H3N2), NA inhibitors combined with rimantadine induced an overall synergistic reduction of extracellular virus yield (P < 0.0001 by overall F test). As determined by regression analysis and three-dimensional representation of the data, zanamivir and rimantadine had maximum synergy at concentrations ranging from 12 to ~75 µM for rimantadine and from 0.015 to ~0.03 µM for zanamivir (Fig. 3). Oseltamivir carboxylate and rimantadine reached maximum synergy at concentrations of 18 to ~80 µM for rimantadine and <0.05 µM for oseltamivir carboxylate (results not shown). Peramivir and rimantadine reached maximum synergy at 10 to ~70 µM for rimantadine and 0.0001 to ~0.0002 µM for peramivir (results not shown).



View larger version (48K):
[in this window]
[in a new window]
  		FIG. 3. Contour plot showing the interaction of zanamivir and rimantadine in reducing the extracellular yield of the A/Panama/2007/99 (H3N2) virus in MDCK cells. The drugs acted synergistically at all concentrations tested. The contour plot shows the area of maximum synergy. The numbers in the contour curves are the differences of the values from those of the expected model for the mixtures of two agents, assuming Loewe additivity and the regression response model that fits the data.

Effect of NA inhibitor-rimantadine combinations on cell-associated virus yield. In these experiments, we tested seven concentrations of oseltamivir carboxylate (0.001 to 1 µM) and six concentrations of rimantadine (2.5 to 80 µM), either alone or combined, by cell ELISA. This method is frequently used to test the anti-influenza efficacies of agents in vitro. Table 3 shows the percent reduction of cell-associated A/New Caledonia/20/99 (H1N1) virus yield at selected concentrations of oseltamivir carboxylate and rimantadine. The effects of both drugs were dose dependent. The effect of single-agent rimantadine was small. When rimantadine was combined with oseltamivir carboxylate, the inhibition of virus replication was enhanced at most concentrations and was pronounced (35%) at a 0.01 µM concentration of oseltamivir carboxylate combined with a 20 µM concentration of rimantadine (Table 3).


View this table:
[in this window]
[in a new window]
  		TABLE 3. Effect of oseltamivir carboxylate-rimantadine combinations on the yield of cell-associated influenza A/New Caledonia/20/99 (H1N1) virus in MDCK cells

To characterize the mode of drug interaction in the reduction of cell-associated virus yield, we performed the same regression analysis as that used for the extracellular virus yield. On the resulting contour plot (Fig. 4), the strongest synergistic effect of the two drugs was seen when the highest dose of rimantadine was combined with the lowest dose of oseltamivir carboxylate and when the lowest dose of rimantadine was combined with the highest dose of oseltamivir carboxylate. Maximum synergy was observed when rimantadine at <60 µM was combined with oseltamivir carboxylate at <0.01 µM. Contrary to our findings in the inhibition of extracellular virus, there was a region of dose combinations that showed antagonistic interaction. Maximum antagonism was observed at low concentrations (<0.6 µM) of both agents.



View larger version (48K):
[in this window]
[in a new window]
  		FIG. 4. Contour plot showing the interaction between oseltamivir carboxylate and rimantadine in reducing cell-associated influenza A/New Caledonia/20/99 (H1N1) virus yield in MDCK cells. The numbers in the contour curves are the differences of the values from those of the expected model for the mixtures of two agents, assuming Loewe additivity and the regression response model that fits the data.

Because the modes of drug interaction appeared to be different for the two different assays (cellular versus extracellular virus yield), we conducted a separate set of experiments in which a single assay system measured the reduction of cell-associated and extracellular H1N1 virus yield by oseltamivir carboxylate (Fig. 5A) and rimantadine (Fig. 5B). The two agents inhibited both extracellular and cell-associated virus in a dose-dependent manner, but their efficacy differed. At 10 µM, oseltamivir carboxylate completely inhibited extracellular virus but reduced cell-associated virus only from 7.0 to 4.5 log10 TCID50/0.2 ml. Rimantadine reduced extracellular and cell-associated H1N1 virus yields to a more comparable extent at different drug concentrations; although the reduction of extracellular yield was more dose responsive, there was no significant difference between the levels of extracellular and cell-associated virus inhibition at the highest concentration tested.



View larger version (16K):
[in this window]
[in a new window]
  		FIG. 5. Reduction of extracellular and cell-associated A/New Caledonia/20/99 (H1N1) virus yield in MDCK cells by oseltamivir carboxylate (A) and rimantadine (B). Extracellular virus yield (•) was measured as log10 TCID50/0.2 ml of culture supernatant. Cell-associated virus yield ({blacksquare}) was measured as log10 TCID50/0.2 ml of cell suspension after three rounds of freeze-thawing. Values are the means ± standard deviations for two or three independent assays.

Cytotoxicity. The NA inhibitors used as single agents caused no cytotoxicity in MDCK cells at the range of concentrations tested. Rimantadine caused some cytotoxicity (~5% reduction of live cells) at the highest concentration tested (80 µM), whether used alone or in combination with the NA inhibitors. However, no enhanced cytotoxicity was seen when the drugs were used in combination.


arrow
DISCUSSION
 
Our results show that combination treatment with an NA inhibitor (zanamivir, oseltamivir carboxylate, or peramivir) and rimantadine markedly reduces the extracellular H1N1 and H3N2 influenza virus yield in MDCK cells in comparison to the yield obtained after treatment with either single drug. The drugs were found to interact both additively and synergistically. The three-dimensional approach that we employed allowed a complete analysis of all tested drug concentrations and resulting biological effects and is considered to be the most suitable model for analysis of drug interactions (16).

Importantly, synergism between NA inhibitors and rimantadine was observed at a wide range of concentrations, against both the H1N1 and H3N2 influenza virus subtypes. The influenza A/Panama/2007/99 (H3N2) strain was more sensitive to the combination, which synergistically reduced extracellular virus yield in MDCK cells at all concentrations tested. The extracellular yield of influenza A/New Caledonia/20/99 (H1N1) virus was reduced both additively and synergistically; however, the patterns of reactivity differed slightly from those of the H3N2 virus. Influenza viruses have been reported to differ in their sensitivities to zanamivir, oseltamivir carboxylate, and peramivir (8, 34, 41). This finding can be explained to some extent by the differences among strains in the types of amino acid residues surrounding the enzyme active center of NA and by the balance between the HA affinity to cellular receptors and the NA enzymatic activity. These effects are also closely related to the structure of the NA inhibitors, a fact which is considered important for achieving an energetically favorable interaction with the influenza NA glycoprotein (41). Our results showed that two drug pairs (zanamivir-rimantadine and oseltamivir carboxylate-rimantadine) interact similarly to inhibit recovery of extracellular H1N1 virus in MDCK cells but that peramivir interacts with rimantadine differently to inhibit extracellular virus. A likely explanation is the different chemical structure of the NA inhibitors (2, 20, 38). These structural characteristics may affect the orientation of the drugs' active groups and thus alter the NA inhibitors' binding capacity for the amino acids in the active site of the enzyme.

The exact mechanism by which these two drugs synergistically reduce influenza virus yield in MDCK cells is not clear. Although additional experiments are needed, we hypothesize that the inhibition of the M2 protein function by the amantadine and its derivative rimantadine causes a decrease in pH within intracellular compartments and thus may play a role in the maturation of HA glycoprotein in trans-Golgi regions of the exocytic pathway (5). Treatment with amantadine and rimantadine could cause specific conversion of the native HA conformation to the low-pH HA conformation (36); HA in this conformation has a tendency to aggregate and therefore may interfere with the pinching off of transport vesicles, causing enlargement of trans-Golgi structures in a manner analogous to the inhibition of virus release from the cells. This effect could not only decrease the numbers of virus particles released from the cells but also alter the balance between HA and NA so that less NA inhibitor is required to achieve the same biological effect.

We assayed the inhibition of both extracellular and cell-associated virus in cell culture. Utilization of the two different biological assays allows us to determine the most reliable and suitable method of analysis for evaluation of the mode of anti-influenza drug interaction in vitro. The NA inhibitors used in combination with rimantadine additively or synergistically reduced extracellular virus yield in MDCK cells. However, they appeared to interact differently in assays of cell-associated virus yield, showing antagonism at certain concentrations. Influenza viruses with low NA activity are known to be released inefficiently from the surfaces of infected cells, thereby limiting virus spread from cell to cell or within the respiratory tract (10, 21, 23). In the presence of the NA inhibitors, we observed the same effects. When the release of virus particles was blocked, the virions aggregated at the cell surface, and their spread from cell to cell was limited. Therefore, assays of the cell-associated virus yield may underestimate the efficacies of the NA inhibitors when used either alone or in combinations. Our results and those reported by Madren and coauthors (22) suggest that the cell ELISA is not optimal for use in analyzing anti-influenza drug interactions. Moreover, such an analysis can be limited by the 18-h incubation period, the multiplicity of infection (0.01 to 0.1 PFU/cell) used in cell ELISAs, and the limited number of virus replication cycles that may occur during this time period. Treatment with NA inhibitor can protect only neighboring cells from secondary infection; therefore, if all or most cells are initially infected with virus, the effect of the drugs may be underestimated. It is also important to remember that the cell ELISA is more dependent on the use of a constant number of cells for analysis and is characterized by more variable results than the plaque reduction assay. The method of analysis may be crucial for evaluation of the drug interactions.

Combination therapy against influenza virus infection has been investigated (13, 42, 44). However, most studies have tested experimental compounds, such as polyoxometalate PM-523 (30), nucleoside analogue 2'-deoxy-2'-fluoroguanosine (22), or infusions made from the natural antiviral agent Flos verbasci (29). Earlier studies showed that zanamivir exerts additive effects with rimantadine, ribavirin, or 2'-deoxy-2'-fluoroguanosine against influenza A viruses in MDCK cells (22). However, additional information is needed about combination therapy with antiviral agents approved for use against influenza virus infection. The NA inhibitor peramivir was recently reported to interact favorably with ribavirin to reduce extracellular influenza A virus yield in cell culture and in mice (35). The present study is the first to use regression analysis and three-dimensional data representation to analyze the interaction of NA inhibitors and rimantadine against influenza virus infection in MDCK cells at a wide range of concentrations. Our results showed that pairing zanamivir, oseltamivir carboxylate, or peramivir with rimantadine reduces extracellular H1N1 and H3N2 virus yields in MDCK cells either additively or synergistically. Because study of the reduction of virus yield in vitro is considered to be a crucial step in drug evaluation, our observations could serve as baseline information for future studies in vivo. The degree of inhibition and the type of drug interaction are reported to differ with the virus strain, drug concentration, multiplicity of infection, and type of cell culture (13). Our results demonstrate that the observed findings also depend on the biological effect chosen as an endpoint and the method of detection of this biological effect.

Overall, our results support the idea that virus yield can be synergistically reduced by combination therapy with antiviral drugs that target different viral proteins and have different mechanisms of action. However, studies with animal models are needed to determine the advantages of the drug combinations tested here.


arrow
ACKNOWLEDGMENTS
 
This work was supported by grants AI95357 and AI57570-01 from the National Institute of Allergy and Infectious Diseases, by Cancer Center Support (CORE) grant CA-21765 from the National Institutes of Health, by the American Lebanese Syrian Associated Charities, and by the R. W. Johnson Pharmaceutical Research Institute.

We thank Christoph Scholtissek for valuable discussion of the results, Tiebin Lin for statistical analysis, and Sharon Naron for excellent editorial assistance.


arrow
FOOTNOTES
 
* Corresponding author. Mailing address: Department of Infectious Diseases, St. Jude Children's Research Hospital, 332 N. Lauderdale, Memphis, TN 38105-2794. Phone: (901) 495-3400. Fax: (901) 523-2622. E-mail: robert.webster{at}stjude.org. Back


arrow
REFERENCES
 
    1
  1. Abdelbasit, K. M., and P. L. Plackett. 1982. Experimental design for joint action. Biometrics 38:171-179.[CrossRef]
    2. 2
  2. Babu, Y. S., P. Chand, S. Bantia, P. Kotian, A. Dehghani, Y. El-Kattan, T. H. Lin, T. L. Hutchison, A. J. Elliott, C. D. Parker, S. L. Ananth, L. L. Horn, G. W. Laver, and J. A. Montgomery. 2000. BCX-1812 (RWJ-270201): discovery of a novel, highly potent, orally active, and selective influenza neuraminidase inhibitor through structure-based drug design. J. Med. Chem. 43:3482-3486.[CrossRef][Medline]
    3. 3
  3. Bantia, S., C. D. Parker, S. L. Ananth, L. L. Horn, K. Andries, P. Chand, P. L. Kotian, A. Dehghani, Y. El-Kattan, T. Lin, T. L. Hutchison, J. A. Montgomery, D. L. Kellog, and Y. S. Babu. 2001. Comparison of the anti-influenza virus activity of RWJ-270201 with those of oseltamivir and zanamivir. Antimicrob. Agents Chemother. 45:1162-1167.[Abstract/Free Full Text]
    4. 4
  4. Belshe, R. B., H. M. Smith, C. B. Hall, R. Betts, and A. J. Hay. 1988. Genetic basis of resistance to rimantadine emerging during treatment of influenza virus infection. J. Virol. 62:1508-1512.[Abstract/Free Full Text]
    5. 5
  5. Ciampor, F., P. M. Bayley, M. V. Nermut, E. M. A. Hirst, R. J. Sugrue, and A. J. Hay. 1992. Evidence that the amantadine-induced, M2-mediated conversion of influenza A virus hemagglutinin to the low pH conformation occurs in an acidic trans Golgi compartment. Virology 188:14-24.[CrossRef][Medline]
    6. 6
  6. Colman, P. M., P. A. Hoyne, and M. C. Lawrence. 1993. Sequence and structure alignment of paramyxovirus hemagglutinin-neuraminidase with influenza virus neuraminidase. J. Virol. 67:2972-2980.[Abstract/Free Full Text]
    7. 7
  7. Galegov, G. A., N. L. Pushkarskaya, N. P. Obrosova-Serova, and V. M. Zhdanov. 1977. Combined action of ribavirin and rimantadine in experimental myxovirus infection. Experientia 33:905-906.[CrossRef][Medline]
    8. 8
  8. Govorkova, E. A., I. A. Leneva, O. G. Goloubeva, K. Bush, and R. G. Webster. 2001. Comparison of the efficacies of RWJ-270201, zanamivir, and oseltamivir against H5N1, H9N2, and other avian influenza viruses. Antimicrob. Agents Chemother. 45:2723-2732.[Abstract/Free Full Text]
    9. 9
  9. Greco, W., G. Bravo, and J. C. Parsons. 1995. The search for synergy: a critical review from a response surface perspective. Pharmacol. Rev. 47:331-385.[Medline]
    10. 10
  10. Gubareva, L. V., L. Kaiser, and F. G. Hayden. 2000. Influenza virus neuraminidase inhibitors. Lancet 4:827-835.
    11. 11
  11. Hayden, F. G., R. G. Douglas, Jr., and R. Simons. 1980. Enhancement of activity against influenza viruses by combinations of antiviral agents. Antimicrob. Agents Chemother. 18:536-541.[Abstract/Free Full Text]
    12. 12
  12. Hayden, F. G., A. N. Slepushkin, and N. L. Pushkarskaya. 1984. Combined interferon-{alpha}2, rimantadine hydrochloride, and ribavirin inhibition of influenza virus replication in vitro. Antimicrob. Agents Chemother. 25:53-57.[Abstract/Free Full Text]
    13. 13
  13. Hayden, F. G. 1986. Combinations of antiviral agents for treatment of influenza virus infections. J. Antimicrob. Chemother. 18(Suppl. B):77-83.
    14. 14
  14. Hayden, F. G., and H. J. Hay. 1992. Emergence and transmission of influenza A viruses resistant to amantadine and rimantadine. Curr. Top. Microbiol. Immunol. 176:119-130.[Medline]
    15. 15
  15. Hayden, F.G. 1996. Amantadine and rimantadine—clinical aspects, p. 59-77. In D. D. Richman (ed.), Antiviral drug resistance. John Wiley & Sons, Inc., New York, N.Y.
    16. 16
  16. Kanzawa, F., K. Nishio, K. Fukuoka, M. Fukuda, T. Kunimoto, and N. Saijo. 1997. Evaluation of synergism by a novel three-dimensional model for the combined action of cisplatin and etoposide on the growth of a human small-cell lung-cancer cell line, SBC-3. Int. J. Cancer 71:311-319.[CrossRef][Medline]
    17. 17
  17. Keech, M., A. J. Scott, and P. J. Ryan. 1998. The impact of influenza and influenza-like illness on productivity and healthcare resource utilization in a working population. Occup. Med. (London) 48:85-90.
    18. 18
  18. Kim, C. U., W. Lew, M. A. Williams, H. Liu, L. Zhang, S. Swaminathan, N. Bischofberger, M. S. Chen, D. B. Mendel, C. Y. Tai, W. G. Laver, and R. C. Stevens. 1997. Influenza neuraminidase inhibitors possessing a novel hydrophobic interaction in the enzyme active site: design, synthesis, and structural analysis of carbocyclic sialic acid analogues with potent anti-influenza activity. J. Am. Chem. Soc. 119:681-690.[CrossRef][Medline]
    19. 19
  19. Lee, C., M. Loeb, A. Phillips, J. Nesbitt, K. Smith, M. Fearon, M. A. McArthur, T. Mazzulli, Y. Li, and A. McGeer. 2000. Zanamivir use during transmission of amantadine-resistant influenza A in a nursing home. Infect. Control Hosp. Epidemiol. 21:700-704.[CrossRef][Medline]
    20. 20
  20. Lew, W., X. Chen, and C. U. Kim. 2000. Discovery and development of GS4104 (oseltamivir): an orally active influenza neuraminidase inhibitor. Curr. Med. Chem. 7:663-672.[Medline]
    21. 21
  21. Liu, C., and G. M. Air. 1993. Selection and characterization of a neuraminidase-minus mutant of influenza virus and its rescue by cloned neuraminidase genes. Virology 194:403-407.[CrossRef][Medline]
    22. 22
  22. Madren, L. K., C. Shipman, Jr., and F. G. Hayden. 1995. In vitro inhibitory effects of combinations of anti-influenza agents. Antivir. Chem. Chemother. 6:109-113.
    23. 23
  23. McKimm-Breschkin, J. L., T. J. Blick, A. Sahasrabudhe, T. Tiong, D. Marshall, G. J. Hart, R. C. Bethell, and C. R. Penn. 1996. Generation and characterization of variants of NWS/G70C influenza virus after in vitro passage in 4-amino-Neu5Ac2en and 4-guanidino-Neu5Ac2en. Antimicrob. Agents Chemother. 40:40-46.[Abstract]
    24. 24
  24. McKimm-Breschkin, J. L. 2000. Resistance of influenza viruses to neuraminidase inhibitors—a review. Antivir. Res. 47:1-17.[CrossRef][Medline]
    25. 25
  25. Mendel, D. B., C. Y. Tai, P. A. Escarpe, W. X. Li, R. W. Sidwell, J. H. Huffman, C. Sweet, K. J. Jakeman, J. Merson, S. A. Lacy, W. Lew, M. A. Williams, L. Zhang, M. S. Chen, N. Bishofberger, and C. U. Kim. 1998. Oral administration of a prodrug of the influenza virus neuraminidase inhibitor GS4071 protects mice and ferrets against influenza infection. Antimicrob. Agents Chemother. 42:640-646.[Abstract/Free Full Text]
    26. 26
  26. Monto, A. S., D. M. Fleming, D. Henry, R. de Groot, M. Makela, T. Klein, M. Elliott, O. N. Keene, and C. Y. Man. 1999. Efficacy and safety of the neuraminidase inhibitor zanamivir in the treatment of influenza A and B virus infections. J. Infect. Dis. 180:254-261.[CrossRef][Medline]
    27. 27
  27. Prichard, M. N., and C. Shipman, Jr. 1990. A tree-dimensional model to analyze drug-drug interactions. Antivir. Res. 14:181-206.[CrossRef][Medline]
    28. 28
  28. Reed, L. J., and H. Muench. 1938. A simple method for estimating fifty percent endpoints. Am. J. Hyg. 27:493-497.
    29. 29
  29. Serkedjieva, J. 2000. Combined anti-influenza virus activity of Flos verbasci infusion and amantadine derivatives. Phytother. Res. 14:571-574.[CrossRef][Medline]
    30. 30
  30. Shigeta, S., S. Mori, J. Watanabe, S. Soeda, K. Takahashi, and T. Yamase. 1997. Synergistic anti-influenza virus A (H1N1) activities of PM-523 (polyoxometalate) and ribavirin in vitro and in vivo. Antimicrob. Agents Chemother. 41:1423-1427.[Abstract]
    31. 31
  31. Shortridge, K. F. 1999. Poultry and the influenza H5N1 outbreak in Hong Kong, 1997: abridged chronology and virus isolation. Vaccine 17(Suppl. 1):S26-S29.
    32. 32
  32. Sidwell, R. W., J. H. Huffman, D. L. Barnard, K. W. Bailey, M. N. Wong, A. Morrison, T. Syndergaard, and C. U. Kim. 1998. Inhibition of influenza virus infections in mice by GS4104, an orally effective influenza virus neuraminidase inhibitor. Antivir. Res. 37:107-120.[CrossRef][Medline]
    33. 33
  33. Sims, L. D., T. M. Ellis, K. K. Liu, K. Dryting, H. Wong, M. Peiris, Y. Guan, and K. F. Shortridge. 2003. Avian influenza in Hong Kong 1997-2002. Avian Dis. 47(Suppl. 3):S832-S838.
    34. 34
  34. Smee, D. F., J. H. Huffman, A. C. Morrison, D. L. Barnard, and R. W. Sidwell. 2001. Cyclopentane neuraminidase inhibitors with potent in vitro anti-influenza virus activities. Antimicrob. Agents Chemother. 45:743-748.[Abstract/Free Full Text]
    35. 35
  35. Smee, D. F., K. W. Bailey, A. C. Morrison, and R. W. Sidwell. 2002. Combination treatment of influenza A virus infections in cell culture and in mice with the cyclopentane neuraminidase inhibitor RWJ-270201 and ribavirin. Chemotherapy 48:88-93.[CrossRef][Medline]
    36. 36
  36. Sugrue, R. J., G. Bahadur, M. C. Zambon, M. Hall-Smith, A. R. Duglas, and A. J. Hay. 1990. Specific structural alterations of the influenza haemagglutinin by amantadine. EMBO J. 9:3469-3476.[Medline]
    37. 37
  37. Tan, M., H. B. Fang, G. L. Tian, and P. J. Houghton. 2003. Experimental design and sample size determination for testing synergism in drug combination studies based on uniform measures. Stat. Med. 22:2091-2100.[CrossRef][Medline]
    38. 38
  38. Varghese, J. N., J. L. McKimm-Breschkin, J. B. Caldwell, A. A. Kortt, P. M. Colman. 1992. The structure of the complex between influenza virus neuraminidase and sialic acid, the viral receptor. Proteins 14:327-332.[CrossRef][Medline]
    39. 39
  39. von Itzstein, M., W.-Y. Wu, G. K. Kok, M. S. Pegg, J. C. Dyason, B. Jin, T. Van Phan, M. L. Smithe, H. F. White, S. W. Oliver, P. M. Colman, J. N. Varghese, D. M. Ryan, J. M. Woods, R. C. Bethell, V. J. Hotham, J. M. Cameron, and C. R. Penn. 1993. Rational design of potent sialidase-based inhibitors of influenza virus protection. Nature 363:418-423.[CrossRef][Medline]
    40. 40
  40. Wang, C., K. Takeuchi, L. H. Pinto, and R. A. Lamb. 1993. Ion channel activity of influenza A virus M2 protein: characterization of the amantadine block. J. Virol. 67:5585-5594.[Abstract/Free Full Text]
    41. 41
  41. Wang, M. Z., C. Y. Tai, and D. B. Mendel. 2002. Mechanism by which mutations at His-274 alter sensitivity of influenza A virus N1 neuraminidase to oseltamivir carboxylate and zanamivir. Antimicrob. Agents Chemother. 46:3809-3816.[Abstract/Free Full Text]
    42. 42
  42. Wilson, S. Z., V. Knight, P. R. Wyde, S. Drake, and R. B. Couch. 1980. Amantadine and ribavirin aerosol treatment of influenza A and B infection in mice. Antimicrob. Agents Chemother. 17:642-648.[Abstract/Free Full Text]
    43. 43
  43. Woods, J. M., R. C. Bethell, J. A. V. Coates, N. Healy, S. A. Hiscox, B. A. Pearson, D. M. Ryan, J. Ticehurst, S. M. Walcott, and C. R. Penn. 1993. 4-Guanidino-2,4-dideoxy-2,3-dehydro-N-acethylneuraminic acid is a highly effective inhibitor both of the sialidase (neuraminidase) and of growth of wide range of influenza A and B viruses in vitro. Antimicrob. Agents Chemother. 37:1473-1479.[Abstract/Free Full Text]
    44. 44
  44. Zhirnov, O. P. 1987. High protection of animals lethally infected with influenza virus by aprotinin-rimantadine combinations. J. Med. Virol. 21:161-167.[Medline]

Antimicrobial Agents and Chemotherapy, December 2004, p. 4855-4863, Vol. 48, No. 12
0066-4804/04/$08.00+0     DOI: 10.1128/AAC.48.12.4855-4863.2004
Copyright © 2004, American Society for Microbiology. All Rights Reserved.



This article has been cited by other articles:

  • Nguyen, J. T., Hoopes, J. D., Smee, D. F., Prichard, M. N., Driebe, E. M., Engelthaler, D. M., Le, M. H., Keim, P. S., Spence, R. P., Went, G. T. (2009). Triple Combination of Oseltamivir, Amantadine, and Ribavirin Displays Synergistic Activity against Multiple Influenza Virus Strains In Vitro. Antimicrob. Agents Chemother. 53: 4115-4126 [Abstract] [Full Text]  
  • Smee, D. F., Hurst, B. L., Wong, M.-H., Bailey, K. W., Morrey, J. D. (2009). Effects of Double Combinations of Amantadine, Oseltamivir, and Ribavirin on Influenza A (H5N1) Virus Infections in Cell Culture and in Mice. Antimicrob. Agents Chemother. 53: 2120-2128 [Abstract] [Full Text]  
  • Ilyushina, N. A., Hay, A., Yilmaz, N., Boon, A. C. M., Webster, R. G., Govorkova, E. A. (2008). Oseltamivir-Ribavirin Combination Therapy for Highly Pathogenic H5N1 Influenza Virus Infection in Mice. Antimicrob. Agents Chemother. 52: 3889-3897 [Abstract] [Full Text]  
  • Gaudreault, E., Gosselin, J. (2008). Leukotriene B4 Induces Release of Antimicrobial Peptides in Lungs of Virally Infected Mice. J. Immunol. 180: 6211-6221 [Abstract] [Full Text]  
  • Tsiodras, S., Mooney, J. D, Hatzakis, A. (2007). Role of combination antiviral therapy in pandemic influenza and stockpiling implications. BMJ 334: 293-294 [Full Text]  
  • Wei, L., Sandbulte, M. R., Thomas, P. G., Webby, R. J., Homayouni, R., Pfeffer, L. M. (2006). NF{kappa}B Negatively Regulates Interferon-induced Gene Expression and Anti-influenza Activity. J. Biol. Chem. 281: 11678-11684 [Abstract] [Full Text]  
  • Wong, S. S. Y., Yuen, K.-y. (2006). Avian Influenza Virus Infections in Humans. Chest 129: 156-168 [Abstract] [Full Text]  

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrowReprints and Permissions
Right arrow Copyright Information
Right arrow Books from ASM Press
Right arrow MicrobeWorld
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Govorkova, E. A.
Right arrow Articles by Webster, R. G.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Govorkova, E. A.
Right arrow Articles by Webster, R. G.

Copyright © 2009 by the American Society for Microbiology.   For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»