Compelling personal testimony about the clinical benefits experienced by Duchenne muscular dystrophy (DMD) patients taking PTC Therapeutics Inc.'s Translarna (ataluren) was not enough to overcome less-than-compelling randomized clinical trial data, a US FDA advisory committee concluded Sept. 28.
Ten of 11 members of the Peripheral and Central Nervous System Drugs Advisory Committee voted that while it is possible ataluren may be effective in treating dystrophinopathies, particularly DMD, resulting from nonsense mutations in the dystrophin gene, the data are inconclusive and more work would be needed to establish efficacy.
The panelists pointed to the failure of two randomized, placebo-controlled trials – studies 007 and 020 – to meet their prespecified primary endpoints. Although post hoc, exploratory analyses suggesting benefit were interesting, they were not persuasive evidence of efficacy, committee members said.
Advisory Committee Vote
The best interpretation of the information presented today regarding the use of ataluren for the treatment of dystrophinopathies resulting from nonsense mutations in the dystrophin gene is that:
- a. The data suggest that ataluren is not effective – 0 votes
- b. Although it is possible that ataluren may be effective, the data are inconclusive, and more work would be needed to establish whether ataluren is effective – 10 votes
- c. The data are sufficient to conclude that ataluren is effective – 1 vote
"I came back to the question of whether there was substantial evidence in this case, and it seemed like the evidence that kept coming out was mostly from post hoc, re-examinations of existing trials after the data had been revealed," said committee member Aaron Kesselheim, associate professor of medicine at Harvard.
"I'm not one to be fixed on any particular p-value," Kesselheim said. "For me the critical issue here was that in a lot of the studies that we saw, when the studies were set up and evaluated prospectively … there was not a clear effect, and then retrospectively it looked like there were certain areas where there were clear effects, and that kind of re-analysis can be misleading."
"I think the FDA does often exercise flexibility, but there is some statutory threshold of evidence of efficacy and safety required before approval," said panel chairman G. Caleb Alexander, associate professor of epidemiology and medicine at Johns Hopkins. "When I was looking at the question about whether or not there was evidence of effectiveness, I thought the data were inconclusive."
"I'm not sure any number of post hoc analyses can replace the confidence that's provided by a well-controlled, randomized trial that meets its primary, pre-specified endpoints." – Johns Hopkins' Alexander
"I'm not sure any number of post hoc analyses can replace the confidence that's provided by a well-controlled, randomized trial that meets its primary, prespecified endpoints," Alexander added.
Patients Tell A More Robust Story Of Efficacy
The frailty of the clinical trial results – at least as prespecified from a statistical perspective – stood in stark contrast to stories during the open public hearing about the experiences of more than 20 patients who received ataluren in clinical trials.
Caregivers and patients, some of whom have taken ataluren for 10 years, said the drug has slowed disease progression and maintained walking ability far longer than predicted based upon the natural history of the disease. They also testified to improvements in ambulation, strength and energy when patients started or restarted ataluren, and to deterioration when they stopped the drug.
There were a total of 28 open public hearing slots, some of which had multiple speakers representing multiple families. Only one speaker urged against approval.
"I found the public testimony compelling. I found the data disappointing," said panelist Mark Green, a professor of neurology, anesthesiology and rehabilitation medicine at Icahn School of Medicine at Mt. Sinai.
"I was impressed by the heartrending testimony of many of the people in the public and I thought that was very compelling," said Joel Perlmutter, neurology professor at Washington University. "But I'm also reminded if that kind of response occurred in all these people that were treated with this in the study, we wouldn't have had failure to find statistically significant results. So although it's compelling and it's emotional, it wasn't really supported by the data that we had at hand to review."
Perlmutter said the post hoc analyses were intriguing but raised validity questions. "I think there's a lot of interesting potential with this," he said. "This could be great, but I don't think we have the data yet to approve that."
The panel's patient representative, Jeffrey Watkins, was the only committee member to vote that the data were sufficient to conclude ataluren is effective.
"It's not effective in everybody, it's not a cure. But it's certainly effective, I believe, in some people." – Patient representative Watkins
Watkins said he was persuaded by written comments from clinicians who reported observing a positive impact with ataluren in dozens of patients. "I put a lot of validity to their observations."
"The second powerful evidence that I heard was the deleterious effect of stopping the drug, patients who had reversed symptoms and when they get back on they would see improvement," he said. "I interpreted that there's sufficient data to conclude it's effective. It's not effective in everybody, it's not a cure. But it's certainly effective, I believe, in some people."
No Regulatory Guarantees With DMD
FDA is expected to make a decision on PTC's new drug application by Oct. 24. The NDA has been the subject of two refusal-to-file letters and was filed over protest. (Also see "Exondys Revisited? Translarna Brings Efficacy Woes Into US Panel Review" - Pink Sheet, 26 Sep, 2017.)
The panel's vote, coupled with FDA reviewers' seemingly strongly held view that the efficacy data are insufficient to support approval, would suggest a complete response letter will be forthcoming. However, the regulatory history around DMD drugs means that such an action is not necessarily a done deal in this very rare and highly disabling disease with a very active patient advocacy community.
For example, Sarepta Therapeutics Inc.'s exon-skipping DMD drug Exondys 51 (eteplirsen) also received a negative advisory committee review. However, Center for Drug Evaluation and Research Director Janet Woodcock overruled review staff and granted eteplirsen accelerated approval on the basis of dystrophin production.
Unlike the eteplirsen meeting, Woodcock did not attend the advisory committee review of ataluren. In addition, FDA officials said the dystrophin production data for ataluren were not interpretable due to a number of methodological shortcomings and, consequently, accelerated approval was not on the table. (See sidebar)
FDA Concern About 'Scientific Method'
In contrast to FDA's frequent criticisms of the eteplirsen development program, agency staff repeatedly commended PTC for conducting two well-designed trials, the second of which (study 020) was intended to explore a subgroup of patients for whom post hoc analyses of the first trial (007) suggested benefit.
However, Division of Neurology Products Director Billy Dunn said that presenting ataluren as efficacious on the basis of a number of post hoc exploratory analyses of failed trials is not in line with appropriate scientific methods.
"We are not here today to endorse or rebut the magnitude of reported change, the meaningfulness of a given observation, or the pattern of effects observed on a panel of exploratory endpoints," Dunn said. "Our concern is much more fundamental and regards the basics of the scientific method, the need to formulate hypothesis, rigorously gather data to prospectively test the predictions based on the hypothesis, and if the data suggest a need to alter the hypothesis, do so and then rigorously test again."
"We have no concerns with identifying promising patterns via exploratory analyses," Dunn continued. "This is the essence of scientific discovery and is something to be encouraged. What should be approached with great caution is the tendency to draw conclusions when the data suggest a need to alter the hypothesis and test again."
Totality Of The Evidence And Flexibility
PTC sought to persuade committee members that the totality of the evidence – including clinical trial data, real-world experience in countries where the drug is approved, and comparisons of ataluren patient outcomes against natural history data – demonstrated that ataluren was effective. The company and its consultants tried to make the case that FDA was not exercising a sufficient amount of flexibility in the context of rare disease patients that are difficult to study.
Murad Husain, PTC's senior vice president of global regulatory affairs, said the agency "provided a balanced statistical review highlighting both the evidence of effectiveness and limitations of our application."
"However, the clinical review has stated a strict definition of statistical significance, p-value of less than 0.05 for the primary endpoint, suggesting that the inability to achieve this level of statistical significance alone should preclude approval. For this, FDA is holding ataluren to a different standard than prior NDA reviews for rare diseases."
PTC consultant Craig McDonald, director of neuromuscular disease clinics at the University of California-Davis and chair of a Duchenne natural history study, said the ataluren data "are remarkable in their consistency."
"If we only look at the four key muscle function endpoints from the two randomized controlled trials, we see that all favor ataluren," McDonald said. "An integrated analysis shows that the likelihood of this result being due to chance alone is 0.8%. This aligns with the statistician's analysis from the FDA briefing book where they state that there is a possible signal of treatment effect."
Xiang Ling, a senior staff fellow in FDA's Division of Biometrics I, acknowledged that the trial data suggested a signal of treatment effect for the low dose that was tested in both studies, with a statistically nonsignificant numerical trend in the primary analysis. In addition, a numerical treatment difference on the six-minute walk distance (6MWD) endpoint was seen in both studies in the subgroup of patients with baseline 6MWD between 300 and 400 meters.
However, these results are difficult to interpret for numerous reasons, Ling said, including the absence of a treatment effect seen with the high ataluren dose in the 007 study, the lack of a multiplicity adjustment for the 300-400m 6MWD subgroup of patients, and dissimilarity in the numerical treatment difference on the 6MWD between the two studies.