US FDA's Benefit/Risk Framework Gets High Marks But Could Be Improved

The US FDA's benefit/risk assessment framework effectively explains agency approval decisions on new drug and biological applications and is useful for product sponsors in their drug development programs, but the agency should take steps to expand their use and increase their availability, a consultant's report concludes.

"The benefit/risk framework was successful in communicating the reasoning behind FDA's regulatory decisions, useful and worthwhile to various audiences, and [was] clear and understandable to most audiences," said Valerie Overton, vice president of Eastern Research Group (ERG), which conducted an assessment of the agency's implementation of the framework.

CDER has started making the templates available for a broader set of applications than just new molecular entities and original biologic applications.

 

 

In terms of potential refinements, representatives from the agency, industry and external stakeholder groups "were interested in having benefit/risk frameworks for more types of applications, to have them be more easily findable as … standalone documents, improve the consistency and the level of detail, and to refine the template in ways to enhance the presentation of content," Overton said at a Sept. 18 FDA meeting on the framework's implementation.

The ERG assessment will serve as a baseline for FDA's commitments relative to the benefit/risk framework under the sixth iteration of the Prescription Drug User Fee Act (PDUFA VI). FDA already is moving forward with some of the suggestions, including making the templates available for applications beyond their current use for new molecular entities (NMEs) and novel biologic applications.

Crossing Over From PDUFA V To VI

FDA began working in 2009 to develop a structured benefit/risk framework, with the goals of better communicating the reasoning behind approval decisions and ensuring that the big picture of benefits and risks is kept in mind by agency staffers throughout a complex, detailed review, said Sara Eggers, an operations research analyst in the Center for Drug Evaluation and Research's (CDER) Office of Strategic Programs.

For a given drug or biologic product application, the framework is used by reviewers to summarize what is known about the targeted condition and current treatment options, as well as the evidence of benefit, risk and risk management considerations for the product under review. (See box)

Under PDUFA V, which runs through Sept.30, FDA was required to: develop a five-year plan to implement a structured assessment in the new drug approval process; hold two public workshops on benefit/risk considerations; and conduct an evaluation assessing the impact of the framework.

Under the PDUFA VI commitment letter, FDA will publish draft guidance by the end of FY 2020 on benefit/risk assessments for new drugs and biologics. The guidance will explain FDA’s decision-making context and framework for benefit/risk assessment, discuss appropriate interactions between a sponsor and FDA during drug development to understand therapeutic context, and describe appropriate approaches to communicate to the public FDA’s thinking on a product’s benefit/risk assessment.

Beginning in FY 2021, FDA must conduct another evaluation, assessing how reviewers apply the framework and identifying best practices for use.

ERG's initial assessment under PDUVA V, key findings from which were summarized at the workshop, will serve as a baseline for the PDUFA VI assessment.

More Than 300 Interviews

FDA's March 2013 draft plan called for integrating the benefit/risk assessment framework into the review process for NME new drug applications (NDAs) and original biologic license applications (BLAs) beginning in fiscal 2014, with use of the framework for efficacy supplements and all original NDAs targeted for FY 2016 and 2017, respectively. (Also see "FDA’s Structured Benefit-Risk Assessment Framework To Come Online In FY 2014" - Pink Sheet, 11 Mar, 2013.)

However, FDA's implementation of the framework has slipped somewhat from the original timeline. While the Center for Biologics Evaluation and Research (CBER) integrated the framework into review templates for original BLAs and efficacy supplements in May 2013, CDER did not implement the new template for NME and original BLA reviews until March 2015.

ERG's evaluation looked at the frameworks completed for 43 NMEs and original BLAs submitted to the agency between March 1, 2015 and Feb. 29, 2016, and for which a first-cycle action had been taken by May 17, 2017.

The consultant conducted interviews with 104 agency staffers involved in drug and biologic reviews, 55 representatives of companies whose products received approval, and 154 representatives from patient groups, healthcare organizations and healthcare providers.

Newer FDA reviewers "felt that this was particularly useful in helping them understand how to present their thinking and cover the key points in a stepwise, logical, concise fashion." – ERG's Overton

 

 

Among the FDA staff interviewed, 75% said the framework was useful in one or more ways. These included organizing thinking about benefits and risks, reminding reviewers to cover key points, training new reviewers in how to think about and weigh benefits and risks, and communicating benefit/risk analysis in a concise, standard fashion up the review chain, Overton said.

"There [were] a lot of positive comments, particularly from the newer reviewers, who felt that this was particularly useful in helping them understand how to present their thinking and cover the key points in a stepwise, logical, concise fashion," she said.

Approximately 25% of the FDA staffers interviewed thought the framework was intended primarily for communicating their analysis and reasoning outside the agency, rather than for internal use, she said.

Jeff Roberts, acting clinical branch chief in CBER's Office of Vaccines Research and Review, said the framework has been helpful for medical officers in thinking through the "bigger picture" relative to vaccines.

Vaccines have a "slightly different benefit/risk perspective, because … our risk tolerance is very low when we're considering licensing a new product that might be used by the entire birth cohort of healthy babies," he said. "We've seen that the framework can work well even with a fairly different benefit/risk perspective."

For Sponsors, A Drug Development Aid

ERG's assessment found that the framework has proven helpful to industry sponsors not only for the specific products reviewed, but also for assessing their pipelines and designing development programs.

Applicants "overwhelming felt that the benefit/risk framework is useful," Overton said. "They cited quite a number of ways in which it could be useful to them, both currently and potentially in the future as more benefit/risk frameworks are developed for more products over time."

For specific products, the individual frameworks verified alignment between the sponsor and FDA on their experiences in the product review. "They were appreciative of seeing in writing in a concise fashion that the way FDA portrayed the review – in terms of what they thought about, how they thought about it, what the discussions were – reflected their own communications with FDA during the review process," Overton said.

The frameworks were a useful tool in communicating FDA's perspective about a drug application to a sponsor's upper management, investors and business partners. In addition, they served as a form of competitive intelligence that aided pipeline development.

For sponsors, the frameworks were a useful tool in communicating FDA's perspective about a drug application to upper management; they also served as a form of competitive intelligence that aided pipeline development.

 

 

"It seemed the benefit/risk framework not only for their own products but for other products enabled them to glean insights about FDAs thinking, what FDA's concerns were, what FDA thought of as … the key factors in their decision, so that they could apply those lessons learned and that thinking to their own development programs to ensure that those development programs are as effective and focused as possible," Overton said.

Applicants noted one limitation with the frameworks is that they are only available for approved products, and not for those subject to complete response letters.

"Applicants generally responded that they would like to see that benefit/risk framework for nonapproved applications as well, and that they would like to see those privately, not published on FDA's website," Overton said.

A Credible Source of Information …

Non-industry external stakeholders said the frameworks promoted transparency and helped them better understand the therapy and decide whether to use or prescribe it. The documents also aided in the interpretation and sharing of information about new therapies.

"Many of these external stakeholders said that they appreciated seeing the opinion of credible, objective experts at FDA," Overton said. "A lot of them pointed out that when they read information about new products … some of them will actually go to the literature and look at FDA's website for the more technical and lengthy explanations of the product, but it's rare to find a concise explanation of how and why a particular product was approved, and it's even rarer to find that from an author that has the credibility and the objectivity that FDA does."

These external stakeholders favored expanding use of the frameworks to include efficacy supplements.

FDA appears to be moving in this direction. Eggers said that beginning this month, CDER will be testing the frameworks on a broader set of applications where appropriate, such as those for new indications or new patient populations.

… If You Can Find It

External stakeholders also want FDA to make the frameworks easier to find.

Currently, they are incorporated at the beginning of multiple review documents for products regulated by CDER, and into a single review document for CBER products. Although these review documents are publicly available on FDA's website, they can be hundreds of pages long.

"Almost all of the folks that we interviewed indicated that they would not have known to look there to find the benefit/risk assessment," Overton said, "and so what they suggested is that FDA publish these as standalone documents that are easily searchable and findable through Google or through a search of the FDA website."

Interviewees across all groups identified a need for more consistency in the documents.

"In some cases, benefit/risk frameworks were at a more summary level and were …one or two or three pages, and in other cases the level of detail was quite greater than that, resulting in frameworks that could be 10, 15, 20 pages long," Overton said.

Respondents also suggested refining the framework template to enhance presentation of content. "These are not foundational changes, but rather kind of tweaks and refinements, to for example add a concise, well-structured conclusion statement to bring everything together in one or two sentences."

Some respondents said the frameworks should include clinical considerations and more details about review issues, particular those for which there were differences of opinion among agency reviewers. Overton said a small number of respondents thought the framework should include more quantitative information and perhaps even focus on quantitative benefit/risk assessment rather than a more qualitative approach.

Eggers suggested there is an opportunity to explore more technical approaches within the qualitative framework to inform benefit/risk assessment in targeted cases.

"Examples could be structured techniques that characterize uncertainties inherent to the assessment," Eggers said. "We find that it is often what makes these decisions challenging – before you ever get to the benefit/risk tradeoffs – is to truly understand what the uncertainties are."