US FDA updated its approach to assessing the accuracy of tests that manufacturers want to sell for point-of-care use in response to a congressional mandate, but a key industry stakeholder says the agency misread lawmakers' message.
The agency issued a draft guidance Nov. 29 on studies to support accuracy for CLIA (Clinical Laboratory Improvement Amendments) waiver submissions. The document, once final, is intended to replace one section of a larger 2008 guidance that sets out the standards for a test-maker to prove that an assay is "simple and accurate" enough to be used by non-lab-trained personnel, for instance, at a community health clinic or at the hospital bedside, with a negligible change of erroneous result.
It’s the "accurate" part of the standard that has been the most significant source of controversy since the 2008 guidance was released. Industry groups say FDA's adopted approach has unnecessarily restrained the point-of-care sector and patient access to tests, and, they argue, it does not follow the law. A coalition was formed in 2014 to change FDA's policy and it was successful in lobbying Congress to include a provision in the December 2016 21st Century Cures Act mandating the agency to revise the "accuracy" section of its 2008 guidance. (Also see "21st Century Cures: Device Provisions" - Medtech Insight, 14 Dec, 2016.)
James Boiani, the attorney who spearheaded the Coalition for CLIA Waiver Reform, says FDA's new draft still gets the issue wrong and does not follow the intent of the law. "We have a lot of work to do on this," he said in an interview. "The guidance still has a very fundamental flaw."
An Accurate Definition?
The crux of the issue is a disagreement over what the CLIA statute intends with the word "accurate."
In the latest draft guidance, the agency says that accuracy "is a widely used and generally understood term in clinical laboratory science," meaning, the closeness of agreement between a test result and an accepted reference-test result.
But industry disagrees that Congress intended for this to be the meaning in the context of CLIA waivers. Because the focus of the statute is on the impact of an untrained user on test accuracy, companies argue, the measure of accuracy should be whether a test is as accurate in the hands of an untrained user as it is with a trained user.
These two interpretations lead to different practical standards. FDA's 2008 CLIA waiver guidance calls for a prospective study comparing a test to an appropriate reference or gold-standard method in the hands of untrained users. Industry says this has created a higher accuracy standard for waived tests than for tests 510(k)-cleared for use in a sophisticated laboratory, which only need to show substantial equivalence to a predicate, and an unneeded burden beyond a study comparing test performance based on users. As a result, the time and cost of gaining a CLIA waiver for a novel test has skyrocketed since 2008, before which the standard was different, industry groups say. (Also see "Coming Together On CLIA Waivers? Industry Hopes To Recruit Patients, Providers To Push Reform" - Medtech Insight, 6 Jun, 2014.)
The Cures Act asks the agency to revise "Section V." of its guidance addressing the accuracy standard and, specifically, to incorporate "the appropriate use of comparable performance between a waived user and a moderately complex laboratory user to demonstrate accuracy."
In response, FDA's draft guidance makes some changes. Specifically, it offers the option ("Option 1") to do a study comparing application of the test by untrained users versus trained users, if the original marketing submission application, such as a 510(k), includes a study comparing the test to a calibration or reference method in trained users. "If the sponsor chooses to demonstrate the accuracy of the test when performed by trained operators as part of the marketing submission through comparison to a traceable calibration method (or reference method), the sponsor can leverage these data in combination with a new study to demonstrate agreement between results of the test performed by untrained operators and trained operators in the waiver by application submission," the draft guidance states. "Valid statistical methods may then be used to estimate the accuracy of the test when performed by untrained operators using a combined analysis of the data from both studies."
But when a standard 510(k) approach is employed, i.e., the test is compared to a predicate rather than a reference method to support marketing authorization, the CLIA waiver application must directly compare the test in untrained users to a reference method to demonstrate accuracy ("Option 2"), the draft guidance states.
In any case, the document notes, "For those sponsors that want the best study to determine accuracy, the study design described in Option 2 is preferred for all types of tests (qualitative, semi-quantitative and quantitative), even when accuracy was determined in the marketing application with trained users."
Boiani says this adjustment does not amount to a meaningful change and that he would not anticipate any improvement in the rate of CLIA-waived tests reaching the market if the document were finalized. "This is not going to fix the problem," he said.
The statement that accuracy "is a 'generally understood term in clinical laboratory science' ... that is the [FDA] Office of Chief Counsel trying to build an argument that the plain language of the word 'accuracy' allows them to interpret it this way," Boiani claims. But, "based on the legislative history of FDAMA [which updated the CLIA-waiver provision], based on 21st Century Cures and the legislative history there, and statements by Sen. [Richard] Burr and others who are working on it, it is extremely clear that is not what Congress intended."
But the draft guidance on CLIA waiver accuracy standards can't be looked at in isolation. Simultaneous to its posting, FDA also issued a draft guidance on "Recommendations for Dual 510(k) and CLIA Waiver by Application Studies" that the agency says describes an approach that will "speed up the process of bringing simple and accurate IVD devices to CLIA-waived settings, which will better serve patients and providers."
The conventional CLIA waiver approach is sequential – first get a 510(k) clearance (or PMA approval) and then submit a distinct "CLIA waiver by application." The 2012 user-fee reauthorization (MDUFA III) established a new dual-submission pathway to address both clearance and waiver in one review process.
The dual-submission pathway has generally been perceived as long-winded and not the most efficient path to market for most products. Between FY 2013 and FY 2016, FDA processed a total of five dual submissions, compared to 37 distinct CLIA-waiver applications. But the two pathways almost equalized in FY 2017 when six dual submissions were accepted versus seven standalone CLIA-waiver applications, according to data released by FDA last month.
The new draft guidance suggests that the agency views the dual-submission approach as a way around some of the inefficiencies that arise with standard, stepwise process for new IVDs. "FDA believes this pathway is in many instances the least burdensome and fastest approach for manufacturers to obtain a CLIA-waived categorization in addition to 510(k) clearance for new in vitro diagnostic devices," the draft guidance states. The guidance incorporates FDA's experience with the dual submission pathway under MDUFA III to find efficiencies, the agency says. Applying the recommendation in the document, "an applicant may choose to conduct a single set of comparison and reproducibility studies with untrained operators to satisfy associated requirements for both 510(k) and CLIA Waiver by Application," FDA explains.
Under MDUFA IV, FDA committed to completing 90% of dual 510(k)-CLIA submissions within 180 days. The performance goal for standalone CLIA waiver applications is 90% in 150 days.
Boiani isn't convinced, however, that the dual submissions pathway will have a broad-based impact on getting more point-of-care tests to market. "In some instances, I think, the dual submission approach can have a lot of benefit, but I think it can't be the only solution."
The combined process is still going to take longer, sometimes significantly so, than simply getting a 510(k) clearance. For many companies, the attorney noted, getting on the market more quickly and then expanding to a broader point-of-care market base is preferable to support a revenue stream and, for smaller companies in particular, investor interest. "If I am going to do a 510(k), it is going to get reviewed a lot faster, and we will have a marketable product, maybe restricted to who I can sell it to because it is not CLIA-waivered, but we will have a marketable product a year or years ahead of where I would if I go ahead with this dual submission process," Boiani said. "For a lot of innovative companies, that matters."
In addition, the fundamental accuracy standard remains in play. "You can still be running into situations, maybe you are comparing to your predicate and you're performing as well as the predicate, but you are not reaching this 95% accuracy," he noted. "They still need to start in the right place, and the right place is getting accuracy defined as comparable performance between trained and untrained users using the same device."
Routes To Reform?
Boiani's reaction is an early sign that the comment period for these two draft documents, which ends Jan. 29, could be an active one. A primary effort by the Coalition for CLIA Waiver Reform is to engage with the end-user community and patient groups. For instance, they have worked with the Convenient Care Association, a trade association for retail-based health clinics, and the National Association of Worksite Health Centers, which facilitates onsite health and wellness centers for employers. Boiani expects those groups and others will be "very vocal" about the impact of the policy on restricting their access to needed tests.
Another cohort that could conceivably get involved in the process is members of Congress, if, in fact, there is a perception that the guidance does not follow the intent of current statute. Sens. Richard Burr, R-NC, and Al-Franken, D-Minn., introduced the legislation including the CLIA-waiver provision that was ultimately included in the Cures Act. Burr was also involved, as a member of the US House, in the CLIA-waiver provisions included in the FDA Modernization Act of 1997. "I hope they weigh in," Boiani said.
Another factor is questions over the specific FDA personnel who will be involved in revising the draft guidance documents. Alberto Gutierrez retired as head of the Office of In Vitro Diagnostics and Radiological Health (OIR) in September, and FDA is searching for a replacement. (Also see "Dx Departure: US FDA's Diagnostics Chief Gutierrez Steps Down" - Medtech Insight, 28 Sep, 2017.) The approach taken to the CLIA waiver draft guidance, Boiani says, "definitely speaks to a need to get someone from outside the agency to come in and bring some new perspectives, because this is more of the same."
From the editors of The Gray Sheet