Mock audits, or pilot audits, are not permitted under the EU's new Medical Device and IVD Regulations. That seemed to be the consensus of a notified body panel at the Knect 365MedTech Summit in Brussels on June 11.
Yet, one notified body speaker at the meeting spoke openly about how his notified body has been conducting mock audits. Also, during the coffee breaks, companies were tellingMedtech Insight that they were still preparing for such audits with EU notified bodies and that these audits were scheduled imminently.
Furthermore, a show of hands in the auditorium showed that some 5%-10% of delegates at the meeting were still hoping that they could benefit from a mock audit from their notified body.
"The new regulations are not an upgrade – they are hugely disruptive, and preparing for them is like never having been on the market before," TÜV-SÜD's Bassil Akra says.
So what is going on? Why is there such confusion? And what is the official position? If mock audits are permitted, are they being done for the benefit of the notified bodies, or the benefit of companies?
The answers to these questions, it seems, are not straightforward and ambiguity has led to a situation where some notified bodies and companies may seem to have benefited pilot audits than others. This, it seems, is because of the interpretation by local regulatory authorities, or by the notified bodies themselves, of what is permitted.
The reason this issue is so contentious is because some companies fear that others may have been given an unfair competitive advantage if they have gone through such a process and already had their products reviewed – even if to a limited extent– against the requirements of the Medical Devices and IVD Regulations.
Also, any notified body that has carried out some form of assessments of a company against the new MDR and IVDR is clearly going to be better prepared in its application and for its joint assessment by the designating authorities than a notified body that has not.
'Clearly' Not Allowed!
Guy Buijzen, president of the EU medical device and IVD notified bodies association, TEAM-NB, was in no doubt about the status of mock audits. "It is very clear … they are not allowed" before the notified body is designated, he told the meeting. Buijzen is also regulatory affairs manager and senior project manager at the notified body, DEKRA, and a former chairman of NB-Med (the European Commission-led notified bodies group).
Buijzen acknowledged that "various competent authorities" allow their notified bodies to have some of their assessment procedures "tested" in a mock review using customers' technical documentation as part of the preparation for the joint assessment in the designation process.
But, he stressed,the European Commission and competent authorities see mock audits as "consulting activities" and therefore not permitted.
Companies need to take this fact on board, he said. Timelines are very demanding in terms of technical documentation assessment procedures. Companies themselves internally need to be as critical as notified bodies are. Notified bodies do not have the time for several rounds of questions and answers once we begin working against the new requirements.
One particular area in which companies really need to invest, Buijzen added, is in clinical data. The show of hands here – if the audience is representative of the various medical device classes – suggests that not enough firms are investing sufficiently in clinical, he challenged. This is a totally new system. Companies need to step up to it, he said.
Dr Bassil Akra, of notified body TÜV-SÜD Product Service, agreed with Buijzen that mock audits/assessments are not allowed based on the feedback received from the joint assessment team.
"The European Commission does not want notified bodies to do audits and/or early assessments without receiving an official application for conformity assessment according to the new legislative framework – which is not possible without being notified for the new Regulations," he added.
The Commission believes that notified bodies should not be involved in early independent preparation activities. Instead these should be performed by medical device consultancy companies, Akra, VP of the global focus teams at the German notified body TÜV SÜD Product Service, told meeting attendees.
However, in Akra's view, although this is the regulators' current interpretation,itdoes not make sense that mock or pilot services are not allowed, taking into consideration the short period that has been given for implementation, as well as the uncertainties associated with the new regulations.
"Not permitting them raises the risk that manufacturers will not have their devices compliant in time, since they would not understand the expectations at an early stage," Akra said.
This is a critical issue to resolve, he asserted. "The new regulations are not an upgrade – they are hugely disruptive, and preparing for them is like never having been on the market before," he said.
What Notified Bodies Can Do
Akra confirmed that notified bodies cannot and are not doing consultancy activities, but, in his view, they "can do early independent activities raising non-conformities at an early stage." He posed the question of who, else was going to prepare the device industry for the wave of new requirements. "Someone is needed to tell you now – and not in 2020 – whether your approach is right or wrong," he asserted.
And while many companies may decide to hire consultants to have an independent assessment of their interpretation of the new regulations and of their compliance, he said, nothing offers firms a guarantee that it will be certified at the end of the day.
The big challenge with the new regulations is that they are evolving; every day there is a new interpretation of the texts, Akra asserted, and there are also inconsistencies when it comes to the translation into the different European languages of the texts.
BSI's 25 Or So Mock Audits
Ever pragmatic, the UK's MHRA seems to have used this loophole that notified bodies can raise questions at the early stage to permit its notified bodies to perform some kind of pre-assessment testing.
Gary Slack, senior VP, global medical devices of BSI (the large notified body based in the UK), explained that BSI has indeed been doing some types of mock audits with its companies. It has been taking guidance from the UK authority responsible for devices, the Medicines and Healthcare products Regulatory Authority (MHRA), as part of its efforts to prepare to meet the challenges of being a notified body under the Medical Device and IVD Regulations.
He explained via teleconference at the Knect365 meeting that there are six technical teams at BSI, each representing a different, broad products sector (one of them is focused on IVDs). Each of these teams selected a handful of manufacturers – small companies among them – to assess as a way to help the notified body check and understand the new processes that it needs to conduct under the new MDR and IVDR.
"We were careful to devise a structural gap analysis only" in a mock audit, says BSI's Gary Slack. "We were not able to make recommendations as we are not yet designated."
There were some 25 of these types of audits conducted in total, Slack later told Medtech Insight.
"We performed a type of gap-analysis exercise to understand the applications of the new processes that we are putting in place to meet the essential requirements of the new Regulations and also to allow our customers to do a gap analysis toward meeting the requirements of the regulations that come in 2020," Slack said.
But BSI was instructed by its competent authority to keep assessments to a minimum and not do work that would contribute to a final MDR audit review. The European Commission and the competent authorities have all made it clear that such audits should not stray into the realm of pre-audits against the MDR, Slack acknowledged.
This means that BSI raised the first line of questioning to the companies – a familiar first stage within the normal conformity assessment process involving the undertaking of an initial review of the manufacturers' technical documentation – it did not raise a second line of questioning or go any deeper into the conformity assessment process.
"We were careful to devise a structural gap analysis only," Slack said, specifically avoiding preparing any individual company for their actual audit still to come. "We were not able to make recommendations as we are not yet designated," Slack added.
He noted that this "mock audit" process has revealed some interesting differences in interpretation among different companies of the requirements of the MDR and IVDR, and this has been a valuable learning point.
For BSI, the process was useful; there were takeaways that have helped the organization improve and do the processes more efficiently, Slack noted.
"I don't think we will be doing any more mock audits as we are not permitted to do too many," he said.
TEAM-NB: 'It Depends'
Speaking on behalf of the EU medical device and IVD notified body association, TEAM-NB, the group's director Françoise Schlemmer told Medtech Insight that the answer to whether mock audits are allowed or not depends on what is meant by the term "mock audit."
"No audit can take place against the MDR or IVDR before a notified body is designated," she confirmed, adding that not all the details of the regulations are in place yet, as there are some essential delegated and implementing acts pending.
If the intention of the mock audit is to perform assessments against MDR and IVDR that could be taken into consideration to then issue the final conformity assessment certificate with shorter audits once the notified body has been designated, then this is not allowed, she said.
This was clarified at a meeting last November at a notified bodies-only session, she said, and is explained in a report. But that report is not publicly available, she admitted.
If, however, the mock audit involves audits against MDR and IVDR that are intended as a gap analysis to identify what the manufacturer still needs to do to answer the new requirements and the process has no influence on the duration of the official audit against MDR/IVDR that will take place after the notified body designation, then that is allowed, Schlemmer explained.
So far so good. This seems to make sense.
As for ongoing mock audits, Schlemmer suggested that "it is always possible for a manufacturer to request a “mock up” or “blank” audit where notified bodies assess the requirements still to address.
At this point, understanding the difference between what counts as permissible and what does not before the notified body is designated starts to become more complicated.
Schlemmer further explained that "the gap analysis/blank audit/mock audit or whatever you call it - is a less formal way that allows manufacturers to point out areas where they have doubts," and can ask the notified body to take a look. This will ensure that when the audit takes place against the new Regulations, that there are likely to be fewer non-conformities, or at least the objective is that there is no major non-conformities," she said.
Again, this becomes confusing since this would surely shorten the duration of the final audit under the MDR/IVDR compared to if the mock audit had not been undertaken.
In response to the question of whether the main objective of these mock audits is to help the notified body in its preparation for work against the new regulations, rather than the manufacturer, Schlemmer said this is not the case. "It is definitely intended to help the manufacturer get it right at a first go," she said. And she added that such mock audits can continue even now.
Notified bodies are in favor of these efforts, but the organizations have so much pressure and so many audits and assessments that need to be performed, and little time to do everything, she added.
Schlemmer's comments seems to contradict statements made by Slack from BSI, for example; he suggested that the premise on which mock audits are permitted is to aid notified bodies in their designation application.
Are Things Any Clearer?
There is no doubt that the debate over mock audits is a finely nuanced one, and that confusion likely remains over whether a "mock audit"or "trial audit" or "pre-assessment" falls is allowed?
It is hard to imagine the European Commission or national authorities penalizing a company for work already completed in this increasingly overstretched regulatory environment – unless it is blatantly obvious that there have been efforts to finalize conformity assessments on behalf of companies and bias toward individual manufacturers.
There is the potential for companies to flood notified bodies with requests to have notified bodies perform a gap analysis or "early independent activities," raising non-conformities at an early stage.
But with bottlenecks predicted even without such early requests, many manufacturers are likely to see the notified bodies' gates squeezed shut against the bulge of applications. This is likely to happen as the testing and certification bodies focus their efforts on recertification against the current directives while also coping with joint audits by the designating authorities, and closing out their own non-conformities to finally become notified bodies against the new MDR and IVDR.
Regardless of whether the process has been fair, it seems that the show must go on and manufacturers and notified bodies alike will have plenty of other concerns on which to focus their attention.
Medtech Insight did not receive a response on the official position of the European Commission and the UK's MHRA before going to press.
From the editors of Clinica