lexoro Case Study: Automated Clinical Evaluation Suite

The fundamental goal of Clinical Evaluation is to evaluate and analyze clinical data. In doing so, it is crucial to provide evidence for the clinical safety and performance of the product. Through the interaction of different departments and areas within a company (e.g., Risk Management and Post Market Surveillance), various inputs serve as parameters for a successful evaluation. Since clinical evaluations – depending on the classification of the respective product – have to be repeated and renewed on a regular basis, they provide indispensable insights throughout the entire product life cycle. Thus, it can always be ensured that a product or device functions according to the rules of the MDR and does not pose a risk to patients or users.

The Medical Device Regulation is above all and is therefore associated with most of the pain points and responsible for many difficulties for manufacturers. These pain points and difficulties for manufacturers – especially in the area of clinical evaluation – are as follows:

• New rules of classification bring more products into higher risk classes and thus increase the effort and requirements for clinical evaluation and necessary documentation and reports.
• Medical device manufacturers need to invest more in clinical studies and investigations for their products with lower risk classification. This in turn requires more time, staff or resources – especially for a broader range of products.
• An optimal CER structure must take into account MDR requirements as well as MDCG guidance, industry standards and MedDev aspects that remain relevant.
• In addition, there is the increased need for CE-marked products that require Post-Market Clinical Follow-Up (PMCF). Conversely, this again increases the effort for companies and also increases the database that must be evaluated and implemented for documentation.

lexoro has focused on helping companies & departments accelerate and optimize workflows & work steps in the wake of increasing requirements and challenges in regulatory affairs in regulated industries.

Our lexoro approach: we look for processes or tasks that are currently inefficient and would benefit from simplification or improvement. By restructuring them into automatable parts, we can develop a powerful and user-friendly solution that helps experts with tedious tasks, thus increasing efficiency.

This Intelligent Automation approach, in combination with our focus areas, was successfully deployed & implemented in the project and delivered results that largely resolved the pain points of the employees.

After the initial discussions at the start of the collaboration, it quickly became apparent that automation for clinical evaluation in the experts department would make sense in the form of modular automation sprints.

From the lexoro focus areas – „data processing & document generation“ and „information acquisition & evaluation“ – it was possible to develop an optimal concept for the customer in 3 modules. The all-in-one, web-based Clinical Evaluation Suite.

Briefly summarized, the 3 modules enable the automation of the Clinical Evaluation – starting from the information search in databases & literature, to the evaluation & classification of the results, to the populating of tables and the generation of the Clinical Evaluation Report (CER). These modules can be used independently of each other, can be combined individually, and can thus be flexibly selected for the pain points & difficulties within the department.

Additional automation technologies – particularly in the form of Natural Language Generation and Data Science, as well as Machine Learning – can be used to leverage even more potential for process optimization. For the customer, the main problem areas were limited to these 3 process steps, which initially made the 3 modules an ideal fit for the automation strategy.

But how do the 3 modules – „Search“, „Evaluate“ and „Report“ – work in detail?

The Search Module offers the possibility to perform automated searches & investigations. In particular, it is a matter of information acquisition and data collection in various necessary sources – such as classic databases and literature – in different languages. The so-called crawler technology behind this is driven by Robotic Process Automation. Typical advantages of RPA are the error-free repetitive execution of predefined standard processes as well as the efficiency in terms of lead time. Thus, time savings of up to 90% can be achieved in many work steps with purely RPA-driven processes. In addition, RPA and the crawlers behind it offer the possibility of being flexibly and individually applied to workflows – exactly according to the customer’s ideas. For the 3-stage clinical evaluation project, for example, the RPA database crawler could be flexibly „set“ to Medline, FDA Recall and Embase. Further databases are only a question of requirements and the integration is therefore no problem. Typical foreign databases can also be connected: the ANSM in France, MHRA in the United Kingdom or the PDMA in Japan also serve as a solid basis for the lexoro Clinical Evaluation Crawler.

Regarding the process or the procedure for the employees: first, some manual processes take place for preparation – displayed in blue here. These include the selection of the databases to be searched and the definition of the search strategy. Here, various keywords can be selected, such as the product type, new approvals, incidents or others. In addition, the necessary time period or a specific product name can be selected. In general, there are two options for integrating the search strategy into the crawler: first, the strategy can be uploaded to the system as an Excel file in order to transfer the data. The second possibility offers the direct input of the search strategy in a web interface – as shown in the exemplary Software Interface.

When the search is started, the crawler works completely independently. The search strategy is processed fully automatically and the necessary databases and literature sources are searched. The hits of the search are collected and finally all results are displayed in a list for an overview, which can either be further processed directly in the system in Module 2: Evaluate or made available to the employees via export. As is usual with lexoro solutions, the structure of the Excel export can be adapted individually & customer-specifically. In the given project, for example, the year of publication, author and title of the publication as well as abstract were searched, collected and exported.

The advantages summarized: The RPA crawler technology processes your search strategy without errors. The result is an automated database search in real time with which you can achieve up to 90% time savings. Your employees are relieved of tedious standard tasks and can devote themselves to other activities.

The second module of the automation for clinical evaluation deals with the classification & evaluation of hits and search results from e.g. database searches. As already mentioned, the modules can be combined individually. For example, the second module can also be integrated without module 1, as long as a databasis for the evaluation is available. Usually, however, both modules are selected together, since on the one hand the interaction and the necessary parameters can be perfectly coordinated and on the other hand the search module with the high degree of automation and the quickly achievable return on investment represents the optimal entry into the world of automation for many customers.

In addition to classic RPA tools, which are responsible for integrating search results into the lexoro classifier, for example, the technology in the second module revolves mainly around Artificial Intelligence. lexoro relies primarily on Natural Language Processing in the area of AI with aspects such as Information Extraction, Named Entity Recognition, Relationship Extraction and Sentiment Analysis. This allows valuable and, above all, relevant key terms to be identified & extracted from the texts and literature. Based on this information, the lexoro classification algorithm can make decisions. By using criteria that are individually tailored to the customer, the relevant processes can thus be automated. In the case of clinical evaluation, it is particularly about AI-assisted inclusion & exclusion evaluation as well as appraisal. In this way, collected literature and database hits can be checked for „relevant“ vs. „not relevant“.

In the case of the classifier, we often still speak of so-called partially automated support, since the AI technologies cannot yet determine the relevance for all occurring cases with 100% accuracy and it would also not make sense – especially in the medical environment with a very low error tolerance – to leave the human factor completely out of the equation. For this reason, lexoro classifiers often use methods to have employees check again for „uncertain“ cases – the so-called human-in-the-loop concept.

Nevertheless, the evaluation and exclusion of „definitely not relevant“ articles and publications can save enormous resources and accelerate the overall process. After the lexoro algorithm has evaluated the necessary hit lists, the questionable results are passed on to a member of staff for review and the rest are presented in a list (e.g. Excel or Word) and passed on to the relevant department.

Table filling and document generation characterize the third module: Report. In the Clinical Evaluation module, the Clinical Evaluation Report (CER) is generated to conclude and summarize the results. This is analyzed clinical data obtained either from a clinical trial of your product or from the results of other studies on essentially equivalent products. Since the CER is often based on the same or at least a similar structure within a company, the automation of text blocks and the populating of chapters lends itself very well.

In the course of the project, in cooperation with the customer, it was worked out which chapters & parts of the CER are suitable for automatic populating of the template. In the current solution, it is now only necessary to select the template for a specific chapter. After that, lexoro’s RPA technology takes the wheel again. First, the necessary and relevant data is collected. If the document generation module is used independently of the other two modules, the automation can independently compile a file base here, which is used as the basis for subsequent populating. In the current project, however, all 3 modules were integrated and so there was only the need to collect the results from the previous two modules, adapt them if necessary and then prepare them for populating the templates. For the customer it was important here that the automation has access to a specific in-house system in order to be able to collect important documents.

As soon as all files & information meet the quality requirements, a fully automatic populating of the desired chapters and/or tables takes place in the CER. Here, an automated transfer of textual sections and chapters into the CER is carried out, whereby any customer-specific templates are supported due to the very flexible possibilities of document generation.

In summary, the lexoro document generation tool exactly meets the customer’s challenges for the CER. Fast and easy integration of text passages from the previously searched and screened documents & results and this automatically into the correct chapters of the CER. Due to the high flexibility and universal applicability of the CER tool, more chapters and functions will probably be added to the tool in the future. Also in this module up to 70% time saving or automation level can be achieved. A real gamechanger.

End-to-end automation was implemented for the customer. From the initial database search and literature review, to the evaluation and classification with information extraction, to the creation & populating of chapters of the CER with the relevant text passages.

Your advantages & effects at a glance:

• Standardization of clinical evaluation with compliance-oriented collection & processing of clinical data
• Customizable modules for different areas of the Clinical Evaluation process with high flexibility and focus on customer specific requirements
• All-in-one, end-to-end and web-based solution with secure data processing and readiness in 4-6 weeks
• Time savings of 70% and thus relief for the clinical evaluation staff

The complete scope & explanation of the solution with all the supplementary small features would exceed the length of this paper. However, this is exactly what convinced the customer in the initial project phase: the individual lexoro approach, whereby almost all of the customer’s wishes and special requirements could be integrated into the system.

The first step is a free brainstorming session (teams, zoom, or similar) to discuss the problems, goals and how to proceed. Helpful information in advance: a precise process explanation and possible input & output files or information that will be processed during the process to understand the steps exactly. In the further course, the use case and the application scenario are precisely specified – i.e. which processes and work steps can be automated and which benefits and effects can be generated. This includes use case analysis & customization, an initial effort and cost estimate, and possibly a feasibility study if technical feasibility needs to be demonstrated. The third step is to develop a pilot project. The pilot is the entry into the world of automation and offers first important functionalities that can be fully used in daily operations. Due to a reduced set of features, a pilot can be introduced quickly and thus be used directly by the team for productive work steps.

This stage is also used as a test phase of the pilot with subsequent feedback for further development of the system with any changes for an overall solution.

A so-called Minimum Viable Product (MVP) is characterized by further fine-tuning of features, expansion of requirements and functions, and as preparation for deployment or launch of the solution. The „advanced pilot“ is launched after the initial pilot project.

In step 5, extensive planning for the implementation of an overall solution – the so-called product or solution development – takes place according to the agile „sprint model“. As a general rule, an RPA pilot takes about 4-6 weeks – depending on the scope and complexity. For an extensive solution, as described in this example, about 3-4 months are estimated. Your advantage here: our agile sprint model. We divide our projects into sprints of 2 weeks.

This way you can determine the budget yourself and decide which elements should be automated. Thus, you can implement adjustments and enhancements at any time. You have full control and one thing is certain: within a very short period of time, considerable successes are achieved for both your company and your employees.

Through this flexible and individual approach, we offer our customers the possibility to develop their own product with the exact requirements tailored to them. This is at the same time the decisive factor in today’s MDR landscape. By integrating flexible automation solutions, some challenges in medical technology can be addressed. Conventional, out-of-the-box software often doesn’t cover all the requirements you need for complete and compliant clinical evaluation or CER creation. In addition, requirements are constantly changing or becoming more stringent. External expertise is usually only available at short notice and is often associated with the loss of know-how. Know-how that your employees urgently need when these „outsiders“ or even colleagues leave. These challenges are compounded by the increasing number of manual tasks and the general difficulty of the expert labor market. With lexoro solutions and the combined power of industry experience and automation know-how, you can tackle these challenges and prepare for the future.