MedPerf speeds medical AI while protecting data privacy.

MedPerf speeds medical AI while protecting data privacy.

AI in Medicine

Artificial intelligence (AI) has made significant advancements in various industries, and one area where it holds tremendous potential is medicine. However, applying AI, specifically machine learning (ML), to medicine has always been challenging due to the sensitivity of patient data. Researchers and developers have been grappling with how to leverage real-world data sets without compromising privacy. Enter MedPerf, a groundbreaking effort that aims to revolutionize the way AI is trained and implemented in healthcare.

MedPerf, formed by the non-profit MLCommons Association, has introduced a novel approach known as “federated” training of AI. The goal is to strike a balance between data privacy and the need for algorithm developers and clinicians to benefit from real-world data sets and newly developed ML models. In a position paper published in the prestigious scientific journal Nature, MedPerf outlines its mission to solve the data impasse.

The MedPerf benchmarking platform takes AI models and sends them to clinicians who possess the relevant data. The clinicians then evaluate how the model performs against the data. This approach allows AI developers to access private datasets that they would otherwise never have access to, while clinicians gain insights into the potential of AI in predicting patient outcomes. The data remains securely within the clinicians’ facilities, ensuring privacy is preserved.

“This approach aims to catalyze wider adoption of medical AI, leading to more efficacious, reproducible, and cost-effective clinical practice, with ultimately improved patient outcomes,” explains MedPerf in its Nature Machine Intelligence publication. The group, composed of more than 20 companies, including tech giants like Nvidia and Microsoft, along with academic institutions and hospitals across 13 countries and five continents, emphasizes the potential of MedPerf in various biomedical fields beyond radiology and surgery, such as genomics, computational pathology, and natural language processing.

The core principles of MedPerf are presented in a summary schematic provided by MLCommons, showcasing the platform’s potential impact on the healthcare industry. David Kanter, the executive director of MLCommons, expressed his pride in the broad community engagement MedPerf has garnered, emphasizing its potential for improving medical care worldwide.

MedPerf’s Platform

MedPerf’s platform, comprised of MLCubs, offers a secure environment for application containers that ensures the confidentiality and integrity of data. The platform consists of three MLCubes: one for data preparation, one for hosting the model, and one for evaluating the model’s performance. This comprehensive infrastructure enables seamless and secure benchmarking of medical AI models.

MedPerf has partnered with industry leaders such as Hugging Face, a repository of AI models, and Sage Bionetworks, the developer of the Synapse platform for data sharing. These collaborations further enhance MedPerf’s capabilities, facilitating automatic evaluation of models and supporting the execution of community challenges.

The platform’s effectiveness has already been demonstrated through the Federated Tumor Segmentation Challenge, where MedPerf successfully identified brain tumors using MRI images across 32 participating sites on six continents. In addition, pilot studies with academic groups involved in multi-institutional collaborations have further validated MedPerf’s potential.

MedPerf acknowledges its current stage as it transitions from an alpha to a beta phase, with plans to expand its platform and engage more participants. The organization invites healthcare stakeholders to contribute by forming benchmark committees and registering their data on the platform, promising that no data sharing is required.

MedPerf’s code is readily available on GitHub, encouraging transparency and collaboration within the medical AI community. With its innovative and privacy-preserving approach, MedPerf paves the way for widespread adoption of AI in medicine, promising a future of more effective and personalized healthcare.