PhD Student, Health Data in Practice PhD Programme, Wolfson Institute of Population Health
Rheumatoid arthritis (RA) is a chronic inflammatory disorder that primarily affects the joints, affecting 18 million people worldwide. It is an autoimmune condition, where the immune system, which normally protects the body from infections and diseases, mistakenly attacks healthy tissues. Common symptoms include joint swelling, stiffness and pain. If untreated, it can cause severe damage to the joints and their surrounding tissue, and lead to systemic inflammation in the lungs, heart, eyes, and blood vessels. There is no definitive cure.
There are numerous disease-modifying anti-rheumatic drugs (DMARDs) that can be prescribed to treat rheumatoid arthritis. These drugs include as methotrexate, anti-tumor necrosis factor drugs, and biologics. However, prescription is a trial-and-error process: a patient is administered a drug for a trial period to see if there is a positive response. If the symptoms persist, the drug is discontinued, and the patient is administered a different drug. This process is repeated sequentially, over a period of time that can take years. Meanwhile for unresponsive patients, the disease progresses, and quality of life decreases. Additionally, each medication can bring its own host of undesirable side effects.
Funded through the Barts Biomedical Research Centre, interdisciplinary researchers in DERI, the William Harvey Research Institute and Barts NHS Trust have been working on novel approaches to identify the right drug for the right patient.
Over the last 15 years, successive clinical trials have taken place, with patients in different stages of RA. Drug response in RA patients was measured, and joint tissue and blood samples were collected. The tissue was imaged under microscopy, producing highly detailed histopathology images, also known as whole slide images, which were stained to highlight different cells in the tissue. RNA sequencing was also applied to the tissue samples, to capture gene expression.
Instead of following a sequence of drugs, the aim is to stratify patients by response type. This way, the patient can immediately be assigned to the drug most likely to work, rather than wait for successive therapeutic failure with ongoing disease progression and assorted side effects.
The research team developed a novel technique, called MUSTANG, which processes the multiple stained histopathology images and classifies the patient based on their most likely RA pathotype. The AI method had to overcome two challenges:
To address these challenges, MUSTANG identifies the tissue in each image, and breaks the tissue regions up into small patches. Each patch is encoded using a convolutional neural network, a type of deep learning AI system. Then, a graph is formed over the encoded patches and across all images for a patient. Finally, a graph neural network performs inference to make a prediction at the patient level. A summary of the MUSTANG processing is below.
We have shown MUSTANG produces state-of-the-art results in predicting the patient’s pathotype, achieving 92% Area Under the ROC curve. Additionally, the method runs in roughly 11 minutes. These results are very promising.
MUSTANG excels at making patient-level predictions from multiple whole slide images. In future work, the team plans to incorporate additional data into the AI model, including the RNA sequencing data. The expectation is that this will further improve results, by building on tissue information with gene expression information, including detailed information about genes and pathways what define response to therapy. The goal is to improve the method further, then translate the research into practice for drug prescription, providing a new standard of care in personalized medicine.
This research was published in the British Machine Vision Conference, held in Aberdeen UK in November 2023. A reference is below.
Amaya Gallagher-Syed, Luca Rossi, Felice Rivellese, Costantino Pitzalis, Myles Lewis, Michael Barnes, Gregory Slabaugh "Multi-Stain Self-Attention Graph Multiple Instance Learning Pipeline for Histopathology Whole Slide Images", 34th British Machine Vision Conference, Aberdeen. 2023.
For further information please see the project GitHub page which includes source code, and the online conference proceedings: